2024-02-26T05:52:29.962 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-26T05:52:29.963 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421 --owner scheduled_vshankar@teuthology --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} -- /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/orig.config.yaml 2024-02-26T05:52:30.022 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T05:52:30.112 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421 branch: wip-vshankar-testing-20240216.060239 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} email: vshankar@redhat.com first_in_suite: false job_id: '7573421' kernel: client: branch: testing flavor: default kdb: true sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240216.060239 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240216.060239 sha1: ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2171 sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 sleep_before_teardown: 0 subset: 54/1024 suite: fs:workload suite_branch: wip-vshankar-testing-20240216.060239 suite_path: /home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa suite_relpath: qa suite_repo: https://github.com/vshankar/ceph.git suite_sha1: ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrDu0evKNPSsjtnw5GLpAqIRSFCiLkfZE6SMlt9B9o8F6Ou+2RhSpooORwsJMkfew7OsrufFf5jwsomNr6VMrs= smithi080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMaAGpLNMrVubKGPygxK0ILlx9L8LVuxUCwiomS/0g/y5jedb+QLejRwrgvB/M60iRatlQ9eCV5ouV5fo/CLULM= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLuu6LRYtl16aE5cYvFgLTTTUrx+Bph3hXT94rBMJ2R07130X7x2/ieiMX9PxPQTLULqfn443KJuh6MrPvxex98= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-26_05:44:42 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-26T05:52:30.113 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa; will attempt to use it 2024-02-26T05:52:30.113 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks 2024-02-26T05:52:30.114 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-26T05:52:30.115 INFO:teuthology.task.internal:Checking packages... 2024-02-26T05:52:30.139 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '25296a8f176256aea5ed8b5850b25167dce0ae33' 2024-02-26T05:52:30.139 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-26T05:52:30.139 INFO:teuthology.packaging:ref: None 2024-02-26T05:52:30.139 INFO:teuthology.packaging:tag: None 2024-02-26T05:52:30.139 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240216.060239 2024-02-26T05:52:30.139 INFO:teuthology.packaging:sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T05:52:30.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240216.060239 2024-02-26T05:52:30.283 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1514.g25296a8f 2024-02-26T05:52:30.284 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-26T05:52:30.290 INFO:teuthology.task.internal:no buildpackages task found 2024-02-26T05:52:30.291 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-26T05:52:30.304 INFO:teuthology.task.internal:Saving configuration 2024-02-26T05:52:30.326 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-26T05:52:30.332 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T05:52:30.356 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:46:04.186119', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrDu0evKNPSsjtnw5GLpAqIRSFCiLkfZE6SMlt9B9o8F6Ou+2RhSpooORwsJMkfew7OsrufFf5jwsomNr6VMrs='} 2024-02-26T05:52:30.377 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:46:04.184874', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMaAGpLNMrVubKGPygxK0ILlx9L8LVuxUCwiomS/0g/y5jedb+QLejRwrgvB/M60iRatlQ9eCV5ouV5fo/CLULM='} 2024-02-26T05:52:30.399 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:46:04.187169', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLuu6LRYtl16aE5cYvFgLTTTUrx+Bph3hXT94rBMJ2R07130X7x2/ieiMX9PxPQTLULqfn443KJuh6MrPvxex98='} 2024-02-26T05:52:30.399 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-26T05:52:30.404 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-26T05:52:30.404 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-26T05:52:30.405 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-26T05:52:30.405 INFO:teuthology.run_tasks:Running task console_log... 2024-02-26T05:52:30.505 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc72dc9b0d0>, signals=[15]) 2024-02-26T05:52:30.506 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-26T05:52:30.511 INFO:teuthology.task.internal:Opening connections... 2024-02-26T05:52:30.511 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T05:52:30.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:52:30.619 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi080.front.sepia.ceph.com 2024-02-26T05:52:30.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:52:30.708 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-02-26T05:52:30.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:52:30.794 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-26T05:52:30.801 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-02-26T05:52:30.820 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-02-26T05:52:30.821 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T05:52:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-02-26T05:52:30.880 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T05:52:30.880 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T05:52:30.880 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T05:52:30.880 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-02-26T05:52:30.904 DEBUG:teuthology.orchestra.run.smithi080:> uname -m 2024-02-26T05:52:30.922 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2024-02-26T05:52:30.922 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-02-26T05:52:30.978 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="CentOS Stream" 2024-02-26T05:52:30.978 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="9" 2024-02-26T05:52:30.978 INFO:teuthology.orchestra.run.smithi080.stdout:ID="centos" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE="rhel fedora" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="9" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:PLATFORM_ID="platform:el9" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:ANSI_COLOR="0;31" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:LOGO="fedora-logo-icon" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://centos.org/" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T05:52:30.979 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T05:52:30.979 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-02-26T05:52:31.013 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-02-26T05:52:31.030 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-02-26T05:52:31.030 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-02-26T05:52:31.087 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-02-26T05:52:31.087 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-02-26T05:52:31.087 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-02-26T05:52:31.087 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-02-26T05:52:31.087 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T05:52:31.088 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T05:52:31.088 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-02-26T05:52:31.113 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-26T05:52:31.123 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-26T05:52:31.163 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-26T05:52:31.164 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-02-26T05:52:31.166 DEBUG:teuthology.orchestra.run.smithi080:> test '!' -e /home/ubuntu/cephtest 2024-02-26T05:52:31.168 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-02-26T05:52:31.185 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-26T05:52:31.190 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-26T05:52:31.190 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-02-26T05:52:31.223 DEBUG:teuthology.orchestra.run.smithi080:> test -z $(ls -A /var/lib/ceph) 2024-02-26T05:52:31.225 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-02-26T05:52:31.335 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-26T05:52:31.407 INFO:teuthology.run_tasks:Running task kernel... 2024-02-26T05:52:31.422 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '7c0fbba6c508351b48a34af7de7b074eb29ad0f9'} 2024-02-26T05:52:31.422 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-02-26T05:52:31.422 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-02-26T05:52:31.423 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T05:52:31.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:52:31.440 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-02-26T05:52:31.878 DEBUG:teuthology.task.kernel:sha1 for client.0 is 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T05:52:31.878 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "7c0fbba6c508351b48a34af7de7b074eb29ad0f9"... 2024-02-26T05:52:31.878 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-02-26T05:52:31.896 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-361.el9.x86_64 2024-02-26T05:52:31.896 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T05:52:31.897 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-26T05:52:31.897 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum upgrade -y linux-firmware 2024-02-26T05:52:34.063 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 17 MB/s | 8.1 MB 00:00 2024-02-26T05:52:35.888 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.5 MB 00:00 2024-02-26T05:52:44.886 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 961 kB/s | 8.0 MB 00:08 2024-02-26T05:52:48.395 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-02-26T05:52:53.804 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.1 MB 00:01 2024-02-26T05:52:55.188 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 95 kB/s | 15 kB 00:00 2024-02-26T05:52:56.406 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 19 MB/s | 20 MB 00:01 2024-02-26T05:53:01.756 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-02-26T05:53:04.106 INFO:teuthology.orchestra.run.smithi049.stderr:Modular dependency problems: 2024-02-26T05:53:04.106 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-02-26T05:53:04.106 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 1: conflicting requests 2024-02-26T05:53:04.106 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.106 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 2: conflicting requests 2024-02-26T05:53:04.107 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.107 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 3: conflicting requests 2024-02-26T05:53:04.107 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.107 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 4: conflicting requests 2024-02-26T05:53:04.108 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.108 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 5: conflicting requests 2024-02-26T05:53:04.108 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.108 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 6: conflicting requests 2024-02-26T05:53:04.108 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.108 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 7: conflicting requests 2024-02-26T05:53:04.108 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.109 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 8: conflicting requests 2024-02-26T05:53:04.109 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.109 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 9: conflicting requests 2024-02-26T05:53:04.109 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.110 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 10: conflicting requests 2024-02-26T05:53:04.110 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.110 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 11: conflicting requests 2024-02-26T05:53:04.110 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.111 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 12: conflicting requests 2024-02-26T05:53:04.111 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.111 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 13: conflicting requests 2024-02-26T05:53:04.111 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.111 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 14: conflicting requests 2024-02-26T05:53:04.111 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.112 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 15: conflicting requests 2024-02-26T05:53:04.112 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.112 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 16: conflicting requests 2024-02-26T05:53:04.112 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.112 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 17: conflicting requests 2024-02-26T05:53:04.113 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.113 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 18: conflicting requests 2024-02-26T05:53:04.113 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.113 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 19: conflicting requests 2024-02-26T05:53:04.113 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.113 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 20: conflicting requests 2024-02-26T05:53:04.114 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.114 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 21: conflicting requests 2024-02-26T05:53:04.114 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.114 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 22: conflicting requests 2024-02-26T05:53:04.114 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.115 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 23: conflicting requests 2024-02-26T05:53:04.115 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.115 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 24: conflicting requests 2024-02-26T05:53:04.115 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.115 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 25: conflicting requests 2024-02-26T05:53:04.115 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.117 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 26: conflicting requests 2024-02-26T05:53:04.117 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.117 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 27: conflicting requests 2024-02-26T05:53:04.117 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.117 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 28: conflicting requests 2024-02-26T05:53:04.117 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.117 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 29: conflicting requests 2024-02-26T05:53:04.117 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 30: conflicting requests 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 31: conflicting requests 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 32: conflicting requests 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 33: conflicting requests 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 34: conflicting requests 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 35: conflicting requests 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 36: conflicting requests 2024-02-26T05:53:04.118 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 38: conflicting requests 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.119 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.120 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.121 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.122 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.122 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T05:53:04.685 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 CentOS-BaseOS 354 M 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 1 Package 2024-02-26T05:53:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T05:53:04.695 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 354 M 2024-02-26T05:53:04.695 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-02-26T05:53:25.022 INFO:teuthology.orchestra.run.smithi049.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 17 MB/s | 354 MB 00:20 2024-02-26T05:53:25.023 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-02-26T05:53:25.024 INFO:teuthology.orchestra.run.smithi049.stdout:Total 17 MB/s | 354 MB 00:20 2024-02-26T05:53:25.024 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-02-26T05:53:25.106 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-02-26T05:53:25.106 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-02-26T05:53:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-02-26T05:53:26.733 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-02-26T05:53:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-02-26T05:53:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T05:53:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T05:53:56.941 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T05:53:56.941 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T05:53:57.367 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T05:53:57.367 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T05:53:57.367 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-02-26T05:53:57.367 INFO:teuthology.orchestra.run.smithi049.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-02-26T05:53:57.367 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T05:53:57.367 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-02-26T05:53:57.648 INFO:teuthology.task.kernel:Downloading kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.0... 2024-02-26T05:53:57.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T05:53:57.794 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ 2024-02-26T05:53:57.795 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-26T05:53:57.797 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi049.front.sepia.ceph.com to complete... 2024-02-26T05:54:34.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26 05:54:34 URL:https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm [335934793/335934793] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-26T05:54:34.165 INFO:teuthology.orchestra.run.smithi049.stderr:FINISHED --2024-02-26 05:54:34-- 2024-02-26T05:54:34.165 INFO:teuthology.orchestra.run.smithi049.stderr:Total wall clock time: 36s 2024-02-26T05:54:34.165 INFO:teuthology.orchestra.run.smithi049.stderr:Downloaded: 1 files, 320M in 36s (8.91 MB/s) 2024-02-26T05:54:34.184 INFO:teuthology.task.kernel:Installing kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.0... 2024-02-26T05:54:34.184 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-26T05:54:36.217 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying... ######################################## 2024-02-26T05:54:36.297 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing... ######################################## 2024-02-26T05:54:36.382 INFO:teuthology.orchestra.run.smithi049.stdout:Updating / installing... 2024-02-26T05:54:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-6.8.0_rc4_g7c0fbba6c508-1 ######################################## 2024-02-26T05:54:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-26T05:55:21.359 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-26T05:55:21.359 DEBUG:teuthology.orchestra.run.smithi049:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-26T05:55:21.443 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/System.map 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/config 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-26T05:55:21.444 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu 2024-02-26T05:55:21.445 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm.ko 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ansi_cprng.ko 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/anubis.ko 2024-02-26T05:55:21.446 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/arc4.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_memcpy.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_pq.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_tx.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_xor.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/raid6test.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authenc.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authencesn.ko 2024-02-26T05:55:21.447 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blake2b_generic.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_common.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_generic.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/camellia_generic.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast5_generic.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast6_generic.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast_common.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ccm.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cmac.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crc32_generic.ko 2024-02-26T05:55:21.448 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_engine.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_user.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cts.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/des_generic.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/dh_generic.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/echainiv.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/essiv.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/fcrypt.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/khazad.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4hc.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/md4.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/michael_mic.ko 2024-02-26T05:55:21.449 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcbc.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcrypt.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/rmd160.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/seed.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/serpent_generic.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tcrypt.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tea.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_common.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_generic.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/vmac.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/wp512.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xcbc.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xor.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xxhash_generic.ko 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi 2024-02-26T05:55:21.450 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_pad.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/custom_method.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/ec_sys.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/platform_profile.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbs.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbshc.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/video.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/acard-ahci.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ahci_platform.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ata_generic.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/libahci_platform.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_acpi.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ali.ko 2024-02-26T05:55:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_amd.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_artop.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atiixp.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atp867x.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd640.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd64x.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cypress.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_efar.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt366.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt37x.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it8213.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it821x.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_jmicron.ko 2024-02-26T05:55:21.452 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_marvell.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_mpiix.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_netcell.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ninja32.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87410.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87415.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_oldpiix.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_opti.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_optidma.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pcmcia.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_piccolo.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_rdc.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sch.ko 2024-02-26T05:55:21.453 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_serverworks.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sil680.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sis.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sl82c105.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_triflex.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_via.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pdc_adma.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_inic162x.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_mv.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_nv.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_promise.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_qstor.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil24.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sis.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_svw.ko 2024-02-26T05:55:21.454 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sx4.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_uli.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_via.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_vsc.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay/ks0108.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma/bcma.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe/aoe.ko 2024-02-26T05:55:21.455 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/brd.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd/drbd.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/floppy.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/loop.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/nbd.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk/null_blk.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/pktcdvd.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/rbd.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/virtio_blk.ko 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback 2024-02-26T05:55:21.456 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkfront.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram/zram.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hangcheck-timer.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/via-rng.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-26T05:55:21.457 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/lp.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave/mwave.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ppdev.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tlclk.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm.ko 2024-02-26T05:55:21.458 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/uv_mmtimer.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/virtio_console.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-26T05:55:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp.ko 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common 2024-02-26T05:55:21.460 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-aes.ko 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-sha.ko 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca/dca.ko 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-26T05:55:21.461 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/amd64_edac.ko 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/e752x_edac.ko 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/edac_mce_amd.ko 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3000_edac.ko 2024-02-26T05:55:21.462 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3200_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5100_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5400_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7300_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7core_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i82975x_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/ie31200_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/sb_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/x38_edac.ko 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware 2024-02-26T05:55:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/edd.ko 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-ich.ko 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-appleir.ko 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-aureal.ko 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-axff.ko 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-dr.ko 2024-02-26T05:55:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elecom.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elo.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-emsff.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gaff.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gt683r.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gyration.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtekff.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-hyperv.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-icade.ko 2024-02-26T05:55:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-keytouch.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-kye.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lcpower.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-led.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lenovo.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-multitouch.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-ortek.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-penmount.ko 2024-02-26T05:55:21.466 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-petalynx.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-picolcd.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-pl.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-plantronics.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-primax.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-rmi.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-common.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-26T05:55:21.467 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-saitek.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-samsung.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-26T05:55:21.468 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sjoy.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sony.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-speedlink.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-steelseries.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sunplus.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tivo.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tmff.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-topseed.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-twinhan.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-uclogic.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-waltop.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-wiimote.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-xinmo.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zpff.ko 2024-02-26T05:55:21.469 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zydacron.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/uhid.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/wacom.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_balloon.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_utils.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_vmbus.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru3.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7414.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7418.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adc128d818.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1025.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1026.ko 2024-02-26T05:55:21.470 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1029.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1031.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm9240.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ads7828.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7410.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7411.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7462.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7470.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7475.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7x10.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/amc6821.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/applesmc.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asb100.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asc7621.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-26T05:55:21.471 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/atxp1.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/coretemp.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dme1737.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds1621.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds620.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc1403.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc6w201.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71805f.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71882fg.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f75375s.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fam15h_power.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fschmd.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g760a.ko 2024-02-26T05:55:21.472 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g762.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl518sm.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl520sm.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5500_temp.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5k_amb.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmaem.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmpex.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina209.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina2xx.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/it87.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k10temp.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k8temp.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lineage-pem.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm63.ko 2024-02-26T05:55:21.473 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm73.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm75.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm77.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm78.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm80.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm83.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm85.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm87.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm90.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm92.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm93.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95234.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95241.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95245.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc2945.ko 2024-02-26T05:55:21.474 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4151.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4215.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4222.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4245.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4260.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4261.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max16065.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1619.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1668.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max197.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6639.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6650.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6697.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/mcp3021.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6683.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775-core.ko 2024-02-26T05:55:21.475 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct7802.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87360.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87427.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pcf8591.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-26T05:55:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/powr1220.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5627.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5636.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht15.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht21.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/shtc1.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sis5595.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47b397.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m1.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m192.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/thmc50.ko 2024-02-26T05:55:21.477 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp102.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp103.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp401.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp421.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via-cputemp.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via686a.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt1211.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt8231.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627ehf.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627hf.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83781d.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83791d.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83792d.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83793.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83795.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l785ts.ko 2024-02-26T05:55:21.478 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l786ng.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-26T05:55:21.479 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-26T05:55:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-dev.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-mux.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-smbus.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-stub.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_core.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4 2024-02-26T05:55:21.481 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp 2024-02-26T05:55:21.482 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/gameport.ko 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/joydev.ko 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse 2024-02-26T05:55:21.483 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/appletouch.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/bcm5974.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/cyapatp.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/sermouse.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-26T05:55:21.484 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/altera_ps2.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/arc_ps2.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/serio_raw.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/sparse-keymap.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-blinkm.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lm3530.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lp3944.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-ss4200.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-26T05:55:21.485 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache/bcache.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-bio-prison.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache-smq.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-crypt.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-delay.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-flakey.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-log-userspace.ko 2024-02-26T05:55:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-multipath.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-queue-length.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-raid.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-round-robin.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-service-time.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-switch.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-thin-pool.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-verity.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid0.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid1.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid10.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid456.ko 2024-02-26T05:55:21.487 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptbase.ko 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptctl.ko 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptfc.ko 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptlan.ko 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptsas.ko 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptscsih.ko 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptspi.ko 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd 2024-02-26T05:55:21.488 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_ich.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_sch.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/mfd-core.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/sm501.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/viperboard.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/vx855.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/wl1273-core.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds9802als.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds990x.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/bh1770glc.ko 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710 2024-02-26T05:55:21.489 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710/cb710.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/at24.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/max6875.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/enclosure.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/hpilo.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29003.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29020.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-26T05:55:21.490 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-me.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-txe.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_7xx1.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_core.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tsl2550.ko 2024-02-26T05:55:21.491 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_balloon.ko 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding/bonding.ko 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx 2024-02-26T05:55:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dummy.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/eql.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-26T05:55:21.493 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-26T05:55:21.494 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom 2024-02-26T05:55:21.495 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna 2024-02-26T05:55:21.496 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-26T05:55:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-26T05:55:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dnet.ko 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ethoc.ko 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/fealnx.ko 2024-02-26T05:55:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-26T05:55:21.500 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/jme.ko 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-26T05:55:21.501 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-26T05:55:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-26T05:55:21.503 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-26T05:55:21.504 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-26T05:55:21.505 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-26T05:55:21.506 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/geneve.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ifb.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvlan.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvtap.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio.ko 2024-02-26T05:55:21.507 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mii.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/net_failover.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/netconsole.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/nlmon.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/amd.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-26T05:55:21.508 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm87xx.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/broadcom.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/cicada.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/davicom.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/dp83640.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/et1011c.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/icplus.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/lxt.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/marvell.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/micrel.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/national.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/phylink.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/qsemi.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/realtek.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/smsc.ko 2024-02-26T05:55:21.509 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/ste10Xp.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/vitesse.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_async.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppoe.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppox.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pptp.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slhc.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slip.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/sungem_phy.ko 2024-02-26T05:55:21.510 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tap.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_random.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tun.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/veth.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/virtio_net.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan/vxlan.ko 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback 2024-02-26T05:55:21.511 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netfront.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb/ntb.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme-core.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_cs.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_pc.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_serial.ko 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug 2024-02-26T05:55:21.512 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/xen-pcifront.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/i82092.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/pd6729.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acerhdf.ko 2024-02-26T05:55:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/hdaps.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel_ips.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-26T05:55:21.514 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-26T05:55:21.515 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps_parport.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/pps_core.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp_kvm.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-26T05:55:21.516 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-em3027.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-max6900.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-26T05:55:21.517 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-x1205.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-9xxx.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-sas.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-xxxx.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/BusLogic.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/a100u2w.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/advansys.ko 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx 2024-02-26T05:55:21.518 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/am53c974.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/atp870u.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa/bfa.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ch.ko 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor 2024-02-26T05:55:21.519 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dc395x.ko 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-26T05:55:21.520 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dmx3191d.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esp_scsi.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic/fnic.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hpsa.ko 2024-02-26T05:55:21.521 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hptiop.ko 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hv_storvsc.ko 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/initio.ko 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ipr.ko 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ips.ko 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci/isci.ko 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc 2024-02-26T05:55:21.522 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc/libfc.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas/libsas.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas 2024-02-26T05:55:21.523 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvumi.ko 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pmcraid.ko 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla1280.ko 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-26T05:55:21.524 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/raid_class.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_debug.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ses.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/st.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/stex.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/virtio_scsi.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/wd719x.ko 2024-02-26T05:55:21.525 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb/ssb.ko 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_file.ko 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_iblock.ko 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_mod.ko 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_pscsi.ko 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_user.ko 2024-02-26T05:55:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-26T05:55:21.527 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-26T05:55:21.528 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_gsm.ko 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_hdlc.ko 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/nozomi.ko 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/arc_uart.ko 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs 2024-02-26T05:55:21.529 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio.ko 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_aec.ko 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_cif.ko 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_pci_generic.ko 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_sercos3.ko 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb 2024-02-26T05:55:21.530 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-acm.ko 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usblp.ko 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usbtmc.ko 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/mdc800.ko 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/microtek.ko 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/adutux.ko 2024-02-26T05:55:21.531 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/appledisplay.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi26.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi62.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ezusb.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/idmouse.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/iowarrior.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ldusb.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/legousbtower.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-26T05:55:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usb3503.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usblcd.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/uss720.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/yurex.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy/phy-generic.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/aircable.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ark3116.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-26T05:55:21.533 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ch341.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cp210x.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cyberjack.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/empeg.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_ti.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipaq.ko 2024-02-26T05:55:21.534 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipw.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ir-usb.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mct_u232.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7720.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7840.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/navman.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/omninet.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/opticon.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/option.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/oti6858.ko 2024-02-26T05:55:21.535 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/pl2303.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcaux.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcserial.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/quatech2.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/safe_serial.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/sierra.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ssu100.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/symbolserial.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_debug.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/visor.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/whiteheat.ko 2024-02-26T05:55:21.536 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/uas.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-karma.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-26T05:55:21.537 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/usb-storage.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_net.ko 2024-02-26T05:55:21.538 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_scsi.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/apple_bl.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/lcd.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_balloon.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_mmio.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_ring.ko 2024-02-26T05:55:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2482.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2490.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-26T05:55:21.540 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/wire.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/hpwdt.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/i6300esb.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ib700wdt.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ibmasr.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-26T05:55:21.541 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it87_wdt.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/machzwd.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/nv_tco.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/softdog.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/via_wdt.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/wdt_pci.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/xen_wdt.ko 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen 2024-02-26T05:55:21.542 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-evtchn.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntalloc.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntdev.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-privcmd.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p/9p.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs/affs.ko 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs 2024-02-26T05:55:21.543 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs/befs.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/binfmt_misc.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs/btrfs.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles/cachefiles.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph/ceph.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda/coda.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs/cramfs.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm/dlm.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-26T05:55:21.544 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs/f2fs.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/fat.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/msdos.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/vfat.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/cuse.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/fuse.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2/gfs2.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs/hfs.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus/hfsplus.ko 2024-02-26T05:55:21.545 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs/isofs.ko 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs/jfs.ko 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd/lockd.ko 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix/minix.ko 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs/netfs.ko 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-26T05:55:21.546 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfs.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv3.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv4.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/grace.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/nfs_acl.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd/nfsd.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2/nilfs2.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-celtic.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-centeuro.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-croatian.ko 2024-02-26T05:55:21.547 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-cyrillic.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-gaelic.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-greek.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-iceland.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-inuit.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-roman.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-romanian.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-turkish.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1250.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1251.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1255.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp737.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp775.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp850.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp852.ko 2024-02-26T05:55:21.548 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp855.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp857.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp860.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp861.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp862.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp863.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp864.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp865.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp866.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp869.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp874.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp932.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp936.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp949.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp950.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_euc-jp.ko 2024-02-26T05:55:21.549 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-1.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-13.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-14.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-15.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-2.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-3.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-4.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-5.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-6.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-7.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-9.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-r.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-ru.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-u.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-26T05:55:21.550 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_utf8.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs/overlay.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore/ramoops.ko 2024-02-26T05:55:21.551 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs/reiserfs.ko 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs/romfs.ko 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client/cifs.ko 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_arc4.ko 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_md4.ko 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs/squashfs.ko 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv/sysv.ko 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf 2024-02-26T05:55:21.552 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf/udf.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs/ufs.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs/xfs.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu/rcutorture.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/torture.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/asn1_encoder.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc-itu-t.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc8.ko 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto 2024-02-26T05:55:21.553 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libarc4.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libdes.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/libcrc32c.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lru_cache.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4_compress.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4hc_compress.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math/cordic.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6/raid6_pq.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_bm.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_fsm.ko 2024-02-26T05:55:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_kmp.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd/zstd_compress.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm/hwpoison-inject.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/6lowpan.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_dest.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_fragment.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_hop.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_mobility.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_routing.ko 2024-02-26T05:55:21.555 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_udp.ko 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/garp.ko 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/mrp.ko 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/p8022.ko 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/psnap.ko 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/stp.ko 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q/8021q.ko 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p 2024-02-26T05:55:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_fd.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_rdma.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_virtio.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv/batman-adv.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/br_netfilter.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/bridge.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-26T05:55:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-26T05:55:21.558 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtables.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph/libceph.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/failover.ko 2024-02-26T05:55:21.559 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/pktgen.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_diag.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv4.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv6.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver/dns_resolver.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/dsa_core.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_brcm.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_dsa.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_none.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_trailer.ko 2024-02-26T05:55:21.560 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ah4.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/esp4.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/fou.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/gre.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/inet_diag.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_gre.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_tunnel.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_vti.ko 2024-02-26T05:55:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipcomp.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipip.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-26T05:55:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_bic.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_dctcp.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_diag.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_highspeed.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_htcp.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_hybla.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_illinois.ko 2024-02-26T05:55:21.563 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_lp.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_scalable.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_vegas.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_veno.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_westwood.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_yeah.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tunnel4.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_diag.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_tunnel.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ah6.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/esp6.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/fou6.ko 2024-02-26T05:55:21.564 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_tunnel.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_vti.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ipcomp6.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-26T05:55:21.565 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-26T05:55:21.566 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/sit.ko 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/tunnel6.ko 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key/af_key.ko 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_core.ko 2024-02-26T05:55:21.567 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_eth.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip6.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_netlink.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ppp.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc/llc.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154/mac802154.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls/mpls_gso.ko 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter 2024-02-26T05:55:21.568 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-26T05:55:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-26T05:55:21.570 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-26T05:55:21.571 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conncount.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-26T05:55:21.572 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_log_syslog.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-26T05:55:21.573 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_irc.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_sip.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_tables.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_log.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_chain_nat.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_compat.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_ct.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fib.ko 2024-02-26T05:55:21.574 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_hash.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_limit.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_log.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_masq.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_nat.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_numgen.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_queue.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_quota.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_redir.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject_inet.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_socket.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_tunnel.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_AUDIT.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-26T05:55:21.575 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CT.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_DSCP.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HL.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HMARK.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LED.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LOG.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NETMAP.ko 2024-02-26T05:55:21.576 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFLOG.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_RATEEST.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_SECMARK.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TEE.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TPROXY.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TRACE.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_addrtype.ko 2024-02-26T05:55:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_bpf.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cgroup.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cluster.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_comment.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connbytes.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlabel.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlimit.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connmark.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_conntrack.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cpu.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dccp.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_devgroup.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dscp.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ecn.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_esp.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hashlimit.ko 2024-02-26T05:55:21.578 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_helper.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hl.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipcomp.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_iprange.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipvs.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_l2tp.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_length.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_limit.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mac.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mark.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_multiport.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nat.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nfacct.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_osf.ko 2024-02-26T05:55:21.579 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_owner.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_physdev.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_pkttype.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_policy.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_quota.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_rateest.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_realm.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_recent.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_sctp.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_set.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_socket.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_state.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_statistic.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_string.ko 2024-02-26T05:55:21.580 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_tcpmss.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_time.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_u32.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink/netlink_diag.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh/nsh.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/openvswitch.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-geneve.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-gre.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-vxlan.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet/af_packet_diag.ko 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds 2024-02-26T05:55:21.581 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_rdma.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_tcp.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_csum.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_gact.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_mirred.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_nat.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_pedit.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_police.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_simple.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_skbedit.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_vlan.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_basic.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_bpf.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_flow.ko 2024-02-26T05:55:21.582 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_fw.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_route.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_u32.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_cmp.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_ipset.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_meta.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_nbyte.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_text.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_u32.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_choke.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_codel.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_drr.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_fq.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_gred.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hfsc.ko 2024-02-26T05:55:21.583 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hhf.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_htb.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_ingress.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio_lib.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_multiq.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_netem.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_pie.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_plug.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_prio.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_qfq.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_red.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfb.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfq.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_tbf.ko 2024-02-26T05:55:21.584 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_teql.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp_diag.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/sunrpc.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/diag.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/tipc.ko 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix 2024-02-26T05:55:21.585 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix/unix_diag.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys/trusted.ko 2024-02-26T05:55:21.586 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib/irqbypass.ko 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias.bin 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.alias.bin 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.bin 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.modinfo 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep.bin 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.devname 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.order 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.softdep 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols.bin 2024-02-26T05:55:21.587 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T05:55:21.592 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc4-g7c0fbba6c508 2024-02-26T05:55:21.592 DEBUG:teuthology.orchestra.run.smithi049:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-26T05:55:21.616 INFO:teuthology.orchestra.run.smithi049.stdout:postinstall scriptlet (using /bin/sh): 2024-02-26T05:55:21.616 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-02-26T05:55:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: /usr/bin/kernel-install add 6.8.0-rc4-g7c0fbba6c508 /lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T05:55:21.616 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2024-02-26T05:55:21.616 INFO:teuthology.orchestra.run.smithi049.stdout:for file in vmlinuz System.map config; do 2024-02-26T05:55:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508"; then 2024-02-26T05:55:21.616 INFO:teuthology.orchestra.run.smithi049.stdout: cp "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508" 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout: fi 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:done 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:new-kernel-pkg --remove 6.8.0-rc4-g7c0fbba6c508 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc4-g7c0fbba6c508.img 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-install remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/update-bootloader --remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T05:55:21.617 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2024-02-26T05:55:21.618 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-02-26T05:55:21.699 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-02-26T05:55:21.699 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-02-26T05:55:21.699 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-02-26T05:55:21.699 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 68.el9 2024-02-26T05:55:21.699 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-02-26T05:55:21.699 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-26T05:55:21.699 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8264254 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-02-26T05:55:21.700 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T05:55:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-26T05:55:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-26T05:55:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-26T05:55:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-02-26T05:55:21.701 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T05:55:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-02-26T05:55:21.702 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-26T05:55:21.702 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 6.8.0-rc4-g7c0fbba6c508 2024-02-26T05:55:21.702 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-26T05:55:22.343 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-02-26T05:55:23.814 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-26T05:55:23.841 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-02-26T05:55:23.844 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-02-26T05:55:23.859 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.4aomgfhJAj 2024-02-26T05:55:23.859 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4aomgfhJAj 2024-02-26T05:55:23.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.4aomgfhJAj 2024-02-26T05:55:24.078 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.4aomgfhJAj is 6KB 2024-02-26T05:55:24.126 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.4aomgfhJAj 2024-02-26T05:55:24.141 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-02-26T05:55:24.212 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-26T05:55:24.212 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-26T05:55:24.212 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-26T05:55:24.213 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508.conf 2024-02-26T05:55:24.214 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508 2024-02-26T05:55:24.385 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-02-26T05:55:54.387 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-26T05:55:54.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T05:55:54.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:56:12.736 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-02-26T05:56:21.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T05:56:21.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:56:24.796 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-02-26T05:56:36.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T05:56:36.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:56:39.900 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-02-26T05:56:54.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T05:56:54.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:56:57.984 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-02-26T05:57:15.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T05:57:15.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:57:34.396 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-02-26T05:57:55.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T05:57:55.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:57:55.781 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-02-26T05:57:56.563 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T05:57:56.563 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc4-g7c0fbba6c508"... 2024-02-26T05:57:56.563 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-02-26T05:57:56.607 INFO:teuthology.orchestra.run.smithi049.stdout:6.8.0-rc4-g7c0fbba6c508 2024-02-26T05:57:56.607 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc4-g7c0fbba6c508 vs 6.8.0-rc4-g7c0fbba6c508 2024-02-26T05:57:56.607 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-26T05:57:56.607 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-26T05:57:57.608 DEBUG:teuthology.parallel:result is None 2024-02-26T05:57:57.608 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-26T05:57:57.615 INFO:teuthology.task.internal:Creating test directory... 2024-02-26T05:57:57.615 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T05:57:57.621 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T05:57:57.624 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T05:57:57.662 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-26T05:57:57.668 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-26T05:57:57.678 INFO:teuthology.task.internal:Creating archive directory... 2024-02-26T05:57:57.678 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T05:57:57.708 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T05:57:57.710 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T05:57:57.941 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-26T05:57:57.949 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-26T05:57:57.949 DEBUG:teuthology.orchestra.run.smithi080:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T05:57:57.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:57:57.966 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T05:57:57.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:57:57.983 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T05:57:57.989 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T05:57:58.008 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T05:57:58.045 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T05:57:58.064 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T05:57:58.079 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T05:57:58.096 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T05:57:58.149 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T05:57:58.254 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T05:57:58.258 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-26T05:57:58.268 INFO:teuthology.task.internal:Configuring sudo... 2024-02-26T05:57:58.268 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T05:57:58.274 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T05:57:58.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T05:57:58.354 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-26T05:57:58.366 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-26T05:57:58.366 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T05:57:58.372 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T05:57:58.376 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T05:57:58.408 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T05:57:58.525 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T05:57:58.624 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T05:57:58.697 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T05:57:58.795 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T05:57:58.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T05:57:58.871 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T05:57:58.915 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T05:57:58.966 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T05:57:59.057 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T05:57:59.100 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T05:57:59.101 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T05:57:59.171 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T05:57:59.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T05:57:59.272 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T05:57:59.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T05:57:59.401 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T05:57:59.402 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T05:57:59.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-02-26T05:57:59.477 DEBUG:teuthology.orchestra.run.smithi080:> sudo service rsyslog restart 2024-02-26T05:57:59.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-02-26T05:57:59.523 INFO:teuthology.orchestra.run.smithi080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T05:57:59.546 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T05:57:59.584 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T05:57:59.870 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-26T05:57:59.877 INFO:teuthology.task.internal:Starting timer... 2024-02-26T05:57:59.877 INFO:teuthology.run_tasks:Running task pcp... 2024-02-26T05:57:59.888 INFO:teuthology.run_tasks:Running task selinux... 2024-02-26T05:57:59.896 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-26T05:57:59.896 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-02-26T05:58:00.023 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-02-26T05:58:00.026 DEBUG:teuthology.orchestra.run.smithi080:> sudo service auditd rotate 2024-02-26T05:58:00.092 INFO:teuthology.orchestra.run.smithi080.stdout:Rotating logs: 2024-02-26T05:58:00.094 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-02-26T05:58:00.153 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-02-26T05:58:00.156 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-26T05:58:00.156 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-02-26T05:58:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-02-26T05:58:00.205 DEBUG:teuthology.orchestra.run.smithi080:> /usr/sbin/getenforce 2024-02-26T05:58:00.232 INFO:teuthology.orchestra.run.smithi080.stdout:Permissive 2024-02-26T05:58:00.232 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-02-26T05:58:00.254 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-02-26T05:58:00.255 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi080.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive'} 2024-02-26T05:58:00.255 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T05:58:00.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:58:00.338 DEBUG:teuthology.orchestra.run.smithi080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T05:58:00.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:58:00.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T05:58:00.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:58:00.397 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-26T05:58:00.397 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-02-26T05:58:00.486 DEBUG:teuthology.orchestra.run.smithi080:> sudo /usr/sbin/setenforce permissive 2024-02-26T05:58:00.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-02-26T05:58:00.615 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-26T05:58:00.689 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-26T05:58:00.714 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-26T05:58:00.724 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi080.front.sepia.ceph.com,smithi115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-26T06:05:27.911 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi080.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com')] 2024-02-26T06:05:28.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T06:05:28.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T06:05:28.256 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-02-26T06:05:28.424 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-02-26T06:05:28.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-02-26T06:05:28.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T06:05:28.514 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-02-26T06:05:28.589 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-02-26T06:05:28.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-02-26T06:05:28.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T06:05:28.683 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-02-26T06:05:28.752 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-02-26T06:05:28.754 INFO:teuthology.run_tasks:Running task clock... 2024-02-26T06:05:28.764 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-26T06:05:28.765 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T06:05:28.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:05:28.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T06:05:28.773 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:05:28.776 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T06:05:28.776 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:05:28.808 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T06:05:28.825 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T06:05:28.826 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T06:05:28.843 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T06:05:28.857 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: ntpd: command not found 2024-02-26T06:05:28.871 INFO:teuthology.orchestra.run.smithi080.stdout:506 Cannot talk to daemon 2024-02-26T06:05:28.872 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-02-26T06:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T06:05:28.886 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-02-26T06:05:28.887 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T06:05:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T06:05:28.903 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T06:05:28.920 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T06:05:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T06:05:28.947 INFO:teuthology.orchestra.run.smithi080.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:05:28.950 INFO:teuthology.orchestra.run.smithi080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:05:28.950 INFO:teuthology.orchestra.run.smithi080.stdout:=============================================================================== 2024-02-26T06:05:28.950 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:28.950 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:28.950 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:28.950 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:28.962 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:05:28.965 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:05:28.965 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-02-26T06:05:28.965 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:28.965 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:28.965 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:28.965 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:29.052 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-02-26T06:05:29.105 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-02-26T06:05:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T06:05:29.227 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T06:05:29.364 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:05:29.374 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:05:29.374 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-02-26T06:05:29.374 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:29.374 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:29.374 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:29.374 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:05:29.375 INFO:teuthology.run_tasks:Running task install... 2024-02-26T06:05:29.386 DEBUG:teuthology.task.install:project ceph 2024-02-26T06:05:29.386 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33'}} 2024-02-26T06:05:29.386 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33'} 2024-02-26T06:05:29.386 INFO:teuthology.task.install:Using flavor: default 2024-02-26T06:05:29.394 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-26T06:05:29.394 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-26T06:05:29.395 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T06:05:29.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:05:29.397 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T06:05:29.397 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:05:29.398 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T06:05:29.399 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:05:29.658 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T06:05:29.659 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T06:05:29.665 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T06:05:29.665 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T06:05:29.673 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T06:05:29.674 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T06:05:29.781 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T06:05:29.781 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:05:29.781 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T06:05:29.810 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T06:05:29.811 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T06:05:29.869 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T06:05:29.870 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:05:29.870 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T06:05:29.878 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T06:05:29.878 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:05:29.878 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T06:05:29.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T06:05:29.910 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T06:05:29.910 DEBUG:teuthology.orchestra.run.smithi080:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T06:05:29.962 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T06:05:29.975 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T06:05:29.975 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T06:05:29.978 DEBUG:teuthology.orchestra.run.smithi080:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T06:05:30.028 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-02-26T06:05:30.030 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-02-26T06:05:30.058 DEBUG:teuthology.orchestra.run.smithi080:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T06:05:30.124 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T06:05:30.126 INFO:teuthology.orchestra.run.smithi080.stdout:check_obsoletes = 1 2024-02-26T06:05:30.126 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum clean all 2024-02-26T06:05:30.265 INFO:teuthology.orchestra.run.smithi115.stdout:74 files removed 2024-02-26T06:05:30.276 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T06:05:30.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T06:05:30.362 INFO:teuthology.orchestra.run.smithi080.stdout:74 files removed 2024-02-26T06:05:30.375 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-02-26T06:05:30.379 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-02-26T06:05:30.388 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T06:05:30.747 INFO:teuthology.orchestra.run.smithi049.stdout:74 files removed 2024-02-26T06:05:30.788 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T06:05:31.175 INFO:teuthology.orchestra.run.smithi080.stdout:ceph packages for x86_64 166 kB/s | 82 kB 00:00 2024-02-26T06:05:31.228 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 127 kB/s | 82 kB 00:00 2024-02-26T06:05:31.943 INFO:teuthology.orchestra.run.smithi080.stdout:ceph noarch packages 14 kB/s | 10 kB 00:00 2024-02-26T06:05:32.037 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-02-26T06:05:32.092 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 107 kB/s | 82 kB 00:00 2024-02-26T06:05:32.293 INFO:teuthology.orchestra.run.smithi080.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-02-26T06:05:32.421 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-02-26T06:05:32.801 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 16 kB/s | 10 kB 00:00 2024-02-26T06:05:33.037 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS-9 - AppStream 11 MB/s | 8.1 MB 00:00 2024-02-26T06:05:33.140 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-02-26T06:05:33.426 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-02-26T06:05:34.446 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 8.6 MB/s | 8.1 MB 00:00 2024-02-26T06:05:34.915 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.5 MB 00:00 2024-02-26T06:05:34.921 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.5 MB 00:00 2024-02-26T06:05:35.881 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-02-26T06:05:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.5 MB 00:00 2024-02-26T06:05:37.511 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.0 MB 00:02 2024-02-26T06:05:38.918 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-02-26T06:05:39.440 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 19 MB 00:02 2024-02-26T06:05:39.579 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-02-26T06:05:43.807 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-02-26T06:05:45.076 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-02-26T06:05:45.339 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 15 kB 00:00 2024-02-26T06:05:45.673 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.1 MB 00:02 2024-02-26T06:05:47.315 INFO:teuthology.orchestra.run.smithi080.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-26T06:05:47.349 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2024-02-26T06:05:48.675 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2024-02-26T06:05:49.694 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-02-26T06:05:51.603 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-02-26T06:05:52.432 INFO:teuthology.orchestra.run.smithi080.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-02-26T06:05:52.888 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 19 MB/s | 20 MB 00:01 2024-02-26T06:05:53.633 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-02-26T06:05:55.006 INFO:teuthology.orchestra.run.smithi080.stderr:Modular dependency problems: 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 1: conflicting requests 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 2: conflicting requests 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 3: conflicting requests 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 4: conflicting requests 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 5: conflicting requests 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 6: conflicting requests 2024-02-26T06:05:55.007 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 7: conflicting requests 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 8: conflicting requests 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 9: conflicting requests 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 10: conflicting requests 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 11: conflicting requests 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 12: conflicting requests 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.008 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 13: conflicting requests 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 14: conflicting requests 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 15: conflicting requests 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 16: conflicting requests 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 17: conflicting requests 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 18: conflicting requests 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 19: conflicting requests 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.009 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 20: conflicting requests 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 21: conflicting requests 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 22: conflicting requests 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 23: conflicting requests 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 24: conflicting requests 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 25: conflicting requests 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 26: conflicting requests 2024-02-26T06:05:55.010 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 27: conflicting requests 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 28: conflicting requests 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 29: conflicting requests 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 30: conflicting requests 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 31: conflicting requests 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 32: conflicting requests 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.011 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 33: conflicting requests 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 34: conflicting requests 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 35: conflicting requests 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 36: conflicting requests 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.012 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 38: conflicting requests 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T06:05:55.013 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.014 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.015 INFO:teuthology.orchestra.run.smithi080.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T06:05:55.016 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.016 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.016 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.016 INFO:teuthology.orchestra.run.smithi080.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T06:05:55.418 INFO:teuthology.orchestra.run.smithi080.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:05:55.419 INFO:teuthology.orchestra.run.smithi080.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:05:55.432 INFO:teuthology.orchestra.run.smithi080.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T06:05:55.432 INFO:teuthology.orchestra.run.smithi080.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T06:05:55.433 INFO:teuthology.orchestra.run.smithi080.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T06:05:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T06:05:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T06:05:55.438 INFO:teuthology.orchestra.run.smithi080.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T06:05:55.439 INFO:teuthology.orchestra.run.smithi080.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T06:05:55.529 INFO:teuthology.orchestra.run.smithi080.stdout:Dependencies resolved. 2024-02-26T06:05:55.554 INFO:teuthology.orchestra.run.smithi080.stdout:========================================================================================= 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: Package Arch Version Repository Size 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout:========================================================================================= 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout:Installing: 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T06:05:55.555 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T06:05:55.556 INFO:teuthology.orchestra.run.smithi080.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T06:05:55.557 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout:Upgrading: 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:05:55.558 INFO:teuthology.orchestra.run.smithi080.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout:Installing dependencies: 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T06:05:55.559 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T06:05:55.560 INFO:teuthology.orchestra.run.smithi080.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T06:05:55.561 INFO:teuthology.orchestra.run.smithi080.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T06:05:55.562 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T06:05:55.563 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T06:05:55.564 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T06:05:55.565 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T06:05:55.566 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout:Installing weak dependencies: 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction Summary 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout:========================================================================================= 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout:Install 150 Packages 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout:Upgrade 16 Packages 2024-02-26T06:05:55.567 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:05:55.568 INFO:teuthology.orchestra.run.smithi080.stdout:Total download size: 221 M 2024-02-26T06:05:55.568 INFO:teuthology.orchestra.run.smithi080.stdout:Downloading Packages: 2024-02-26T06:05:56.196 INFO:teuthology.orchestra.run.smithi115.stderr:Modular dependency problems: 2024-02-26T06:05:56.196 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-02-26T06:05:56.196 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 1: conflicting requests 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 2: conflicting requests 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 3: conflicting requests 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 4: conflicting requests 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 5: conflicting requests 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.197 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 6: conflicting requests 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 7: conflicting requests 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 8: conflicting requests 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 9: conflicting requests 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 10: conflicting requests 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 11: conflicting requests 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 12: conflicting requests 2024-02-26T06:05:56.198 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 13: conflicting requests 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 14: conflicting requests 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 15: conflicting requests 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 16: conflicting requests 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 17: conflicting requests 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 18: conflicting requests 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 19: conflicting requests 2024-02-26T06:05:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 20: conflicting requests 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 21: conflicting requests 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 22: conflicting requests 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 23: conflicting requests 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 24: conflicting requests 2024-02-26T06:05:56.200 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 25: conflicting requests 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 26: conflicting requests 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 27: conflicting requests 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 28: conflicting requests 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 29: conflicting requests 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 30: conflicting requests 2024-02-26T06:05:56.201 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.203 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 31: conflicting requests 2024-02-26T06:05:56.203 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.203 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 32: conflicting requests 2024-02-26T06:05:56.203 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.203 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 33: conflicting requests 2024-02-26T06:05:56.203 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.203 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 34: conflicting requests 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 35: conflicting requests 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 36: conflicting requests 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.204 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 38: conflicting requests 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.205 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T06:05:56.206 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.207 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.207 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.207 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.207 INFO:teuthology.orchestra.run.smithi115.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T06:05:56.207 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.207 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.207 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.207 INFO:teuthology.orchestra.run.smithi115.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T06:05:56.291 INFO:teuthology.orchestra.run.smithi080.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-02-26T06:05:56.628 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:05:56.629 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:05:56.642 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T06:05:56.643 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T06:05:56.644 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T06:05:56.645 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T06:05:56.647 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T06:05:56.650 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T06:05:56.651 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T06:05:56.737 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-02-26T06:05:56.750 INFO:teuthology.orchestra.run.smithi080.stdout:(2/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 7.7 MB/s | 5.3 MB 00:00 2024-02-26T06:05:56.753 INFO:teuthology.orchestra.run.smithi115.stdout:========================================================================================= 2024-02-26T06:05:56.753 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-02-26T06:05:56.753 INFO:teuthology.orchestra.run.smithi115.stdout:========================================================================================= 2024-02-26T06:05:56.753 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-02-26T06:05:56.753 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T06:05:56.753 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T06:05:56.753 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T06:05:56.754 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T06:05:56.755 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T06:05:56.756 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T06:05:56.757 INFO:teuthology.orchestra.run.smithi115.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T06:05:56.758 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T06:05:56.759 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T06:05:56.760 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T06:05:56.760 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T06:05:56.760 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T06:05:56.760 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T06:05:56.760 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T06:05:56.760 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T06:05:56.760 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T06:05:56.761 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T06:05:56.762 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T06:05:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T06:05:56.764 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T06:05:56.765 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T06:05:56.766 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout:========================================================================================= 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout:Install 150 Packages 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 16 Packages 2024-02-26T06:05:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:05:56.768 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 221 M 2024-02-26T06:05:56.768 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-02-26T06:05:56.817 INFO:teuthology.orchestra.run.smithi080.stdout:(3/166): ceph-immutable-object-cache-19.0.0-151 2.1 MB/s | 145 kB 00:00 2024-02-26T06:05:57.067 INFO:teuthology.orchestra.run.smithi080.stdout:(4/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 8.5 MB/s | 2.1 MB 00:00 2024-02-26T06:05:57.251 INFO:teuthology.orchestra.run.smithi080.stdout:(5/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 923 kB/s | 886 kB 00:00 2024-02-26T06:05:57.267 INFO:teuthology.orchestra.run.smithi115.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-02-26T06:05:57.284 INFO:teuthology.orchestra.run.smithi080.stdout:(6/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-02-26T06:05:57.510 INFO:teuthology.orchestra.run.smithi080.stdout:(7/166): ceph-common-19.0.0-1514.g25296a8f.el9. 14 MB/s | 20 MB 00:01 2024-02-26T06:05:58.059 INFO:teuthology.orchestra.run.smithi115.stdout:(2/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 1.1 MB/s | 886 kB 00:00 2024-02-26T06:05:58.301 INFO:teuthology.orchestra.run.smithi115.stdout:(3/166): ceph-immutable-object-cache-19.0.0-151 601 kB/s | 145 kB 00:00 2024-02-26T06:05:58.626 INFO:teuthology.orchestra.run.smithi080.stdout:(8/166): ceph-radosgw-19.0.0-1514.g25296a8f.el9 8.7 MB/s | 9.7 MB 00:01 2024-02-26T06:05:58.702 INFO:teuthology.orchestra.run.smithi080.stdout:(9/166): ceph-selinux-19.0.0-1514.g25296a8f.el9 336 kB/s | 25 kB 00:00 2024-02-26T06:05:59.146 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-02-26T06:05:59.426 INFO:teuthology.orchestra.run.smithi115.stdout:(4/166): ceph-common-19.0.0-1514.g25296a8f.el9. 8.2 MB/s | 20 MB 00:02 2024-02-26T06:05:59.651 INFO:teuthology.orchestra.run.smithi115.stdout:(5/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 6.6 MB/s | 1.5 MB 00:00 2024-02-26T06:06:00.035 INFO:teuthology.orchestra.run.smithi115.stdout:(6/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 1.2 MB/s | 2.1 MB 00:01 2024-02-26T06:06:00.151 INFO:teuthology.orchestra.run.smithi115.stdout:(7/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x86 9.4 MB/s | 4.7 MB 00:00 2024-02-26T06:06:00.168 INFO:teuthology.orchestra.run.smithi080.stdout:(10/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x8 5.8 MB/s | 17 MB 00:02 2024-02-26T06:06:00.252 INFO:teuthology.orchestra.run.smithi080.stdout:(11/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 2.6 MB/s | 217 kB 00:00 2024-02-26T06:06:00.327 INFO:teuthology.orchestra.run.smithi080.stdout:(12/166): cephfs-shell-19.0.0-1514.g25296a8f.el 361 kB/s | 27 kB 00:00 2024-02-26T06:06:00.394 INFO:teuthology.orchestra.run.smithi080.stdout:(13/166): libcephfs-devel-19.0.0-1514.g25296a8f 467 kB/s | 31 kB 00:00 2024-02-26T06:06:00.553 INFO:teuthology.orchestra.run.smithi080.stdout:(14/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 4.6 MB/s | 743 kB 00:00 2024-02-26T06:06:00.636 INFO:teuthology.orchestra.run.smithi080.stdout:(15/166): libcephsqlite-19.0.0-1514.g25296a8f.e 1.9 MB/s | 163 kB 00:00 2024-02-26T06:06:00.720 INFO:teuthology.orchestra.run.smithi080.stdout:(16/166): librados-devel-19.0.0-1514.g25296a8f. 1.5 MB/s | 127 kB 00:00 2024-02-26T06:06:00.870 INFO:teuthology.orchestra.run.smithi080.stdout:(17/166): libradosstriper1-19.0.0-1514.g25296a8 3.2 MB/s | 498 kB 00:00 2024-02-26T06:06:01.227 INFO:teuthology.orchestra.run.smithi115.stdout:(8/166): ceph-radosgw-19.0.0-1514.g25296a8f.el9 9.1 MB/s | 9.7 MB 00:01 2024-02-26T06:06:01.328 INFO:teuthology.orchestra.run.smithi115.stdout:(9/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 1.2 MB/s | 5.3 MB 00:04 2024-02-26T06:06:01.352 INFO:teuthology.orchestra.run.smithi115.stdout:(10/166): ceph-selinux-19.0.0-1514.g25296a8f.el 201 kB/s | 25 kB 00:00 2024-02-26T06:06:01.436 INFO:teuthology.orchestra.run.smithi115.stdout:(11/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 2.6 MB/s | 217 kB 00:00 2024-02-26T06:06:01.511 INFO:teuthology.orchestra.run.smithi115.stdout:(12/166): cephfs-shell-19.0.0-1514.g25296a8f.el 361 kB/s | 27 kB 00:00 2024-02-26T06:06:01.578 INFO:teuthology.orchestra.run.smithi115.stdout:(13/166): libcephfs-devel-19.0.0-1514.g25296a8f 468 kB/s | 31 kB 00:00 2024-02-26T06:06:01.621 INFO:teuthology.orchestra.run.smithi080.stdout:(18/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 6.8 MB/s | 5.1 MB 00:00 2024-02-26T06:06:01.678 INFO:teuthology.orchestra.run.smithi115.stdout:(14/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 7.3 MB/s | 743 kB 00:00 2024-02-26T06:06:01.696 INFO:teuthology.orchestra.run.smithi080.stdout:(19/166): python3-ceph-argparse-19.0.0-1514.g25 603 kB/s | 45 kB 00:00 2024-02-26T06:06:01.728 INFO:teuthology.orchestra.run.smithi115.stdout:(15/166): libcephsqlite-19.0.0-1514.g25296a8f.e 3.2 MB/s | 163 kB 00:00 2024-02-26T06:06:01.771 INFO:teuthology.orchestra.run.smithi080.stdout:(20/166): python3-ceph-common-19.0.0-1514.g2529 1.7 MB/s | 131 kB 00:00 2024-02-26T06:06:01.830 INFO:teuthology.orchestra.run.smithi080.stdout:(21/166): python3-cephfs-19.0.0-1514.g25296a8f. 2.7 MB/s | 161 kB 00:00 2024-02-26T06:06:01.855 INFO:teuthology.orchestra.run.smithi115.stdout:(16/166): librados-devel-19.0.0-1514.g25296a8f. 1.0 MB/s | 127 kB 00:00 2024-02-26T06:06:01.905 INFO:teuthology.orchestra.run.smithi080.stdout:(22/166): python3-rados-19.0.0-1514.g25296a8f.e 4.2 MB/s | 320 kB 00:00 2024-02-26T06:06:01.935 INFO:teuthology.orchestra.run.smithi049.stderr:Modular dependency problems: 2024-02-26T06:06:01.936 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-02-26T06:06:01.936 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 1: conflicting requests 2024-02-26T06:06:01.936 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.936 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 2: conflicting requests 2024-02-26T06:06:01.936 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.936 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 3: conflicting requests 2024-02-26T06:06:01.936 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.936 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 4: conflicting requests 2024-02-26T06:06:01.939 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.939 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 5: conflicting requests 2024-02-26T06:06:01.939 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 6: conflicting requests 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 7: conflicting requests 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 8: conflicting requests 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 9: conflicting requests 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 10: conflicting requests 2024-02-26T06:06:01.940 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.941 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 11: conflicting requests 2024-02-26T06:06:01.941 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.941 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 12: conflicting requests 2024-02-26T06:06:01.941 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.941 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 13: conflicting requests 2024-02-26T06:06:01.941 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.941 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 14: conflicting requests 2024-02-26T06:06:01.941 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 15: conflicting requests 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 16: conflicting requests 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 17: conflicting requests 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 18: conflicting requests 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 19: conflicting requests 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 20: conflicting requests 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 21: conflicting requests 2024-02-26T06:06:01.942 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 22: conflicting requests 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 23: conflicting requests 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 24: conflicting requests 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 25: conflicting requests 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 26: conflicting requests 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 27: conflicting requests 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.943 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 28: conflicting requests 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 29: conflicting requests 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 30: conflicting requests 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 31: conflicting requests 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 32: conflicting requests 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 33: conflicting requests 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 34: conflicting requests 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.944 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 35: conflicting requests 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 36: conflicting requests 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 38: conflicting requests 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.945 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.946 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.947 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.948 INFO:teuthology.orchestra.run.smithi049.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T06:06:01.949 INFO:teuthology.orchestra.run.smithi115.stdout:(17/166): libradosstriper1-19.0.0-1514.g25296a8 6.0 MB/s | 498 kB 00:00 2024-02-26T06:06:01.989 INFO:teuthology.orchestra.run.smithi080.stdout:(23/166): python3-rbd-19.0.0-1514.g25296a8f.el9 3.5 MB/s | 298 kB 00:00 2024-02-26T06:06:02.073 INFO:teuthology.orchestra.run.smithi080.stdout:(24/166): python3-rgw-19.0.0-1514.g25296a8f.el9 1.2 MB/s | 99 kB 00:00 2024-02-26T06:06:02.164 INFO:teuthology.orchestra.run.smithi080.stdout:(25/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 930 kB/s | 85 kB 00:00 2024-02-26T06:06:02.531 INFO:teuthology.orchestra.run.smithi080.stdout:(26/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x8 908 kB/s | 4.7 MB 00:05 2024-02-26T06:06:02.565 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:06:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:06:02.571 INFO:teuthology.orchestra.run.smithi115.stdout:(18/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 8.1 MB/s | 5.1 MB 00:00 2024-02-26T06:06:02.591 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T06:06:02.592 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T06:06:02.594 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T06:06:02.595 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T06:06:02.598 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T06:06:02.602 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T06:06:02.604 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T06:06:02.663 INFO:teuthology.orchestra.run.smithi115.stdout:(19/166): python3-ceph-argparse-19.0.0-1514.g25 494 kB/s | 45 kB 00:00 2024-02-26T06:06:02.706 INFO:teuthology.orchestra.run.smithi080.stdout:(27/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 5.7 MB/s | 3.1 MB 00:00 2024-02-26T06:06:02.709 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-02-26T06:06:02.721 INFO:teuthology.orchestra.run.smithi115.stdout:(20/166): python3-ceph-common-19.0.0-1514.g2529 2.2 MB/s | 131 kB 00:00 2024-02-26T06:06:02.731 INFO:teuthology.orchestra.run.smithi080.stdout:(28/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 853 kB/s | 171 kB 00:00 2024-02-26T06:06:02.734 INFO:teuthology.orchestra.run.smithi049.stdout:========================================================================================= 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout:========================================================================================= 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T06:06:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T06:06:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-02-26T06:06:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T06:06:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T06:06:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T06:06:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T06:06:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T06:06:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T06:06:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T06:06:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T06:06:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T06:06:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout:========================================================================================= 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout:Install 150 Packages 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 16 Packages 2024-02-26T06:06:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:06:02.748 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 221 M 2024-02-26T06:06:02.748 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-02-26T06:06:02.757 INFO:teuthology.orchestra.run.smithi080.stdout:(29/166): ceph-grafana-dashboards-19.0.0-1514.g 621 kB/s | 31 kB 00:00 2024-02-26T06:06:02.780 INFO:teuthology.orchestra.run.smithi115.stdout:(21/166): python3-cephfs-19.0.0-1514.g25296a8f. 2.7 MB/s | 161 kB 00:00 2024-02-26T06:06:02.872 INFO:teuthology.orchestra.run.smithi115.stdout:(22/166): python3-rados-19.0.0-1514.g25296a8f.e 3.4 MB/s | 320 kB 00:00 2024-02-26T06:06:02.891 INFO:teuthology.orchestra.run.smithi080.stdout:(30/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 868 kB/s | 138 kB 00:00 2024-02-26T06:06:02.956 INFO:teuthology.orchestra.run.smithi115.stdout:(23/166): python3-rbd-19.0.0-1514.g25296a8f.el9 3.5 MB/s | 298 kB 00:00 2024-02-26T06:06:03.039 INFO:teuthology.orchestra.run.smithi115.stdout:(24/166): python3-rgw-19.0.0-1514.g25296a8f.el9 1.2 MB/s | 99 kB 00:00 2024-02-26T06:06:03.050 INFO:teuthology.orchestra.run.smithi080.stdout:(31/166): ceph-mgr-dashboard-19.0.0-1514.g25296 6.1 MB/s | 1.8 MB 00:00 2024-02-26T06:06:03.089 INFO:teuthology.orchestra.run.smithi115.stdout:(25/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 1.7 MB/s | 85 kB 00:00 2024-02-26T06:06:03.125 INFO:teuthology.orchestra.run.smithi080.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1514.g25 3.2 MB/s | 247 kB 00:00 2024-02-26T06:06:03.209 INFO:teuthology.orchestra.run.smithi080.stdout:(33/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 627 kB/s | 52 kB 00:00 2024-02-26T06:06:03.390 INFO:teuthology.orchestra.run.smithi115.stdout:(26/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 10 MB/s | 3.1 MB 00:00 2024-02-26T06:06:03.440 INFO:teuthology.orchestra.run.smithi115.stdout:(27/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 3.4 MB/s | 171 kB 00:00 2024-02-26T06:06:03.482 INFO:teuthology.orchestra.run.smithi115.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1514.g 759 kB/s | 31 kB 00:00 2024-02-26T06:06:03.532 INFO:teuthology.orchestra.run.smithi115.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 2.7 MB/s | 138 kB 00:00 2024-02-26T06:06:03.626 INFO:teuthology.orchestra.run.smithi080.stdout:(34/166): ceph-test-19.0.0-1514.g25296a8f.el9.x 9.6 MB/s | 47 MB 00:04 2024-02-26T06:06:03.651 INFO:teuthology.orchestra.run.smithi080.stdout:(35/166): ceph-prometheus-alerts-19.0.0-1514.g2 35 kB/s | 15 kB 00:00 2024-02-26T06:06:03.690 INFO:teuthology.orchestra.run.smithi115.stdout:(30/166): ceph-mgr-dashboard-19.0.0-1514.g25296 11 MB/s | 1.8 MB 00:00 2024-02-26T06:06:03.812 INFO:teuthology.orchestra.run.smithi080.stdout:(36/166): ceph-volume-19.0.0-1514.g25296a8f.el9 1.4 MB/s | 264 kB 00:00 2024-02-26T06:06:03.846 INFO:teuthology.orchestra.run.smithi080.stdout:(37/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 3.8 MB/s | 748 kB 00:00 2024-02-26T06:06:03.896 INFO:teuthology.orchestra.run.smithi080.stdout:(38/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 218 kB/s | 18 kB 00:00 2024-02-26T06:06:03.931 INFO:teuthology.orchestra.run.smithi049.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-02-26T06:06:04.004 INFO:teuthology.orchestra.run.smithi080.stdout:(39/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 111 kB/s | 18 kB 00:00 2024-02-26T06:06:04.039 INFO:teuthology.orchestra.run.smithi080.stdout:(40/166): fuse-2.9.9-15.el9.x86_64.rpm 686 kB/s | 80 kB 00:00 2024-02-26T06:06:04.164 INFO:teuthology.orchestra.run.smithi080.stdout:(41/166): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-02-26T06:06:04.189 INFO:teuthology.orchestra.run.smithi080.stdout:(42/166): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-02-26T06:06:04.223 INFO:teuthology.orchestra.run.smithi080.stdout:(43/166): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-02-26T06:06:04.248 INFO:teuthology.orchestra.run.smithi080.stdout:(44/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 218 kB 00:00 2024-02-26T06:06:04.273 INFO:teuthology.orchestra.run.smithi080.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-02-26T06:06:04.307 INFO:teuthology.orchestra.run.smithi115.stdout:(31/166): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-02-26T06:06:04.315 INFO:teuthology.orchestra.run.smithi080.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-02-26T06:06:04.357 INFO:teuthology.orchestra.run.smithi080.stdout:(47/166): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-02-26T06:06:04.366 INFO:teuthology.orchestra.run.smithi115.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1514.g25 4.2 MB/s | 247 kB 00:00 2024-02-26T06:06:04.408 INFO:teuthology.orchestra.run.smithi080.stdout:(48/166): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-02-26T06:06:04.416 INFO:teuthology.orchestra.run.smithi115.stdout:(33/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 1.0 MB/s | 52 kB 00:00 2024-02-26T06:06:04.456 INFO:teuthology.orchestra.run.smithi049.stdout:(2/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 1.7 MB/s | 886 kB 00:00 2024-02-26T06:06:04.458 INFO:teuthology.orchestra.run.smithi080.stdout:(49/166): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-02-26T06:06:04.466 INFO:teuthology.orchestra.run.smithi115.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1514.g2 307 kB/s | 15 kB 00:00 2024-02-26T06:06:04.492 INFO:teuthology.orchestra.run.smithi080.stdout:(50/166): python3-cryptography-36.0.1-4.el9.x86 5.0 MB/s | 1.2 MB 00:00 2024-02-26T06:06:04.525 INFO:teuthology.orchestra.run.smithi115.stdout:(35/166): ceph-volume-19.0.0-1514.g25296a8f.el9 4.4 MB/s | 264 kB 00:00 2024-02-26T06:06:04.556 INFO:teuthology.orchestra.run.smithi049.stdout:(3/166): ceph-immutable-object-cache-19.0.0-151 1.4 MB/s | 145 kB 00:00 2024-02-26T06:06:04.609 INFO:teuthology.orchestra.run.smithi080.stdout:(51/166): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-02-26T06:06:04.634 INFO:teuthology.orchestra.run.smithi115.stdout:(36/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 6.8 MB/s | 748 kB 00:00 2024-02-26T06:06:04.634 INFO:teuthology.orchestra.run.smithi080.stdout:(52/166): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-02-26T06:06:04.693 INFO:teuthology.orchestra.run.smithi080.stdout:(53/166): elfutils-libelf-devel-0.190-2.el9.x86 370 kB/s | 21 kB 00:00 2024-02-26T06:06:04.700 INFO:teuthology.orchestra.run.smithi115.stdout:(37/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 274 kB/s | 18 kB 00:00 2024-02-26T06:06:04.726 INFO:teuthology.orchestra.run.smithi080.stdout:(54/166): boost-program-options-1.75.0-8.el9.x8 912 kB/s | 107 kB 00:00 2024-02-26T06:06:04.785 INFO:teuthology.orchestra.run.smithi080.stdout:(55/166): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-02-26T06:06:04.826 INFO:teuthology.orchestra.run.smithi115.stdout:(38/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 141 kB/s | 18 kB 00:00 2024-02-26T06:06:04.893 INFO:teuthology.orchestra.run.smithi080.stdout:(56/166): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-02-26T06:06:04.952 INFO:teuthology.orchestra.run.smithi080.stdout:(57/166): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-02-26T06:06:05.053 INFO:teuthology.orchestra.run.smithi080.stdout:(58/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-02-26T06:06:05.120 INFO:teuthology.orchestra.run.smithi080.stdout:(59/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 680 kB/s | 45 kB 00:00 2024-02-26T06:06:05.123 INFO:teuthology.orchestra.run.smithi049.stdout:(4/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 3.8 MB/s | 2.1 MB 00:00 2024-02-26T06:06:05.245 INFO:teuthology.orchestra.run.smithi080.stdout:(60/166): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-02-26T06:06:05.412 INFO:teuthology.orchestra.run.smithi080.stdout:(61/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.8 MB/s | 3.0 MB 00:00 2024-02-26T06:06:05.490 INFO:teuthology.orchestra.run.smithi049.stdout:(5/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-02-26T06:06:05.504 INFO:teuthology.orchestra.run.smithi080.stdout:(62/166): libtool-2.4.6-45.el9.x86_64.rpm 2.2 MB/s | 582 kB 00:00 2024-02-26T06:06:05.518 INFO:teuthology.orchestra.run.smithi115.stdout:(39/166): fuse-2.9.9-15.el9.x86_64.rpm 116 kB/s | 80 kB 00:00 2024-02-26T06:06:05.529 INFO:teuthology.orchestra.run.smithi080.stdout:(63/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 218 kB/s | 26 kB 00:00 2024-02-26T06:06:05.588 INFO:teuthology.orchestra.run.smithi080.stdout:(64/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-02-26T06:06:05.647 INFO:teuthology.orchestra.run.smithi080.stdout:(65/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-02-26T06:06:05.667 INFO:teuthology.orchestra.run.smithi049.stdout:(6/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 2.5 MB/s | 5.3 MB 00:02 2024-02-26T06:06:05.697 INFO:teuthology.orchestra.run.smithi080.stdout:(66/166): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-02-26T06:06:05.803 INFO:teuthology.orchestra.run.smithi080.stdout:(67/166): lua-5.4.4-4.el9.x86_64.rpm 875 kB/s | 188 kB 00:00 2024-02-26T06:06:06.181 INFO:teuthology.orchestra.run.smithi080.stdout:(68/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 11 MB/s | 4.1 MB 00:00 2024-02-26T06:06:06.401 INFO:teuthology.orchestra.run.smithi115.stdout:(40/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x8 2.6 MB/s | 17 MB 00:06 2024-02-26T06:06:06.427 INFO:teuthology.orchestra.run.smithi115.stdout:(41/166): jq-1.6-16.el9.x86_64.rpm 206 kB/s | 187 kB 00:00 2024-02-26T06:06:06.466 INFO:teuthology.orchestra.run.smithi049.stdout:(7/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x86 4.8 MB/s | 4.7 MB 00:00 2024-02-26T06:06:06.489 INFO:teuthology.orchestra.run.smithi080.stdout:(69/166): openblas-openmp-0.3.21-2.el9.x86_64.r 6.0 MB/s | 4.7 MB 00:00 2024-02-26T06:06:06.623 INFO:teuthology.orchestra.run.smithi080.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 6.6 MB/s | 905 kB 00:00 2024-02-26T06:06:06.665 INFO:teuthology.orchestra.run.smithi080.stdout:(71/166): postgresql-13.11-1.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-02-26T06:06:06.774 INFO:teuthology.orchestra.run.smithi080.stdout:(72/166): postgresql-private-libs-13.11-1.el9.x 899 kB/s | 135 kB 00:00 2024-02-26T06:06:06.932 INFO:teuthology.orchestra.run.smithi080.stdout:(73/166): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-02-26T06:06:07.042 INFO:teuthology.orchestra.run.smithi080.stdout:(74/166): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-02-26T06:06:07.141 INFO:teuthology.orchestra.run.smithi080.stdout:(75/166): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2024-02-26T06:06:07.224 INFO:teuthology.orchestra.run.smithi049.stdout:(8/166): ceph-common-19.0.0-1514.g25296a8f.el9. 5.4 MB/s | 20 MB 00:03 2024-02-26T06:06:07.259 INFO:teuthology.orchestra.run.smithi080.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-02-26T06:06:07.292 INFO:teuthology.orchestra.run.smithi049.stdout:(9/166): ceph-selinux-19.0.0-1514.g25296a8f.el9 382 kB/s | 25 kB 00:00 2024-02-26T06:06:07.367 INFO:teuthology.orchestra.run.smithi080.stdout:(77/166): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-02-26T06:06:07.418 INFO:teuthology.orchestra.run.smithi080.stdout:(78/166): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-02-26T06:06:07.527 INFO:teuthology.orchestra.run.smithi080.stdout:(79/166): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-02-26T06:06:07.543 INFO:teuthology.orchestra.run.smithi080.stdout:(80/166): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-02-26T06:06:07.552 INFO:teuthology.orchestra.run.smithi115.stdout:(42/166): libquadmath-11.4.1-3.el9.x86_64.rpm 174 kB/s | 195 kB 00:01 2024-02-26T06:06:07.610 INFO:teuthology.orchestra.run.smithi080.stdout:(81/166): python3-markupsafe-1.1.1-12.el9.x86_6 522 kB/s | 35 kB 00:00 2024-02-26T06:06:07.727 INFO:teuthology.orchestra.run.smithi080.stdout:(82/166): ceph-mgr-diskprediction-local-19.0.0- 1.5 MB/s | 7.4 MB 00:04 2024-02-26T06:06:07.752 INFO:teuthology.orchestra.run.smithi080.stdout:(83/166): python3-mako-1.1.4-6.el9.noarch.rpm 762 kB/s | 172 kB 00:00 2024-02-26T06:06:07.828 INFO:teuthology.orchestra.run.smithi080.stdout:(84/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-02-26T06:06:07.861 INFO:teuthology.orchestra.run.smithi080.stdout:(85/166): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-02-26T06:06:07.895 INFO:teuthology.orchestra.run.smithi080.stdout:(86/166): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-02-26T06:06:07.920 INFO:teuthology.orchestra.run.smithi080.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-02-26T06:06:07.995 INFO:teuthology.orchestra.run.smithi080.stdout:(88/166): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-02-26T06:06:08.037 INFO:teuthology.orchestra.run.smithi080.stdout:(89/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-02-26T06:06:08.069 INFO:teuthology.orchestra.run.smithi115.stdout:(43/166): libgfortran-11.4.1-3.el9.x86_64.rpm 482 kB/s | 803 kB 00:01 2024-02-26T06:06:08.137 INFO:teuthology.orchestra.run.smithi080.stdout:(90/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.7 MB/s | 5.1 MB 00:00 2024-02-26T06:06:08.179 INFO:teuthology.orchestra.run.smithi080.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 359 kB/s | 51 kB 00:00 2024-02-26T06:06:08.204 INFO:teuthology.orchestra.run.smithi080.stdout:(92/166): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-02-26T06:06:08.238 INFO:teuthology.orchestra.run.smithi080.stdout:(93/166): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-02-26T06:06:08.292 INFO:teuthology.orchestra.run.smithi049.stdout:(10/166): ceph-radosgw-19.0.0-1514.g25296a8f.el 5.3 MB/s | 9.7 MB 00:01 2024-02-26T06:06:08.305 INFO:teuthology.orchestra.run.smithi080.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 664 kB/s | 44 kB 00:00 2024-02-26T06:06:08.330 INFO:teuthology.orchestra.run.smithi080.stdout:(95/166): python3-toml-0.10.2-6.el9.noarch.rpm 332 kB/s | 42 kB 00:00 2024-02-26T06:06:08.388 INFO:teuthology.orchestra.run.smithi080.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 599 kB/s | 35 kB 00:00 2024-02-26T06:06:08.408 INFO:teuthology.orchestra.run.smithi049.stdout:(11/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 1.8 MB/s | 217 kB 00:00 2024-02-26T06:06:08.464 INFO:teuthology.orchestra.run.smithi080.stdout:(97/166): qatlib-23.11.0-1.el9.x86_64.rpm 1.3 MB/s | 216 kB 00:00 2024-02-26T06:06:08.489 INFO:teuthology.orchestra.run.smithi080.stdout:(98/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 442 kB/s | 44 kB 00:00 2024-02-26T06:06:08.508 INFO:teuthology.orchestra.run.smithi049.stdout:(12/166): cephfs-shell-19.0.0-1514.g25296a8f.el 272 kB/s | 27 kB 00:00 2024-02-26T06:06:08.527 INFO:teuthology.orchestra.run.smithi115.stdout:(44/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 224 kB/s | 218 kB 00:00 2024-02-26T06:06:08.556 INFO:teuthology.orchestra.run.smithi080.stdout:(99/166): uuid-1.6.2-55.el9.x86_64.rpm 868 kB/s | 58 kB 00:00 2024-02-26T06:06:08.625 INFO:teuthology.orchestra.run.smithi049.stdout:(13/166): libcephfs-devel-19.0.0-1514.g25296a8f 269 kB/s | 31 kB 00:00 2024-02-26T06:06:08.640 INFO:teuthology.orchestra.run.smithi080.stdout:(100/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-02-26T06:06:08.674 INFO:teuthology.orchestra.run.smithi080.stdout:(101/166): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-02-26T06:06:08.749 INFO:teuthology.orchestra.run.smithi080.stdout:(102/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-02-26T06:06:08.782 INFO:teuthology.orchestra.run.smithi080.stdout:(103/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 448 kB/s | 64 kB 00:00 2024-02-26T06:06:08.851 INFO:teuthology.orchestra.run.smithi049.stdout:(14/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x8 5.3 MB/s | 17 MB 00:03 2024-02-26T06:06:08.910 INFO:teuthology.orchestra.run.smithi049.stdout:(15/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 2.6 MB/s | 743 kB 00:00 2024-02-26T06:06:08.941 INFO:teuthology.orchestra.run.smithi080.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 116 kB/s | 22 kB 00:00 2024-02-26T06:06:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:(16/166): libcephsqlite-19.0.0-1514.g25296a8f.e 1.6 MB/s | 163 kB 00:00 2024-02-26T06:06:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 269 kB/s | 253 kB 00:00 2024-02-26T06:06:09.018 INFO:teuthology.orchestra.run.smithi049.stdout:(17/166): librados-devel-19.0.0-1514.g25296a8f. 1.1 MB/s | 127 kB 00:00 2024-02-26T06:06:09.135 INFO:teuthology.orchestra.run.smithi049.stdout:(18/166): libradosstriper1-19.0.0-1514.g25296a8 2.7 MB/s | 498 kB 00:00 2024-02-26T06:06:09.225 INFO:teuthology.orchestra.run.smithi080.stdout:(105/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-02-26T06:06:09.244 INFO:teuthology.orchestra.run.smithi049.stdout:(19/166): python3-ceph-argparse-19.0.0-1514.g25 420 kB/s | 45 kB 00:00 2024-02-26T06:06:09.266 INFO:teuthology.orchestra.run.smithi080.stdout:(106/166): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-02-26T06:06:09.368 INFO:teuthology.orchestra.run.smithi049.stdout:(20/166): python3-ceph-common-19.0.0-1514.g2529 1.0 MB/s | 131 kB 00:00 2024-02-26T06:06:09.486 INFO:teuthology.orchestra.run.smithi049.stdout:(21/166): python3-cephfs-19.0.0-1514.g25296a8f. 1.4 MB/s | 161 kB 00:00 2024-02-26T06:06:09.492 INFO:teuthology.orchestra.run.smithi080.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 86 kB/s | 19 kB 00:00 2024-02-26T06:06:09.517 INFO:teuthology.orchestra.run.smithi080.stdout:(108/166): abseil-cpp-20211102.0-3.el9.x86_64.r 952 kB/s | 548 kB 00:00 2024-02-26T06:06:09.603 INFO:teuthology.orchestra.run.smithi049.stdout:(22/166): python3-rados-19.0.0-1514.g25296a8f.e 2.7 MB/s | 320 kB 00:00 2024-02-26T06:06:09.617 INFO:teuthology.orchestra.run.smithi080.stdout:(109/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-02-26T06:06:09.692 INFO:teuthology.orchestra.run.smithi080.stdout:(110/166): gperftools-libs-2.9.1-2.el9.x86_64.r 661 kB/s | 309 kB 00:00 2024-02-26T06:06:09.711 INFO:teuthology.orchestra.run.smithi049.stdout:(23/166): python3-rbd-19.0.0-1514.g25296a8f.el9 2.7 MB/s | 298 kB 00:00 2024-02-26T06:06:09.719 INFO:teuthology.orchestra.run.smithi080.stdout:(111/166): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-02-26T06:06:09.769 INFO:teuthology.orchestra.run.smithi115.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 141 kB/s | 106 kB 00:00 2024-02-26T06:06:09.793 INFO:teuthology.orchestra.run.smithi080.stdout:(112/166): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-02-26T06:06:09.818 INFO:teuthology.orchestra.run.smithi080.stdout:(113/166): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-02-26T06:06:09.853 INFO:teuthology.orchestra.run.smithi049.stdout:(24/166): python3-rgw-19.0.0-1514.g25296a8f.el9 707 kB/s | 99 kB 00:00 2024-02-26T06:06:09.927 INFO:teuthology.orchestra.run.smithi080.stdout:(114/166): python3-asyncssh-2.13.2-4.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-02-26T06:06:09.968 INFO:teuthology.orchestra.run.smithi080.stdout:(115/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-02-26T06:06:09.970 INFO:teuthology.orchestra.run.smithi049.stdout:(25/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 5.4 MB/s | 5.1 MB 00:00 2024-02-26T06:06:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:(26/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 604 kB/s | 85 kB 00:00 2024-02-26T06:06:10.027 INFO:teuthology.orchestra.run.smithi080.stdout:(116/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-02-26T06:06:10.102 INFO:teuthology.orchestra.run.smithi080.stdout:(117/166): libarrow-9.0.0-10.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-02-26T06:06:10.121 INFO:teuthology.orchestra.run.smithi049.stdout:(27/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 1.3 MB/s | 171 kB 00:00 2024-02-26T06:06:10.128 INFO:teuthology.orchestra.run.smithi080.stdout:(118/166): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-02-26T06:06:10.153 INFO:teuthology.orchestra.run.smithi080.stdout:(119/166): python3-cachetools-4.2.4-1.el9.noarc 175 kB/s | 32 kB 00:00 2024-02-26T06:06:10.203 INFO:teuthology.orchestra.run.smithi080.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-02-26T06:06:10.237 INFO:teuthology.orchestra.run.smithi080.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-02-26T06:06:10.237 INFO:teuthology.orchestra.run.smithi049.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1514.g 271 kB/s | 31 kB 00:00 2024-02-26T06:06:10.262 INFO:teuthology.orchestra.run.smithi080.stdout:(122/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-02-26T06:06:10.296 INFO:teuthology.orchestra.run.smithi080.stdout:(123/166): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-02-26T06:06:10.332 INFO:teuthology.orchestra.run.smithi080.stdout:(124/166): python3-google-auth-2.28.0-1.el9.noa 2.1 MB/s | 203 kB 00:00 2024-02-26T06:06:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 1.2 MB/s | 138 kB 00:00 2024-02-26T06:06:10.396 INFO:teuthology.orchestra.run.smithi080.stdout:(125/166): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-02-26T06:06:10.422 INFO:teuthology.orchestra.run.smithi080.stdout:(126/166): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-02-26T06:06:10.464 INFO:teuthology.orchestra.run.smithi080.stdout:(127/166): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-02-26T06:06:10.489 INFO:teuthology.orchestra.run.smithi080.stdout:(128/166): python3-jaraco-classes-3.2.1-5.el9.n 191 kB/s | 18 kB 00:00 2024-02-26T06:06:10.514 INFO:teuthology.orchestra.run.smithi080.stdout:(129/166): python3-jaraco-collections-3.0.0-8.e 250 kB/s | 23 kB 00:00 2024-02-26T06:06:10.546 INFO:teuthology.orchestra.run.smithi049.stdout:(30/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 5.4 MB/s | 3.1 MB 00:00 2024-02-26T06:06:10.556 INFO:teuthology.orchestra.run.smithi080.stdout:(130/166): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-02-26T06:06:10.581 INFO:teuthology.orchestra.run.smithi080.stdout:(131/166): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-02-26T06:06:10.594 INFO:teuthology.orchestra.run.smithi115.stdout:(47/166): python3-pycparser-2.20-6.el9.noarch.r 164 kB/s | 135 kB 00:00 2024-02-26T06:06:10.623 INFO:teuthology.orchestra.run.smithi080.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-02-26T06:06:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:(133/166): python3-logutils-0.3.5-21.el9.noarch 460 kB/s | 46 kB 00:00 2024-02-26T06:06:10.674 INFO:teuthology.orchestra.run.smithi080.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 854 kB/s | 79 kB 00:00 2024-02-26T06:06:10.703 INFO:teuthology.orchestra.run.smithi115.stdout:(48/166): python3-cryptography-36.0.1-4.el9.x86 577 kB/s | 1.2 MB 00:02 2024-02-26T06:06:10.707 INFO:teuthology.orchestra.run.smithi080.stdout:(135/166): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-02-26T06:06:10.755 INFO:teuthology.orchestra.run.smithi049.stdout:(31/166): ceph-mgr-dashboard-19.0.0-1514.g25296 4.5 MB/s | 1.8 MB 00:00 2024-02-26T06:06:10.758 INFO:teuthology.orchestra.run.smithi080.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-02-26T06:06:10.775 INFO:teuthology.orchestra.run.smithi080.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-02-26T06:06:10.800 INFO:teuthology.orchestra.run.smithi080.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-02-26T06:06:10.872 INFO:teuthology.orchestra.run.smithi049.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1514.g25 2.1 MB/s | 247 kB 00:00 2024-02-26T06:06:10.925 INFO:teuthology.orchestra.run.smithi080.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 169 kB/s | 28 kB 00:00 2024-02-26T06:06:10.951 INFO:teuthology.orchestra.run.smithi080.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 175 kB/s | 31 kB 00:00 2024-02-26T06:06:10.976 INFO:teuthology.orchestra.run.smithi080.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-02-26T06:06:10.980 INFO:teuthology.orchestra.run.smithi049.stdout:(33/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 487 kB/s | 52 kB 00:00 2024-02-26T06:06:11.026 INFO:teuthology.orchestra.run.smithi080.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-02-26T06:06:11.051 INFO:teuthology.orchestra.run.smithi080.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-02-26T06:06:11.076 INFO:teuthology.orchestra.run.smithi080.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-02-26T06:06:11.106 INFO:teuthology.orchestra.run.smithi049.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1514.g2 123 kB/s | 15 kB 00:00 2024-02-26T06:06:11.143 INFO:teuthology.orchestra.run.smithi080.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-02-26T06:06:11.169 INFO:teuthology.orchestra.run.smithi080.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 765 kB/s | 90 kB 00:00 2024-02-26T06:06:11.194 INFO:teuthology.orchestra.run.smithi080.stdout:(147/166): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-02-26T06:06:11.248 INFO:teuthology.orchestra.run.smithi049.stdout:(35/166): ceph-volume-19.0.0-1514.g25296a8f.el9 1.8 MB/s | 264 kB 00:00 2024-02-26T06:06:11.261 INFO:teuthology.orchestra.run.smithi080.stdout:(148/166): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-02-26T06:06:11.286 INFO:teuthology.orchestra.run.smithi080.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-02-26T06:06:11.457 INFO:teuthology.orchestra.run.smithi049.stdout:(36/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 3.5 MB/s | 748 kB 00:00 2024-02-26T06:06:11.461 INFO:teuthology.orchestra.run.smithi115.stdout:(49/166): python3-requests-2.25.1-8.el9.noarch. 144 kB/s | 125 kB 00:00 2024-02-26T06:06:11.565 INFO:teuthology.orchestra.run.smithi049.stdout:(37/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 170 kB/s | 18 kB 00:00 2024-02-26T06:06:11.611 INFO:teuthology.orchestra.run.smithi115.stdout:(50/166): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-02-26T06:06:11.678 INFO:teuthology.orchestra.run.smithi115.stdout:(51/166): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-02-26T06:06:11.721 INFO:teuthology.orchestra.run.smithi115.stdout:(52/166): python3-urllib3-1.26.5-5.el9.noarch.r 211 kB/s | 215 kB 00:01 2024-02-26T06:06:11.746 INFO:teuthology.orchestra.run.smithi115.stdout:(53/166): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-02-26T06:06:11.791 INFO:teuthology.orchestra.run.smithi049.stdout:(38/166): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-02-26T06:06:11.796 INFO:teuthology.orchestra.run.smithi115.stdout:(54/166): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-02-26T06:06:11.821 INFO:teuthology.orchestra.run.smithi115.stdout:(55/166): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-02-26T06:06:11.833 INFO:teuthology.orchestra.run.smithi049.stdout:(39/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 66 kB/s | 18 kB 00:00 2024-02-26T06:06:11.846 INFO:teuthology.orchestra.run.smithi115.stdout:(56/166): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-02-26T06:06:11.867 INFO:teuthology.orchestra.run.smithi049.stdout:(40/166): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 80 kB 00:00 2024-02-26T06:06:11.888 INFO:teuthology.orchestra.run.smithi115.stdout:(57/166): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-02-26T06:06:11.916 INFO:teuthology.orchestra.run.smithi049.stdout:(41/166): jq-1.6-16.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-02-26T06:06:11.930 INFO:teuthology.orchestra.run.smithi115.stdout:(58/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-02-26T06:06:11.970 INFO:teuthology.orchestra.run.smithi080.stdout:(150/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-02-26T06:06:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:(59/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-02-26T06:06:11.992 INFO:teuthology.orchestra.run.smithi049.stdout:(42/166): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-02-26T06:06:12.003 INFO:teuthology.orchestra.run.smithi080.stdout:(151/166): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-02-26T06:06:12.023 INFO:teuthology.orchestra.run.smithi115.stdout:(60/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-02-26T06:06:12.025 INFO:teuthology.orchestra.run.smithi049.stdout:(43/166): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-02-26T06:06:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:(61/166): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-02-26T06:06:12.098 INFO:teuthology.orchestra.run.smithi115.stdout:(62/166): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-02-26T06:06:12.118 INFO:teuthology.orchestra.run.smithi049.stdout:(44/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.7 MB/s | 218 kB 00:00 2024-02-26T06:06:12.124 INFO:teuthology.orchestra.run.smithi115.stdout:(63/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 437 kB/s | 26 kB 00:00 2024-02-26T06:06:12.149 INFO:teuthology.orchestra.run.smithi115.stdout:(64/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-02-26T06:06:12.151 INFO:teuthology.orchestra.run.smithi049.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-02-26T06:06:12.174 INFO:teuthology.orchestra.run.smithi115.stdout:(65/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-02-26T06:06:12.200 INFO:teuthology.orchestra.run.smithi115.stdout:(66/166): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-02-26T06:06:12.224 INFO:teuthology.orchestra.run.smithi115.stdout:(67/166): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-02-26T06:06:12.270 INFO:teuthology.orchestra.run.smithi080.stdout:(152/166): NetworkManager-1.45.91-1.el9.x86_64. 7.6 MB/s | 2.3 MB 00:00 2024-02-26T06:06:12.312 INFO:teuthology.orchestra.run.smithi080.stdout:(153/166): NetworkManager-libnm-1.45.91-1.el9.x 5.9 MB/s | 1.8 MB 00:00 2024-02-26T06:06:12.333 INFO:teuthology.orchestra.run.smithi115.stdout:(68/166): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-02-26T06:06:12.337 INFO:teuthology.orchestra.run.smithi080.stdout:(154/166): NetworkManager-team-1.45.91-1.el9.x8 635 kB/s | 42 kB 00:00 2024-02-26T06:06:12.363 INFO:teuthology.orchestra.run.smithi080.stdout:(155/166): NetworkManager-tui-1.45.91-1.el9.x86 4.8 MB/s | 247 kB 00:00 2024-02-26T06:06:12.388 INFO:teuthology.orchestra.run.smithi080.stdout:(156/166): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-02-26T06:06:12.401 INFO:teuthology.orchestra.run.smithi115.stdout:(69/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 23 MB/s | 4.1 MB 00:00 2024-02-26T06:06:12.413 INFO:teuthology.orchestra.run.smithi080.stdout:(157/166): elfutils-debuginfod-client-0.190-2.e 726 kB/s | 36 kB 00:00 2024-02-26T06:06:12.443 INFO:teuthology.orchestra.run.smithi115.stdout:(70/166): postgresql-13.11-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-02-26T06:06:12.455 INFO:teuthology.orchestra.run.smithi080.stdout:(158/166): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-02-26T06:06:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:(71/166): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 905 kB 00:00 2024-02-26T06:06:12.480 INFO:teuthology.orchestra.run.smithi080.stdout:(159/166): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-02-26T06:06:12.502 INFO:teuthology.orchestra.run.smithi115.stdout:(72/166): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 135 kB 00:00 2024-02-26T06:06:12.506 INFO:teuthology.orchestra.run.smithi080.stdout:(160/166): libacl-2.3.1-4.el9.x86_64.rpm 458 kB/s | 23 kB 00:00 2024-02-26T06:06:12.561 INFO:teuthology.orchestra.run.smithi115.stdout:(73/166): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-02-26T06:06:12.653 INFO:teuthology.orchestra.run.smithi115.stdout:(74/166): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-02-26T06:06:12.668 INFO:teuthology.orchestra.run.smithi049.stdout:(46/166): ceph-test-19.0.0-1514.g25296a8f.el9.x 8.8 MB/s | 47 MB 00:05 2024-02-26T06:06:12.678 INFO:teuthology.orchestra.run.smithi115.stdout:(75/166): python3-attrs-20.3.0-7.el9.noarch.rpm 749 kB/s | 87 kB 00:00 2024-02-26T06:06:12.698 INFO:teuthology.orchestra.run.smithi080.stdout:(161/166): openssl-3.0.7-25.el9.x86_64.rpm 5.6 MB/s | 1.2 MB 00:00 2024-02-26T06:06:12.710 INFO:teuthology.orchestra.run.smithi049.stdout:(47/166): python3-ply-3.11-14.el9.noarch.rpm 191 kB/s | 106 kB 00:00 2024-02-26T06:06:12.728 INFO:teuthology.orchestra.run.smithi115.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-02-26T06:06:12.777 INFO:teuthology.orchestra.run.smithi049.stdout:(48/166): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2024-02-26T06:06:12.802 INFO:teuthology.orchestra.run.smithi049.stdout:(49/166): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-02-26T06:06:12.823 INFO:teuthology.orchestra.run.smithi080.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2024-02-26T06:06:12.837 INFO:teuthology.orchestra.run.smithi115.stdout:(77/166): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-02-26T06:06:12.844 INFO:teuthology.orchestra.run.smithi049.stdout:(50/166): python3-requests-2.25.1-8.el9.noarch. 942 kB/s | 125 kB 00:00 2024-02-26T06:06:12.862 INFO:teuthology.orchestra.run.smithi115.stdout:(78/166): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-02-26T06:06:12.887 INFO:teuthology.orchestra.run.smithi049.stdout:(51/166): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-02-26T06:06:12.887 INFO:teuthology.orchestra.run.smithi115.stdout:(79/166): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-02-26T06:06:12.913 INFO:teuthology.orchestra.run.smithi115.stdout:(80/166): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-02-26T06:06:12.915 INFO:teuthology.orchestra.run.smithi080.stdout:(163/166): librados2-19.0.0-1514.g25296a8f.el9. 2.0 MB/s | 3.4 MB 00:01 2024-02-26T06:06:12.938 INFO:teuthology.orchestra.run.smithi115.stdout:(81/166): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-02-26T06:06:12.963 INFO:teuthology.orchestra.run.smithi115.stdout:(82/166): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-02-26T06:06:12.990 INFO:teuthology.orchestra.run.smithi080.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 994 kB/s | 74 kB 00:00 2024-02-26T06:06:13.022 INFO:teuthology.orchestra.run.smithi115.stdout:(83/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-02-26T06:06:13.036 INFO:teuthology.orchestra.run.smithi049.stdout:(52/166): boost-program-options-1.75.0-8.el9.x8 717 kB/s | 107 kB 00:00 2024-02-26T06:06:13.098 INFO:teuthology.orchestra.run.smithi080.stdout:(165/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.9 MB/s | 1.1 MB 00:00 2024-02-26T06:06:13.111 INFO:teuthology.orchestra.run.smithi049.stdout:(53/166): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-02-26T06:06:13.137 INFO:teuthology.orchestra.run.smithi049.stdout:(54/166): elfutils-libelf-devel-0.190-2.el9.x86 216 kB/s | 21 kB 00:00 2024-02-26T06:06:13.139 INFO:teuthology.orchestra.run.smithi115.stdout:(84/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-02-26T06:06:13.140 INFO:teuthology.orchestra.run.smithi080.stdout:(166/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 3.4 MB/s | 2.2 MB 00:00 2024-02-26T06:06:13.145 INFO:teuthology.orchestra.run.smithi080.stdout:-------------------------------------------------------------------------------- 2024-02-26T06:06:13.146 INFO:teuthology.orchestra.run.smithi080.stdout:Total 13 MB/s | 221 MB 00:17 2024-02-26T06:06:13.156 INFO:teuthology.orchestra.run.smithi115.stdout:(85/166): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-02-26T06:06:13.189 INFO:teuthology.orchestra.run.smithi115.stdout:(86/166): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-02-26T06:06:13.196 INFO:teuthology.orchestra.run.smithi049.stdout:(55/166): flexiblas-3.0.4-8.el9.x86_64.rpm 567 kB/s | 33 kB 00:00 2024-02-26T06:06:13.215 INFO:teuthology.orchestra.run.smithi115.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-02-26T06:06:13.229 INFO:teuthology.orchestra.run.smithi049.stdout:(56/166): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-02-26T06:06:13.240 INFO:teuthology.orchestra.run.smithi115.stdout:(88/166): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-02-26T06:06:13.262 INFO:teuthology.orchestra.run.smithi049.stdout:(57/166): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-02-26T06:06:13.274 INFO:teuthology.orchestra.run.smithi115.stdout:(89/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-02-26T06:06:13.299 INFO:teuthology.orchestra.run.smithi115.stdout:(90/166): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-02-26T06:06:13.310 INFO:teuthology.orchestra.run.smithi049.stdout:(58/166): flexiblas-openblas-openmp-3.0.4-8.el9 228 kB/s | 18 kB 00:00 2024-02-26T06:06:13.325 INFO:teuthology.orchestra.run.smithi115.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-02-26T06:06:13.346 INFO:teuthology.orchestra.run.smithi049.stdout:(59/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-02-26T06:06:13.350 INFO:teuthology.orchestra.run.smithi115.stdout:(92/166): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-02-26T06:06:13.372 INFO:teuthology.orchestra.run.smithi049.stdout:(60/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 745 kB/s | 45 kB 00:00 2024-02-26T06:06:13.383 INFO:teuthology.orchestra.run.smithi115.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-02-26T06:06:13.417 INFO:teuthology.orchestra.run.smithi115.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-02-26T06:06:13.459 INFO:teuthology.orchestra.run.smithi115.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 5.1 MB/s | 216 kB 00:00 2024-02-26T06:06:13.501 INFO:teuthology.orchestra.run.smithi115.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 837 kB/s | 35 kB 00:00 2024-02-26T06:06:13.522 INFO:teuthology.orchestra.run.smithi049.stdout:(61/166): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-02-26T06:06:13.543 INFO:teuthology.orchestra.run.smithi115.stdout:(97/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-26T06:06:13.605 INFO:teuthology.orchestra.run.smithi049.stdout:(62/166): libtool-2.4.6-45.el9.x86_64.rpm 2.4 MB/s | 582 kB 00:00 2024-02-26T06:06:13.631 INFO:teuthology.orchestra.run.smithi049.stdout:(63/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 237 kB/s | 26 kB 00:00 2024-02-26T06:06:13.696 INFO:teuthology.orchestra.run.smithi049.stdout:(64/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 528 kB/s | 47 kB 00:00 2024-02-26T06:06:13.718 INFO:teuthology.orchestra.run.smithi115.stdout:(98/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-02-26T06:06:13.781 INFO:teuthology.orchestra.run.smithi049.stdout:(65/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.1 MB/s | 3.0 MB 00:00 2024-02-26T06:06:13.815 INFO:teuthology.orchestra.run.smithi049.stdout:(66/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-02-26T06:06:13.848 INFO:teuthology.orchestra.run.smithi049.stdout:(67/166): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-02-26T06:06:13.865 INFO:teuthology.orchestra.run.smithi049.stdout:(68/166): openblas-0.3.21-2.el9.x86_64.rpm 424 kB/s | 35 kB 00:00 2024-02-26T06:06:14.127 INFO:teuthology.orchestra.run.smithi115.stdout:(99/166): ceph-test-19.0.0-1514.g25296a8f.el9.x 3.7 MB/s | 47 MB 00:12 2024-02-26T06:06:14.152 INFO:teuthology.orchestra.run.smithi115.stdout:(100/166): uuid-1.6.2-55.el9.x86_64.rpm 133 kB/s | 58 kB 00:00 2024-02-26T06:06:14.177 INFO:teuthology.orchestra.run.smithi115.stdout:(101/166): socat-1.7.4.1-5.el9.x86_64.rpm 481 kB/s | 305 kB 00:00 2024-02-26T06:06:14.202 INFO:teuthology.orchestra.run.smithi115.stdout:(102/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-02-26T06:06:14.228 INFO:teuthology.orchestra.run.smithi115.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-02-26T06:06:14.232 INFO:teuthology.orchestra.run.smithi049.stdout:(69/166): postgresql-13.11-1.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-02-26T06:06:14.253 INFO:teuthology.orchestra.run.smithi115.stdout:(104/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 747 kB/s | 94 kB 00:00 2024-02-26T06:06:14.355 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction check 2024-02-26T06:06:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 3.8 MB/s | 905 kB 00:00 2024-02-26T06:06:14.533 INFO:teuthology.orchestra.run.smithi049.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 135 kB 00:00 2024-02-26T06:06:14.545 INFO:teuthology.orchestra.run.smithi115.stdout:(105/166): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-02-26T06:06:14.612 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction check succeeded. 2024-02-26T06:06:14.612 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction test 2024-02-26T06:06:14.708 INFO:teuthology.orchestra.run.smithi049.stdout:(72/166): openblas-openmp-0.3.21-2.el9.x86_64.r 5.3 MB/s | 4.7 MB 00:00 2024-02-26T06:06:14.787 INFO:teuthology.orchestra.run.smithi115.stdout:(106/166): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-02-26T06:06:14.831 INFO:teuthology.orchestra.run.smithi049.stdout:(73/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 4.2 MB/s | 4.1 MB 00:00 2024-02-26T06:06:14.878 INFO:teuthology.orchestra.run.smithi115.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 213 kB/s | 19 kB 00:00 2024-02-26T06:06:14.908 INFO:teuthology.orchestra.run.smithi049.stdout:(74/166): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-02-26T06:06:15.001 INFO:teuthology.orchestra.run.smithi049.stdout:(75/166): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-02-26T06:06:15.062 INFO:teuthology.orchestra.run.smithi115.stdout:(108/166): gperftools-libs-2.9.1-2.el9.x86_64.r 597 kB/s | 309 kB 00:00 2024-02-26T06:06:15.134 INFO:teuthology.orchestra.run.smithi049.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-02-26T06:06:15.145 INFO:teuthology.orchestra.run.smithi115.stdout:(109/166): protobuf-compiler-3.14.0-13.el9.x86_ 941 kB/s | 863 kB 00:00 2024-02-26T06:06:15.171 INFO:teuthology.orchestra.run.smithi115.stdout:(110/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-02-26T06:06:15.246 INFO:teuthology.orchestra.run.smithi115.stdout:(111/166): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-02-26T06:06:15.276 INFO:teuthology.orchestra.run.smithi049.stdout:(77/166): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-02-26T06:06:15.296 INFO:teuthology.orchestra.run.smithi115.stdout:(112/166): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-02-26T06:06:15.351 INFO:teuthology.orchestra.run.smithi049.stdout:(78/166): python3-jsonpatch-1.21-16.el9.noarch. 346 kB/s | 26 kB 00:00 2024-02-26T06:06:15.363 INFO:teuthology.orchestra.run.smithi115.stdout:(113/166): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-02-26T06:06:15.426 INFO:teuthology.orchestra.run.smithi049.stdout:(79/166): python3-jsonpointer-2.0-4.el9.noarch. 259 kB/s | 19 kB 00:00 2024-02-26T06:06:15.472 INFO:teuthology.orchestra.run.smithi115.stdout:(114/166): python3-asyncssh-2.13.2-4.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-02-26T06:06:15.531 INFO:teuthology.orchestra.run.smithi115.stdout:(115/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-02-26T06:06:15.556 INFO:teuthology.orchestra.run.smithi115.stdout:(116/166): liboath-2.6.7-2.el9.x86_64.rpm 118 kB/s | 49 kB 00:00 2024-02-26T06:06:15.581 INFO:teuthology.orchestra.run.smithi115.stdout:(117/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 399 kB/s | 43 kB 00:00 2024-02-26T06:06:15.586 INFO:teuthology.orchestra.run.smithi049.stdout:(80/166): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-02-26T06:06:15.669 INFO:teuthology.orchestra.run.smithi049.stdout:(81/166): python3-markupsafe-1.1.1-12.el9.x86_6 429 kB/s | 35 kB 00:00 2024-02-26T06:06:15.731 INFO:teuthology.orchestra.run.smithi115.stdout:(118/166): python3-cachetools-4.2.4-1.el9.noarc 160 kB/s | 32 kB 00:00 2024-02-26T06:06:15.757 INFO:teuthology.orchestra.run.smithi115.stdout:(119/166): python3-certifi-2021.10.8-2.el9.noar 75 kB/s | 15 kB 00:00 2024-02-26T06:06:15.782 INFO:teuthology.orchestra.run.smithi115.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 856 kB/s | 172 kB 00:00 2024-02-26T06:06:15.865 INFO:teuthology.orchestra.run.smithi115.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-02-26T06:06:15.890 INFO:teuthology.orchestra.run.smithi115.stdout:(122/166): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-02-26T06:06:15.982 INFO:teuthology.orchestra.run.smithi115.stdout:(123/166): python3-google-auth-2.28.0-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-02-26T06:06:16.033 INFO:teuthology.orchestra.run.smithi115.stdout:(124/166): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-02-26T06:06:16.052 INFO:teuthology.orchestra.run.smithi049.stdout:(82/166): postgresql-server-13.11-1.el9.x86_64. 3.8 MB/s | 5.7 MB 00:01 2024-02-26T06:06:16.083 INFO:teuthology.orchestra.run.smithi115.stdout:(125/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 930 kB/s | 303 kB 00:00 2024-02-26T06:06:16.109 INFO:teuthology.orchestra.run.smithi115.stdout:(126/166): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-02-26T06:06:16.125 INFO:teuthology.orchestra.run.smithi115.stdout:(127/166): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-02-26T06:06:16.159 INFO:teuthology.orchestra.run.smithi115.stdout:(128/166): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-02-26T06:06:16.184 INFO:teuthology.orchestra.run.smithi115.stdout:(129/166): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-02-26T06:06:16.202 INFO:teuthology.orchestra.run.smithi049.stdout:(83/166): python3-babel-2.9.1-2.el9.noarch.rpm 4.6 MB/s | 6.0 MB 00:01 2024-02-26T06:06:16.209 INFO:teuthology.orchestra.run.smithi115.stdout:(130/166): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-02-26T06:06:16.236 INFO:teuthology.orchestra.run.smithi049.stdout:(84/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-02-26T06:06:16.243 INFO:teuthology.orchestra.run.smithi115.stdout:(131/166): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-02-26T06:06:16.302 INFO:teuthology.orchestra.run.smithi115.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-02-26T06:06:16.311 INFO:teuthology.orchestra.run.smithi049.stdout:(85/166): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-02-26T06:06:16.327 INFO:teuthology.orchestra.run.smithi115.stdout:(133/166): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-02-26T06:06:16.352 INFO:teuthology.orchestra.run.smithi115.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-02-26T06:06:16.370 INFO:teuthology.orchestra.run.smithi049.stdout:(86/166): python3-packaging-20.9-5.el9.noarch.r 581 kB/s | 77 kB 00:00 2024-02-26T06:06:16.395 INFO:teuthology.orchestra.run.smithi049.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 510 kB/s | 42 kB 00:00 2024-02-26T06:06:16.402 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction test succeeded. 2024-02-26T06:06:16.403 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction 2024-02-26T06:06:16.410 INFO:teuthology.orchestra.run.smithi115.stdout:(135/166): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-02-26T06:06:16.444 INFO:teuthology.orchestra.run.smithi115.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-02-26T06:06:16.469 INFO:teuthology.orchestra.run.smithi115.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-02-26T06:06:16.496 INFO:teuthology.orchestra.run.smithi049.stdout:(88/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-02-26T06:06:16.511 INFO:teuthology.orchestra.run.smithi115.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-02-26T06:06:16.529 INFO:teuthology.orchestra.run.smithi049.stdout:(89/166): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-02-26T06:06:16.545 INFO:teuthology.orchestra.run.smithi115.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-02-26T06:06:16.570 INFO:teuthology.orchestra.run.smithi115.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-02-26T06:06:16.588 INFO:teuthology.orchestra.run.smithi049.stdout:(90/166): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-02-26T06:06:16.654 INFO:teuthology.orchestra.run.smithi115.stdout:(141/166): python3-rsa-4.9-2.el9.noarch.rpm 546 kB/s | 59 kB 00:00 2024-02-26T06:06:16.655 INFO:teuthology.orchestra.run.smithi049.stdout:(91/166): python3-requests-oauthlib-1.3.0-12.el 813 kB/s | 54 kB 00:00 2024-02-26T06:06:16.679 INFO:teuthology.orchestra.run.smithi115.stdout:(142/166): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-02-26T06:06:16.746 INFO:teuthology.orchestra.run.smithi115.stdout:(143/166): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-02-26T06:06:16.771 INFO:teuthology.orchestra.run.smithi115.stdout:(144/166): python3-routes-2.5.1-5.el9.noarch.rp 726 kB/s | 188 kB 00:00 2024-02-26T06:06:16.830 INFO:teuthology.orchestra.run.smithi115.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-02-26T06:06:16.855 INFO:teuthology.orchestra.run.smithi115.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-02-26T06:06:16.905 INFO:teuthology.orchestra.run.smithi115.stdout:(147/166): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-02-26T06:06:16.922 INFO:teuthology.orchestra.run.smithi115.stdout:(148/166): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-02-26T06:06:16.956 INFO:teuthology.orchestra.run.smithi115.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-26T06:06:16.972 INFO:teuthology.orchestra.run.smithi049.stdout:(92/166): python3-pytz-2021.1-5.el9.noarch.rpm 115 kB/s | 51 kB 00:00 2024-02-26T06:06:17.189 INFO:teuthology.orchestra.run.smithi049.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 193 kB/s | 42 kB 00:00 2024-02-26T06:06:17.189 INFO:teuthology.orchestra.run.smithi115.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-02-26T06:06:17.272 INFO:teuthology.orchestra.run.smithi049.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 536 kB/s | 44 kB 00:00 2024-02-26T06:06:17.356 INFO:teuthology.orchestra.run.smithi115.stdout:(151/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 7.8 MB/s | 3.1 MB 00:00 2024-02-26T06:06:17.414 INFO:teuthology.orchestra.run.smithi049.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 1.5 MB/s | 216 kB 00:00 2024-02-26T06:06:17.490 INFO:teuthology.orchestra.run.smithi115.stdout:(152/166): librados2-19.0.0-1514.g25296a8f.el9. 5.9 MB/s | 3.4 MB 00:00 2024-02-26T06:06:17.556 INFO:teuthology.orchestra.run.smithi049.stdout:(96/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.7 MB/s | 5.1 MB 00:01 2024-02-26T06:06:17.573 INFO:teuthology.orchestra.run.smithi049.stdout:(97/166): qatlib-service-23.11.0-1.el9.x86_64.r 221 kB/s | 35 kB 00:00 2024-02-26T06:06:17.607 INFO:teuthology.orchestra.run.smithi049.stdout:(98/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 899 kB/s | 44 kB 00:00 2024-02-26T06:06:17.665 INFO:teuthology.orchestra.run.smithi049.stdout:(99/166): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-02-26T06:06:17.699 INFO:teuthology.orchestra.run.smithi049.stdout:(100/166): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-02-26T06:06:17.716 INFO:teuthology.orchestra.run.smithi049.stdout:(101/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-02-26T06:06:17.750 INFO:teuthology.orchestra.run.smithi049.stdout:(102/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-02-26T06:06:17.766 INFO:teuthology.orchestra.run.smithi049.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 911 kB/s | 45 kB 00:00 2024-02-26T06:06:17.817 INFO:teuthology.orchestra.run.smithi049.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 338 kB/s | 22 kB 00:00 2024-02-26T06:06:18.017 INFO:teuthology.orchestra.run.smithi049.stdout:(105/166): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-02-26T06:06:18.207 INFO:teuthology.orchestra.run.smithi115.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 61 kB/s | 42 kB 00:00 2024-02-26T06:06:19.123 INFO:teuthology.orchestra.run.smithi115.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 270 kB/s | 247 kB 00:00 2024-02-26T06:06:19.192 INFO:teuthology.orchestra.run.smithi049.stdout:(106/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.4 MB/s | 16 MB 00:02 2024-02-26T06:06:19.823 INFO:teuthology.orchestra.run.smithi115.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 102 kB/s | 71 kB 00:00 2024-02-26T06:06:20.457 INFO:teuthology.orchestra.run.smithi115.stdout:(156/166): NetworkManager-libnm-1.45.91-1.el9.x 599 kB/s | 1.8 MB 00:03 2024-02-26T06:06:20.491 INFO:teuthology.orchestra.run.smithi115.stdout:(157/166): elfutils-debuginfod-client-0.190-2.e 55 kB/s | 36 kB 00:00 2024-02-26T06:06:20.858 INFO:teuthology.orchestra.run.smithi115.stdout:(158/166): NetworkManager-1.45.91-1.el9.x86_64. 634 kB/s | 2.3 MB 00:03 2024-02-26T06:06:21.308 INFO:teuthology.orchestra.run.smithi115.stdout:(159/166): elfutils-libelf-0.190-2.el9.x86_64.r 226 kB/s | 192 kB 00:00 2024-02-26T06:06:21.441 INFO:teuthology.orchestra.run.smithi115.stdout:(160/166): libacl-2.3.1-4.el9.x86_64.rpm 40 kB/s | 23 kB 00:00 2024-02-26T06:06:21.467 INFO:teuthology.orchestra.run.smithi115.stdout:(161/166): elfutils-libs-0.190-2.el9.x86_64.rpm 262 kB/s | 256 kB 00:00 2024-02-26T06:06:22.475 INFO:teuthology.orchestra.run.smithi115.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 321 kB/s | 323 kB 00:01 2024-02-26T06:06:22.920 INFO:teuthology.orchestra.run.smithi080.stdout: Preparing : 1/1 2024-02-26T06:06:23.035 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T06:06:23.073 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T06:06:23.211 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T06:06:23.378 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T06:06:23.403 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T06:06:23.447 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T06:06:23.642 INFO:teuthology.orchestra.run.smithi115.stdout:(163/166): openssl-3.0.7-25.el9.x86_64.rpm 532 kB/s | 1.2 MB 00:02 2024-02-26T06:06:23.725 INFO:teuthology.orchestra.run.smithi115.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 898 kB/s | 74 kB 00:00 2024-02-26T06:06:23.781 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T06:06:23.854 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:06:23.958 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:06:23.988 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:06:24.079 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:06:24.109 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:06:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T06:06:24.276 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T06:06:24.448 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T06:06:24.491 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T06:06:24.520 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:06:24.584 INFO:teuthology.orchestra.run.smithi115.stdout:(165/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 521 kB/s | 1.1 MB 00:02 2024-02-26T06:06:24.623 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:06:24.646 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:06:24.708 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:06:24.730 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:06:24.745 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:06:24.937 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:06:24.967 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:06:24.992 INFO:teuthology.orchestra.run.smithi115.stdout:(166/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 621 kB/s | 2.2 MB 00:03 2024-02-26T06:06:24.995 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-02-26T06:06:24.995 INFO:teuthology.orchestra.run.smithi115.stdout:Total 7.8 MB/s | 221 MB 00:28 2024-02-26T06:06:25.071 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T06:06:25.178 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:06:25.238 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:06:25.313 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:06:25.373 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:06:25.582 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:06:25.691 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T06:06:25.791 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T06:06:25.819 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T06:06:25.854 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T06:06:25.918 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T06:06:25.953 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T06:06:25.996 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T06:06:26.029 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T06:06:26.057 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T06:06:26.126 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T06:06:26.179 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T06:06:26.200 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-02-26T06:06:26.209 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T06:06:26.245 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T06:06:26.321 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T06:06:26.356 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T06:06:26.399 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T06:06:26.428 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T06:06:26.457 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-02-26T06:06:26.457 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-02-26T06:06:26.496 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T06:06:26.567 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T06:06:26.639 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T06:06:26.771 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T06:06:26.827 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T06:06:27.340 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T06:06:27.422 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T06:06:27.451 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T06:06:27.480 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T06:06:27.529 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T06:06:27.575 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T06:06:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T06:06:27.749 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T06:06:27.795 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T06:06:27.849 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T06:06:27.921 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T06:06:27.954 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T06:06:27.990 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T06:06:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T06:06:28.074 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T06:06:28.115 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T06:06:28.157 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T06:06:28.281 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-02-26T06:06:28.281 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-02-26T06:06:28.289 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T06:06:28.330 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T06:06:28.360 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T06:06:28.403 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T06:06:28.494 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T06:06:28.535 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T06:06:28.575 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T06:06:28.601 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T06:06:28.626 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T06:06:28.696 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T06:06:28.824 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T06:06:28.858 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T06:06:28.886 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T06:06:28.920 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T06:06:28.959 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T06:06:28.991 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T06:06:29.021 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T06:06:29.055 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T06:06:29.089 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T06:06:29.130 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T06:06:29.156 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T06:06:29.241 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T06:06:29.286 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T06:06:29.318 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T06:06:29.356 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T06:06:29.452 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T06:06:29.502 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T06:06:30.220 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T06:06:30.266 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T06:06:30.350 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T06:06:30.446 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T06:06:30.499 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T06:06:30.576 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T06:06:30.610 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T06:06:30.786 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T06:06:30.816 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T06:06:30.892 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T06:06:31.512 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T06:06:31.620 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T06:06:32.815 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T06:06:32.912 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T06:06:32.939 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T06:06:32.973 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T06:06:33.277 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T06:06:33.302 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:06:33.393 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:06:33.423 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T06:06:33.459 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T06:06:33.913 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T06:06:33.947 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:06:34.009 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:06:34.032 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T06:06:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-02-26T06:06:35.020 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T06:06:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T06:06:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T06:06:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T06:06:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T06:06:35.288 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T06:06:35.606 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T06:06:35.663 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:06:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:06:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:06:35.905 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:06:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:06:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T06:06:36.094 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T06:06:36.189 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:06:36.270 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T06:06:36.318 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T06:06:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:06:36.362 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:06:36.474 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:06:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:06:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:06:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:06:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:06:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:06:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:06:36.902 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:06:36.914 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T06:06:36.940 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:06:36.987 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:06:37.005 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:06:37.005 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T06:06:37.005 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T06:06:37.005 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:37.033 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:06:37.050 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:06:37.117 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:06:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:06:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:06:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T06:06:37.601 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T06:06:37.647 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T06:06:37.682 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T06:06:37.745 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T06:06:37.789 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T06:06:37.832 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T06:06:37.865 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T06:06:37.901 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T06:06:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T06:06:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T06:06:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T06:06:38.090 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T06:06:38.167 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T06:06:38.201 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T06:06:38.245 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T06:06:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T06:06:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T06:06:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T06:06:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T06:06:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T06:06:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T06:06:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T06:06:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T06:06:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T06:06:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T06:06:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T06:06:39.396 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T06:06:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T06:06:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T06:06:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T06:06:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T06:06:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T06:06:39.774 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T06:06:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T06:06:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T06:06:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T06:06:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T06:06:39.979 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T06:06:40.110 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T06:06:40.168 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T06:06:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T06:06:40.398 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T06:06:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T06:06:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T06:06:40.763 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T06:06:40.789 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T06:06:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T06:06:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T06:06:41.003 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T06:06:41.045 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T06:06:41.073 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T06:06:41.108 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T06:06:41.146 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T06:06:41.178 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T06:06:41.208 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T06:06:41.243 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T06:06:41.276 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T06:06:41.317 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T06:06:41.354 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T06:06:41.440 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T06:06:41.484 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T06:06:41.524 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T06:06:41.571 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T06:06:41.667 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T06:06:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T06:06:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T06:06:42.481 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T06:06:42.514 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T06:06:42.612 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T06:06:42.664 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T06:06:42.750 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T06:06:42.783 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T06:06:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T06:06:42.990 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T06:06:43.066 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T06:06:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T06:06:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T06:06:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T06:06:45.094 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T06:06:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T06:06:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T06:06:45.461 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T06:06:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:06:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:06:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T06:06:45.634 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T06:06:46.087 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T06:06:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:06:46.192 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:06:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T06:06:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:06:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:06:48.770 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:06:48.771 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /sys 2024-02-26T06:06:48.771 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /proc 2024-02-26T06:06:48.771 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /mnt 2024-02-26T06:06:48.771 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /var/tmp 2024-02-26T06:06:48.771 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /home 2024-02-26T06:06:48.771 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /root 2024-02-26T06:06:48.771 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /tmp 2024-02-26T06:06:48.771 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:06:48.934 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:06:48.949 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:06:48.949 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:06:48.949 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T06:06:48.949 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:48.964 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:06:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:06:49.046 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T06:06:49.046 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T06:06:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:06:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:06:50.413 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:06:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:06:50.438 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:06:50.438 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T06:06:50.438 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:06:50.438 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:06:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:50.638 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:06:50.660 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:06:50.661 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:06:50.661 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T06:06:50.661 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:06:50.661 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:06:50.661 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:51.106 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:06:51.128 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:06:51.128 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:06:51.128 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T06:06:51.128 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:06:51.128 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:06:51.128 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:51.244 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T06:06:51.292 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T06:06:51.348 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T06:06:51.530 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T06:06:51.581 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T06:06:51.660 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T06:06:51.756 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T06:06:51.894 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T06:06:51.931 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T06:06:51.964 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T06:06:51.997 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T06:06:52.031 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T06:06:52.059 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T06:06:52.484 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:06:52.517 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:06:52.549 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T06:06:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T06:06:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:06:52.863 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:06:52.899 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:06:53.860 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:06:53.892 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:06:53.992 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:06:54.153 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T06:06:54.177 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:06:54.199 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:06:54.199 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:06:54.200 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T06:06:54.200 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:06:54.200 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:06:54.200 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:54.244 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:06:54.255 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:06:54.352 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T06:06:58.714 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T06:06:58.779 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T06:06:59.632 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T06:06:59.658 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:06:59.681 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:06:59.681 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:06:59.681 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T06:06:59.681 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:06:59.681 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:06:59.681 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:59.725 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:06:59.746 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:06:59.747 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:06:59.747 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T06:06:59.747 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:06:59.793 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:06:59.818 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:06:59.819 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:06:59.819 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T06:06:59.819 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:06:59.819 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:06:59.819 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:07:00.127 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:07:00.149 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:07:00.149 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:00.149 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T06:07:00.149 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:07:00.149 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:07:00.150 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:07:00.202 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T06:07:00.242 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T06:07:00.275 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T06:07:00.302 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T06:07:00.485 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T06:07:00.549 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T06:07:00.581 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T06:07:00.613 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T06:07:00.643 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T06:07:00.669 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:07:00.669 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-02-26T06:07:00.669 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-02-26T06:07:00.669 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-02-26T06:07:00.669 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-02-26T06:07:00.669 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-02-26T06:07:00.669 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-02-26T06:07:00.670 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-02-26T06:07:00.670 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:00.697 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:07:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T06:07:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T06:07:00.844 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:07:00.859 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:07:00.859 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:00.859 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T06:07:00.859 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:07:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:07:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:07:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T06:07:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T06:07:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:07:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:07:02.339 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:02.339 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T06:07:02.339 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:07:02.340 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:07:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T06:07:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T06:07:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T06:07:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:07:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T06:07:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:07:02.570 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:02.570 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T06:07:02.570 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:07:02.571 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:07:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:02.589 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T06:07:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T06:07:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T06:07:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:07:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:07:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T06:07:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T06:07:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T06:07:02.842 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T06:07:02.842 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:07:02.926 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:07:02.955 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T06:07:02.958 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:07:02.959 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:07:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:07:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:07:03.037 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:03.037 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T06:07:03.037 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:07:03.037 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:07:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:03.137 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:07:03.138 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:07:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T06:07:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T06:07:03.221 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:07:03.246 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T06:07:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T06:07:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T06:07:03.299 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T06:07:03.361 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:07:03.457 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T06:07:03.507 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T06:07:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T06:07:03.674 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T06:07:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T06:07:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T06:07:03.890 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T06:07:03.922 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T06:07:03.957 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T06:07:03.985 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T06:07:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:07:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:07:04.473 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T06:07:04.533 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T06:07:04.556 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:07:04.786 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:07:04.815 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:07:05.776 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:07:05.832 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:07:05.932 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:07:06.103 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T06:07:06.135 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:07:06.158 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:07:06.158 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:06.158 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T06:07:06.158 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:07:06.158 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:07:06.158 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:06.202 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:07:06.213 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:07:06.309 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T06:07:08.554 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:07:08.554 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T06:07:08.554 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T06:07:08.555 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T06:07:08.556 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T06:07:08.557 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T06:07:08.558 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T06:07:08.559 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T06:07:08.560 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T06:07:08.561 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T06:07:08.562 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T06:07:08.563 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T06:07:08.564 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T06:07:08.565 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T06:07:08.566 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T06:07:08.567 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T06:07:08.568 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T06:07:08.569 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T06:07:08.570 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T06:07:08.570 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T06:07:08.570 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T06:07:08.570 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T06:07:08.570 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T06:07:10.674 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T06:07:10.737 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T06:07:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T06:07:11.625 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:07:11.647 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:07:11.648 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:11.648 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T06:07:11.648 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:07:11.648 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:07:11.648 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:07:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:07:11.714 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:11.714 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T06:07:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:11.722 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T06:07:11.722 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:07:11.722 INFO:teuthology.orchestra.run.smithi080.stdout:Upgraded: 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.723 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout:Installed: 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.724 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.725 INFO:teuthology.orchestra.run.smithi080.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: jq-1.6-16.el9.x86_64 2024-02-26T06:07:11.726 INFO:teuthology.orchestra.run.smithi080.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.727 INFO:teuthology.orchestra.run.smithi080.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T06:07:11.728 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T06:07:11.729 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T06:07:11.730 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T06:07:11.731 INFO:teuthology.orchestra.run.smithi080.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.732 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T06:07:11.733 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.734 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.734 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:11.734 INFO:teuthology.orchestra.run.smithi080.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T06:07:11.734 INFO:teuthology.orchestra.run.smithi080.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T06:07:11.734 INFO:teuthology.orchestra.run.smithi080.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T06:07:11.734 INFO:teuthology.orchestra.run.smithi080.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T06:07:11.734 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T06:07:11.734 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T06:07:11.735 INFO:teuthology.orchestra.run.smithi080.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T06:07:11.735 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-02-26T06:07:11.735 INFO:teuthology.orchestra.run.smithi080.stdout:Complete! 2024-02-26T06:07:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:07:11.782 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:07:11.782 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:11.782 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T06:07:11.782 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:07:11.782 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:07:11.782 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:07:12.107 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:07:12.107 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:07:12.107 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T06:07:12.107 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:07:12.107 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:07:12.107 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T06:07:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T06:07:12.211 DEBUG:teuthology.parallel:result is None 2024-02-26T06:07:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T06:07:12.260 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T06:07:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T06:07:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T06:07:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T06:07:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T06:07:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T06:07:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:07:12.690 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T06:07:12.716 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T06:07:13.328 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:07:13.364 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:07:13.390 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:07:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T06:07:14.278 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T06:07:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T06:07:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T06:07:14.480 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T06:07:14.505 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T06:07:14.531 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T06:07:14.556 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T06:07:14.616 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T06:07:14.616 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:07:14.706 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:07:14.735 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T06:07:14.761 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T06:07:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T06:07:14.811 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T06:07:14.812 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:07:14.896 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:07:14.921 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T06:07:14.927 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:07:14.927 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:07:15.107 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:07:15.107 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:07:15.190 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:07:15.215 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T06:07:15.233 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T06:07:15.266 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T06:07:15.330 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:07:20.380 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:07:20.380 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T06:07:20.380 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T06:07:20.380 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T06:07:20.380 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T06:07:20.380 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T06:07:20.380 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T06:07:20.381 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T06:07:20.381 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:07:20.381 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:07:20.381 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:07:20.381 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T06:07:20.381 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T06:07:20.382 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T06:07:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T06:07:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T06:07:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T06:07:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T06:07:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T06:07:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T06:07:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T06:07:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T06:07:20.384 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T06:07:20.385 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T06:07:20.386 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T06:07:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T06:07:20.388 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T06:07:20.389 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T06:07:20.390 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T06:07:20.391 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T06:07:20.392 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T06:07:20.393 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T06:07:20.394 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T06:07:20.395 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T06:07:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T06:07:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:23.057 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-02-26T06:07:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T06:07:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T06:07:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: jq-1.6-16.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T06:07:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T06:07:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T06:07:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T06:07:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T06:07:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T06:07:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T06:07:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:07:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T06:07:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T06:07:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T06:07:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T06:07:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T06:07:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T06:07:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T06:07:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:07:23.071 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-02-26T06:07:23.550 DEBUG:teuthology.parallel:result is None 2024-02-26T06:08:32.014 INFO:teuthology.orchestra.run.smithi049.stdout:[MIRROR] abseil-cpp-20211102.0-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/a/abseil-cpp-20211102.0-3.el9.x86_64.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-02-26T06:08:32.014 INFO:teuthology.orchestra.run.smithi049.stdout:[MIRROR] gperftools-libs-2.9.1-2.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/g/gperftools-libs-2.9.1-2.el9.x86_64.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-02-26T06:08:32.014 INFO:teuthology.orchestra.run.smithi049.stdout:[MIRROR] grpc-data-1.46.7-10.el9.noarch.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/g/grpc-data-1.46.7-10.el9.noarch.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-02-26T06:08:32.618 INFO:teuthology.orchestra.run.smithi049.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 149 B/s | 19 kB 02:13 2024-02-26T06:08:32.776 INFO:teuthology.orchestra.run.smithi049.stdout:(108/166): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 kB/s | 309 kB 02:14 2024-02-26T06:08:32.810 INFO:teuthology.orchestra.run.smithi049.stdout:(109/166): abseil-cpp-20211102.0-3.el9.x86_64.r 4.1 kB/s | 548 kB 02:14 2024-02-26T06:08:33.093 INFO:teuthology.orchestra.run.smithi049.stdout:(110/166): libarrow-9.0.0-10.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-02-26T06:08:33.110 INFO:teuthology.orchestra.run.smithi049.stdout:(111/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 79 kB/s | 26 kB 00:00 2024-02-26T06:08:33.127 INFO:teuthology.orchestra.run.smithi049.stdout:(112/166): liboath-2.6.7-2.el9.x86_64.rpm 154 kB/s | 49 kB 00:00 2024-02-26T06:08:33.219 INFO:teuthology.orchestra.run.smithi049.stdout:(113/166): libunwind-1.6.2-1.el9.x86_64.rpm 748 kB/s | 67 kB 00:00 2024-02-26T06:08:33.260 INFO:teuthology.orchestra.run.smithi049.stdout:(114/166): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-02-26T06:08:33.345 INFO:teuthology.orchestra.run.smithi049.stdout:(115/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-02-26T06:08:33.394 INFO:teuthology.orchestra.run.smithi049.stdout:(116/166): python3-asyncssh-2.13.2-4.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-02-26T06:08:33.436 INFO:teuthology.orchestra.run.smithi049.stdout:(117/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 250 kB/s | 43 kB 00:00 2024-02-26T06:08:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:(118/166): python3-cachetools-4.2.4-1.el9.noarc 192 kB/s | 32 kB 00:00 2024-02-26T06:08:33.562 INFO:teuthology.orchestra.run.smithi049.stdout:(119/166): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-02-26T06:08:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-02-26T06:08:33.629 INFO:teuthology.orchestra.run.smithi049.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-02-26T06:08:33.680 INFO:teuthology.orchestra.run.smithi049.stdout:(122/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.6 MB/s | 303 kB 00:00 2024-02-26T06:08:33.738 INFO:teuthology.orchestra.run.smithi049.stdout:(123/166): python3-colorama-0.4.4-12.el9.noarch 251 kB/s | 35 kB 00:00 2024-02-26T06:08:33.772 INFO:teuthology.orchestra.run.smithi049.stdout:(124/166): python3-google-auth-2.28.0-1.el9.noa 1.4 MB/s | 203 kB 00:00 2024-02-26T06:08:33.889 INFO:teuthology.orchestra.run.smithi049.stdout:(125/166): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-02-26T06:08:33.916 INFO:teuthology.orchestra.run.smithi049.stdout:(126/166): python3-grpcio-tools-1.46.7-10.el9.x 836 kB/s | 144 kB 00:00 2024-02-26T06:08:33.956 INFO:teuthology.orchestra.run.smithi049.stdout:(127/166): python3-jaraco-8.2.1-3.el9.noarch.rp 61 kB/s | 11 kB 00:00 2024-02-26T06:08:33.998 INFO:teuthology.orchestra.run.smithi049.stdout:(128/166): python3-jaraco-classes-3.2.1-5.el9.n 166 kB/s | 18 kB 00:00 2024-02-26T06:08:34.056 INFO:teuthology.orchestra.run.smithi049.stdout:(129/166): python3-jaraco-collections-3.0.0-8.e 234 kB/s | 23 kB 00:00 2024-02-26T06:08:34.131 INFO:teuthology.orchestra.run.smithi049.stdout:(130/166): python3-jaraco-functools-3.5.0-2.el9 112 kB/s | 19 kB 00:00 2024-02-26T06:08:34.173 INFO:teuthology.orchestra.run.smithi049.stdout:(131/166): python3-jaraco-text-3.2.0-6.el9.noar 112 kB/s | 20 kB 00:00 2024-02-26T06:08:34.282 INFO:teuthology.orchestra.run.smithi049.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 6.9 MB/s | 1.0 MB 00:00 2024-02-26T06:08:34.324 INFO:teuthology.orchestra.run.smithi049.stdout:(133/166): python3-logutils-0.3.5-21.el9.noarch 243 kB/s | 46 kB 00:00 2024-02-26T06:08:34.349 INFO:teuthology.orchestra.run.smithi049.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 451 kB/s | 79 kB 00:00 2024-02-26T06:08:34.391 INFO:teuthology.orchestra.run.smithi049.stdout:(135/166): python3-natsort-7.1.1-5.el9.noarch.r 538 kB/s | 58 kB 00:00 2024-02-26T06:08:34.491 INFO:teuthology.orchestra.run.smithi049.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-02-26T06:08:34.533 INFO:teuthology.orchestra.run.smithi049.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 90 kB/s | 16 kB 00:00 2024-02-26T06:08:34.558 INFO:teuthology.orchestra.run.smithi049.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 540 kB/s | 90 kB 00:00 2024-02-26T06:08:34.609 INFO:teuthology.orchestra.run.smithi049.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 245 kB/s | 28 kB 00:00 2024-02-26T06:08:34.667 INFO:teuthology.orchestra.run.smithi049.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 286 kB/s | 31 kB 00:00 2024-02-26T06:08:34.701 INFO:teuthology.orchestra.run.smithi049.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-02-26T06:08:34.743 INFO:teuthology.orchestra.run.smithi049.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-02-26T06:08:34.785 INFO:teuthology.orchestra.run.smithi049.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 312 kB/s | 36 kB 00:00 2024-02-26T06:08:34.852 INFO:teuthology.orchestra.run.smithi049.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 342 kB/s | 51 kB 00:00 2024-02-26T06:08:34.885 INFO:teuthology.orchestra.run.smithi049.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-02-26T06:08:34.919 INFO:teuthology.orchestra.run.smithi049.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 676 kB/s | 90 kB 00:00 2024-02-26T06:08:34.986 INFO:teuthology.orchestra.run.smithi049.stdout:(147/166): python3-werkzeug-2.0.3-3.el9.1.noarc 3.2 MB/s | 427 kB 00:00 2024-02-26T06:08:35.028 INFO:teuthology.orchestra.run.smithi049.stdout:(148/166): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-02-26T06:08:35.061 INFO:teuthology.orchestra.run.smithi049.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-02-26T06:08:35.254 INFO:teuthology.orchestra.run.smithi049.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2024-02-26T06:08:35.462 INFO:teuthology.orchestra.run.smithi049.stdout:(151/166): NetworkManager-1.45.91-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-02-26T06:08:35.629 INFO:teuthology.orchestra.run.smithi049.stdout:(152/166): NetworkManager-libnm-1.45.91-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-02-26T06:08:35.696 INFO:teuthology.orchestra.run.smithi049.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 643 kB/s | 42 kB 00:00 2024-02-26T06:08:35.788 INFO:teuthology.orchestra.run.smithi049.stdout:(154/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 4.3 MB/s | 3.1 MB 00:00 2024-02-26T06:08:35.863 INFO:teuthology.orchestra.run.smithi049.stdout:(155/166): librados2-19.0.0-1514.g25296a8f.el9. 4.1 MB/s | 3.4 MB 00:00 2024-02-26T06:08:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:(156/166): NetworkManager-tui-1.45.91-1.el9.x86 1.2 MB/s | 247 kB 00:00 2024-02-26T06:08:35.931 INFO:teuthology.orchestra.run.smithi049.stdout:(157/166): acl-2.3.1-4.el9.x86_64.rpm 506 kB/s | 71 kB 00:00 2024-02-26T06:08:35.998 INFO:teuthology.orchestra.run.smithi049.stdout:(158/166): elfutils-debuginfod-client-0.190-2.e 273 kB/s | 36 kB 00:00 2024-02-26T06:08:36.057 INFO:teuthology.orchestra.run.smithi049.stdout:(159/166): elfutils-libelf-0.190-2.el9.x86_64.r 1.2 MB/s | 192 kB 00:00 2024-02-26T06:08:36.082 INFO:teuthology.orchestra.run.smithi049.stdout:(160/166): libacl-2.3.1-4.el9.x86_64.rpm 279 kB/s | 23 kB 00:00 2024-02-26T06:08:36.123 INFO:teuthology.orchestra.run.smithi049.stdout:(161/166): elfutils-libs-0.190-2.el9.x86_64.rpm 1.3 MB/s | 256 kB 00:00 2024-02-26T06:08:36.232 INFO:teuthology.orchestra.run.smithi049.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2024-02-26T06:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout:(163/166): openssl-3.0.7-25.el9.x86_64.rpm 5.2 MB/s | 1.2 MB 00:00 2024-02-26T06:08:36.374 INFO:teuthology.orchestra.run.smithi049.stdout:(164/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 7.4 MB/s | 2.2 MB 00:00 2024-02-26T06:08:36.491 INFO:teuthology.orchestra.run.smithi049.stdout:(165/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 373 kB/s | 74 kB 00:00 2024-02-26T06:08:36.550 INFO:teuthology.orchestra.run.smithi049.stdout:(166/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.4 MB/s | 1.1 MB 00:00 2024-02-26T06:08:36.562 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-02-26T06:08:36.563 INFO:teuthology.orchestra.run.smithi049.stdout:Total 1.4 MB/s | 221 MB 02:33 2024-02-26T06:08:40.189 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-02-26T06:08:40.580 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-02-26T06:08:40.580 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-02-26T06:08:43.283 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-02-26T06:08:43.283 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-02-26T06:08:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-02-26T06:08:50.838 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T06:08:50.923 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T06:08:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T06:08:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T06:08:51.132 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T06:08:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T06:08:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T06:08:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:08:51.933 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:08:51.964 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:08:52.138 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:08:52.195 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:08:52.344 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T06:08:52.445 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T06:08:52.730 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T06:08:52.787 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T06:08:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:08:53.036 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:08:53.064 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:08:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:08:53.240 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:08:53.583 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:08:53.625 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:08:53.950 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T06:08:54.164 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:08:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:08:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:08:54.491 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:08:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:08:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T06:08:55.153 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T06:08:55.189 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T06:08:55.236 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T06:08:55.328 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T06:08:55.375 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T06:08:55.437 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T06:08:55.480 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T06:08:55.512 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T06:08:55.628 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T06:08:55.728 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T06:08:55.772 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T06:08:55.836 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T06:08:55.955 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T06:08:55.993 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T06:08:56.043 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T06:08:56.091 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T06:08:56.218 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T06:08:56.278 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T06:08:56.381 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T06:08:56.596 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T06:08:56.658 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T06:08:57.384 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T06:08:57.973 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T06:08:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T06:08:58.116 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T06:08:58.205 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T06:08:58.270 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T06:08:58.362 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T06:08:58.618 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T06:08:58.694 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T06:08:58.773 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T06:08:58.894 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T06:08:58.936 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T06:08:59.001 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T06:08:59.075 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T06:08:59.117 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T06:08:59.186 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T06:08:59.258 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T06:08:59.462 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T06:08:59.513 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T06:08:59.585 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T06:08:59.641 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T06:08:59.784 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T06:08:59.825 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T06:08:59.875 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T06:08:59.941 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T06:08:59.974 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T06:09:00.177 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T06:09:00.357 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T06:09:00.404 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T06:09:00.435 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T06:09:00.491 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T06:09:00.552 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T06:09:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T06:09:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T06:09:00.711 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T06:09:00.754 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T06:09:00.830 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T06:09:00.878 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T06:09:01.025 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T06:09:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T06:09:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T06:09:01.216 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T06:09:01.377 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T06:09:01.466 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T06:09:02.494 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T06:09:02.572 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T06:09:02.614 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T06:09:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T06:09:02.818 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T06:09:02.931 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T06:09:02.974 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T06:09:03.196 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T06:09:03.220 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T06:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T06:09:04.170 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T06:09:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T06:09:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T06:09:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T06:09:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T06:09:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T06:09:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T06:09:06.563 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:09:06.686 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:09:06.721 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T06:09:06.777 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T06:09:07.336 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T06:09:07.366 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:09:07.501 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:09:07.528 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T06:09:10.209 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:09:10.248 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:09:10.929 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:09:10.977 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:09:11.178 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:09:11.178 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T06:09:11.178 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T06:09:11.178 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:11.261 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-02-26T06:09:24.776 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:25.005 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:09:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:09:25.057 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:25.057 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T06:09:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:26.776 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:09:26.889 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:09:26.889 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:26.890 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T06:09:26.890 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:09:26.890 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:09:26.890 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:27.134 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:09:27.229 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:09:27.229 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:27.229 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T06:09:27.229 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:09:27.229 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:09:27.229 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:27.768 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:09:27.842 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:09:27.842 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:27.842 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T06:09:27.842 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:09:27.843 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:09:27.843 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:27.986 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T06:09:28.050 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T06:09:28.127 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T06:09:28.390 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T06:09:28.457 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T06:09:28.548 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T06:09:28.678 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T06:09:28.860 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T06:09:28.938 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T06:09:28.982 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T06:09:29.029 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T06:09:29.081 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T06:09:29.113 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T06:09:29.852 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:09:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:09:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T06:09:30.062 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T06:09:30.101 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:09:30.451 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:09:30.485 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:09:31.835 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:09:32.086 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T06:09:32.129 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T06:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:32.304 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:09:32.345 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:09:32.509 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T06:09:37.585 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T06:09:37.669 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T06:09:38.689 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T06:09:38.729 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:09:38.804 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:09:38.804 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:38.804 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T06:09:38.804 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:09:38.804 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:09:38.805 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:38.845 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:09:38.949 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:09:38.949 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:38.949 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T06:09:38.950 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:39.021 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:09:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:09:39.131 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:39.131 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T06:09:39.131 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:09:39.131 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:09:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:09:39.644 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:09:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:09:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T06:09:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:09:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:09:39.645 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T06:09:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T06:09:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T06:09:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T06:09:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T06:09:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T06:09:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T06:09:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T06:09:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T06:09:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:09:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T06:09:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T06:09:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:09:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:09:41.794 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T06:09:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T06:09:43.423 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T06:09:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T06:09:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T06:09:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T06:09:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T06:09:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T06:09:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T06:09:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:09:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:09:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T06:09:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T06:09:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T06:09:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T06:09:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:09:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:09:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T06:09:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:09:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:09:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:09:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:09:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:09:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T06:09:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T06:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T06:09:45.011 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:09:45.780 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:09:45.781 INFO:teuthology.orchestra.run.smithi049.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-26T06:09:45.781 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T06:09:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T06:09:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T06:09:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T06:09:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T06:09:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T06:09:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T06:09:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T06:09:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T06:09:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T06:09:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T06:09:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T06:09:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T06:09:53.199 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T06:09:53.200 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.201 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T06:09:53.203 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-16.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T06:09:53.204 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T06:09:53.205 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T06:09:53.206 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T06:09:53.207 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T06:09:53.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T06:09:53.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T06:09:53.211 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T06:09:53.212 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T06:09:53.212 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T06:09:53.212 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:09:53.212 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-02-26T06:09:53.842 DEBUG:teuthology.parallel:result is None 2024-02-26T06:09:53.842 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:09:53.984 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T06:09:54.041 INFO:teuthology.orchestra.run.smithi049.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T06:09:54.041 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T06:09:54.042 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T06:09:54.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:09:54.183 DEBUG:teuthology.orchestra.run.smithi080:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T06:09:54.209 INFO:teuthology.orchestra.run.smithi080.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T06:09:54.209 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T06:09:54.210 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T06:09:54.211 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:09:54.345 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T06:09:54.371 INFO:teuthology.orchestra.run.smithi115.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T06:09:54.371 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T06:09:54.371 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T06:09:54.373 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-26T06:09:54.373 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:09:54.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T06:09:54.462 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:09:54.462 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T06:09:54.496 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:09:54.496 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T06:09:54.530 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-26T06:09:54.530 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:09:54.530 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T06:09:54.643 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T06:09:54.745 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:09:54.746 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T06:09:54.778 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T06:09:54.846 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:09:54.847 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T06:09:54.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T06:09:54.947 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-26T06:09:54.947 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:09:54.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T06:09:55.030 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T06:09:55.155 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:09:55.155 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T06:09:55.184 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T06:09:55.251 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:09:55.251 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T06:09:55.279 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T06:09:55.346 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-26T06:09:55.347 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:09:55.347 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T06:09:55.429 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T06:09:55.543 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:09:55.544 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T06:09:55.571 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T06:09:55.637 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:09:55.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T06:09:55.664 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T06:09:55.731 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-26T06:09:55.836 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'subvols': {'create': 2}} 2024-02-26T06:09:55.836 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:09:55.837 INFO:tasks.cephadm:Cluster fsid is a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:09:55.837 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-26T06:09:55.837 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.49', 'mon.b': '172.21.15.80', 'mon.c': '172.21.15.115'} 2024-02-26T06:09:55.837 INFO:tasks.cephadm:First mon is mon.a on smithi049 2024-02-26T06:09:55.837 INFO:tasks.cephadm:First mgr is x 2024-02-26T06:09:55.838 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-26T06:09:55.838 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2024-02-26T06:09:55.914 DEBUG:teuthology.orchestra.run.smithi080:> sudo hostname $(hostname -s) 2024-02-26T06:09:55.941 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-02-26T06:09:55.967 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-26T06:09:55.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:09:56.106 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76813/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1514-g25296a8f', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1514.g25296a8f'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-16 07:18:11.982893', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240216.060239', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-26T06:09:56.329 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-vshankar-testing-20240216.060239, sha1 25296a8f176256aea5ed8b5850b25167dce0ae33 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:09:56.330 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm 2024-02-26T06:09:56.331 DEBUG:teuthology.orchestra.run.smithi049:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:57.303 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 06:09 /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:57.304 DEBUG:teuthology.orchestra.run.smithi080:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:58.247 INFO:teuthology.orchestra.run.smithi080.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 06:09 /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:58.248 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:58.773 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 06:09 /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:58.773 DEBUG:teuthology.orchestra.run.smithi049:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:58.837 DEBUG:teuthology.orchestra.run.smithi080:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:58.855 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T06:09:58.916 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 on all hosts... 2024-02-26T06:09:58.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T06:09:58.923 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T06:09:58.926 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T06:09:59.674 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:09:59.680 INFO:teuthology.orchestra.run.smithi080.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:10:01.153 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:10:51.480 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-02-26T06:10:51.480 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T06:10:51.480 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T06:10:51.480 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-02-26T06:10:51.480 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T06:10:51.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-02-26T06:10:51.480 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-02-26T06:10:51.559 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:10:51.560 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T06:10:51.560 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T06:10:51.560 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2024-02-26T06:10:51.560 INFO:teuthology.orchestra.run.smithi049.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T06:10:51.560 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:10:51.560 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:10:56.457 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-02-26T06:10:56.458 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T06:10:56.458 INFO:teuthology.orchestra.run.smithi080.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T06:10:56.458 INFO:teuthology.orchestra.run.smithi080.stdout: "repo_digests": [ 2024-02-26T06:10:56.458 INFO:teuthology.orchestra.run.smithi080.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T06:10:56.458 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-02-26T06:10:56.458 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-02-26T06:10:56.483 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2024-02-26T06:10:56.601 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph 2024-02-26T06:10:56.631 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-02-26T06:10:56.666 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2024-02-26T06:10:56.779 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 777 /etc/ceph 2024-02-26T06:10:56.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-02-26T06:10:56.836 INFO:tasks.cephadm:Writing seed config... 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-26T06:10:56.838 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-26T06:10:56.839 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-26T06:10:56.840 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-26T06:10:56.841 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-26T06:10:56.841 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-26T06:10:56.841 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-26T06:10:56.841 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-02-26T06:10:56.841 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:10:56.842 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-26T06:10:56.901 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a9ae7f00-d46d-11ee-95c3-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-26T06:10:56.902 DEBUG:teuthology.orchestra.run.smithi049:mon.a> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a.service 2024-02-26T06:10:56.949 DEBUG:teuthology.orchestra.run.smithi049:mgr.x> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.x.service 2024-02-26T06:10:56.994 INFO:tasks.cephadm:Bootstrapping... 2024-02-26T06:10:56.994 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 -v bootstrap --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.49 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:10:57.290 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-02-26T06:10:57.290 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33', '-v', 'bootstrap', '--fsid', 'a9ae7f00-d46d-11ee-95c3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.49', '--skip-admin-label'] 2024-02-26T06:10:57.356 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T06:10:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-26T06:10:57.357 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying podman|docker is present... 2024-02-26T06:10:57.414 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T06:10:57.415 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying lvm2 is present... 2024-02-26T06:10:57.415 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying time synchronization is in place... 2024-02-26T06:10:57.433 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-26T06:10:57.433 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-26T06:10:57.449 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-26T06:10:57.450 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-02-26T06:10:57.467 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-02-26T06:10:57.484 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-02-26T06:10:57.484 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-02-26T06:10:57.484 INFO:teuthology.orchestra.run.smithi049.stdout:Repeating the final host check... 2024-02-26T06:10:57.542 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T06:10:57.542 INFO:teuthology.orchestra.run.smithi049.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-26T06:10:57.542 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl is present 2024-02-26T06:10:57.542 INFO:teuthology.orchestra.run.smithi049.stdout:lvcreate is present 2024-02-26T06:10:57.558 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-26T06:10:57.558 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-26T06:10:57.576 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-26T06:10:57.577 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-02-26T06:10:57.615 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-02-26T06:10:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-02-26T06:10:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:Host looks OK 2024-02-26T06:10:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:Cluster fsid: a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:10:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:Acquiring lock 140517226012624 on /run/cephadm/a9ae7f00-d46d-11ee-95c3-87774f69a715.lock 2024-02-26T06:10:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:Lock 140517226012624 acquired on /run/cephadm/a9ae7f00-d46d-11ee-95c3-87774f69a715.lock 2024-02-26T06:10:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 3300 ... 2024-02-26T06:10:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 6789 ... 2024-02-26T06:10:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:Base mon IP(s) is [172.21.15.49:3300, 172.21.15.49:6789], mon addrv is [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-02-26T06:10:57.626 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.49 metric 100 2024-02-26T06:10:57.626 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.49 metric 100 2024-02-26T06:10:57.634 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-26T06:10:57.634 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-26T06:10:57.634 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-26T06:10:57.642 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-26T06:10:57.642 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-26T06:10:57.642 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-26T06:10:57.642 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-26T06:10:57.642 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-02-26T06:10:57.642 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-26T06:10:57.643 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-02-26T06:10:57.643 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-02-26T06:10:57.643 INFO:teuthology.orchestra.run.smithi049.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-26T06:10:57.644 INFO:teuthology.orchestra.run.smithi049.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-26T06:10:57.645 INFO:teuthology.orchestra.run.smithi049.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:10:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 2024-02-26T06:10:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:10:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Getting image source signatures 2024-02-26T06:10:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-26T06:10:58.333 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:3023f2ea73c33e75bbdc7c6f4507678b80f8a5636a9a11c7f009befea7a064c0 2024-02-26T06:10:58.333 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying config sha256:a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 2024-02-26T06:10:58.333 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-26T06:10:59.596 INFO:teuthology.orchestra.run.smithi049.stdout:ceph: stdout ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev) 2024-02-26T06:10:59.596 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph version: ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev) 2024-02-26T06:10:59.596 INFO:teuthology.orchestra.run.smithi049.stdout:Extracting ceph user uid/gid from container image... 2024-02-26T06:11:00.796 INFO:teuthology.orchestra.run.smithi049.stdout:stat: stdout 167 167 2024-02-26T06:11:00.796 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial keys... 2024-02-26T06:11:02.014 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQB1K9xlt1dzERAADrrnvFLT7U9IvFn2sbPJYg== 2024-02-26T06:11:03.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQB2K9xld3FyGxAAL6rGPpo6P0oSVnSzq42MTg== 2024-02-26T06:11:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQB3K9xl1VP9IhAAF441MM9dppOyB4qrytUhsg== 2024-02-26T06:11:04.178 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial monmap... 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool for a [v2:172.21.15.49:3300,v1:172.21.15.49:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = quincy 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: set fsid to a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-26T06:11:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:11:05.435 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mon... 2024-02-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi049.stdout:create mon.a on 2024-02-26T06:11:07.002 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-26T06:11:07.247 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-26T06:11:07.546 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715.target. 2024-02-26T06:11:07.546 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715.target. 2024-02-26T06:11:07.959 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a 2024-02-26T06:11:07.960 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a.service: Unit ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a.service not loaded. 2024-02-26T06:11:08.344 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715.target.wants/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a.service → /etc/systemd/system/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@.service. 2024-02-26T06:11:08.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:08 smithi049 podman[31296]: 2024-02-26 06:11:08.75025788 +0000 UTC m=+0.141679868 container create feab2706645f81a203ed433528ac06216e9d7826412b470c75999d0a64085d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5) 2024-02-26T06:11:08.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:08 smithi049 podman[31296]: 2024-02-26 06:11:08.995289353 +0000 UTC m=+0.386711398 container init feab2706645f81a203ed433528ac06216e9d7826412b470c75999d0a64085d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-26T06:11:09.114 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T06:11:09.114 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-02-26T06:11:09.132 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T06:11:09.132 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-02-26T06:11:09.132 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-02-26T06:11:09.132 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-02-26T06:11:09.133 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon to start... 2024-02-26T06:11:09.134 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon... 2024-02-26T06:11:09.285 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:09 smithi049 podman[31296]: 2024-02-26 06:11:09.00636224 +0000 UTC m=+0.397784195 container start feab2706645f81a203ed433528ac06216e9d7826412b470c75999d0a64085d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-26T06:11:09.285 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:09 smithi049 bash[31296]: feab2706645f81a203ed433528ac06216e9d7826412b470c75999d0a64085d62 2024-02-26T06:11:09.285 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:09 smithi049 ceph-mon[31315]: mkfs a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:09.285 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:09 smithi049 systemd[1]: Started Ceph mon.a for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:11:09.285 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:09 smithi049 ceph-mon[31315]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T06:11:10.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:10 smithi049 ceph-mon[31315]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T06:11:10.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:10 smithi049 ceph-mon[31315]: monmap e1: 1 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:11:10.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:10 smithi049 ceph-mon[31315]: fsmap 2024-02-26T06:11:10.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:10 smithi049 ceph-mon[31315]: osdmap e1: 0 total, 0 up, 0 in 2024-02-26T06:11:10.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:10 smithi049 ceph-mon[31315]: mgrmap e1: no daemons active 2024-02-26T06:11:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:13 smithi049 ceph-mon[31315]: from='client.? 172.21.15.49:0/3101568841' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-26T06:11:18.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout cluster: 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout id: a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout services: 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 4s) 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout data: 2024-02-26T06:11:18.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-26T06:11:18.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-26T06:11:18.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-26T06:11:18.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pgs: 2024-02-26T06:11:18.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:18.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.031+0000 7fa1cd600700 1 Processor -- start 2024-02-26T06:11:18.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.031+0000 7fa1cd600700 1 -- start start 2024-02-26T06:11:18.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.031+0000 7fa1cd600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c807dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:18.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.031+0000 7fa1cd600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c807e270 con 0x7fa1c807d880 2024-02-26T06:11:18.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.032+0000 7fa1c6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c807dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:18.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.033+0000 7fa1c6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c807dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55468/0 (socket says 172.21.15.49:55468) 2024-02-26T06:11:18.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.033+0000 7fa1c6a00700 1 -- 172.21.15.49:0/338574427 learned_addr learned my addr 172.21.15.49:0/338574427 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:18.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.033+0000 7fa1c6a00700 1 -- 172.21.15.49:0/338574427 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c807ea80 con 0x7fa1c807d880 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.033+0000 7fa1c6a00700 1 --2- 172.21.15.49:0/338574427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c807dca0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80098a0 tx=0x7fa1b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5bb4649dd55a35 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.034+0000 7fa1c5600700 1 -- 172.21.15.49:0/338574427 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1b8004160 con 0x7fa1c807d880 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.034+0000 7fa1c5600700 1 -- 172.21.15.49:0/338574427 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b80042c0 con 0x7fa1c807d880 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.034+0000 7fa1c5600700 1 -- 172.21.15.49:0/338574427 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1b80045a0 con 0x7fa1c807d880 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.034+0000 7fa1cd600700 1 -- 172.21.15.49:0/338574427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 msgr2=0x7fa1c807dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.034+0000 7fa1cd600700 1 --2- 172.21.15.49:0/338574427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c807dca0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80098a0 tx=0x7fa1b8009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.034+0000 7fa1cd600700 1 -- 172.21.15.49:0/338574427 shutdown_connections 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.034+0000 7fa1cd600700 1 --2- 172.21.15.49:0/338574427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c807dca0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.034+0000 7fa1cd600700 1 -- 172.21.15.49:0/338574427 >> 172.21.15.49:0/338574427 conn(0x7fa1c807ce30 msgr2=0x7fa1c81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.035+0000 7fa1cd600700 1 -- 172.21.15.49:0/338574427 shutdown_connections 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.035+0000 7fa1cd600700 1 -- 172.21.15.49:0/338574427 wait complete. 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.036+0000 7fa1cd600700 1 Processor -- start 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.036+0000 7fa1cd600700 1 -- start start 2024-02-26T06:11:18.750 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.036+0000 7fa1cd600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c81a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.036+0000 7fa1cd600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c81a15e0 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.037+0000 7fa1c6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c81a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.037+0000 7fa1c6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c81a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55482/0 (socket says 172.21.15.49:55482) 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.037+0000 7fa1c6a00700 1 -- 172.21.15.49:0/3101568841 learned_addr learned my addr 172.21.15.49:0/3101568841 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.037+0000 7fa1c6a00700 1 -- 172.21.15.49:0/3101568841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b8009550 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.037+0000 7fa1c6a00700 1 --2- 172.21.15.49:0/3101568841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c81a10a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa1c807e560 tx=0x7fa1b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.037+0000 7fa1bf400700 1 -- 172.21.15.49:0/3101568841 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1b8011630 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.037+0000 7fa1bf400700 1 -- 172.21.15.49:0/3101568841 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b8011790 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.038+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c81a17e0 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.038+0000 7fa1bf400700 1 -- 172.21.15.49:0/3101568841 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1b8011a30 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.038+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c81a1c80 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.038+0000 7fa1bf400700 1 -- 172.21.15.49:0/3101568841 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa1b8022070 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.038+0000 7fa1bf400700 1 -- 172.21.15.49:0/3101568841 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa1b801c450 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.039+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c807edf0 con 0x7fa1c807d880 2024-02-26T06:11:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.042+0000 7fa1bf400700 1 -- 172.21.15.49:0/3101568841 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa1b8011b90 con 0x7fa1c807d880 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.124+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa1c81a4780 con 0x7fa1c807d880 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.124+0000 7fa1bf400700 1 -- 172.21.15.49:0/3101568841 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+310 (secure 0 0 0) 0x7fa1b8011d30 con 0x7fa1c807d880 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.125+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 msgr2=0x7fa1c81a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.125+0000 7fa1cd600700 1 --2- 172.21.15.49:0/3101568841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c81a10a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa1c807e560 tx=0x7fa1b8004750 comp rx=0 tx=0).stop 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.126+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 shutdown_connections 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.126+0000 7fa1cd600700 1 --2- 172.21.15.49:0/3101568841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1c807d880 0x7fa1c81a10a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.126+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 >> 172.21.15.49:0/3101568841 conn(0x7fa1c807ce30 msgr2=0x7fa1c8106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.126+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 shutdown_connections 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:13.126+0000 7fa1cd600700 1 -- 172.21.15.49:0/3101568841 wait complete. 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:mon is available 2024-02-26T06:11:18.752 INFO:teuthology.orchestra.run.smithi049.stdout:Assimilating anything we can from ceph.conf... 2024-02-26T06:11:19.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:19 smithi049 ceph-mon[31315]: from='client.? 172.21.15.49:0/2520754402' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-26T06:11:19.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:19 smithi049 ceph-mon[31315]: from='client.? 172.21.15.49:0/2520754402' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-26T06:11:20.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:20.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-02-26T06:11:20.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:20.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-02-26T06:11:20.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-26T06:11:20.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.337+0000 7f5fec800700 1 Processor -- start 2024-02-26T06:11:20.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.337+0000 7f5fec800700 1 -- start start 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.338+0000 7f5fec800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe407b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.338+0000 7f5fec800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe407b8f0 con 0x7f5fe407ce70 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.338+0000 7f5fe9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe407b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.338+0000 7f5fe9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe407b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55496/0 (socket says 172.21.15.49:55496) 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.338+0000 7f5fe9e00700 1 -- 172.21.15.49:0/1350612405 learned_addr learned my addr 172.21.15.49:0/1350612405 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.339+0000 7f5fe9e00700 1 -- 172.21.15.49:0/1350612405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe407ba30 con 0x7f5fe407ce70 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.339+0000 7f5fe9e00700 1 --2- 172.21.15.49:0/1350612405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe407b320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80098a0 tx=0x7f5fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f5d447dab39fbf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.339+0000 7f5fe8a00700 1 -- 172.21.15.49:0/1350612405 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fd8004160 con 0x7f5fe407ce70 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.340+0000 7f5fe8a00700 1 -- 172.21.15.49:0/1350612405 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd80042c0 con 0x7f5fe407ce70 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.340+0000 7f5fe8a00700 1 -- 172.21.15.49:0/1350612405 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fd80045a0 con 0x7f5fe407ce70 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.340+0000 7f5fec800700 1 -- 172.21.15.49:0/1350612405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 msgr2=0x7f5fe407b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.340+0000 7f5fec800700 1 --2- 172.21.15.49:0/1350612405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe407b320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80098a0 tx=0x7f5fd8009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.341+0000 7f5fec800700 1 -- 172.21.15.49:0/1350612405 shutdown_connections 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.341+0000 7f5fec800700 1 --2- 172.21.15.49:0/1350612405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe407b320 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.341+0000 7f5fec800700 1 -- 172.21.15.49:0/1350612405 >> 172.21.15.49:0/1350612405 conn(0x7f5fe4102d00 msgr2=0x7f5fe4105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:20.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.341+0000 7f5fec800700 1 -- 172.21.15.49:0/1350612405 shutdown_connections 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.341+0000 7f5fec800700 1 -- 172.21.15.49:0/1350612405 wait complete. 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.342+0000 7f5fec800700 1 Processor -- start 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.342+0000 7f5fec800700 1 -- start start 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.342+0000 7f5fec800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.342+0000 7f5fec800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419d2c0 con 0x7f5fe407ce70 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.343+0000 7f5fe9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.343+0000 7f5fe9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55500/0 (socket says 172.21.15.49:55500) 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.343+0000 7f5fe9e00700 1 -- 172.21.15.49:0/2520754402 learned_addr learned my addr 172.21.15.49:0/2520754402 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.343+0000 7f5fe9e00700 1 -- 172.21.15.49:0/2520754402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8009550 con 0x7f5fe407ce70 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.343+0000 7f5fe9e00700 1 --2- 172.21.15.49:0/2520754402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe419cd80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5fe407c390 tx=0x7f5fd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.344+0000 7f5fdea00700 1 -- 172.21.15.49:0/2520754402 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fd8011630 con 0x7f5fe407ce70 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.344+0000 7f5fdea00700 1 -- 172.21.15.49:0/2520754402 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd8011790 con 0x7f5fe407ce70 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.344+0000 7f5fdea00700 1 -- 172.21.15.49:0/2520754402 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fd8011a30 con 0x7f5fe407ce70 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.344+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe419d4c0 con 0x7f5fe407ce70 2024-02-26T06:11:20.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.344+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe419d960 con 0x7f5fe407ce70 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.344+0000 7f5fdea00700 1 -- 172.21.15.49:0/2520754402 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5fd8021070 con 0x7f5fe407ce70 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.344+0000 7f5fdea00700 1 -- 172.21.15.49:0/2520754402 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5fd801b7d0 con 0x7f5fe407ce70 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.345+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe4196c10 con 0x7f5fe407ce70 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.348+0000 7f5fdea00700 1 -- 172.21.15.49:0/2520754402 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5fd801b990 con 0x7f5fe407ce70 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.426+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5fe407c000 con 0x7f5fe407ce70 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.434+0000 7f5fdea00700 1 -- 172.21.15.49:0/2520754402 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f5fd801bbe0 con 0x7f5fe407ce70 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.435+0000 7f5fdea00700 1 -- 172.21.15.49:0/2520754402 <== mon.0 v2:172.21.15.49:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fd801bd80 con 0x7f5fe407ce70 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.437+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 msgr2=0x7f5fe419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.437+0000 7f5fec800700 1 --2- 172.21.15.49:0/2520754402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe419cd80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5fe407c390 tx=0x7f5fd8004750 comp rx=0 tx=0).stop 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.437+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 shutdown_connections 2024-02-26T06:11:20.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.437+0000 7f5fec800700 1 --2- 172.21.15.49:0/2520754402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fe407ce70 0x7f5fe419cd80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:20.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.437+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 >> 172.21.15.49:0/2520754402 conn(0x7f5fe4102d00 msgr2=0x7f5fe4104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:20.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.437+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 shutdown_connections 2024-02-26T06:11:20.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:19.437+0000 7f5fec800700 1 -- 172.21.15.49:0/2520754402 wait complete. 2024-02-26T06:11:20.692 INFO:teuthology.orchestra.run.smithi049.stdout:Generating new minimal ceph.conf... 2024-02-26T06:11:21.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:21 smithi049 ceph-mon[31315]: from='client.? 172.21.15.49:0/3774358222' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.289+0000 7fad71600700 1 Processor -- start 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.289+0000 7fad71600700 1 -- start start 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.290+0000 7fad71600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.290+0000 7fad71600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c10a260 con 0x7fad6c109870 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.290+0000 7fad6aa00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.291+0000 7fad6aa00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47178/0 (socket says 172.21.15.49:47178) 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.291+0000 7fad6aa00700 1 -- 172.21.15.49:0/3866509412 learned_addr learned my addr 172.21.15.49:0/3866509412 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.291+0000 7fad6aa00700 1 -- 172.21.15.49:0/3866509412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c10aa20 con 0x7fad6c109870 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.291+0000 7fad6aa00700 1 --2- 172.21.15.49:0/3866509412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c109c90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0098a0 tx=0x7fad5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6131251dbe347ef7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.292+0000 7fad69600700 1 -- 172.21.15.49:0/3866509412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad5c004160 con 0x7fad6c109870 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.292+0000 7fad69600700 1 -- 172.21.15.49:0/3866509412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad5c0042c0 con 0x7fad6c109870 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.292+0000 7fad69600700 1 -- 172.21.15.49:0/3866509412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad5c011ac0 con 0x7fad6c109870 2024-02-26T06:11:22.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.292+0000 7fad71600700 1 -- 172.21.15.49:0/3866509412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 msgr2=0x7fad6c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.292+0000 7fad71600700 1 --2- 172.21.15.49:0/3866509412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c109c90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0098a0 tx=0x7fad5c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.293+0000 7fad71600700 1 -- 172.21.15.49:0/3866509412 shutdown_connections 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.293+0000 7fad71600700 1 --2- 172.21.15.49:0/3866509412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c109c90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.293+0000 7fad71600700 1 -- 172.21.15.49:0/3866509412 >> 172.21.15.49:0/3866509412 conn(0x7fad6c07ce30 msgr2=0x7fad6c1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.293+0000 7fad71600700 1 -- 172.21.15.49:0/3866509412 shutdown_connections 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.293+0000 7fad71600700 1 -- 172.21.15.49:0/3866509412 wait complete. 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.294+0000 7fad71600700 1 Processor -- start 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.294+0000 7fad71600700 1 -- start start 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.294+0000 7fad71600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.294+0000 7fad71600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c19db40 con 0x7fad6c109870 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.295+0000 7fad6aa00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.295+0000 7fad6aa00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47194/0 (socket says 172.21.15.49:47194) 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.295+0000 7fad6aa00700 1 -- 172.21.15.49:0/3774358222 learned_addr learned my addr 172.21.15.49:0/3774358222 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:22.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.295+0000 7fad6aa00700 1 -- 172.21.15.49:0/3774358222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c009550 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.295+0000 7fad6aa00700 1 --2- 172.21.15.49:0/3774358222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c19d600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fad5c000c00 tx=0x7fad5c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.296+0000 7fad63400700 1 -- 172.21.15.49:0/3774358222 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad5c003aa0 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.296+0000 7fad63400700 1 -- 172.21.15.49:0/3774358222 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad5c003c00 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.296+0000 7fad63400700 1 -- 172.21.15.49:0/3774358222 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad5c0248a0 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.296+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c19dd40 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.296+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c19e1e0 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.296+0000 7fad63400700 1 -- 172.21.15.49:0/3774358222 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fad5c02d3f0 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.296+0000 7fad63400700 1 -- 172.21.15.49:0/3774358222 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fad5c019770 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.297+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6c10dc50 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.300+0000 7fad63400700 1 -- 172.21.15.49:0/3774358222 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fad5c01ac60 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.380+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fad6c1a0d40 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.381+0000 7fad63400700 1 -- 172.21.15.49:0/3774358222 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fad5c035090 con 0x7fad6c109870 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.382+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 msgr2=0x7fad6c19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.382+0000 7fad71600700 1 --2- 172.21.15.49:0/3774358222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c19d600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fad5c000c00 tx=0x7fad5c011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:22.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.382+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 shutdown_connections 2024-02-26T06:11:22.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.382+0000 7fad71600700 1 --2- 172.21.15.49:0/3774358222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad6c109870 0x7fad6c19d600 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:22.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.382+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 >> 172.21.15.49:0/3774358222 conn(0x7fad6c07ce30 msgr2=0x7fad6c106bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:22.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.383+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 shutdown_connections 2024-02-26T06:11:22.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:21.383+0000 7fad71600700 1 -- 172.21.15.49:0/3774358222 wait complete. 2024-02-26T06:11:22.058 INFO:teuthology.orchestra.run.smithi049.stdout:Restarting the monitor... 2024-02-26T06:11:22.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:22 smithi049 systemd[1]: Stopping Ceph mon.a for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:11:22.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31311]: 2024-02-26T06:11:22.468+0000 7f34c7c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:11:22.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31311]: 2024-02-26T06:11:22.468+0000 7f34c7c00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-26T06:11:22.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:22 smithi049 podman[31570]: 2024-02-26 06:11:22.623712653 +0000 UTC m=+0.260407365 container died feab2706645f81a203ed433528ac06216e9d7826412b470c75999d0a64085d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-26T06:11:23.341 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:23 smithi049 podman[31570]: 2024-02-26 06:11:23.172115147 +0000 UTC m=+0.808809876 container cleanup feab2706645f81a203ed433528ac06216e9d7826412b470c75999d0a64085d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-26T06:11:23.341 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:23 smithi049 bash[31570]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a 2024-02-26T06:11:23.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:23 smithi049 podman[31584]: 2024-02-26 06:11:23.338836321 +0000 UTC m=+0.699770116 container remove feab2706645f81a203ed433528ac06216e9d7826412b470c75999d0a64085d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-26T06:11:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:23 smithi049 systemd[1]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-26T06:11:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:23 smithi049 systemd[1]: Stopped Ceph mon.a for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:11:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:23 smithi049 systemd[1]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a.service: Consumed 1.925s CPU time. 2024-02-26T06:11:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:23 smithi049 systemd[1]: Starting Ceph mon.a for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:11:24.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 podman[31677]: 2024-02-26 06:11:24.272912095 +0000 UTC m=+0.127010570 container create 12392c53e5257df8d623e29a315c737dcd2f0fcbc6e5720b3990525108683127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-26T06:11:24.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 podman[31677]: 2024-02-26 06:11:24.18450478 +0000 UTC m=+0.038603223 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:11:24.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 podman[31677]: 2024-02-26 06:11:24.585049636 +0000 UTC m=+0.439148141 container init 12392c53e5257df8d623e29a315c737dcd2f0fcbc6e5720b3990525108683127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-26T06:11:24.684 INFO:teuthology.orchestra.run.smithi049.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-26T06:11:24.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 podman[31677]: 2024-02-26 06:11:24.596724888 +0000 UTC m=+0.450823342 container start 12392c53e5257df8d623e29a315c737dcd2f0fcbc6e5720b3990525108683127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-26T06:11:24.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: set uid:gid to 167:167 (ceph:ceph) 2024-02-26T06:11:24.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev), process ceph-mon, pid 2 2024-02-26T06:11:24.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: pidfile_write: ignore empty --pid-file 2024-02-26T06:11:24.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: load: jerasure load: lrc 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: RocksDB version: 7.9.2 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Git sha 0 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Compile date 2024-02-16 06:18:30 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: DB SUMMARY 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: DB Session ID: RSNNG0ECWY08X97AY304 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: CURRENT file: CURRENT 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: IDENTITY file: IDENTITY 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100597 ; 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.error_if_exists: 0 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.create_if_missing: 0 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.paranoid_checks: 1 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-26T06:11:24.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.env: 0x55be67261ae0 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.fs: PosixFileSystem 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.info_log: 0x55be67d61900 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_file_opening_threads: 16 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.statistics: (nil) 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.use_fsync: 0 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_log_file_size: 0 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.keep_log_file_num: 1000 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.recycle_log_file_num: 0 2024-02-26T06:11:24.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.allow_fallocate: 1 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.allow_mmap_reads: 0 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.allow_mmap_writes: 0 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.use_direct_reads: 0 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.create_missing_column_families: 0 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.db_log_dir: 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.wal_dir: 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-26T06:11:24.923 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.advise_random_on_open: 1 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.db_write_buffer_size: 0 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.write_buffer_manager: 0x55be67d35400 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.rate_limiter: (nil) 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.wal_recovery_mode: 2 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.enable_thread_tracking: 0 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.enable_pipelined_write: 0 2024-02-26T06:11:24.924 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.unordered_write: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.row_cache: None 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.wal_filter: None 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.allow_ingest_behind: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.two_write_queues: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.manual_wal_flush: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.wal_compression: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.atomic_flush: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-26T06:11:24.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.log_readahead_size: 0 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.best_efforts_recovery: 0 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.allow_data_in_errors: 0 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.db_host_id: __hostname__ 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_background_jobs: 2 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_background_compactions: -1 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_subcompactions: 1 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_total_wal_size: 0 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-26T06:11:24.926 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_open_files: -1 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bytes_per_sync: 0 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_readahead_size: 0 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_background_flushes: -1 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Compression algorithms supported: 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: kZSTD supported: 0 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: kXpressCompression supported: 0 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: kLZ4HCCompression supported: 1 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: kZlibCompression supported: 1 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: kSnappyCompression supported: 1 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: kLZ4Compression supported: 1 2024-02-26T06:11:24.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: kBZip2Compression supported: 0 2024-02-26T06:11:24.928 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-26T06:11:24.928 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-26T06:11:24.928 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-26T06:11:24.928 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-26T06:11:24.928 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-26T06:11:24.928 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.merge_operator: 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_filter: None 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_filter_factory: None 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.sst_partitioner_factory: None 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55be67d61a00) 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-26T06:11:24.929 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_type: 0 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_index_type: 0 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_shortening: 1 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: checksum: 4 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: no_block_cache: 0 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache: 0x55be67d65090 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_options: 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: capacity : 536870912 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_shard_bits : 4 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: strict_capacity_limit : 0 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_compressed: (nil) 2024-02-26T06:11:24.930 INFO:journalctl@ceph.mon.a.smithi049.stdout: persistent_cache: (nil) 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size: 4096 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size_deviation: 10 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_restart_interval: 16 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_block_restart_interval: 1 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: metadata_block_size: 4096 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: partition_filters: 0 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: use_delta_encoding: 1 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: filter_policy: bloomfilter 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: whole_key_filtering: 1 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: verify_compression: 0 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: format_version: 5 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_index_compression: 1 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_align: 0 2024-02-26T06:11:24.931 INFO:journalctl@ceph.mon.a.smithi049.stdout: max_auto_readahead_size: 262144 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout: prepopulate_block_cache: 0 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout: initial_auto_readahead_size: 8192 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.write_buffer_size: 33554432 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_write_buffer_number: 2 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression: NoCompression 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression: Disabled 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.prefix_extractor: nullptr 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.num_levels: 7 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-26T06:11:24.932 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.level: 32767 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.strategy: 0 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-26T06:11:24.933 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.enabled: false 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.target_file_size_base: 67108864 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-26T06:11:24.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.arena_block_size: 1048576 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.disable_auto_compactions: 0 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-26T06:11:24.935 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.table_properties_collectors: 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.inplace_update_support: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.bloom_locality: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.max_successive_merges: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.paranoid_file_checks: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.force_consistency_checks: 1 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.report_bg_io_stats: 0 2024-02-26T06:11:24.936 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.ttl: 2592000 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.enable_blob_files: false 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.min_blob_size: 0 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.blob_file_size: 268435456 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.blob_file_starting_level: 0 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-26T06:11:24.937 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 24b19699-afa4-4034-b4a3-8ded8d9f1f66 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708927884655979, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708927884658201, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95570, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 93556, "index_size": 296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13934, "raw_average_key_size": 49, "raw_value_size": 85990, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708927884, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "24b19699-afa4-4034-b4a3-8ded8d9f1f66", "db_session_id": "RSNNG0ECWY08X97AY304", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708927884658374, "job": 1, "event": "recovery_finished"} 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55be67e56000 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: DB pointer 0x55be67e40000 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** DB Stats ** 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T06:11:24.938 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: L0 2/0 95.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: Sum 2/0 95.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 60.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-02-26T06:11:24.939 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative compaction: 0.00 GB write, 13.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval compaction: 0.00 GB write, 13.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache BinnedLRUCache@0x55be67d65090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: starting mon.a rank 0 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:24.940 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: mon.a@-1(???) e1 preinit fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: mon.a@-1(???).mds e1 new map 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: mon.a@-1(???).mds e1 print_map 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout: e1 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout: legacy client fscid: -1 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout: No filesystems configured 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 ceph-mon[31696]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 bash[31677]: 12392c53e5257df8d623e29a315c737dcd2f0fcbc6e5720b3990525108683127 2024-02-26T06:11:24.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:24 smithi049 systemd[1]: Started Ceph mon.a for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:11:26.064 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:25 smithi049 ceph-mon[31696]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T06:11:26.064 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:25 smithi049 ceph-mon[31696]: monmap e1: 1 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:11:26.064 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:25 smithi049 ceph-mon[31696]: fsmap 2024-02-26T06:11:26.065 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:25 smithi049 ceph-mon[31696]: osdmap e1: 0 total, 0 up, 0 in 2024-02-26T06:11:26.065 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:25 smithi049 ceph-mon[31696]: mgrmap e1: no daemons active 2024-02-26T06:11:26.065 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:25 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/622058704' entity='client.admin' 2024-02-26T06:11:26.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.320+0000 7fdd80c00700 1 Processor -- start 2024-02-26T06:11:26.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.320+0000 7fdd80c00700 1 -- start start 2024-02-26T06:11:26.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.320+0000 7fdd80c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c109c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:26.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.321+0000 7fdd80c00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c10a210 con 0x7fdd7c109820 2024-02-26T06:11:26.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.322+0000 7fdd79e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:26.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.322+0000 7fdd79e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47200/0 (socket says 172.21.15.49:47200) 2024-02-26T06:11:26.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.322+0000 7fdd79e00700 1 -- 172.21.15.49:0/3532632000 learned_addr learned my addr 172.21.15.49:0/3532632000 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:26.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.323+0000 7fdd79e00700 1 -- 172.21.15.49:0/3532632000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c10a9d0 con 0x7fdd7c109820 2024-02-26T06:11:26.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.323+0000 7fdd79e00700 1 --2- 172.21.15.49:0/3532632000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c109c40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0098a0 tx=0x7fdd6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8a24ae4eb65522 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:26.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.323+0000 7fdd78a00700 1 -- 172.21.15.49:0/3532632000 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd6c004160 con 0x7fdd7c109820 2024-02-26T06:11:26.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.323+0000 7fdd78a00700 1 -- 172.21.15.49:0/3532632000 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd6c0042c0 con 0x7fdd7c109820 2024-02-26T06:11:26.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.323+0000 7fdd78a00700 1 -- 172.21.15.49:0/3532632000 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd6c011ac0 con 0x7fdd7c109820 2024-02-26T06:11:26.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.324+0000 7fdd80c00700 1 -- 172.21.15.49:0/3532632000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 msgr2=0x7fdd7c109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:26.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.324+0000 7fdd80c00700 1 --2- 172.21.15.49:0/3532632000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c109c40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0098a0 tx=0x7fdd6c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:26.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.324+0000 7fdd80c00700 1 -- 172.21.15.49:0/3532632000 shutdown_connections 2024-02-26T06:11:26.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.324+0000 7fdd80c00700 1 --2- 172.21.15.49:0/3532632000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c109c40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:26.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.324+0000 7fdd80c00700 1 -- 172.21.15.49:0/3532632000 >> 172.21.15.49:0/3532632000 conn(0x7fdd7c07ce30 msgr2=0x7fdd7c1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:26.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.324+0000 7fdd80c00700 1 -- 172.21.15.49:0/3532632000 shutdown_connections 2024-02-26T06:11:26.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.325+0000 7fdd80c00700 1 -- 172.21.15.49:0/3532632000 wait complete. 2024-02-26T06:11:26.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.326+0000 7fdd80c00700 1 Processor -- start 2024-02-26T06:11:26.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.326+0000 7fdd80c00700 1 -- start start 2024-02-26T06:11:26.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.326+0000 7fdd80c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:26.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.326+0000 7fdd80c00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c19dad0 con 0x7fdd7c109820 2024-02-26T06:11:26.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.326+0000 7fdd79e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:26.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.327+0000 7fdd79e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47202/0 (socket says 172.21.15.49:47202) 2024-02-26T06:11:26.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.327+0000 7fdd79e00700 1 -- 172.21.15.49:0/622058704 learned_addr learned my addr 172.21.15.49:0/622058704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:26.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.327+0000 7fdd79e00700 1 -- 172.21.15.49:0/622058704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009550 con 0x7fdd7c109820 2024-02-26T06:11:26.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.327+0000 7fdd79e00700 1 --2- 172.21.15.49:0/622058704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c19d590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c000c00 tx=0x7fdd6c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:26.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.328+0000 7fdd72a00700 1 -- 172.21.15.49:0/622058704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd6c003aa0 con 0x7fdd7c109820 2024-02-26T06:11:26.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.328+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c19dcd0 con 0x7fdd7c109820 2024-02-26T06:11:26.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.328+0000 7fdd72a00700 1 -- 172.21.15.49:0/622058704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd6c003c00 con 0x7fdd7c109820 2024-02-26T06:11:26.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.328+0000 7fdd72a00700 1 -- 172.21.15.49:0/622058704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd6c0248a0 con 0x7fdd7c109820 2024-02-26T06:11:26.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.328+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c19e170 con 0x7fdd7c109820 2024-02-26T06:11:26.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.328+0000 7fdd72a00700 1 -- 172.21.15.49:0/622058704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdd6c02d3f0 con 0x7fdd7c109820 2024-02-26T06:11:26.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.328+0000 7fdd72a00700 1 -- 172.21.15.49:0/622058704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdd6c0197d0 con 0x7fdd7c109820 2024-02-26T06:11:26.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.329+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7c10dc00 con 0x7fdd7c109820 2024-02-26T06:11:26.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.333+0000 7fdd72a00700 1 -- 172.21.15.49:0/622058704 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdd6c01ac60 con 0x7fdd7c109820 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.410+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fdd7c10de10 con 0x7fdd7c109820 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.414+0000 7fdd72a00700 1 -- 172.21.15.49:0/622058704 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdd6c0350c0 con 0x7fdd7c109820 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.415+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 msgr2=0x7fdd7c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.415+0000 7fdd80c00700 1 --2- 172.21.15.49:0/622058704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c19d590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c000c00 tx=0x7fdd6c011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.415+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 shutdown_connections 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.415+0000 7fdd80c00700 1 --2- 172.21.15.49:0/622058704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd7c109820 0x7fdd7c19d590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.415+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 >> 172.21.15.49:0/622058704 conn(0x7fdd7c07ce30 msgr2=0x7fdd7c106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.416+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 shutdown_connections 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:25.416+0000 7fdd80c00700 1 -- 172.21.15.49:0/622058704 wait complete. 2024-02-26T06:11:26.085 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-26T06:11:26.113 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:11:26.113 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mgr... 2024-02-26T06:11:26.115 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-26T06:11:26.116 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-26T06:11:26.565 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.x 2024-02-26T06:11:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.x.service: Unit ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.x.service not loaded. 2024-02-26T06:11:26.834 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715.target.wants/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@.service. 2024-02-26T06:11:27.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:26 smithi049 systemd[1]: Starting Ceph mgr.x for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:11:27.514 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:27 smithi049 podman[31907]: 2024-02-26 06:11:27.158533022 +0000 UTC m=+0.038552671 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:11:27.515 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:27 smithi049 podman[31907]: 2024-02-26 06:11:27.265962565 +0000 UTC m=+0.145982226 container create 03ccb02c6201fd4a4c8eceb8c35bc030a2a7d3ba675a7bcee2e0133f6127bb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-26T06:11:27.515 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:27 smithi049 podman[31907]: 2024-02-26 06:11:27.51424206 +0000 UTC m=+0.394261728 container init 03ccb02c6201fd4a4c8eceb8c35bc030a2a7d3ba675a7bcee2e0133f6127bb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-26T06:11:27.655 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T06:11:27.656 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-02-26T06:11:27.677 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T06:11:27.677 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-02-26T06:11:27.677 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-02-26T06:11:27.677 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-02-26T06:11:27.698 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T06:11:27.698 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-02-26T06:11:27.716 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T06:11:27.716 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-02-26T06:11:27.716 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-02-26T06:11:27.716 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-26T06:11:27.716 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr to start... 2024-02-26T06:11:27.716 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr... 2024-02-26T06:11:27.869 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:27 smithi049 podman[31907]: 2024-02-26 06:11:27.524699555 +0000 UTC m=+0.404719185 container start 03ccb02c6201fd4a4c8eceb8c35bc030a2a7d3ba675a7bcee2e0133f6127bb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-26T06:11:27.869 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:27 smithi049 bash[31907]: 03ccb02c6201fd4a4c8eceb8c35bc030a2a7d3ba675a7bcee2e0133f6127bb2c 2024-02-26T06:11:27.869 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:27 smithi049 systemd[1]: Started Ceph mgr.x for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:11:28.464 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:28 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:28.358+0000 7f2eef7ac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:11:28.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:28 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4009648723' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T06:11:28.842 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:28 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:28.492+0000 7f2eef7ac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:11:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T06:11:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T06:11:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T06:11:29.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:29.166 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:29.173 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T06:11:09.087418+0000", 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.373+0000 7f05c2600700 1 Processor -- start 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.373+0000 7f05c2600700 1 -- start start 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.373+0000 7f05c2600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc07b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.373+0000 7f05c2600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc07b840 con 0x7f05bc07ce50 2024-02-26T06:11:29.174 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.373+0000 7f05bb400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc07b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.373+0000 7f05bb400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc07b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47230/0 (socket says 172.21.15.49:47230) 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.373+0000 7f05bb400700 1 -- 172.21.15.49:0/513031779 learned_addr learned my addr 172.21.15.49:0/513031779 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.373+0000 7f05bb400700 1 -- 172.21.15.49:0/513031779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc07b980 con 0x7f05bc07ce50 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.374+0000 7f05bb400700 1 --2- 172.21.15.49:0/513031779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc07b300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0098a0 tx=0x7f05ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600eb0e4c8c5ba53 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.374+0000 7f05ba000700 1 -- 172.21.15.49:0/513031779 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05ac004160 con 0x7f05bc07ce50 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.374+0000 7f05ba000700 1 -- 172.21.15.49:0/513031779 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05ac0042c0 con 0x7f05bc07ce50 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.374+0000 7f05ba000700 1 -- 172.21.15.49:0/513031779 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05ac011ac0 con 0x7f05bc07ce50 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.375+0000 7f05c2600700 1 -- 172.21.15.49:0/513031779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 msgr2=0x7f05bc07b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.375+0000 7f05c2600700 1 --2- 172.21.15.49:0/513031779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc07b300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0098a0 tx=0x7f05ac009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.375+0000 7f05c2600700 1 -- 172.21.15.49:0/513031779 shutdown_connections 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.375+0000 7f05c2600700 1 --2- 172.21.15.49:0/513031779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc07b300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.375+0000 7f05c2600700 1 -- 172.21.15.49:0/513031779 >> 172.21.15.49:0/513031779 conn(0x7f05bc102c70 msgr2=0x7f05bc105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.375+0000 7f05c2600700 1 -- 172.21.15.49:0/513031779 shutdown_connections 2024-02-26T06:11:29.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.375+0000 7f05c2600700 1 -- 172.21.15.49:0/513031779 wait complete. 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.376+0000 7f05c2600700 1 Processor -- start 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.376+0000 7f05c2600700 1 -- start start 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.377+0000 7f05c2600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.377+0000 7f05c2600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc19db10 con 0x7f05bc07ce50 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.377+0000 7f05bb400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.377+0000 7f05bb400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47246/0 (socket says 172.21.15.49:47246) 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.377+0000 7f05bb400700 1 -- 172.21.15.49:0/4009648723 learned_addr learned my addr 172.21.15.49:0/4009648723 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.377+0000 7f05bb400700 1 -- 172.21.15.49:0/4009648723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009550 con 0x7f05bc07ce50 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.378+0000 7f05bb400700 1 --2- 172.21.15.49:0/4009648723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc19d5d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009130 tx=0x7f05ac011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.378+0000 7f05b3e00700 1 -- 172.21.15.49:0/4009648723 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05ac003aa0 con 0x7f05bc07ce50 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.378+0000 7f05b3e00700 1 -- 172.21.15.49:0/4009648723 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05ac003c00 con 0x7f05bc07ce50 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.378+0000 7f05b3e00700 1 -- 172.21.15.49:0/4009648723 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05ac024800 con 0x7f05bc07ce50 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.378+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc19dd10 con 0x7f05bc07ce50 2024-02-26T06:11:29.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.378+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc19e1b0 con 0x7f05bc07ce50 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.378+0000 7f05b3e00700 1 -- 172.21.15.49:0/4009648723 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f05ac02d3f0 con 0x7f05bc07ce50 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.379+0000 7f05b3e00700 1 -- 172.21.15.49:0/4009648723 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f05ac019760 con 0x7f05bc07ce50 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.379+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc10da00 con 0x7f05bc07ce50 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.382+0000 7f05b3e00700 1 -- 172.21.15.49:0/4009648723 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f05ac024e00 con 0x7f05bc07ce50 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.463+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f05bc1a0de0 con 0x7f05bc07ce50 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.465+0000 7f05b3e00700 1 -- 172.21.15.49:0/4009648723 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f05ac01abd0 con 0x7f05bc07ce50 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.467+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 msgr2=0x7f05bc19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.467+0000 7f05c2600700 1 --2- 172.21.15.49:0/4009648723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc19d5d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009130 tx=0x7f05ac011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.467+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 shutdown_connections 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.467+0000 7f05c2600700 1 --2- 172.21.15.49:0/4009648723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc07ce50 0x7f05bc19d5d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.467+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 >> 172.21.15.49:0/4009648723 conn(0x7f05bc102c70 msgr2=0x7f05bc103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.467+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 shutdown_connections 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:28.467+0000 7f05c2600700 1 -- 172.21.15.49:0/4009648723 wait complete. 2024-02-26T06:11:29.177 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (1/15)... 2024-02-26T06:11:29.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:29.349+0000 7f2eef7ac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:11:29.803 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:29.479+0000 7f2eef7ac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:11:29.803 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:29.671+0000 7f2eef7ac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:11:29.803 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:29.803+0000 7f2eef7ac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:11:30.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:29.918+0000 7f2eef7ac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:11:30.538 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:30.271+0000 7f2eef7ac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:11:30.539 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:30.419+0000 7f2eef7ac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:11:30.896 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:30.538+0000 7f2eef7ac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:11:30.896 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:30.667+0000 7f2eef7ac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:11:30.896 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:30.895+0000 7f2eef7ac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:11:32.094 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:31 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/997838199' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T06:11:32.366 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:32 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:32.315+0000 7f2eef7ac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T06:11:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:32.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T06:11:32.583 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:32.583 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T06:11:32.583 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T06:11:32.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T06:11:32.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T06:11:32.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T06:11:32.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T06:11:32.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:32.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T06:11:32.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T06:11:32.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:32.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T06:11:09.087418+0000", 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.752+0000 7fb979200700 1 Processor -- start 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.752+0000 7fb979200700 1 -- start start 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.753+0000 7fb979200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97407b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.753+0000 7fb979200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97407b840 con 0x7fb97407ce50 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.753+0000 7fb971e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97407b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.753+0000 7fb971e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97407b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45760/0 (socket says 172.21.15.49:45760) 2024-02-26T06:11:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.753+0000 7fb971e00700 1 -- 172.21.15.49:0/789770444 learned_addr learned my addr 172.21.15.49:0/789770444 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.754+0000 7fb971e00700 1 -- 172.21.15.49:0/789770444 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97407b980 con 0x7fb97407ce50 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.754+0000 7fb971e00700 1 --2- 172.21.15.49:0/789770444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97407b300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb9640098a0 tx=0x7fb964009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3aa855371a31ea server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.754+0000 7fb970a00700 1 -- 172.21.15.49:0/789770444 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb964004160 con 0x7fb97407ce50 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.755+0000 7fb970a00700 1 -- 172.21.15.49:0/789770444 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9640042c0 con 0x7fb97407ce50 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.755+0000 7fb970a00700 1 -- 172.21.15.49:0/789770444 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb964011ac0 con 0x7fb97407ce50 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.755+0000 7fb979200700 1 -- 172.21.15.49:0/789770444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 msgr2=0x7fb97407b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.755+0000 7fb979200700 1 --2- 172.21.15.49:0/789770444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97407b300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb9640098a0 tx=0x7fb964009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.755+0000 7fb979200700 1 -- 172.21.15.49:0/789770444 shutdown_connections 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.755+0000 7fb979200700 1 --2- 172.21.15.49:0/789770444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97407b300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.755+0000 7fb979200700 1 -- 172.21.15.49:0/789770444 >> 172.21.15.49:0/789770444 conn(0x7fb974102c70 msgr2=0x7fb974105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.756+0000 7fb979200700 1 -- 172.21.15.49:0/789770444 shutdown_connections 2024-02-26T06:11:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.756+0000 7fb979200700 1 -- 172.21.15.49:0/789770444 wait complete. 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.757+0000 7fb979200700 1 Processor -- start 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.757+0000 7fb979200700 1 -- start start 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.757+0000 7fb979200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.757+0000 7fb979200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419db10 con 0x7fb97407ce50 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.758+0000 7fb971e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.758+0000 7fb971e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45766/0 (socket says 172.21.15.49:45766) 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.758+0000 7fb971e00700 1 -- 172.21.15.49:0/997838199 learned_addr learned my addr 172.21.15.49:0/997838199 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.758+0000 7fb971e00700 1 -- 172.21.15.49:0/997838199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964009550 con 0x7fb97407ce50 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.758+0000 7fb971e00700 1 --2- 172.21.15.49:0/997838199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97419d5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb964009130 tx=0x7fb964011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.759+0000 7fb96aa00700 1 -- 172.21.15.49:0/997838199 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb964003aa0 con 0x7fb97407ce50 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.759+0000 7fb96aa00700 1 -- 172.21.15.49:0/997838199 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb964003c00 con 0x7fb97407ce50 2024-02-26T06:11:32.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.759+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97419dd10 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.759+0000 7fb96aa00700 1 -- 172.21.15.49:0/997838199 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb964024800 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.759+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97419e1b0 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.759+0000 7fb96aa00700 1 -- 172.21.15.49:0/997838199 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb96402d3f0 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.759+0000 7fb96aa00700 1 -- 172.21.15.49:0/997838199 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb964019760 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.760+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97410da00 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.763+0000 7fb96aa00700 1 -- 172.21.15.49:0/997838199 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb96401abd0 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.840+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb9741a0d20 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.841+0000 7fb96aa00700 1 -- 172.21.15.49:0/997838199 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb96401ad70 con 0x7fb97407ce50 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.842+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 msgr2=0x7fb97419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.842+0000 7fb979200700 1 --2- 172.21.15.49:0/997838199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97419d5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb964009130 tx=0x7fb964011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.842+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 shutdown_connections 2024-02-26T06:11:32.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.842+0000 7fb979200700 1 --2- 172.21.15.49:0/997838199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb97407ce50 0x7fb97419d5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:32.597 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.842+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 >> 172.21.15.49:0/997838199 conn(0x7fb974102c70 msgr2=0x7fb974103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:32.597 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.842+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 shutdown_connections 2024-02-26T06:11:32.597 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:31.842+0000 7fb979200700 1 -- 172.21.15.49:0/997838199 wait complete. 2024-02-26T06:11:32.597 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (2/15)... 2024-02-26T06:11:32.669 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:32 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:32.499+0000 7f2eef7ac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:11:33.076 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:32 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:32.951+0000 7f2eef7ac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:11:33.343 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:33.076+0000 7f2eef7ac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:11:33.602 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:33.343+0000 7f2eef7ac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:11:33.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:33.602+0000 7f2eef7ac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:11:34.325 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:33.991+0000 7f2eef7ac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:11:34.325 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:34 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:34.325+0000 7f2eef7ac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:11:34.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:34 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:34.650+0000 7f2eef7ac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:11:35.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:35 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3815453804' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T06:11:36.078 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:35 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:35.762+0000 7f2eef7ac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:11:36.098 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:36.098 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:11:36.098 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:11:36.098 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T06:11:36.098 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T06:11:36.098 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T06:11:36.099 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T06:11:36.099 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:36.099 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T06:11:36.099 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T06:11:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T06:11:36.101 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T06:11:36.102 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T06:11:09.087418+0000", 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T06:11:36.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.223+0000 7f6c51400700 1 Processor -- start 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.223+0000 7f6c51400700 1 -- start start 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.224+0000 7f6c51400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.224+0000 7f6c51400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c07b840 con 0x7f6c4c07ce50 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.224+0000 7f6c4a800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.224+0000 7f6c4a800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45768/0 (socket says 172.21.15.49:45768) 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.224+0000 7f6c4a800700 1 -- 172.21.15.49:0/4051386892 learned_addr learned my addr 172.21.15.49:0/4051386892 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.225+0000 7f6c4a800700 1 -- 172.21.15.49:0/4051386892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c07b980 con 0x7f6c4c07ce50 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.225+0000 7f6c4a800700 1 --2- 172.21.15.49:0/4051386892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098a0 tx=0x7f6c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c229352d3653fb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.225+0000 7f6c49400700 1 -- 172.21.15.49:0/4051386892 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c004160 con 0x7f6c4c07ce50 2024-02-26T06:11:36.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.225+0000 7f6c49400700 1 -- 172.21.15.49:0/4051386892 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c3c0042c0 con 0x7f6c4c07ce50 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.225+0000 7f6c49400700 1 -- 172.21.15.49:0/4051386892 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c011ac0 con 0x7f6c4c07ce50 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.226+0000 7f6c51400700 1 -- 172.21.15.49:0/4051386892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 msgr2=0x7f6c4c07b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.226+0000 7f6c51400700 1 --2- 172.21.15.49:0/4051386892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098a0 tx=0x7f6c3c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.226+0000 7f6c51400700 1 -- 172.21.15.49:0/4051386892 shutdown_connections 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.226+0000 7f6c51400700 1 --2- 172.21.15.49:0/4051386892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.226+0000 7f6c51400700 1 -- 172.21.15.49:0/4051386892 >> 172.21.15.49:0/4051386892 conn(0x7f6c4c102c70 msgr2=0x7f6c4c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.226+0000 7f6c51400700 1 -- 172.21.15.49:0/4051386892 shutdown_connections 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.226+0000 7f6c51400700 1 -- 172.21.15.49:0/4051386892 wait complete. 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.228+0000 7f6c51400700 1 Processor -- start 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.228+0000 7f6c51400700 1 -- start start 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.229+0000 7f6c51400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:36.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.229+0000 7f6c51400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c19db10 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.229+0000 7f6c4a800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.229+0000 7f6c4a800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45778/0 (socket says 172.21.15.49:45778) 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.229+0000 7f6c4a800700 1 -- 172.21.15.49:0/3815453804 learned_addr learned my addr 172.21.15.49:0/3815453804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.229+0000 7f6c4a800700 1 -- 172.21.15.49:0/3815453804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c009550 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.229+0000 7f6c4a800700 1 --2- 172.21.15.49:0/3815453804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c003f70 tx=0x7f6c3c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.230+0000 7f6c43400700 1 -- 172.21.15.49:0/3815453804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c0039f0 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.230+0000 7f6c43400700 1 -- 172.21.15.49:0/3815453804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c3c003b50 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.230+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c19dd10 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.230+0000 7f6c43400700 1 -- 172.21.15.49:0/3815453804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c01a760 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.230+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c19e1b0 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.230+0000 7f6c43400700 1 -- 172.21.15.49:0/3815453804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6c3c01ac90 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.230+0000 7f6c43400700 1 -- 172.21.15.49:0/3815453804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6c3c0199b0 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.231+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c10da00 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.234+0000 7f6c43400700 1 -- 172.21.15.49:0/3815453804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6c3c0249f0 con 0x7f6c4c07ce50 2024-02-26T06:11:36.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.314+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6c4c10dc10 con 0x7f6c4c07ce50 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.315+0000 7f6c43400700 1 -- 172.21.15.49:0/3815453804 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f6c3c019b70 con 0x7f6c4c07ce50 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.316+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 msgr2=0x7f6c4c19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.316+0000 7f6c51400700 1 --2- 172.21.15.49:0/3815453804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c003f70 tx=0x7f6c3c011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.316+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 shutdown_connections 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.316+0000 7f6c51400700 1 --2- 172.21.15.49:0/3815453804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d5d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.316+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 >> 172.21.15.49:0/3815453804 conn(0x7f6c4c102c70 msgr2=0x7f6c4c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.317+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 shutdown_connections 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:35.317+0000 7f6c51400700 1 -- 172.21.15.49:0/3815453804 wait complete. 2024-02-26T06:11:36.107 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (3/15)... 2024-02-26T06:11:36.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:36.506+0000 7f2eef7ac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:11:36.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:36.643+0000 7f2eef7ac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:11:37.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:36.938+0000 7f2eef7ac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:11:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: Activating manager daemon x 2024-02-26T06:11:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: mgrmap e2: x(active, starting, since 0.0116351s) 2024-02-26T06:11:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: Manager daemon x is now available 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' 2024-02-26T06:11:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:36 smithi049 ceph-mon[31696]: from='mgr.14100 172.21.15.49:0/1391868662' entity='mgr.x' 2024-02-26T06:11:39.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:38 smithi049 ceph-mon[31696]: mgrmap e3: x(active, since 1.02167s) 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T06:11:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T06:11:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-02-26T06:11:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T06:11:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T06:11:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T06:11:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T06:11:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T06:11:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T06:11:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T06:11:09.087418+0000", 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.706+0000 7fe05bc00700 1 Processor -- start 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.706+0000 7fe05bc00700 1 -- start start 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.707+0000 7fe05bc00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe054107a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.707+0000 7fe05bc00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe054108060 con 0x7fe054107670 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.707+0000 7fe059200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe054107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.707+0000 7fe059200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe054107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45846/0 (socket says 172.21.15.49:45846) 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.707+0000 7fe059200700 1 -- 172.21.15.49:0/1125082518 learned_addr learned my addr 172.21.15.49:0/1125082518 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.707+0000 7fe059200700 1 -- 172.21.15.49:0/1125082518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe054108820 con 0x7fe054107670 2024-02-26T06:11:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.708+0000 7fe059200700 1 --2- 172.21.15.49:0/1125082518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe054107a90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe0480098a0 tx=0x7fe048009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4228c1ed7b05e19d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.708+0000 7fe053400700 1 -- 172.21.15.49:0/1125082518 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe048004160 con 0x7fe054107670 2024-02-26T06:11:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.708+0000 7fe053400700 1 -- 172.21.15.49:0/1125082518 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0480042c0 con 0x7fe054107670 2024-02-26T06:11:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.708+0000 7fe053400700 1 -- 172.21.15.49:0/1125082518 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe048011ac0 con 0x7fe054107670 2024-02-26T06:11:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.709+0000 7fe05bc00700 1 -- 172.21.15.49:0/1125082518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 msgr2=0x7fe054107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.709+0000 7fe05bc00700 1 --2- 172.21.15.49:0/1125082518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe054107a90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe0480098a0 tx=0x7fe048009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.709+0000 7fe05bc00700 1 -- 172.21.15.49:0/1125082518 shutdown_connections 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.709+0000 7fe05bc00700 1 --2- 172.21.15.49:0/1125082518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe054107a90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.709+0000 7fe05bc00700 1 -- 172.21.15.49:0/1125082518 >> 172.21.15.49:0/1125082518 conn(0x7fe054102c70 msgr2=0x7fe054105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.709+0000 7fe05bc00700 1 -- 172.21.15.49:0/1125082518 shutdown_connections 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.709+0000 7fe05bc00700 1 -- 172.21.15.49:0/1125082518 wait complete. 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.710+0000 7fe05bc00700 1 Processor -- start 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.710+0000 7fe05bc00700 1 -- start start 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.710+0000 7fe05bc00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe05419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.710+0000 7fe05bc00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419b980 con 0x7fe054107670 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.711+0000 7fe059200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe05419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.711+0000 7fe059200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe05419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45860/0 (socket says 172.21.15.49:45860) 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.711+0000 7fe059200700 1 -- 172.21.15.49:0/4260776336 learned_addr learned my addr 172.21.15.49:0/4260776336 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.711+0000 7fe059200700 1 -- 172.21.15.49:0/4260776336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048009550 con 0x7fe054107670 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.711+0000 7fe059200700 1 --2- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe05419b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe048000c00 tx=0x7fe048011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.712+0000 7fe051600700 1 -- 172.21.15.49:0/4260776336 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe048003aa0 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.712+0000 7fe051600700 1 -- 172.21.15.49:0/4260776336 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe048003c00 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.712+0000 7fe051600700 1 -- 172.21.15.49:0/4260776336 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0480248a0 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.712+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05419bb80 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.712+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05419c020 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.713+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05410ba50 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.713+0000 7fe051600700 1 -- 172.21.15.49:0/4260776336 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 3) v1 ==== 45815+0+0 (secure 0 0 0) 0x7fe04801ac60 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.714+0000 7fe051600700 1 --2- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fe03c038dc0 0x7fe03c03b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.714+0000 7fe051600700 1 -- 172.21.15.49:0/4260776336 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe04804ca80 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.714+0000 7fe053e00700 1 --2- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fe03c038dc0 0x7fe03c03b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.715+0000 7fe053e00700 1 --2- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fe03c038dc0 0x7fe03c03b280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe040006fd0 tx=0x7fe040006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.720+0000 7fe051600700 1 -- 172.21.15.49:0/4260776336 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe04801a430 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:38.964+0000 7fe051600700 1 -- 172.21.15.49:0/4260776336 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7fe048017ca0 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.032+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe05410bc60 con 0x7fe054107670 2024-02-26T06:11:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.033+0000 7fe051600700 1 -- 172.21.15.49:0/4260776336 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fe048018920 con 0x7fe054107670 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.037+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fe03c038dc0 msgr2=0x7fe03c03b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.037+0000 7fe05bc00700 1 --2- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fe03c038dc0 0x7fe03c03b280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe040006fd0 tx=0x7fe040006e40 comp rx=0 tx=0).stop 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.037+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 msgr2=0x7fe05419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.037+0000 7fe05bc00700 1 --2- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe05419b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe048000c00 tx=0x7fe048011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.037+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 shutdown_connections 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.037+0000 7fe05bc00700 1 --2- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fe03c038dc0 0x7fe03c03b280 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.037+0000 7fe05bc00700 1 --2- 172.21.15.49:0/4260776336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe054107670 0x7fe05419b440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.037+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 >> 172.21.15.49:0/4260776336 conn(0x7fe054102c70 msgr2=0x7fe054103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.038+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 shutdown_connections 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:39.038+0000 7fe05bc00700 1 -- 172.21.15.49:0/4260776336 wait complete. 2024-02-26T06:11:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:mgr is available 2024-02-26T06:11:40.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:39 smithi049 ceph-mon[31696]: mgrmap e4: x(active, since 2s) 2024-02-26T06:11:40.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:39 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4260776336' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T06:11:41.162 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:40 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/406144192' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-02-26T06:11:41.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.281+0000 7fd4b3800700 1 Processor -- start 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.282+0000 7fd4b3800700 1 -- start start 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.282+0000 7fd4b3800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.282+0000 7fd4b3800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac07ae10 con 0x7fd4ac108180 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.283+0000 7fd4b0e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.283+0000 7fd4b0e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60956/0 (socket says 172.21.15.49:60956) 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.283+0000 7fd4b0e00700 1 -- 172.21.15.49:0/3605703040 learned_addr learned my addr 172.21.15.49:0/3605703040 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.283+0000 7fd4b0e00700 1 -- 172.21.15.49:0/3605703040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac07af50 con 0x7fd4ac108180 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.283+0000 7fd4b0e00700 1 --2- 172.21.15.49:0/3605703040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac10a570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009e80 tx=0x7fd4a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f546b762164bf60e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:41.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.284+0000 7fd4ab400700 1 -- 172.21.15.49:0/3605703040 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4a000e070 con 0x7fd4ac108180 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.284+0000 7fd4ab400700 1 -- 172.21.15.49:0/3605703040 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4a000b8c0 con 0x7fd4ac108180 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.284+0000 7fd4ab400700 1 -- 172.21.15.49:0/3605703040 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4a0004550 con 0x7fd4ac108180 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.284+0000 7fd4b3800700 1 -- 172.21.15.49:0/3605703040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 msgr2=0x7fd4ac10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.284+0000 7fd4b3800700 1 --2- 172.21.15.49:0/3605703040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac10a570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009e80 tx=0x7fd4a000b330 comp rx=0 tx=0).stop 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.285+0000 7fd4b3800700 1 -- 172.21.15.49:0/3605703040 shutdown_connections 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.285+0000 7fd4b3800700 1 --2- 172.21.15.49:0/3605703040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac10a570 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.285+0000 7fd4b3800700 1 -- 172.21.15.49:0/3605703040 >> 172.21.15.49:0/3605703040 conn(0x7fd4ac101b10 msgr2=0x7fd4ac103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.285+0000 7fd4b3800700 1 -- 172.21.15.49:0/3605703040 shutdown_connections 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.285+0000 7fd4b3800700 1 -- 172.21.15.49:0/3605703040 wait complete. 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.286+0000 7fd4b3800700 1 Processor -- start 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.286+0000 7fd4b3800700 1 -- start start 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.287+0000 7fd4b3800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac107a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:41.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.287+0000 7fd4b3800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac107f40 con 0x7fd4ac108180 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.287+0000 7fd4b0e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.287+0000 7fd4b0e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60964/0 (socket says 172.21.15.49:60964) 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.287+0000 7fd4b0e00700 1 -- 172.21.15.49:0/406144192 learned_addr learned my addr 172.21.15.49:0/406144192 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.287+0000 7fd4b0e00700 1 -- 172.21.15.49:0/406144192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0009b30 con 0x7fd4ac108180 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.287+0000 7fd4b0e00700 1 --2- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac107a00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009fd0 tx=0x7fd4a0003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.288+0000 7fd4a9600700 1 -- 172.21.15.49:0/406144192 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4a000e040 con 0x7fd4ac108180 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.288+0000 7fd4a9600700 1 -- 172.21.15.49:0/406144192 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4a00128f0 con 0x7fd4ac108180 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.288+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac106130 con 0x7fd4ac108180 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.288+0000 7fd4a9600700 1 -- 172.21.15.49:0/406144192 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4a001c890 con 0x7fd4ac108180 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.288+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac1065b0 con 0x7fd4ac108180 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.289+0000 7fd4a9600700 1 -- 172.21.15.49:0/406144192 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7fd4a0012a60 con 0x7fd4ac108180 2024-02-26T06:11:41.185 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.289+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac10b840 con 0x7fd4ac108180 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.290+0000 7fd4a9600700 1 --2- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fd494038ee0 0x7fd49403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.290+0000 7fd4a9600700 1 -- 172.21.15.49:0/406144192 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd4a004cf80 con 0x7fd4ac108180 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.290+0000 7fd4abe00700 1 --2- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fd494038ee0 0x7fd49403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.291+0000 7fd4abe00700 1 --2- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fd494038ee0 0x7fd49403b3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd498006fd0 tx=0x7fd498006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.295+0000 7fd4a9600700 1 -- 172.21.15.49:0/406144192 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4a0024c40 con 0x7fd4ac108180 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.517+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd4ac10ba50 con 0x7fd4ac108180 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.520+0000 7fd4a9600700 1 -- 172.21.15.49:0/406144192 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd4a0017020 con 0x7fd4ac108180 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.524+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fd494038ee0 msgr2=0x7fd49403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.524+0000 7fd4b3800700 1 --2- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fd494038ee0 0x7fd49403b3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd498006fd0 tx=0x7fd498006e40 comp rx=0 tx=0).stop 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.524+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 msgr2=0x7fd4ac107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.524+0000 7fd4b3800700 1 --2- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac107a00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009fd0 tx=0x7fd4a0003d70 comp rx=0 tx=0).stop 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.524+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 shutdown_connections 2024-02-26T06:11:41.186 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.524+0000 7fd4b3800700 1 --2- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fd494038ee0 0x7fd49403b3a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:41.187 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.524+0000 7fd4b3800700 1 --2- 172.21.15.49:0/406144192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4ac108180 0x7fd4ac107a00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:41.187 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.524+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 >> 172.21.15.49:0/406144192 conn(0x7fd4ac101b10 msgr2=0x7fd4ac102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:41.187 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.525+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 shutdown_connections 2024-02-26T06:11:41.187 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:40.525+0000 7fd4b3800700 1 -- 172.21.15.49:0/406144192 wait complete. 2024-02-26T06:11:41.187 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling cephadm module... 2024-02-26T06:11:42.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:42 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1779440281' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-26T06:11:42.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: ignoring --setuser ceph since I am not root 2024-02-26T06:11:42.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: ignoring --setgroup ceph since I am not root 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.781+0000 7f7c0cc00700 1 Processor -- start 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.781+0000 7f7c0cc00700 1 -- start start 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.781+0000 7f7c0cc00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c08109c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.781+0000 7f7c0cc00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0810a210 con 0x7f7c08109820 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.782+0000 7f7c05e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c08109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.782+0000 7f7c05e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c08109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60976/0 (socket says 172.21.15.49:60976) 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.782+0000 7f7c05e00700 1 -- 172.21.15.49:0/3199229829 learned_addr learned my addr 172.21.15.49:0/3199229829 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.783+0000 7f7c05e00700 1 -- 172.21.15.49:0/3199229829 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0810a9d0 con 0x7f7c08109820 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.783+0000 7f7c05e00700 1 --2- 172.21.15.49:0/3199229829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c08109c40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80098a0 tx=0x7f7bf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54da6937cafb6fd3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:42.851 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.783+0000 7f7c04a00700 1 -- 172.21.15.49:0/3199229829 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8004160 con 0x7f7c08109820 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c04a00700 1 -- 172.21.15.49:0/3199229829 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf80042c0 con 0x7f7c08109820 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c04a00700 1 -- 172.21.15.49:0/3199229829 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8011ac0 con 0x7f7c08109820 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c0cc00700 1 -- 172.21.15.49:0/3199229829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 msgr2=0x7f7c08109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c0cc00700 1 --2- 172.21.15.49:0/3199229829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c08109c40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80098a0 tx=0x7f7bf8009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c0cc00700 1 -- 172.21.15.49:0/3199229829 shutdown_connections 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c0cc00700 1 --2- 172.21.15.49:0/3199229829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c08109c40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c0cc00700 1 -- 172.21.15.49:0/3199229829 >> 172.21.15.49:0/3199229829 conn(0x7f7c0807ce30 msgr2=0x7f7c081074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c0cc00700 1 -- 172.21.15.49:0/3199229829 shutdown_connections 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.784+0000 7f7c0cc00700 1 -- 172.21.15.49:0/3199229829 wait complete. 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.786+0000 7f7c0cc00700 1 Processor -- start 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.786+0000 7f7c0cc00700 1 -- start start 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.786+0000 7f7c0cc00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c0819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.786+0000 7f7c0cc00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0819dad0 con 0x7f7c08109820 2024-02-26T06:11:42.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.786+0000 7f7c05e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c0819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:42.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.786+0000 7f7c05e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c0819d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60992/0 (socket says 172.21.15.49:60992) 2024-02-26T06:11:42.853 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.786+0000 7f7c05e00700 1 -- 172.21.15.49:0/1779440281 learned_addr learned my addr 172.21.15.49:0/1779440281 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:42.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.787+0000 7f7c05e00700 1 -- 172.21.15.49:0/1779440281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8009550 con 0x7f7c08109820 2024-02-26T06:11:42.855 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.787+0000 7f7c05e00700 1 --2- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c0819d590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000c00 tx=0x7f7bf8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.787+0000 7f7bfea00700 1 -- 172.21.15.49:0/1779440281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8003aa0 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.787+0000 7f7bfea00700 1 -- 172.21.15.49:0/1779440281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf8003c00 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.787+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0819dcd0 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.787+0000 7f7bfea00700 1 -- 172.21.15.49:0/1779440281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bf80248a0 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.788+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0819e170 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.789+0000 7f7bfea00700 1 -- 172.21.15.49:0/1779440281 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f7bf801ac60 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.789+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0810dc00 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.789+0000 7f7bfea00700 1 --2- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f7bec038ee0 0x7f7bec03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.789+0000 7f7bfea00700 1 -- 172.21.15.49:0/1779440281 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7bf804cca0 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.790+0000 7f7c05400700 1 --2- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f7bec038ee0 0x7f7bec03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.790+0000 7f7c05400700 1 --2- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f7bec038ee0 0x7f7bec03b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4006fd0 tx=0x7f7bf4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:41.795+0000 7f7bfea00700 1 -- 172.21.15.49:0/1779440281 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7bf8018530 con 0x7f7c08109820 2024-02-26T06:11:42.856 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.079+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f7c081a0f40 con 0x7f7c08109820 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.133+0000 7f7bfea00700 1 -- 172.21.15.49:0/1779440281 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f7bf8018350 con 0x7f7c08109820 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.136+0000 7f7bfea00700 1 -- 172.21.15.49:0/1779440281 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7bf802d770 con 0x7f7c08109820 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.137+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f7bec038ee0 msgr2=0x7f7bec03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.137+0000 7f7c0cc00700 1 --2- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f7bec038ee0 0x7f7bec03b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4006fd0 tx=0x7f7bf4006e40 comp rx=0 tx=0).stop 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.137+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 msgr2=0x7f7c0819d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.137+0000 7f7c0cc00700 1 --2- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c0819d590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000c00 tx=0x7f7bf8011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.138+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 shutdown_connections 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.138+0000 7f7c0cc00700 1 --2- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f7bec038ee0 0x7f7bec03b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.138+0000 7f7c0cc00700 1 --2- 172.21.15.49:0/1779440281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c08109820 0x7f7c0819d590 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.138+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 >> 172.21.15.49:0/1779440281 conn(0x7f7c0807ce30 msgr2=0x7f7c08106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.138+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 shutdown_connections 2024-02-26T06:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:42.138+0000 7f7c0cc00700 1 -- 172.21.15.49:0/1779440281 wait complete. 2024-02-26T06:11:43.107 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:42.977+0000 7fb5a6227200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:11:43.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:43 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1779440281' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-26T06:11:43.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:43 smithi049 ceph-mon[31696]: mgrmap e5: x(active, since 5s) 2024-02-26T06:11:43.389 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:43 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:43.107+0000 7fb5a6227200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:11:44.273 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:44 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1411601004' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-26T06:11:44.274 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:43 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:43.956+0000 7fb5a6227200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:11:44.274 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:44.085+0000 7fb5a6227200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:11:44.274 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:44.273+0000 7fb5a6227200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:11:44.452 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:11:44.452 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-26T06:11:44.452 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T06:11:44.452 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-26T06:11:44.452 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-26T06:11:44.452 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:11:44.452 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.461+0000 7f3bcda00700 1 Processor -- start 2024-02-26T06:11:44.452 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.461+0000 7f3bcda00700 1 -- start start 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.462+0000 7f3bcda00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc8107a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.462+0000 7f3bcda00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc8108060 con 0x7f3bc8107670 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.462+0000 7f3bc6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc8107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.462+0000 7f3bc6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc8107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32772/0 (socket says 172.21.15.49:32772) 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.462+0000 7f3bc6a00700 1 -- 172.21.15.49:0/3692534580 learned_addr learned my addr 172.21.15.49:0/3692534580 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.463+0000 7f3bc6a00700 1 -- 172.21.15.49:0/3692534580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8108820 con 0x7f3bc8107670 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.463+0000 7f3bc6a00700 1 --2- 172.21.15.49:0/3692534580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc8107a90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80098a0 tx=0x7f3bb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf816542923b5fff server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.463+0000 7f3bc5600700 1 -- 172.21.15.49:0/3692534580 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb8004160 con 0x7f3bc8107670 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.463+0000 7f3bc5600700 1 -- 172.21.15.49:0/3692534580 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bb80042c0 con 0x7f3bc8107670 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.463+0000 7f3bc5600700 1 -- 172.21.15.49:0/3692534580 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb8011ac0 con 0x7f3bc8107670 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.464+0000 7f3bcda00700 1 -- 172.21.15.49:0/3692534580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 msgr2=0x7f3bc8107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.464+0000 7f3bcda00700 1 --2- 172.21.15.49:0/3692534580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc8107a90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80098a0 tx=0x7f3bb8009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.464+0000 7f3bcda00700 1 -- 172.21.15.49:0/3692534580 shutdown_connections 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.464+0000 7f3bcda00700 1 --2- 172.21.15.49:0/3692534580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc8107a90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.464+0000 7f3bcda00700 1 -- 172.21.15.49:0/3692534580 >> 172.21.15.49:0/3692534580 conn(0x7f3bc8102c70 msgr2=0x7f3bc8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.464+0000 7f3bcda00700 1 -- 172.21.15.49:0/3692534580 shutdown_connections 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.464+0000 7f3bcda00700 1 -- 172.21.15.49:0/3692534580 wait complete. 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.465+0000 7f3bcda00700 1 Processor -- start 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.465+0000 7f3bcda00700 1 -- start start 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.466+0000 7f3bcda00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.466+0000 7f3bcda00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc819db80 con 0x7f3bc8107670 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.466+0000 7f3bc6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.466+0000 7f3bc6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32784/0 (socket says 172.21.15.49:32784) 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.466+0000 7f3bc6a00700 1 -- 172.21.15.49:0/1411601004 learned_addr learned my addr 172.21.15.49:0/1411601004 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.466+0000 7f3bc6a00700 1 -- 172.21.15.49:0/1411601004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8009550 con 0x7f3bc8107670 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.467+0000 7f3bc6a00700 1 --2- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc819d640 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000c00 tx=0x7f3bb8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.467+0000 7f3bbf400700 1 -- 172.21.15.49:0/1411601004 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb8003aa0 con 0x7f3bc8107670 2024-02-26T06:11:44.454 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.467+0000 7f3bbf400700 1 -- 172.21.15.49:0/1411601004 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bb8003c00 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.467+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc819dd80 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.467+0000 7f3bbf400700 1 -- 172.21.15.49:0/1411601004 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bb80248a0 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.467+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc819e220 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.468+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc810ba50 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.469+0000 7f3bbf400700 1 -- 172.21.15.49:0/1411601004 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f3bb802d3f0 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.469+0000 7f3bbf400700 1 --2- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f3bac038ee0 0x7f3bac03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.469+0000 7f3bbf400700 1 -- 172.21.15.49:0/1411601004 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3bb804ce90 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.469+0000 7f3bc6000700 1 -- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f3bac038ee0 msgr2=0x7f3bac03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/765116076 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.469+0000 7f3bc6000700 1 --2- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f3bac038ee0 0x7f3bac03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.474+0000 7f3bbf400700 1 -- 172.21.15.49:0/1411601004 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3bb8010380 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.670+0000 7f3bc6000700 1 -- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f3bac038ee0 msgr2=0x7f3bac03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/765116076 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.670+0000 7f3bc6000700 1 --2- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f3bac038ee0 0x7f3bac03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.752+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3bc81a0ff0 con 0x7f3bc8107670 2024-02-26T06:11:44.455 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.753+0000 7f3bbf400700 1 -- 172.21.15.49:0/1411601004 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3bb804e020 con 0x7f3bc8107670 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f3bac038ee0 msgr2=0x7f3bac03b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 --2- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f3bac038ee0 0x7f3bac03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 msgr2=0x7f3bc819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 --2- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc819d640 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000c00 tx=0x7f3bb8011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 shutdown_connections 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 --2- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7f3bac038ee0 0x7f3bac03b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 --2- 172.21.15.49:0/1411601004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc8107670 0x7f3bc819d640 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 >> 172.21.15.49:0/1411601004 conn(0x7f3bc8102c70 msgr2=0x7f3bc8103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 shutdown_connections 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:43.756+0000 7f3bcda00700 1 -- 172.21.15.49:0/1411601004 wait complete. 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-02-26T06:11:44.456 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 5... 2024-02-26T06:11:44.525 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:44.405+0000 7fb5a6227200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:11:44.525 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:44.525+0000 7fb5a6227200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:11:45.044 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:44.884+0000 7fb5a6227200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:11:45.044 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:45.038+0000 7fb5a6227200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:11:45.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:45.158+0000 7fb5a6227200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:11:45.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:45.287+0000 7fb5a6227200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:11:45.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:45.517+0000 7fb5a6227200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:11:47.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:46.861+0000 7fb5a6227200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:11:47.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:47.042+0000 7fb5a6227200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:11:47.870 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:47.478+0000 7fb5a6227200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:11:47.870 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:47.603+0000 7fb5a6227200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:11:48.127 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:47.870+0000 7fb5a6227200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:11:48.128 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:48.125+0000 7fb5a6227200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:11:48.850 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:48.518+0000 7fb5a6227200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:11:48.851 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:48.849+0000 7fb5a6227200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:11:49.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:49 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:49.166+0000 7fb5a6227200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:11:50.669 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:50.244+0000 7fb5a6227200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:11:51.261 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:50.975+0000 7fb5a6227200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:11:51.261 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:51.111+0000 7fb5a6227200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:11:51.669 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:11:51.407+0000 7fb5a6227200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:11:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: Active manager daemon x restarted 2024-02-26T06:11:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: Activating manager daemon x 2024-02-26T06:11:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: osdmap e2: 0 total, 0 up, 0 in 2024-02-26T06:11:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: mgrmap e6: x(active, starting, since 0.00650618s) 2024-02-26T06:11:51.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:11:51.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:11:51.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:11:51.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:11:51.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:11:51.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: Manager daemon x is now available 2024-02-26T06:11:51.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:51 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:11:52.784 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:52 smithi049 ceph-mon[31696]: Found migration_current of "None". Setting to last migration. 2024-02-26T06:11:52.784 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:52 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:11:52.785 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:52 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:11:52.785 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:52 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:11:52.785 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:52 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:11:52.785 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:52 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:11:52.785 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:52 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:11:52.785 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:52 smithi049 ceph-mon[31696]: mgrmap e7: x(active, since 1.01109s) 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.044+0000 7fa5c5400700 1 Processor -- start 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.044+0000 7fa5c5400700 1 -- start start 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.045+0000 7fa5c5400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c010a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.045+0000 7fa5c5400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c007adf0 con 0x7fa5c01081a0 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.045+0000 7fa5be800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c010a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.045+0000 7fa5be800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c010a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32800/0 (socket says 172.21.15.49:32800) 2024-02-26T06:11:53.143 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.045+0000 7fa5be800700 1 -- 172.21.15.49:0/2985384971 learned_addr learned my addr 172.21.15.49:0/2985384971 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.046+0000 7fa5be800700 1 -- 172.21.15.49:0/2985384971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c007af30 con 0x7fa5c01081a0 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.046+0000 7fa5be800700 1 --2- 172.21.15.49:0/2985384971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c010a590 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00098a0 tx=0x7fa5b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520111a6405866c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.046+0000 7fa5bd400700 1 -- 172.21.15.49:0/2985384971 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5b0004160 con 0x7fa5c01081a0 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.046+0000 7fa5bd400700 1 -- 172.21.15.49:0/2985384971 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5b00042c0 con 0x7fa5c01081a0 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.046+0000 7fa5bd400700 1 -- 172.21.15.49:0/2985384971 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5b0011ac0 con 0x7fa5c01081a0 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.047+0000 7fa5c5400700 1 -- 172.21.15.49:0/2985384971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 msgr2=0x7fa5c010a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.047+0000 7fa5c5400700 1 --2- 172.21.15.49:0/2985384971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c010a590 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00098a0 tx=0x7fa5b0009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.047+0000 7fa5c5400700 1 -- 172.21.15.49:0/2985384971 shutdown_connections 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.047+0000 7fa5c5400700 1 --2- 172.21.15.49:0/2985384971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c010a590 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.047+0000 7fa5c5400700 1 -- 172.21.15.49:0/2985384971 >> 172.21.15.49:0/2985384971 conn(0x7fa5c0101b10 msgr2=0x7fa5c0103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.048+0000 7fa5c5400700 1 -- 172.21.15.49:0/2985384971 shutdown_connections 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.048+0000 7fa5c5400700 1 -- 172.21.15.49:0/2985384971 wait complete. 2024-02-26T06:11:53.144 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.049+0000 7fa5c5400700 1 Processor -- start 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.049+0000 7fa5c5400700 1 -- start start 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.049+0000 7fa5c5400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.049+0000 7fa5c5400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c019b920 con 0x7fa5c01081a0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.049+0000 7fa5be800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.049+0000 7fa5be800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32816/0 (socket says 172.21.15.49:32816) 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.049+0000 7fa5be800700 1 -- 172.21.15.49:0/3482894124 learned_addr learned my addr 172.21.15.49:0/3482894124 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.050+0000 7fa5be800700 1 -- 172.21.15.49:0/3482894124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0009550 con 0x7fa5c01081a0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.050+0000 7fa5be800700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c019b3e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009130 tx=0x7fa5b0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.050+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5b0003aa0 con 0x7fa5c01081a0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.050+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5b0003c00 con 0x7fa5c01081a0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.050+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c019bb20 con 0x7fa5c01081a0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.051+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5b0024800 con 0x7fa5c01081a0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.051+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c019bfc0 con 0x7fa5c01081a0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.052+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fa5b001a430 con 0x7fa5c01081a0 2024-02-26T06:11:53.145 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.052+0000 7fa5b7400700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.052+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 --> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5a403bab0 con 0x7fa5a4038ee0 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.052+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa5b004cdb0 con 0x7fa5c01081a0 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.052+0000 7fa5bde00700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 msgr2=0x7fa5a403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/765116076 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.052+0000 7fa5bde00700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.253+0000 7fa5bde00700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 msgr2=0x7fa5a403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/765116076 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.253+0000 7fa5bde00700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.654+0000 7fa5bde00700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 msgr2=0x7fa5a403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/765116076 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:45.654+0000 7fa5bde00700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:46.455+0000 7fa5bde00700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 msgr2=0x7fa5a403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/765116076 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:46.455+0000 7fa5bde00700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:48.057+0000 7fa5bde00700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 msgr2=0x7fa5a403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/765116076 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:48.057+0000 7fa5bde00700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T06:11:53.146 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:51.260+0000 7fa5bde00700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 msgr2=0x7fa5a403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/765116076 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:51.260+0000 7fa5bde00700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:51.415+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7fa5b0024ac0 con 0x7fa5c01081a0 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:51.415+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 msgr2=0x7fa5a403b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:51.415+0000 7fa5b7400700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.419+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fa5b004e990 con 0x7fa5c01081a0 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.419+0000 7fa5b7400700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fa5a403c7d0 0x7fa5a403ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.420+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5a403bab0 con 0x7fa5a403c7d0 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.421+0000 7fa5bde00700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fa5a403c7d0 0x7fa5a403ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.425+0000 7fa5bde00700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fa5a403c7d0 0x7fa5a403ebc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac003a10 tx=0x7fa5ac009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.427+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mgr.14120 v2:172.21.15.49:6800/4089389406 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fa5a403bab0 con 0x7fa5a403c7d0 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.440+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa5c010b860 con 0x7fa5a403c7d0 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.440+0000 7fa5b7400700 1 -- 172.21.15.49:0/3482894124 <== mgr.14120 v2:172.21.15.49:6800/4089389406 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa5c010b860 con 0x7fa5a403c7d0 2024-02-26T06:11:53.147 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.441+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fa5a403c7d0 msgr2=0x7fa5a403ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.441+0000 7fa5c5400700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fa5a403c7d0 0x7fa5a403ebc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac003a10 tx=0x7fa5ac009290 comp rx=0 tx=0).stop 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.441+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 msgr2=0x7fa5c019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.441+0000 7fa5c5400700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c019b3e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009130 tx=0x7fa5b0011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.442+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 shutdown_connections 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.442+0000 7fa5c5400700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fa5a403c7d0 0x7fa5a403ebc0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.442+0000 7fa5c5400700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c01081a0 0x7fa5c019b3e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.442+0000 7fa5c5400700 1 --2- 172.21.15.49:0/3482894124 >> [v2:172.21.15.49:6800/765116076,v1:172.21.15.49:6801/765116076] conn(0x7fa5a4038ee0 0x7fa5a403b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.442+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 >> 172.21.15.49:0/3482894124 conn(0x7fa5c0101b10 msgr2=0x7fa5c0102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.442+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 shutdown_connections 2024-02-26T06:11:53.148 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:52.442+0000 7fa5c5400700 1 -- 172.21.15.49:0/3482894124 wait complete. 2024-02-26T06:11:53.149 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 5 is available 2024-02-26T06:11:53.149 INFO:teuthology.orchestra.run.smithi049.stdout:Setting orchestrator backend to cephadm... 2024-02-26T06:11:53.743 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:53 smithi049 ceph-mon[31696]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-26T06:11:54.828 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:53 smithi049 ceph-mon[31696]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-26T06:11:54.828 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:53 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:11:54.828 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:53 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:11:54.830 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: [26/Feb/2024:06:11:53] ENGINE Bus STARTING 2024-02-26T06:11:54.830 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: [26/Feb/2024:06:11:53] ENGINE Serving on https://172.21.15.49:7150 2024-02-26T06:11:54.830 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: mgrmap e8: x(active, since 2s) 2024-02-26T06:11:54.830 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: [26/Feb/2024:06:11:53] ENGINE Serving on http://172.21.15.49:8765 2024-02-26T06:11:54.830 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: [26/Feb/2024:06:11:53] ENGINE Bus STARTED 2024-02-26T06:11:54.830 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:11:54.830 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:11:54.831 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:11:54.831 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:54 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:11:54.841 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.833+0000 7f0d23000700 1 Processor -- start 2024-02-26T06:11:54.841 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.834+0000 7f0d23000700 1 -- start start 2024-02-26T06:11:54.841 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.834+0000 7f0d23000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:54.841 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.834+0000 7f0d23000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c1080e0 con 0x7f0d1c1076f0 2024-02-26T06:11:54.841 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.835+0000 7f0d1be00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:54.842 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.835+0000 7f0d1be00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33232/0 (socket says 172.21.15.49:33232) 2024-02-26T06:11:54.842 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.835+0000 7f0d1be00700 1 -- 172.21.15.49:0/3705784591 learned_addr learned my addr 172.21.15.49:0/3705784591 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:54.842 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.835+0000 7f0d1be00700 1 -- 172.21.15.49:0/3705784591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c1088a0 con 0x7f0d1c1076f0 2024-02-26T06:11:54.842 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.836+0000 7f0d1be00700 1 --2- 172.21.15.49:0/3705784591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c107b10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009e80 tx=0x7f0d0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9422c96e1f23e74a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:54.842 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.836+0000 7f0d1b400700 1 -- 172.21.15.49:0/3705784591 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d0c00e070 con 0x7f0d1c1076f0 2024-02-26T06:11:54.842 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.836+0000 7f0d1b400700 1 -- 172.21.15.49:0/3705784591 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d0c00b8c0 con 0x7f0d1c1076f0 2024-02-26T06:11:54.843 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.836+0000 7f0d1b400700 1 -- 172.21.15.49:0/3705784591 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d0c004550 con 0x7f0d1c1076f0 2024-02-26T06:11:54.843 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.837+0000 7f0d23000700 1 -- 172.21.15.49:0/3705784591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 msgr2=0x7f0d1c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:54.843 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.837+0000 7f0d23000700 1 --2- 172.21.15.49:0/3705784591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c107b10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009e80 tx=0x7f0d0c00b330 comp rx=0 tx=0).stop 2024-02-26T06:11:54.843 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.837+0000 7f0d23000700 1 -- 172.21.15.49:0/3705784591 shutdown_connections 2024-02-26T06:11:54.843 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.837+0000 7f0d23000700 1 --2- 172.21.15.49:0/3705784591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c107b10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:54.843 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.837+0000 7f0d23000700 1 -- 172.21.15.49:0/3705784591 >> 172.21.15.49:0/3705784591 conn(0x7f0d1c102cd0 msgr2=0x7f0d1c1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:54.843 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.837+0000 7f0d23000700 1 -- 172.21.15.49:0/3705784591 shutdown_connections 2024-02-26T06:11:54.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.837+0000 7f0d23000700 1 -- 172.21.15.49:0/3705784591 wait complete. 2024-02-26T06:11:54.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.838+0000 7f0d23000700 1 Processor -- start 2024-02-26T06:11:54.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.839+0000 7f0d23000700 1 -- start start 2024-02-26T06:11:54.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.839+0000 7f0d23000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:54.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.839+0000 7f0d23000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c19db30 con 0x7f0d1c1076f0 2024-02-26T06:11:54.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.839+0000 7f0d1be00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:54.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.839+0000 7f0d1be00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c19d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33240/0 (socket says 172.21.15.49:33240) 2024-02-26T06:11:54.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.839+0000 7f0d1be00700 1 -- 172.21.15.49:0/1848095364 learned_addr learned my addr 172.21.15.49:0/1848095364 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:54.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.840+0000 7f0d1be00700 1 -- 172.21.15.49:0/1848095364 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c009b30 con 0x7f0d1c1076f0 2024-02-26T06:11:54.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.840+0000 7f0d1be00700 1 --2- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c19d5f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009fd0 tx=0x7f0d0c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:54.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.840+0000 7f0d19600700 1 -- 172.21.15.49:0/1848095364 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d0c00e040 con 0x7f0d1c1076f0 2024-02-26T06:11:54.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.840+0000 7f0d19600700 1 -- 172.21.15.49:0/1848095364 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d0c0128f0 con 0x7f0d1c1076f0 2024-02-26T06:11:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.840+0000 7f0d19600700 1 -- 172.21.15.49:0/1848095364 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d0c01c770 con 0x7f0d1c1076f0 2024-02-26T06:11:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.840+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c19dd30 con 0x7f0d1c1076f0 2024-02-26T06:11:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.840+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c19e1b0 con 0x7f0d1c1076f0 2024-02-26T06:11:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.841+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c10bad0 con 0x7f0d1c1076f0 2024-02-26T06:11:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.842+0000 7f0d19600700 1 -- 172.21.15.49:0/1848095364 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f0d0c012a60 con 0x7f0d1c1076f0 2024-02-26T06:11:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.842+0000 7f0d19600700 1 --2- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0d00038f30 0x7f0d0003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.842+0000 7f0d19600700 1 -- 172.21.15.49:0/1848095364 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0d0c04cfa0 con 0x7f0d1c1076f0 2024-02-26T06:11:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.842+0000 7f0d13400700 1 --2- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0d00038f30 0x7f0d0003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.843+0000 7f0d13400700 1 --2- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0d00038f30 0x7f0d0003b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d04006fd0 tx=0x7f0d04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:53.848+0000 7f0d19600700 1 -- 172.21.15.49:0/1848095364 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d0c024da0 con 0x7f0d1c1076f0 2024-02-26T06:11:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.090+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d1c103e60 con 0x7f0d00038f30 2024-02-26T06:11:54.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.102+0000 7f0d19600700 1 -- 172.21.15.49:0/1848095364 <== mgr.14120 v2:172.21.15.49:6800/4089389406 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0d1c103e60 con 0x7f0d00038f30 2024-02-26T06:11:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.104+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0d00038f30 msgr2=0x7f0d0003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.104+0000 7f0d23000700 1 --2- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0d00038f30 0x7f0d0003b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d04006fd0 tx=0x7f0d04006e40 comp rx=0 tx=0).stop 2024-02-26T06:11:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.105+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 msgr2=0x7f0d1c19d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.105+0000 7f0d23000700 1 --2- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c19d5f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009fd0 tx=0x7f0d0c003d00 comp rx=0 tx=0).stop 2024-02-26T06:11:54.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.105+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 shutdown_connections 2024-02-26T06:11:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.105+0000 7f0d23000700 1 --2- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0d00038f30 0x7f0d0003b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.105+0000 7f0d23000700 1 --2- 172.21.15.49:0/1848095364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d1c1076f0 0x7f0d1c19d5f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.105+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 >> 172.21.15.49:0/1848095364 conn(0x7f0d1c102cd0 msgr2=0x7f0d1c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:54.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.105+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 shutdown_connections 2024-02-26T06:11:54.850 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:54.105+0000 7f0d23000700 1 -- 172.21.15.49:0/1848095364 wait complete. 2024-02-26T06:11:56.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-26T06:11:56.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.380+0000 7fe24f400700 1 Processor -- start 2024-02-26T06:11:56.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.381+0000 7fe24f400700 1 -- start start 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.381+0000 7fe24f400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe248109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.381+0000 7fe24f400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24810a260 con 0x7fe248109870 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.382+0000 7fe24ca00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe248109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.382+0000 7fe24ca00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe248109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33254/0 (socket says 172.21.15.49:33254) 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.382+0000 7fe24ca00700 1 -- 172.21.15.49:0/3789387626 learned_addr learned my addr 172.21.15.49:0/3789387626 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.382+0000 7fe24ca00700 1 -- 172.21.15.49:0/3789387626 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24810aa20 con 0x7fe248109870 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.383+0000 7fe24ca00700 1 --2- 172.21.15.49:0/3789387626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe248109c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0098a0 tx=0x7fe23c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6b14dc60a34c4e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.383+0000 7fe247400700 1 -- 172.21.15.49:0/3789387626 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe23c004160 con 0x7fe248109870 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.383+0000 7fe247400700 1 -- 172.21.15.49:0/3789387626 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe23c0042c0 con 0x7fe248109870 2024-02-26T06:11:56.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.383+0000 7fe247400700 1 -- 172.21.15.49:0/3789387626 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe23c011ac0 con 0x7fe248109870 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.384+0000 7fe24f400700 1 -- 172.21.15.49:0/3789387626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 msgr2=0x7fe248109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.384+0000 7fe24f400700 1 --2- 172.21.15.49:0/3789387626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe248109c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0098a0 tx=0x7fe23c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.384+0000 7fe24f400700 1 -- 172.21.15.49:0/3789387626 shutdown_connections 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.384+0000 7fe24f400700 1 --2- 172.21.15.49:0/3789387626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe248109c90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.384+0000 7fe24f400700 1 -- 172.21.15.49:0/3789387626 >> 172.21.15.49:0/3789387626 conn(0x7fe24807ce30 msgr2=0x7fe2481074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.384+0000 7fe24f400700 1 -- 172.21.15.49:0/3789387626 shutdown_connections 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.384+0000 7fe24f400700 1 -- 172.21.15.49:0/3789387626 wait complete. 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.385+0000 7fe24f400700 1 Processor -- start 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.386+0000 7fe24f400700 1 -- start start 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.386+0000 7fe24f400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe24819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.386+0000 7fe24f400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24819db40 con 0x7fe248109870 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.386+0000 7fe24ca00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe24819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:56.294 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.386+0000 7fe24ca00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe24819d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33258/0 (socket says 172.21.15.49:33258) 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.386+0000 7fe24ca00700 1 -- 172.21.15.49:0/2143914980 learned_addr learned my addr 172.21.15.49:0/2143914980 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.387+0000 7fe24ca00700 1 -- 172.21.15.49:0/2143914980 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c009550 con 0x7fe248109870 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.387+0000 7fe24ca00700 1 --2- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe24819d600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe23c000c00 tx=0x7fe23c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.387+0000 7fe245600700 1 -- 172.21.15.49:0/2143914980 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe23c003aa0 con 0x7fe248109870 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.387+0000 7fe245600700 1 -- 172.21.15.49:0/2143914980 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe23c003c00 con 0x7fe248109870 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.387+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24819dd40 con 0x7fe248109870 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.387+0000 7fe245600700 1 -- 172.21.15.49:0/2143914980 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe23c0248a0 con 0x7fe248109870 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.388+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24819e1e0 con 0x7fe248109870 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.389+0000 7fe245600700 1 -- 172.21.15.49:0/2143914980 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fe23c01ac60 con 0x7fe248109870 2024-02-26T06:11:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.389+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24810dc50 con 0x7fe248109870 2024-02-26T06:11:56.297 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.389+0000 7fe245600700 1 --2- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fe230038ee0 0x7fe23003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:56.297 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.389+0000 7fe245600700 1 -- 172.21.15.49:0/2143914980 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe23c04d180 con 0x7fe248109870 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.390+0000 7fe247e00700 1 --2- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fe230038ee0 0x7fe23003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.390+0000 7fe247e00700 1 --2- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fe230038ee0 0x7fe23003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe234006fd0 tx=0x7fe234006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.395+0000 7fe245600700 1 -- 172.21.15.49:0/2143914980 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe23c018530 con 0x7fe248109870 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.618+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fe248107300 con 0x7fe230038ee0 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.620+0000 7fe245600700 1 -- 172.21.15.49:0/2143914980 <== mgr.14120 v2:172.21.15.49:6800/4089389406 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fe248107300 con 0x7fe230038ee0 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.623+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fe230038ee0 msgr2=0x7fe23003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.623+0000 7fe24f400700 1 --2- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fe230038ee0 0x7fe23003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe234006fd0 tx=0x7fe234006e40 comp rx=0 tx=0).stop 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.623+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 msgr2=0x7fe24819d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.623+0000 7fe24f400700 1 --2- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe24819d600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe23c000c00 tx=0x7fe23c011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.623+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 shutdown_connections 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.623+0000 7fe24f400700 1 --2- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fe230038ee0 0x7fe23003b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.623+0000 7fe24f400700 1 --2- 172.21.15.49:0/2143914980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe248109870 0x7fe24819d600 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.623+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 >> 172.21.15.49:0/2143914980 conn(0x7fe24807ce30 msgr2=0x7fe248106bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:56.299 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.624+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 shutdown_connections 2024-02-26T06:11:56.299 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:55.624+0000 7fe24f400700 1 -- 172.21.15.49:0/2143914980 wait complete. 2024-02-26T06:11:56.299 INFO:teuthology.orchestra.run.smithi049.stdout:Generating ssh key... 2024-02-26T06:11:56.546 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:56 smithi049 ceph-mon[31696]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:11:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-mon[31696]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:11:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-mon[31696]: Generating ssh key... 2024-02-26T06:11:58.313 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: Generating public/private rsa key pair. 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: Your identification has been saved in /tmp/tmppvygbs0c/key. 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: Your public key has been saved in /tmp/tmppvygbs0c/key.pub. 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: The key fingerprint is: 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: SHA256:b62Dq9EyhBA46fG/OXVvdAagUnJc9TTHYnqSVvXNlQI ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: The key's randomart image is: 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: +---[RSA 3072]----+ 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: | o. . ...E.oooo| 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: |+. . . + . o=o++| 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: |..+ + . . =.o +| 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: | . o o . * . | 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: | o o S . + | 2024-02-26T06:11:58.314 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: | o o o o o | 2024-02-26T06:11:58.315 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: | B o.= + | 2024-02-26T06:11:58.315 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: | + +...+ | 2024-02-26T06:11:58.315 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: | o...o. | 2024-02-26T06:11:58.315 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:11:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: +----[SHA256]-----+ 2024-02-26T06:11:58.645 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.872+0000 7fc92e000700 1 Processor -- start 2024-02-26T06:11:58.646 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.873+0000 7fc92e000700 1 -- start start 2024-02-26T06:11:58.646 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.873+0000 7fc92e000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc928107a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:58.646 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.873+0000 7fc92e000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc928108030 con 0x7fc928107640 2024-02-26T06:11:58.646 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.874+0000 7fc927400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc928107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:58.646 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.874+0000 7fc927400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc928107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33270/0 (socket says 172.21.15.49:33270) 2024-02-26T06:11:58.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.874+0000 7fc927400700 1 -- 172.21.15.49:0/1507186945 learned_addr learned my addr 172.21.15.49:0/1507186945 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:58.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.874+0000 7fc927400700 1 -- 172.21.15.49:0/1507186945 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9281087f0 con 0x7fc928107640 2024-02-26T06:11:58.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.875+0000 7fc927400700 1 --2- 172.21.15.49:0/1507186945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc928107a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9180098a0 tx=0x7fc918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38e4f8439d68f2c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:58.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.875+0000 7fc926000700 1 -- 172.21.15.49:0/1507186945 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc918004160 con 0x7fc928107640 2024-02-26T06:11:58.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.875+0000 7fc926000700 1 -- 172.21.15.49:0/1507186945 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9180042c0 con 0x7fc928107640 2024-02-26T06:11:58.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.875+0000 7fc926000700 1 -- 172.21.15.49:0/1507186945 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc918011ac0 con 0x7fc928107640 2024-02-26T06:11:58.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.876+0000 7fc92e000700 1 -- 172.21.15.49:0/1507186945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 msgr2=0x7fc928107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:58.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.876+0000 7fc92e000700 1 --2- 172.21.15.49:0/1507186945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc928107a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9180098a0 tx=0x7fc918009bb0 comp rx=0 tx=0).stop 2024-02-26T06:11:58.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.876+0000 7fc92e000700 1 -- 172.21.15.49:0/1507186945 shutdown_connections 2024-02-26T06:11:58.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.876+0000 7fc92e000700 1 --2- 172.21.15.49:0/1507186945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc928107a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:58.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.876+0000 7fc92e000700 1 -- 172.21.15.49:0/1507186945 >> 172.21.15.49:0/1507186945 conn(0x7fc928102c40 msgr2=0x7fc928105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:58.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.876+0000 7fc92e000700 1 -- 172.21.15.49:0/1507186945 shutdown_connections 2024-02-26T06:11:58.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.876+0000 7fc92e000700 1 -- 172.21.15.49:0/1507186945 wait complete. 2024-02-26T06:11:58.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.877+0000 7fc92e000700 1 Processor -- start 2024-02-26T06:11:58.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.877+0000 7fc92e000700 1 -- start start 2024-02-26T06:11:58.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.878+0000 7fc92e000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc92819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:58.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.878+0000 7fc92e000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92819b940 con 0x7fc928107640 2024-02-26T06:11:58.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.878+0000 7fc927400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc92819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:58.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.878+0000 7fc927400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc92819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33282/0 (socket says 172.21.15.49:33282) 2024-02-26T06:11:58.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.878+0000 7fc927400700 1 -- 172.21.15.49:0/2624152802 learned_addr learned my addr 172.21.15.49:0/2624152802 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:11:58.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.878+0000 7fc927400700 1 -- 172.21.15.49:0/2624152802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc918009550 con 0x7fc928107640 2024-02-26T06:11:58.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.879+0000 7fc927400700 1 --2- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc92819b400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc918011780 tx=0x7fc918011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:58.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.879+0000 7fc91fe00700 1 -- 172.21.15.49:0/2624152802 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc918003af0 con 0x7fc928107640 2024-02-26T06:11:58.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.879+0000 7fc91fe00700 1 -- 172.21.15.49:0/2624152802 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc918003c50 con 0x7fc928107640 2024-02-26T06:11:58.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.879+0000 7fc91fe00700 1 -- 172.21.15.49:0/2624152802 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9180197e0 con 0x7fc928107640 2024-02-26T06:11:58.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.879+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92819bb40 con 0x7fc928107640 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.879+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92819bfe0 con 0x7fc928107640 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.880+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92810ba20 con 0x7fc928107640 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.880+0000 7fc91fe00700 1 -- 172.21.15.49:0/2624152802 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fc91801aba0 con 0x7fc928107640 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.881+0000 7fc91fe00700 1 --2- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc90c038ee0 0x7fc90c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.881+0000 7fc91fe00700 1 -- 172.21.15.49:0/2624152802 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc91804d3a0 con 0x7fc928107640 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.881+0000 7fc926a00700 1 --2- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc90c038ee0 0x7fc90c03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.882+0000 7fc926a00700 1 --2- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc90c038ee0 0x7fc90c03b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc914006fd0 tx=0x7fc914006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:56.886+0000 7fc91fe00700 1 -- 172.21.15.49:0/2624152802 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc91801a430 con 0x7fc928107640 2024-02-26T06:11:58.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.110+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc928103f00 con 0x7fc90c038ee0 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.962+0000 7fc91fe00700 1 -- 172.21.15.49:0/2624152802 <== mgr.14120 v2:172.21.15.49:6800/4089389406 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc928103f00 con 0x7fc90c038ee0 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.965+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc90c038ee0 msgr2=0x7fc90c03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 --2- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc90c038ee0 0x7fc90c03b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc914006fd0 tx=0x7fc914006e40 comp rx=0 tx=0).stop 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 msgr2=0x7fc92819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 --2- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc92819b400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc918011780 tx=0x7fc918011b40 comp rx=0 tx=0).stop 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 shutdown_connections 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 --2- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc90c038ee0 0x7fc90c03b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 --2- 172.21.15.49:0/2624152802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc928107640 0x7fc92819b400 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 >> 172.21.15.49:0/2624152802 conn(0x7fc928102c40 msgr2=0x7fc9281037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 shutdown_connections 2024-02-26T06:11:58.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:57.966+0000 7fc92e000700 1 -- 172.21.15.49:0/2624152802 wait complete. 2024-02-26T06:11:59.249 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:58 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:11:59.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:11:58 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:12:00.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI6LUyyf7R5ACDWK/pSEdOTtIjYTo69zD8crQYolIeoAU+ad1SaSmQ+ij0SSeZ9nhsgMa3fwNWFfuzwMWigdnsNyI48ZCPgQOKHBzOOiRzSZp8EOllfUFWgc7Ip1tHr1RxVivjG+YW30Wk60iSIzTyg6jWChsd3/QPEDhk+YvxpX4O79/1HDdvMd63AyYo6106tJlxjc8zuhkRhOa/K6yPJGv1dp21+T0m10EUdlzK2PbaeunZRU9Bawlfl6V1aJSrtXoHfSty2X43HrNg26WdqtI4qAiKGd8K4KGUGq4bos6iB68s0kzmAnNnce69zNX7lrGpOSn1N6RgzvdlmXvYq2XYtKDqkf3pnzu4JlKQBvmBIb65y2TmVlMc9tNvZVuZLHmwoY1+TmmOlz2x623tGKUal2fr0khpr/VVSkx83rQLDD4FRdtfyhLzT0PsTg/E2QNtYRGl4QPjs4qWqD8oHNcE/isMM6lm2//tW7YLnLWSRQsN7GBl+fCe6hdZyc8= ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:12:00.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.249+0000 7fb077800700 1 Processor -- start 2024-02-26T06:12:00.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.249+0000 7fb077800700 1 -- start start 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.249+0000 7fb077800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb070106350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.249+0000 7fb077800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07007ae50 con 0x7fb070105f30 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.250+0000 7fb074e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb070106350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.250+0000 7fb074e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb070106350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33284/0 (socket says 172.21.15.49:33284) 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.250+0000 7fb074e00700 1 -- 172.21.15.49:0/3081417280 learned_addr learned my addr 172.21.15.49:0/3081417280 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.250+0000 7fb074e00700 1 -- 172.21.15.49:0/3081417280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb070106890 con 0x7fb070105f30 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.251+0000 7fb074e00700 1 --2- 172.21.15.49:0/3081417280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb070106350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb0640098a0 tx=0x7fb064009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14c250636f788c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.251+0000 7fb06f400700 1 -- 172.21.15.49:0/3081417280 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb064004160 con 0x7fb070105f30 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.251+0000 7fb06f400700 1 -- 172.21.15.49:0/3081417280 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0640042c0 con 0x7fb070105f30 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.251+0000 7fb06f400700 1 -- 172.21.15.49:0/3081417280 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb064011ac0 con 0x7fb070105f30 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.251+0000 7fb077800700 1 -- 172.21.15.49:0/3081417280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 msgr2=0x7fb070106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.251+0000 7fb077800700 1 --2- 172.21.15.49:0/3081417280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb070106350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb0640098a0 tx=0x7fb064009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.252+0000 7fb077800700 1 -- 172.21.15.49:0/3081417280 shutdown_connections 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.252+0000 7fb077800700 1 --2- 172.21.15.49:0/3081417280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb070106350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.252+0000 7fb077800700 1 -- 172.21.15.49:0/3081417280 >> 172.21.15.49:0/3081417280 conn(0x7fb070101b10 msgr2=0x7fb070103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.252+0000 7fb077800700 1 -- 172.21.15.49:0/3081417280 shutdown_connections 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.252+0000 7fb077800700 1 -- 172.21.15.49:0/3081417280 wait complete. 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.253+0000 7fb077800700 1 Processor -- start 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.254+0000 7fb077800700 1 -- start start 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.254+0000 7fb077800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb07019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.254+0000 7fb077800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07019db00 con 0x7fb070105f30 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.254+0000 7fb074e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb07019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.254+0000 7fb074e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb07019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33296/0 (socket says 172.21.15.49:33296) 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.254+0000 7fb074e00700 1 -- 172.21.15.49:0/869574006 learned_addr learned my addr 172.21.15.49:0/869574006 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.255+0000 7fb074e00700 1 -- 172.21.15.49:0/869574006 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb064009550 con 0x7fb070105f30 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.255+0000 7fb074e00700 1 --2- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb07019d5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb064009130 tx=0x7fb064011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.255+0000 7fb06d600700 1 -- 172.21.15.49:0/869574006 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb064003aa0 con 0x7fb070105f30 2024-02-26T06:12:00.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.256+0000 7fb06d600700 1 -- 172.21.15.49:0/869574006 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb064003c00 con 0x7fb070105f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.256+0000 7fb06d600700 1 -- 172.21.15.49:0/869574006 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb064024800 con 0x7fb070105f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.256+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07019dd00 con 0x7fb070105f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.256+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07019e1a0 con 0x7fb070105f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.257+0000 7fb06d600700 1 -- 172.21.15.49:0/869574006 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fb06401a430 con 0x7fb070105f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.257+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb070197260 con 0x7fb070105f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.258+0000 7fb06d600700 1 --2- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fb058038f30 0x7fb05803b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.258+0000 7fb06d600700 1 -- 172.21.15.49:0/869574006 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb06404d150 con 0x7fb070105f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.258+0000 7fb06fe00700 1 --2- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fb058038f30 0x7fb05803b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.259+0000 7fb06fe00700 1 --2- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fb058038f30 0x7fb05803b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb05c006fd0 tx=0x7fb05c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.263+0000 7fb06d600700 1 -- 172.21.15.49:0/869574006 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb06401abd0 con 0x7fb070105f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.477+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb07002da30 con 0x7fb058038f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.478+0000 7fb06d600700 1 -- 172.21.15.49:0/869574006 <== mgr.14120 v2:172.21.15.49:6800/4089389406 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fb07002da30 con 0x7fb058038f30 2024-02-26T06:12:00.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fb058038f30 msgr2=0x7fb05803b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 --2- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fb058038f30 0x7fb05803b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb05c006fd0 tx=0x7fb05c006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 msgr2=0x7fb07019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 --2- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb07019d5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb064009130 tx=0x7fb064011b40 comp rx=0 tx=0).stop 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 shutdown_connections 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 --2- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fb058038f30 0x7fb05803b3f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 --2- 172.21.15.49:0/869574006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb070105f30 0x7fb07019d5c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 >> 172.21.15.49:0/869574006 conn(0x7fb070101b10 msgr2=0x7fb0701943a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 shutdown_connections 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:11:59.481+0000 7fb077800700 1 -- 172.21.15.49:0/869574006 wait complete. 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-26T06:12:00.208 INFO:teuthology.orchestra.run.smithi049.stdout:Adding key to root@localhost authorized_keys... 2024-02-26T06:12:00.209 INFO:teuthology.orchestra.run.smithi049.stdout:Adding host smithi049... 2024-02-26T06:12:00.532 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:00 smithi049 ceph-mon[31696]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:12:01.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:01 smithi049 ceph-mon[31696]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:12:03.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:03 smithi049 ceph-mon[31696]: Deploying cephadm binary to smithi049 2024-02-26T06:12:08.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Added host 'smithi049' with addr '172.21.15.49' 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.832+0000 7f411ba00700 1 Processor -- start 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.833+0000 7f411ba00700 1 -- start start 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.833+0000 7f411ba00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f4114107a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.833+0000 7f411ba00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4114108060 con 0x7f4114107670 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.834+0000 7f4119000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f4114107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.834+0000 7f4119000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f4114107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45392/0 (socket says 172.21.15.49:45392) 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.834+0000 7f4119000700 1 -- 172.21.15.49:0/240077955 learned_addr learned my addr 172.21.15.49:0/240077955 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.834+0000 7f4119000700 1 -- 172.21.15.49:0/240077955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114108820 con 0x7f4114107670 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.835+0000 7f4119000700 1 --2- 172.21.15.49:0/240077955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f4114107a90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4104009e80 tx=0x7f410400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520c12bc65153abf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.835+0000 7f4113e00700 1 -- 172.21.15.49:0/240077955 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f410400e070 con 0x7f4114107670 2024-02-26T06:12:08.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.835+0000 7f4113e00700 1 -- 172.21.15.49:0/240077955 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f410400b8c0 con 0x7f4114107670 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.835+0000 7f4113e00700 1 -- 172.21.15.49:0/240077955 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4104004550 con 0x7f4114107670 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.836+0000 7f411ba00700 1 -- 172.21.15.49:0/240077955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 msgr2=0x7f4114107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.836+0000 7f411ba00700 1 --2- 172.21.15.49:0/240077955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f4114107a90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4104009e80 tx=0x7f410400b330 comp rx=0 tx=0).stop 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.836+0000 7f411ba00700 1 -- 172.21.15.49:0/240077955 shutdown_connections 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.836+0000 7f411ba00700 1 --2- 172.21.15.49:0/240077955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f4114107a90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.836+0000 7f411ba00700 1 -- 172.21.15.49:0/240077955 >> 172.21.15.49:0/240077955 conn(0x7f4114102c70 msgr2=0x7f4114105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.836+0000 7f411ba00700 1 -- 172.21.15.49:0/240077955 shutdown_connections 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.836+0000 7f411ba00700 1 -- 172.21.15.49:0/240077955 wait complete. 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.837+0000 7f411ba00700 1 Processor -- start 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.837+0000 7f411ba00700 1 -- start start 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.838+0000 7f411ba00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f411419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.838+0000 7f411ba00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411419b880 con 0x7f4114107670 2024-02-26T06:12:08.652 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.838+0000 7f4119000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f411419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.838+0000 7f4119000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f411419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45408/0 (socket says 172.21.15.49:45408) 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.838+0000 7f4119000700 1 -- 172.21.15.49:0/2733496981 learned_addr learned my addr 172.21.15.49:0/2733496981 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.838+0000 7f4119000700 1 -- 172.21.15.49:0/2733496981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009b30 con 0x7f4114107670 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.839+0000 7f4119000700 1 --2- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f411419b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4104009fd0 tx=0x7f4104003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.839+0000 7f4112000700 1 -- 172.21.15.49:0/2733496981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f410400e040 con 0x7f4114107670 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.839+0000 7f4112000700 1 -- 172.21.15.49:0/2733496981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41040128f0 con 0x7f4114107670 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.839+0000 7f4112000700 1 -- 172.21.15.49:0/2733496981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f410401c890 con 0x7f4114107670 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.839+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411419ba80 con 0x7f4114107670 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.839+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411419bf00 con 0x7f4114107670 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.840+0000 7f4112000700 1 -- 172.21.15.49:0/2733496981 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f4104012a60 con 0x7f4114107670 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.840+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411410ba50 con 0x7f4114107670 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.841+0000 7f4112000700 1 --2- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f40f8038f30 0x7f40f803b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:08.653 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.841+0000 7f4112000700 1 -- 172.21.15.49:0/2733496981 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f410404d020 con 0x7f4114107670 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.841+0000 7f410be00700 1 --2- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f40f8038f30 0x7f40f803b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.842+0000 7f410be00700 1 --2- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f40f8038f30 0x7f40f803b3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006fd0 tx=0x7f40fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:00.847+0000 7f4112000700 1 -- 172.21.15.49:0/2733496981 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4104024e20 con 0x7f4114107670 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:01.075+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}) v1 -- 0x7f4114103dd0 con 0x7f40f8038f30 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.994+0000 7f4112000700 1 -- 172.21.15.49:0/2733496981 <== mgr.14120 v2:172.21.15.49:6800/4089389406 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f4114103dd0 con 0x7f40f8038f30 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.999+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f40f8038f30 msgr2=0x7f40f803b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.999+0000 7f411ba00700 1 --2- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f40f8038f30 0x7f40f803b3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006fd0 tx=0x7f40fc006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.999+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 msgr2=0x7f411419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.999+0000 7f411ba00700 1 --2- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f411419b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4104009fd0 tx=0x7f4104003d70 comp rx=0 tx=0).stop 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.999+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 shutdown_connections 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.999+0000 7f411ba00700 1 --2- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f40f8038f30 0x7f40f803b3f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.999+0000 7f411ba00700 1 --2- 172.21.15.49:0/2733496981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4114107670 0x7f411419b340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:07.999+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 >> 172.21.15.49:0/2733496981 conn(0x7f4114102c70 msgr2=0x7f41141036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:08.000+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 shutdown_connections 2024-02-26T06:12:08.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:08.000+0000 7f411ba00700 1 -- 172.21.15.49:0/2733496981 wait complete. 2024-02-26T06:12:08.655 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mon service... 2024-02-26T06:12:09.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:08 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:12:09.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:08 smithi049 ceph-mon[31696]: Added host smithi049 2024-02-26T06:12:09.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:08 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.301+0000 7f0468e00700 1 Processor -- start 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.302+0000 7f0468e00700 1 -- start start 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.302+0000 7f0468e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046410a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.302+0000 7f0468e00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046407aed0 con 0x7f04641080c0 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.303+0000 7f0461e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046410a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.303+0000 7f0461e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046410a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45420/0 (socket says 172.21.15.49:45420) 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.303+0000 7f0461e00700 1 -- 172.21.15.49:0/173734650 learned_addr learned my addr 172.21.15.49:0/173734650 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.303+0000 7f0461e00700 1 -- 172.21.15.49:0/173734650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046410a9f0 con 0x7f04641080c0 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.304+0000 7f0461e00700 1 --2- 172.21.15.49:0/173734650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046410a4b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04540098a0 tx=0x7f0454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6b0ad8897e9ae4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.304+0000 7f0461400700 1 -- 172.21.15.49:0/173734650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0454004160 con 0x7f04641080c0 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.304+0000 7f0461400700 1 -- 172.21.15.49:0/173734650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04540042c0 con 0x7f04641080c0 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.304+0000 7f0461400700 1 -- 172.21.15.49:0/173734650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0454011ac0 con 0x7f04641080c0 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.305+0000 7f0468e00700 1 -- 172.21.15.49:0/173734650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 msgr2=0x7f046410a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.305+0000 7f0468e00700 1 --2- 172.21.15.49:0/173734650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046410a4b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04540098a0 tx=0x7f0454009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.305+0000 7f0468e00700 1 -- 172.21.15.49:0/173734650 shutdown_connections 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.305+0000 7f0468e00700 1 --2- 172.21.15.49:0/173734650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046410a4b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.305+0000 7f0468e00700 1 -- 172.21.15.49:0/173734650 >> 172.21.15.49:0/173734650 conn(0x7f0464101b10 msgr2=0x7f0464103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.305+0000 7f0468e00700 1 -- 172.21.15.49:0/173734650 shutdown_connections 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.305+0000 7f0468e00700 1 -- 172.21.15.49:0/173734650 wait complete. 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.306+0000 7f0468e00700 1 Processor -- start 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.306+0000 7f0468e00700 1 -- start start 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.307+0000 7f0468e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.307+0000 7f0468e00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046419db60 con 0x7f04641080c0 2024-02-26T06:12:10.190 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.307+0000 7f0461e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.307+0000 7f0461e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45422/0 (socket says 172.21.15.49:45422) 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.307+0000 7f0461e00700 1 -- 172.21.15.49:0/3079676264 learned_addr learned my addr 172.21.15.49:0/3079676264 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.307+0000 7f0461e00700 1 -- 172.21.15.49:0/3079676264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0454009550 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.308+0000 7f0461e00700 1 --2- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046419d620 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0454009130 tx=0x7f0454011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.308+0000 7f045b400700 1 -- 172.21.15.49:0/3079676264 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0454003aa0 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.308+0000 7f045b400700 1 -- 172.21.15.49:0/3079676264 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0454003c00 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.308+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046419dd60 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.308+0000 7f045b400700 1 -- 172.21.15.49:0/3079676264 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0454024800 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.308+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046419e200 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.309+0000 7f045b400700 1 -- 172.21.15.49:0/3079676264 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f045401abd0 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.309+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046410b850 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.310+0000 7f045b400700 1 --2- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0448038f30 0x7f044803b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.310+0000 7f045b400700 1 -- 172.21.15.49:0/3079676264 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f045404d0d0 con 0x7f04641080c0 2024-02-26T06:12:10.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.310+0000 7f0459400700 1 --2- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0448038f30 0x7f044803b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.311+0000 7f0459400700 1 --2- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0448038f30 0x7f044803b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f044c006fd0 tx=0x7f044c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.316+0000 7f045b400700 1 -- 172.21.15.49:0/3079676264 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0454018760 con 0x7f04641080c0 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.561+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0464102cb0 con 0x7f0448038f30 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.574+0000 7f045b400700 1 -- 172.21.15.49:0/3079676264 <== mgr.14120 v2:172.21.15.49:6800/4089389406 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0464102cb0 con 0x7f0448038f30 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0448038f30 msgr2=0x7f044803b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 --2- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0448038f30 0x7f044803b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f044c006fd0 tx=0x7f044c006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 msgr2=0x7f046419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 --2- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046419d620 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0454009130 tx=0x7f0454011b40 comp rx=0 tx=0).stop 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 shutdown_connections 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 --2- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f0448038f30 0x7f044803b3f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 --2- 172.21.15.49:0/3079676264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04641080c0 0x7f046419d620 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 >> 172.21.15.49:0/3079676264 conn(0x7f0464101b10 msgr2=0x7f04641025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 shutdown_connections 2024-02-26T06:12:10.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:09.577+0000 7f0468e00700 1 -- 172.21.15.49:0/3079676264 wait complete. 2024-02-26T06:12:10.193 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mgr service... 2024-02-26T06:12:10.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:10 smithi049 ceph-mon[31696]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:12:10.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:10 smithi049 ceph-mon[31696]: Saving service mon spec with placement count:5 2024-02-26T06:12:10.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:10 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:12:12.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-26T06:12:12.478 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.034+0000 7ff540a00700 1 Processor -- start 2024-02-26T06:12:12.478 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.035+0000 7ff540a00700 1 -- start start 2024-02-26T06:12:12.478 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.035+0000 7ff540a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c107a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:12.479 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.035+0000 7ff540a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c108060 con 0x7ff53c107670 2024-02-26T06:12:12.479 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.036+0000 7ff539e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:12.479 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.036+0000 7ff539e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57538/0 (socket says 172.21.15.49:57538) 2024-02-26T06:12:12.479 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.036+0000 7ff539e00700 1 -- 172.21.15.49:0/925439110 learned_addr learned my addr 172.21.15.49:0/925439110 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:12.480 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.036+0000 7ff539e00700 1 -- 172.21.15.49:0/925439110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c108820 con 0x7ff53c107670 2024-02-26T06:12:12.480 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.037+0000 7ff539e00700 1 --2- 172.21.15.49:0/925439110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c107a90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0098a0 tx=0x7ff52c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add27329efc687a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:12.480 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.037+0000 7ff538a00700 1 -- 172.21.15.49:0/925439110 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff52c004160 con 0x7ff53c107670 2024-02-26T06:12:12.481 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.037+0000 7ff538a00700 1 -- 172.21.15.49:0/925439110 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff52c0042c0 con 0x7ff53c107670 2024-02-26T06:12:12.481 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.037+0000 7ff538a00700 1 -- 172.21.15.49:0/925439110 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff52c011ac0 con 0x7ff53c107670 2024-02-26T06:12:12.481 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.037+0000 7ff540a00700 1 -- 172.21.15.49:0/925439110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 msgr2=0x7ff53c107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:12.481 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.037+0000 7ff540a00700 1 --2- 172.21.15.49:0/925439110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c107a90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0098a0 tx=0x7ff52c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:12.481 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.038+0000 7ff540a00700 1 -- 172.21.15.49:0/925439110 shutdown_connections 2024-02-26T06:12:12.481 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.038+0000 7ff540a00700 1 --2- 172.21.15.49:0/925439110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c107a90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:12.482 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.038+0000 7ff540a00700 1 -- 172.21.15.49:0/925439110 >> 172.21.15.49:0/925439110 conn(0x7ff53c102c70 msgr2=0x7ff53c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:12.482 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.038+0000 7ff540a00700 1 -- 172.21.15.49:0/925439110 shutdown_connections 2024-02-26T06:12:12.482 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.038+0000 7ff540a00700 1 -- 172.21.15.49:0/925439110 wait complete. 2024-02-26T06:12:12.482 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.039+0000 7ff540a00700 1 Processor -- start 2024-02-26T06:12:12.483 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.039+0000 7ff540a00700 1 -- start start 2024-02-26T06:12:12.483 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.039+0000 7ff540a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:12.483 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.039+0000 7ff540a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c19b950 con 0x7ff53c107670 2024-02-26T06:12:12.483 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.040+0000 7ff539e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:12.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.040+0000 7ff539e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57542/0 (socket says 172.21.15.49:57542) 2024-02-26T06:12:12.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.040+0000 7ff539e00700 1 -- 172.21.15.49:0/2421995367 learned_addr learned my addr 172.21.15.49:0/2421995367 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:12.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.040+0000 7ff539e00700 1 -- 172.21.15.49:0/2421995367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c009550 con 0x7ff53c107670 2024-02-26T06:12:12.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.040+0000 7ff539e00700 1 --2- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c19b410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000c00 tx=0x7ff52c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:12.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.040+0000 7ff532a00700 1 -- 172.21.15.49:0/2421995367 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff52c003aa0 con 0x7ff53c107670 2024-02-26T06:12:12.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.041+0000 7ff532a00700 1 -- 172.21.15.49:0/2421995367 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff52c003c00 con 0x7ff53c107670 2024-02-26T06:12:12.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.041+0000 7ff532a00700 1 -- 172.21.15.49:0/2421995367 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff52c0248a0 con 0x7ff53c107670 2024-02-26T06:12:12.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.041+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c19bb50 con 0x7ff53c107670 2024-02-26T06:12:12.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.041+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c19bff0 con 0x7ff53c107670 2024-02-26T06:12:12.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.042+0000 7ff532a00700 1 -- 172.21.15.49:0/2421995367 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ff52c01a430 con 0x7ff53c107670 2024-02-26T06:12:12.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.042+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c10ba50 con 0x7ff53c107670 2024-02-26T06:12:12.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.043+0000 7ff532a00700 1 --2- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7ff520038f30 0x7ff52003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:12.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.043+0000 7ff532a00700 1 -- 172.21.15.49:0/2421995367 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff52c04d020 con 0x7ff53c107670 2024-02-26T06:12:12.486 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.043+0000 7ff539400700 1 --2- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7ff520038f30 0x7ff52003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:12.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.044+0000 7ff539400700 1 --2- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7ff520038f30 0x7ff52003b3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff528006fd0 tx=0x7ff528006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:12.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.048+0000 7ff532a00700 1 -- 172.21.15.49:0/2421995367 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff52c080d20 con 0x7ff53c107670 2024-02-26T06:12:12.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.293+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff53c103ee0 con 0x7ff520038f30 2024-02-26T06:12:12.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.301+0000 7ff532a00700 1 -- 172.21.15.49:0/2421995367 <== mgr.14120 v2:172.21.15.49:6800/4089389406 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff53c103ee0 con 0x7ff520038f30 2024-02-26T06:12:12.487 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.304+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7ff520038f30 msgr2=0x7ff52003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:12.488 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.304+0000 7ff540a00700 1 --2- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7ff520038f30 0x7ff52003b3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff528006fd0 tx=0x7ff528006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:12.488 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.304+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 msgr2=0x7ff53c19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:12.489 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.304+0000 7ff540a00700 1 --2- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c19b410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000c00 tx=0x7ff52c011b40 comp rx=0 tx=0).stop 2024-02-26T06:12:12.489 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.304+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 shutdown_connections 2024-02-26T06:12:12.489 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.304+0000 7ff540a00700 1 --2- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7ff520038f30 0x7ff52003b3f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:12.489 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.304+0000 7ff540a00700 1 --2- 172.21.15.49:0/2421995367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff53c107670 0x7ff53c19b410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:12.489 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.304+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 >> 172.21.15.49:0/2421995367 conn(0x7ff53c102c70 msgr2=0x7ff53c1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:12.489 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.305+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 shutdown_connections 2024-02-26T06:12:12.489 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:11.305+0000 7ff540a00700 1 -- 172.21.15.49:0/2421995367 wait complete. 2024-02-26T06:12:12.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:12 smithi049 ceph-mon[31696]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:12:12.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:12 smithi049 ceph-mon[31696]: Saving service mgr spec with placement count:2 2024-02-26T06:12:12.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:12 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:12:12.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:12 smithi049 ceph-mon[31696]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:13.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:13 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:12:13.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:13 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4021285517' entity='client.admin' 2024-02-26T06:12:13.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:13 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:12:13.990 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.044+0000 7f3c90800700 1 Processor -- start 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.045+0000 7f3c90800700 1 -- start start 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.045+0000 7f3c90800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c88107a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.045+0000 7f3c90800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c88108020 con 0x7f3c88107630 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.045+0000 7f3c8de00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c88107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.045+0000 7f3c8de00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c88107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57546/0 (socket says 172.21.15.49:57546) 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.045+0000 7f3c8de00700 1 -- 172.21.15.49:0/3244203289 learned_addr learned my addr 172.21.15.49:0/3244203289 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.046+0000 7f3c8de00700 1 -- 172.21.15.49:0/3244203289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c881087e0 con 0x7f3c88107630 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.046+0000 7f3c8de00700 1 --2- 172.21.15.49:0/3244203289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c88107a50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0098a0 tx=0x7f3c7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebb57b2800a81ee server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.047+0000 7f3c8ca00700 1 -- 172.21.15.49:0/3244203289 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c7c004160 con 0x7f3c88107630 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.047+0000 7f3c8ca00700 1 -- 172.21.15.49:0/3244203289 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c7c0042c0 con 0x7f3c88107630 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.047+0000 7f3c8ca00700 1 -- 172.21.15.49:0/3244203289 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c7c011a80 con 0x7f3c88107630 2024-02-26T06:12:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.047+0000 7f3c90800700 1 -- 172.21.15.49:0/3244203289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 msgr2=0x7f3c88107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.047+0000 7f3c90800700 1 --2- 172.21.15.49:0/3244203289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c88107a50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0098a0 tx=0x7f3c7c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.047+0000 7f3c90800700 1 -- 172.21.15.49:0/3244203289 shutdown_connections 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.048+0000 7f3c90800700 1 --2- 172.21.15.49:0/3244203289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c88107a50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.048+0000 7f3c90800700 1 -- 172.21.15.49:0/3244203289 >> 172.21.15.49:0/3244203289 conn(0x7f3c88102c10 msgr2=0x7f3c88105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.048+0000 7f3c90800700 1 -- 172.21.15.49:0/3244203289 shutdown_connections 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.048+0000 7f3c90800700 1 -- 172.21.15.49:0/3244203289 wait complete. 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.049+0000 7f3c90800700 1 Processor -- start 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.049+0000 7f3c90800700 1 -- start start 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.049+0000 7f3c90800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c8819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.049+0000 7f3c90800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8819b900 con 0x7f3c88107630 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.049+0000 7f3c8de00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c8819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.050+0000 7f3c8de00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c8819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57556/0 (socket says 172.21.15.49:57556) 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.050+0000 7f3c8de00700 1 -- 172.21.15.49:0/4021285517 learned_addr learned my addr 172.21.15.49:0/4021285517 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:13.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.050+0000 7f3c8de00700 1 -- 172.21.15.49:0/4021285517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c009550 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.050+0000 7f3c8de00700 1 --2- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c8819b3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c011740 tx=0x7f3c7c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.050+0000 7f3c82a00700 1 -- 172.21.15.49:0/4021285517 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c7c0039e0 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.051+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8819bb00 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.051+0000 7f3c82a00700 1 -- 172.21.15.49:0/4021285517 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c7c003b40 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.051+0000 7f3c82a00700 1 -- 172.21.15.49:0/4021285517 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c7c024710 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.051+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8819bfa0 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.052+0000 7f3c82a00700 1 -- 172.21.15.49:0/4021285517 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f3c7c01a430 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.052+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8810ba10 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.053+0000 7f3c82a00700 1 --2- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f3c70038e90 0x7f3c7003b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.053+0000 7f3c82a00700 1 -- 172.21.15.49:0/4021285517 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3c7c04d240 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.053+0000 7f3c8d400700 1 --2- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f3c70038e90 0x7f3c7003b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.054+0000 7f3c8d400700 1 --2- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f3c70038e90 0x7f3c7003b350 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c78006fd0 tx=0x7f3c78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.058+0000 7f3c82a00700 1 -- 172.21.15.49:0/4021285517 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c7c01a6e0 con 0x7f3c88107630 2024-02-26T06:12:13.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.286+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f3c8810bc20 con 0x7f3c88107630 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.289+0000 7f3c82a00700 1 -- 172.21.15.49:0/4021285517 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f3c7c018350 con 0x7f3c88107630 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f3c70038e90 msgr2=0x7f3c7003b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 --2- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f3c70038e90 0x7f3c7003b350 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c78006fd0 tx=0x7f3c78006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 msgr2=0x7f3c8819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 --2- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c8819b3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c011740 tx=0x7f3c7c011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 shutdown_connections 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 --2- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f3c70038e90 0x7f3c7003b350 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 --2- 172.21.15.49:0/4021285517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c88107630 0x7f3c8819b3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 >> 172.21.15.49:0/4021285517 conn(0x7f3c88102c10 msgr2=0x7f3c881036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 shutdown_connections 2024-02-26T06:12:13.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:13.293+0000 7f3c90800700 1 -- 172.21.15.49:0/4021285517 wait complete. 2024-02-26T06:12:14.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:14 smithi049 ceph-mon[31696]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:15.726 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.620+0000 7fc070200700 1 Processor -- start 2024-02-26T06:12:15.726 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.620+0000 7fc070200700 1 -- start start 2024-02-26T06:12:15.726 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.621+0000 7fc070200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc068109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.621+0000 7fc070200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06810a260 con 0x7fc068109870 2024-02-26T06:12:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.621+0000 7fc06d800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc068109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.621+0000 7fc06d800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc068109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57572/0 (socket says 172.21.15.49:57572) 2024-02-26T06:12:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.621+0000 7fc06d800700 1 -- 172.21.15.49:0/3802494983 learned_addr learned my addr 172.21.15.49:0/3802494983 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.621+0000 7fc06d800700 1 -- 172.21.15.49:0/3802494983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06810aa20 con 0x7fc068109870 2024-02-26T06:12:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.622+0000 7fc06d800700 1 --2- 172.21.15.49:0/3802494983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc068109c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0098a0 tx=0x7fc05c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e80794fcf3c624 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.622+0000 7fc067e00700 1 -- 172.21.15.49:0/3802494983 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc05c004160 con 0x7fc068109870 2024-02-26T06:12:15.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.622+0000 7fc067e00700 1 -- 172.21.15.49:0/3802494983 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc05c0042c0 con 0x7fc068109870 2024-02-26T06:12:15.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.622+0000 7fc067e00700 1 -- 172.21.15.49:0/3802494983 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc05c011a80 con 0x7fc068109870 2024-02-26T06:12:15.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.623+0000 7fc070200700 1 -- 172.21.15.49:0/3802494983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 msgr2=0x7fc068109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:15.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.623+0000 7fc070200700 1 --2- 172.21.15.49:0/3802494983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc068109c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0098a0 tx=0x7fc05c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:15.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.623+0000 7fc070200700 1 -- 172.21.15.49:0/3802494983 shutdown_connections 2024-02-26T06:12:15.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.623+0000 7fc070200700 1 --2- 172.21.15.49:0/3802494983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc068109c90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:15.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.623+0000 7fc070200700 1 -- 172.21.15.49:0/3802494983 >> 172.21.15.49:0/3802494983 conn(0x7fc06807ce30 msgr2=0x7fc0681074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:15.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.623+0000 7fc070200700 1 -- 172.21.15.49:0/3802494983 shutdown_connections 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.623+0000 7fc070200700 1 -- 172.21.15.49:0/3802494983 wait complete. 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.624+0000 7fc070200700 1 Processor -- start 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.624+0000 7fc070200700 1 -- start start 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.625+0000 7fc070200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc06819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.625+0000 7fc070200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06819db20 con 0x7fc068109870 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.625+0000 7fc06d800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc06819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.625+0000 7fc06d800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc06819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57580/0 (socket says 172.21.15.49:57580) 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.625+0000 7fc06d800700 1 -- 172.21.15.49:0/1090699273 learned_addr learned my addr 172.21.15.49:0/1090699273 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.625+0000 7fc06d800700 1 -- 172.21.15.49:0/1090699273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c009550 con 0x7fc068109870 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.626+0000 7fc06d800700 1 --2- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc06819d5e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc05c011740 tx=0x7fc05c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:15.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.626+0000 7fc066000700 1 -- 172.21.15.49:0/1090699273 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc05c0039e0 con 0x7fc068109870 2024-02-26T06:12:15.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.626+0000 7fc066000700 1 -- 172.21.15.49:0/1090699273 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc05c003b40 con 0x7fc068109870 2024-02-26T06:12:15.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.626+0000 7fc066000700 1 -- 172.21.15.49:0/1090699273 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc05c024740 con 0x7fc068109870 2024-02-26T06:12:15.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.626+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06819dd20 con 0x7fc068109870 2024-02-26T06:12:15.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.626+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06819e1c0 con 0x7fc068109870 2024-02-26T06:12:15.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.627+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06810dc50 con 0x7fc068109870 2024-02-26T06:12:15.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.628+0000 7fc066000700 1 -- 172.21.15.49:0/1090699273 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fc05c01ab30 con 0x7fc068109870 2024-02-26T06:12:15.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.629+0000 7fc066000700 1 --2- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc050038ee0 0x7fc05003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:15.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.629+0000 7fc066000700 1 -- 172.21.15.49:0/1090699273 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc05c04d200 con 0x7fc068109870 2024-02-26T06:12:15.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.629+0000 7fc06ce00700 1 --2- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc050038ee0 0x7fc05003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:15.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.630+0000 7fc06ce00700 1 --2- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc050038ee0 0x7fc05003b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc054006fd0 tx=0x7fc054006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.634+0000 7fc066000700 1 -- 172.21.15.49:0/1090699273 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc05c010380 con 0x7fc068109870 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.864+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fc0681a0b00 con 0x7fc068109870 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.870+0000 7fc066000700 1 -- 172.21.15.49:0/1090699273 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fc05c04cb70 con 0x7fc068109870 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.873+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc050038ee0 msgr2=0x7fc05003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.873+0000 7fc070200700 1 --2- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc050038ee0 0x7fc05003b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc054006fd0 tx=0x7fc054006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.873+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 msgr2=0x7fc06819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.873+0000 7fc070200700 1 --2- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc06819d5e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc05c011740 tx=0x7fc05c011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.874+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 shutdown_connections 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.874+0000 7fc070200700 1 --2- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fc050038ee0 0x7fc05003b3a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.874+0000 7fc070200700 1 --2- 172.21.15.49:0/1090699273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc068109870 0x7fc06819d5e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.874+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 >> 172.21.15.49:0/1090699273 conn(0x7fc06807ce30 msgr2=0x7fc068106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.874+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 shutdown_connections 2024-02-26T06:12:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:14.874+0000 7fc070200700 1 -- 172.21.15.49:0/1090699273 wait complete. 2024-02-26T06:12:15.738 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling the dashboard module... 2024-02-26T06:12:16.163 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:15 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1090699273' entity='client.admin' 2024-02-26T06:12:17.141 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:16 smithi049 ceph-mon[31696]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:17.141 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:16 smithi049 ceph-mon[31696]: from='mgr.14120 172.21.15.49:0/1379444384' entity='mgr.x' 2024-02-26T06:12:18.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:17 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2816023440' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-26T06:12:18.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: ignoring --setuser ceph since I am not root 2024-02-26T06:12:18.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: ignoring --setgroup ceph since I am not root 2024-02-26T06:12:18.682 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.647+0000 7f71fd200700 1 Processor -- start 2024-02-26T06:12:18.682 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.648+0000 7f71fd200700 1 -- start start 2024-02-26T06:12:18.683 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.648+0000 7f71fd200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f8107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:18.683 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.648+0000 7f71fd200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81080e0 con 0x7f71f81076f0 2024-02-26T06:12:18.684 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.649+0000 7f71f5e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f8107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:18.684 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.649+0000 7f71f5e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f8107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57582/0 (socket says 172.21.15.49:57582) 2024-02-26T06:12:18.684 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.649+0000 7f71f5e00700 1 -- 172.21.15.49:0/2316912644 learned_addr learned my addr 172.21.15.49:0/2316912644 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:18.684 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.649+0000 7f71f5e00700 1 -- 172.21.15.49:0/2316912644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f81088a0 con 0x7f71f81076f0 2024-02-26T06:12:18.684 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.649+0000 7f71f5e00700 1 --2- 172.21.15.49:0/2316912644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f8107b10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71e80098a0 tx=0x7f71e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c162a58164ef12 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:18.684 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.650+0000 7f71f4a00700 1 -- 172.21.15.49:0/2316912644 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71e8004160 con 0x7f71f81076f0 2024-02-26T06:12:18.684 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.650+0000 7f71f4a00700 1 -- 172.21.15.49:0/2316912644 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e80042c0 con 0x7f71f81076f0 2024-02-26T06:12:18.686 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.650+0000 7f71f4a00700 1 -- 172.21.15.49:0/2316912644 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71e8011a80 con 0x7f71f81076f0 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.650+0000 7f71fd200700 1 -- 172.21.15.49:0/2316912644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 msgr2=0x7f71f8107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.650+0000 7f71fd200700 1 --2- 172.21.15.49:0/2316912644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f8107b10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71e80098a0 tx=0x7f71e8009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.651+0000 7f71fd200700 1 -- 172.21.15.49:0/2316912644 shutdown_connections 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.651+0000 7f71fd200700 1 --2- 172.21.15.49:0/2316912644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f8107b10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.651+0000 7f71fd200700 1 -- 172.21.15.49:0/2316912644 >> 172.21.15.49:0/2316912644 conn(0x7f71f8102cd0 msgr2=0x7f71f81050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.651+0000 7f71fd200700 1 -- 172.21.15.49:0/2316912644 shutdown_connections 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.651+0000 7f71fd200700 1 -- 172.21.15.49:0/2316912644 wait complete. 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.652+0000 7f71fd200700 1 Processor -- start 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.652+0000 7f71fd200700 1 -- start start 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.652+0000 7f71fd200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f807cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.652+0000 7f71fd200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f807d0d0 con 0x7f71f81076f0 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.653+0000 7f71f5e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f807cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.653+0000 7f71f5e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f807cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57596/0 (socket says 172.21.15.49:57596) 2024-02-26T06:12:18.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.653+0000 7f71f5e00700 1 -- 172.21.15.49:0/2816023440 learned_addr learned my addr 172.21.15.49:0/2816023440 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:18.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.653+0000 7f71f5e00700 1 -- 172.21.15.49:0/2816023440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e8009550 con 0x7f71f81076f0 2024-02-26T06:12:18.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.653+0000 7f71f5e00700 1 --2- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f807cb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f71e8000c00 tx=0x7f71e8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:18.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.654+0000 7f71eea00700 1 -- 172.21.15.49:0/2816023440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71e80039e0 con 0x7f71f81076f0 2024-02-26T06:12:18.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.654+0000 7f71eea00700 1 -- 172.21.15.49:0/2816023440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e8003b40 con 0x7f71f81076f0 2024-02-26T06:12:18.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.654+0000 7f71eea00700 1 -- 172.21.15.49:0/2816023440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71e8024820 con 0x7f71f81076f0 2024-02-26T06:12:18.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.654+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f807b2c0 con 0x7f71f81076f0 2024-02-26T06:12:18.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.654+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f807b760 con 0x7f71f81076f0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.655+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f810bad0 con 0x7f71f81076f0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.656+0000 7f71eea00700 1 -- 172.21.15.49:0/2816023440 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f71e801ab80 con 0x7f71f81076f0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.656+0000 7f71eea00700 1 --2- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f71dc038f30 0x7f71dc03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.657+0000 7f71eea00700 1 -- 172.21.15.49:0/2816023440 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f71e804cfd0 con 0x7f71f81076f0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.657+0000 7f71f5400700 1 --2- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f71dc038f30 0x7f71dc03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.658+0000 7f71f5400700 1 --2- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f71dc038f30 0x7f71dc03b3f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71e4006fd0 tx=0x7f71e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.663+0000 7f71eea00700 1 -- 172.21.15.49:0/2816023440 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f71e801a950 con 0x7f71f81076f0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:16.953+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f71f807bcb0 con 0x7f71f81076f0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.879+0000 7f71eea00700 1 -- 172.21.15.49:0/2816023440 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f71e802c350 con 0x7f71f81076f0 2024-02-26T06:12:18.691 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.883+0000 7f71eea00700 1 -- 172.21.15.49:0/2816023440 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f71e801a430 con 0x7f71f81076f0 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.884+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f71dc038f30 msgr2=0x7f71dc03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.884+0000 7f71fd200700 1 --2- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f71dc038f30 0x7f71dc03b3f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71e4006fd0 tx=0x7f71e4006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.884+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 msgr2=0x7f71f807cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.884+0000 7f71fd200700 1 --2- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f807cb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f71e8000c00 tx=0x7f71e8011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.884+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 shutdown_connections 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.884+0000 7f71fd200700 1 --2- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f71dc038f30 0x7f71dc03b3f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.884+0000 7f71fd200700 1 --2- 172.21.15.49:0/2816023440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81076f0 0x7f71f807cb90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.884+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 >> 172.21.15.49:0/2816023440 conn(0x7f71f8102cd0 msgr2=0x7f71f8103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.885+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 shutdown_connections 2024-02-26T06:12:18.692 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:17.885+0000 7f71fd200700 1 -- 172.21.15.49:0/2816023440 wait complete. 2024-02-26T06:12:18.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:18 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2816023440' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-26T06:12:18.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:18 smithi049 ceph-mon[31696]: mgrmap e9: x(active, since 26s) 2024-02-26T06:12:18.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:18.751+0000 7fca336ed200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:12:18.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:18.882+0000 7fca336ed200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:12:19.881 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:19.724+0000 7fca336ed200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:12:19.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:19.853+0000 7fca336ed200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:12:20.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:19 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2730763373' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-26T06:12:20.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:20.043+0000 7fca336ed200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.325+0000 7fde73600700 1 Processor -- start 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.325+0000 7fde73600700 1 -- start start 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.326+0000 7fde73600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c106370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.326+0000 7fde73600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c07ae30 con 0x7fde6c105f50 2024-02-26T06:12:20.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.326+0000 7fde70c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.326+0000 7fde70c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c106370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57636/0 (socket says 172.21.15.49:57636) 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.326+0000 7fde70c00700 1 -- 172.21.15.49:0/1078850434 learned_addr learned my addr 172.21.15.49:0/1078850434 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.327+0000 7fde70c00700 1 -- 172.21.15.49:0/1078850434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c07af70 con 0x7fde6c105f50 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.327+0000 7fde70c00700 1 --2- 172.21.15.49:0/1078850434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c106370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde600098a0 tx=0x7fde60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0653ce62b89a31 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.327+0000 7fde6b400700 1 -- 172.21.15.49:0/1078850434 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde60004160 con 0x7fde6c105f50 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.327+0000 7fde6b400700 1 -- 172.21.15.49:0/1078850434 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde600042c0 con 0x7fde6c105f50 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.327+0000 7fde6b400700 1 -- 172.21.15.49:0/1078850434 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde60011a80 con 0x7fde6c105f50 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.328+0000 7fde73600700 1 -- 172.21.15.49:0/1078850434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 msgr2=0x7fde6c106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.328+0000 7fde73600700 1 --2- 172.21.15.49:0/1078850434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c106370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde600098a0 tx=0x7fde60009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.328+0000 7fde73600700 1 -- 172.21.15.49:0/1078850434 shutdown_connections 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.328+0000 7fde73600700 1 --2- 172.21.15.49:0/1078850434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c106370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.328+0000 7fde73600700 1 -- 172.21.15.49:0/1078850434 >> 172.21.15.49:0/1078850434 conn(0x7fde6c101b10 msgr2=0x7fde6c103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:20.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.328+0000 7fde73600700 1 -- 172.21.15.49:0/1078850434 shutdown_connections 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.328+0000 7fde73600700 1 -- 172.21.15.49:0/1078850434 wait complete. 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.329+0000 7fde73600700 1 Processor -- start 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.329+0000 7fde73600700 1 -- start start 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.330+0000 7fde73600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.330+0000 7fde73600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c19da90 con 0x7fde6c105f50 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.330+0000 7fde70c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.330+0000 7fde70c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c19d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57648/0 (socket says 172.21.15.49:57648) 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.330+0000 7fde70c00700 1 -- 172.21.15.49:0/2730763373 learned_addr learned my addr 172.21.15.49:0/2730763373 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.330+0000 7fde70c00700 1 -- 172.21.15.49:0/2730763373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60009550 con 0x7fde6c105f50 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.331+0000 7fde70c00700 1 --2- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c19d550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde60011740 tx=0x7fde60011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.331+0000 7fde69600700 1 -- 172.21.15.49:0/2730763373 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde60003b20 con 0x7fde6c105f50 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.331+0000 7fde69600700 1 -- 172.21.15.49:0/2730763373 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde6001a910 con 0x7fde6c105f50 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.331+0000 7fde69600700 1 -- 172.21.15.49:0/2730763373 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde60024900 con 0x7fde6c105f50 2024-02-26T06:12:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.331+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c19dc90 con 0x7fde6c105f50 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.331+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c19e130 con 0x7fde6c105f50 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.333+0000 7fde69600700 1 -- 172.21.15.49:0/2730763373 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fde60024b70 con 0x7fde6c105f50 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.333+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c197280 con 0x7fde6c105f50 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.333+0000 7fde69600700 1 --2- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fde54038ee0 0x7fde5403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.333+0000 7fde69600700 1 -- 172.21.15.49:0/2730763373 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fde6004d350 con 0x7fde6c105f50 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.333+0000 7fde6be00700 1 -- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fde54038ee0 msgr2=0x7fde5403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4089389406 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.333+0000 7fde6be00700 1 --2- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fde54038ee0 0x7fde5403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.338+0000 7fde69600700 1 -- 172.21.15.49:0/2730763373 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde60018a40 con 0x7fde6c105f50 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.534+0000 7fde6be00700 1 -- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fde54038ee0 msgr2=0x7fde5403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4089389406 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.534+0000 7fde6be00700 1 --2- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fde54038ee0 0x7fde5403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.625+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fde6c02d940 con 0x7fde6c105f50 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.625+0000 7fde69600700 1 -- 172.21.15.49:0/2730763373 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fde60018350 con 0x7fde6c105f50 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fde54038ee0 msgr2=0x7fde5403b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 --2- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fde54038ee0 0x7fde5403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 msgr2=0x7fde6c19d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 --2- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c19d550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde60011740 tx=0x7fde60011a50 comp rx=0 tx=0).stop 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 shutdown_connections 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 --2- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7fde54038ee0 0x7fde5403b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 --2- 172.21.15.49:0/2730763373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c105f50 0x7fde6c19d550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 >> 172.21.15.49:0/2730763373 conn(0x7fde6c101b10 msgr2=0x7fde6c1943c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.628+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 shutdown_connections 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:19.629+0000 7fde73600700 1 -- 172.21.15.49:0/2730763373 wait complete. 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-02-26T06:12:20.376 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 9... 2024-02-26T06:12:20.521 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:20.181+0000 7fca336ed200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:12:20.521 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:20.303+0000 7fca336ed200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:12:20.791 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:20.664+0000 7fca336ed200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:12:21.066 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:20.814+0000 7fca336ed200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:12:21.066 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:20.935+0000 7fca336ed200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:12:21.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:21.066+0000 7fca336ed200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:12:21.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:21.296+0000 7fca336ed200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:12:23.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:22.709+0000 7fca336ed200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:12:23.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:22.892+0000 7fca336ed200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:12:23.669 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:23.333+0000 7fca336ed200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:12:23.669 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:23.459+0000 7fca336ed200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:12:23.981 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:23.724+0000 7fca336ed200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:12:23.981 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:23.980+0000 7fca336ed200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:12:24.374 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:24.373+0000 7fca336ed200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:12:25.026 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:24.707+0000 7fca336ed200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:12:25.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:25.025+0000 7fca336ed200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:12:26.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:26.106+0000 7fca336ed200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:12:27.168 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:26.837+0000 7fca336ed200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:12:27.168 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:26.973+0000 7fca336ed200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:12:27.270+0000 7fca336ed200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: Active manager daemon x restarted 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: Activating manager daemon x 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: osdmap e3: 0 total, 0 up, 0 in 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: mgrmap e10: x(active, starting, since 0.00922294s) 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:12:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:27 smithi049 ceph-mon[31696]: Manager daemon x is now available 2024-02-26T06:12:28.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:12:28.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:12:28.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:12:28.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:12:28.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:28 smithi049 ceph-mon[31696]: mgrmap e11: x(active, since 1.01066s) 2024-02-26T06:12:28.905 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:12:28.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-26T06:12:28.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-26T06:12:28.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:12:28.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.951+0000 7f29c1000700 1 Processor -- start 2024-02-26T06:12:28.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.951+0000 7f29c1000700 1 -- start start 2024-02-26T06:12:28.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.952+0000 7f29c1000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc107a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:28.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.952+0000 7f29c1000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc108060 con 0x7f29bc107670 2024-02-26T06:12:28.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.952+0000 7f29b9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:28.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.952+0000 7f29b9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54344/0 (socket says 172.21.15.49:54344) 2024-02-26T06:12:28.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.952+0000 7f29b9e00700 1 -- 172.21.15.49:0/1541565640 learned_addr learned my addr 172.21.15.49:0/1541565640 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:28.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.953+0000 7f29b9e00700 1 -- 172.21.15.49:0/1541565640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc108820 con 0x7f29bc107670 2024-02-26T06:12:28.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.953+0000 7f29b9e00700 1 --2- 172.21.15.49:0/1541565640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc107a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0098a0 tx=0x7f29ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c3deb73d995d24 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:28.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.953+0000 7f29b8a00700 1 -- 172.21.15.49:0/1541565640 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29ac004160 con 0x7f29bc107670 2024-02-26T06:12:28.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.953+0000 7f29b8a00700 1 -- 172.21.15.49:0/1541565640 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29ac0042c0 con 0x7f29bc107670 2024-02-26T06:12:28.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.953+0000 7f29b8a00700 1 -- 172.21.15.49:0/1541565640 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29ac011a80 con 0x7f29bc107670 2024-02-26T06:12:28.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.954+0000 7f29c1000700 1 -- 172.21.15.49:0/1541565640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 msgr2=0x7f29bc107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:28.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.954+0000 7f29c1000700 1 --2- 172.21.15.49:0/1541565640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc107a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0098a0 tx=0x7f29ac009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:28.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.954+0000 7f29c1000700 1 -- 172.21.15.49:0/1541565640 shutdown_connections 2024-02-26T06:12:28.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.954+0000 7f29c1000700 1 --2- 172.21.15.49:0/1541565640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc107a90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:28.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.954+0000 7f29c1000700 1 -- 172.21.15.49:0/1541565640 >> 172.21.15.49:0/1541565640 conn(0x7f29bc102c70 msgr2=0x7f29bc105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:28.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.954+0000 7f29c1000700 1 -- 172.21.15.49:0/1541565640 shutdown_connections 2024-02-26T06:12:28.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.954+0000 7f29c1000700 1 -- 172.21.15.49:0/1541565640 wait complete. 2024-02-26T06:12:28.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.955+0000 7f29c1000700 1 Processor -- start 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.955+0000 7f29c1000700 1 -- start start 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.956+0000 7f29c1000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.956+0000 7f29c1000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc07d0d0 con 0x7f29bc107670 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.956+0000 7f29b9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.956+0000 7f29b9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54360/0 (socket says 172.21.15.49:54360) 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.956+0000 7f29b9e00700 1 -- 172.21.15.49:0/828387325 learned_addr learned my addr 172.21.15.49:0/828387325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.956+0000 7f29b9e00700 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac009550 con 0x7f29bc107670 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.957+0000 7f29b9e00700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc07cb90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29ac011740 tx=0x7f29ac011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.957+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29ac0039e0 con 0x7f29bc107670 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.957+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29ac003b40 con 0x7f29bc107670 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.957+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29ac024740 con 0x7f29bc107670 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.957+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc07b2c0 con 0x7f29bc107670 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.957+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc07b760 con 0x7f29bc107670 2024-02-26T06:12:28.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.958+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f29ac01a430 con 0x7f29bc107670 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.959+0000 7f29b2a00700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.959+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29a003bb60 con 0x7f29a0038f90 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.959+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f29ac04d260 con 0x7f29bc107670 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.959+0000 7f29b9400700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 msgr2=0x7f29a003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4089389406 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:20.959+0000 7f29b9400700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:21.160+0000 7f29b9400700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 msgr2=0x7f29a003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4089389406 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:21.160+0000 7f29b9400700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:21.561+0000 7f29b9400700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 msgr2=0x7f29a003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4089389406 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:21.561+0000 7f29b9400700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:22.362+0000 7f29b9400700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 msgr2=0x7f29a003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4089389406 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:22.362+0000 7f29b9400700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:23.964+0000 7f29b9400700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 msgr2=0x7f29a003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4089389406 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:23.964+0000 7f29b9400700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T06:12:28.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:27.166+0000 7f29b9400700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 msgr2=0x7f29a003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4089389406 2024-02-26T06:12:28.918 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:27.167+0000 7f29b9400700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-26T06:12:28.918 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:27.280+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7f29ac01ab30 con 0x7f29bc107670 2024-02-26T06:12:28.918 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:27.280+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 msgr2=0x7f29a003b450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:12:28.918 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:27.280+0000 7f29b2a00700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:28.918 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.281+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7f29ac04ea70 con 0x7f29bc107670 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.281+0000 7f29b2a00700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f29a003ca40 0x7f29a003ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.281+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29a003bb60 con 0x7f29a003ca40 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.282+0000 7f29b9400700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f29a003ca40 0x7f29a003ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.283+0000 7f29b9400700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f29a003ca40 0x7f29a003ee30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f29a8003a10 tx=0x7f29a8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.284+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f29a003bb60 con 0x7f29a003ca40 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.292+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f29bc0632a0 con 0x7f29a003ca40 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.292+0000 7f29b2a00700 1 -- 172.21.15.49:0/828387325 <== mgr.14152 v2:172.21.15.49:6800/1431851577 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f29bc0632a0 con 0x7f29a003ca40 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f29a003ca40 msgr2=0x7f29a003ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f29a003ca40 0x7f29a003ee30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f29a8003a10 tx=0x7f29a8009270 comp rx=0 tx=0).stop 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 msgr2=0x7f29bc07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc07cb90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29ac011740 tx=0x7f29ac011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 shutdown_connections 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f29a003ca40 0x7f29a003ee30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29bc107670 0x7f29bc07cb90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:28.919 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 --2- 172.21.15.49:0/828387325 >> [v2:172.21.15.49:6800/4089389406,v1:172.21.15.49:6801/4089389406] conn(0x7f29a0038f90 0x7f29a003b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:28.920 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 >> 172.21.15.49:0/828387325 conn(0x7f29bc102c70 msgr2=0x7f29bc103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:28.920 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.293+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 shutdown_connections 2024-02-26T06:12:28.920 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:28.294+0000 7f29c1000700 1 -- 172.21.15.49:0/828387325 wait complete. 2024-02-26T06:12:28.920 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 9 is available 2024-02-26T06:12:28.920 INFO:teuthology.orchestra.run.smithi049.stdout:Generating a dashboard self-signed certificate... 2024-02-26T06:12:29.619 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:29 smithi049 ceph-mon[31696]: [26/Feb/2024:06:12:28] ENGINE Bus STARTING 2024-02-26T06:12:29.619 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:29 smithi049 ceph-mon[31696]: [26/Feb/2024:06:12:28] ENGINE Serving on http://172.21.15.49:8765 2024-02-26T06:12:29.619 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:29 smithi049 ceph-mon[31696]: [26/Feb/2024:06:12:28] ENGINE Serving on https://172.21.15.49:7150 2024-02-26T06:12:29.619 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:29 smithi049 ceph-mon[31696]: [26/Feb/2024:06:12:28] ENGINE Bus STARTED 2024-02-26T06:12:30.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:30 smithi049 ceph-mon[31696]: mgrmap e12: x(active, since 2s) 2024-02-26T06:12:30.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:30 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.333+0000 7f37dde00700 1 Processor -- start 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.334+0000 7f37dde00700 1 -- start start 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.334+0000 7f37dde00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d8107a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.334+0000 7f37dde00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d8108060 con 0x7f37d8107670 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.334+0000 7f37d6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d8107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.335+0000 7f37d6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d8107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56840/0 (socket says 172.21.15.49:56840) 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.335+0000 7f37d6a00700 1 -- 172.21.15.49:0/3054511743 learned_addr learned my addr 172.21.15.49:0/3054511743 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.335+0000 7f37d6a00700 1 -- 172.21.15.49:0/3054511743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d8108820 con 0x7f37d8107670 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.335+0000 7f37d6a00700 1 --2- 172.21.15.49:0/3054511743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d8107a90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37c80098a0 tx=0x7f37c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69bd8da09070584 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.337+0000 7f37d5600700 1 -- 172.21.15.49:0/3054511743 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37c8004160 con 0x7f37d8107670 2024-02-26T06:12:31.509 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.337+0000 7f37d5600700 1 -- 172.21.15.49:0/3054511743 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37c80042c0 con 0x7f37d8107670 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.337+0000 7f37d5600700 1 -- 172.21.15.49:0/3054511743 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37c8011a80 con 0x7f37d8107670 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.338+0000 7f37dde00700 1 -- 172.21.15.49:0/3054511743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 msgr2=0x7f37d8107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.338+0000 7f37dde00700 1 --2- 172.21.15.49:0/3054511743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d8107a90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37c80098a0 tx=0x7f37c8009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.338+0000 7f37dde00700 1 -- 172.21.15.49:0/3054511743 shutdown_connections 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.338+0000 7f37dde00700 1 --2- 172.21.15.49:0/3054511743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d8107a90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.338+0000 7f37dde00700 1 -- 172.21.15.49:0/3054511743 >> 172.21.15.49:0/3054511743 conn(0x7f37d8102c70 msgr2=0x7f37d8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.339+0000 7f37dde00700 1 -- 172.21.15.49:0/3054511743 shutdown_connections 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.339+0000 7f37dde00700 1 -- 172.21.15.49:0/3054511743 wait complete. 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.340+0000 7f37dde00700 1 Processor -- start 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.340+0000 7f37dde00700 1 -- start start 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.340+0000 7f37dde00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.340+0000 7f37dde00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d819b920 con 0x7f37d8107670 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.341+0000 7f37d6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.341+0000 7f37d6a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56844/0 (socket says 172.21.15.49:56844) 2024-02-26T06:12:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.341+0000 7f37d6a00700 1 -- 172.21.15.49:0/347837181 learned_addr learned my addr 172.21.15.49:0/347837181 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.341+0000 7f37d6a00700 1 -- 172.21.15.49:0/347837181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c8009550 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.341+0000 7f37d6a00700 1 --2- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d819b3e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f37c8011740 tx=0x7f37c8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.342+0000 7f37dca00700 1 -- 172.21.15.49:0/347837181 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37c80039e0 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.342+0000 7f37dca00700 1 -- 172.21.15.49:0/347837181 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37c8003b40 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.342+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d819bb20 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.342+0000 7f37dca00700 1 -- 172.21.15.49:0/347837181 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37c8024740 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.342+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d819bfc0 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.343+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d810ba50 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.343+0000 7f37dca00700 1 -- 172.21.15.49:0/347837181 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f37c801a430 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.344+0000 7f37dca00700 1 --2- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37bc038f90 0x7f37bc03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.344+0000 7f37dca00700 1 -- 172.21.15.49:0/347837181 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f37c804d2e0 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.345+0000 7f37d6000700 1 --2- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37bc038f90 0x7f37bc03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.345+0000 7f37d6000700 1 --2- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37bc038f90 0x7f37bc03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f37c4006fd0 tx=0x7f37c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.350+0000 7f37dca00700 1 -- 172.21.15.49:0/347837181 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37c8010380 con 0x7f37d8107670 2024-02-26T06:12:31.511 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.579+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f37d8103e50 con 0x7f37bc038f90 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.848+0000 7f37dca00700 1 -- 172.21.15.49:0/347837181 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f37d8103e50 con 0x7f37bc038f90 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37bc038f90 msgr2=0x7f37bc03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 --2- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37bc038f90 0x7f37bc03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f37c4006fd0 tx=0x7f37c4006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 msgr2=0x7f37d819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 --2- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d819b3e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f37c8011740 tx=0x7f37c8011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 shutdown_connections 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 --2- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37bc038f90 0x7f37bc03b450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 --2- 172.21.15.49:0/347837181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8107670 0x7f37d819b3e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 >> 172.21.15.49:0/347837181 conn(0x7f37d8102c70 msgr2=0x7f37d8103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 shutdown_connections 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:30.851+0000 7f37dde00700 1 -- 172.21.15.49:0/347837181 wait complete. 2024-02-26T06:12:31.512 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial admin user... 2024-02-26T06:12:32.115 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:31 smithi049 ceph-mon[31696]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:12:32.115 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:32.115 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$f.6wRdET9e7ZHVJsWJcmau9WuI7r.GJGdjJvjF/bu428tfEXidqc6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708927952, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.113+0000 7f96fc800700 1 Processor -- start 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.114+0000 7f96fc800700 1 -- start start 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.114+0000 7f96fc800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f4107a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.114+0000 7f96fc800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f4108010 con 0x7f96f4107620 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.115+0000 7f96f9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f4107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.115+0000 7f96f9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f4107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56852/0 (socket says 172.21.15.49:56852) 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.115+0000 7f96f9e00700 1 -- 172.21.15.49:0/2522487019 learned_addr learned my addr 172.21.15.49:0/2522487019 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.115+0000 7f96f9e00700 1 -- 172.21.15.49:0/2522487019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f41087d0 con 0x7f96f4107620 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.116+0000 7f96f9e00700 1 --2- 172.21.15.49:0/2522487019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f4107a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96e80098a0 tx=0x7f96e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1439bef365912b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:33.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.116+0000 7f96f8a00700 1 -- 172.21.15.49:0/2522487019 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96e8004160 con 0x7f96f4107620 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.116+0000 7f96f8a00700 1 -- 172.21.15.49:0/2522487019 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96e80042c0 con 0x7f96f4107620 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.116+0000 7f96f8a00700 1 -- 172.21.15.49:0/2522487019 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96e8011a80 con 0x7f96f4107620 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.117+0000 7f96fc800700 1 -- 172.21.15.49:0/2522487019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 msgr2=0x7f96f4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.117+0000 7f96fc800700 1 --2- 172.21.15.49:0/2522487019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f4107a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96e80098a0 tx=0x7f96e8009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.118+0000 7f96fc800700 1 -- 172.21.15.49:0/2522487019 shutdown_connections 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.118+0000 7f96fc800700 1 --2- 172.21.15.49:0/2522487019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f4107a40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.118+0000 7f96fc800700 1 -- 172.21.15.49:0/2522487019 >> 172.21.15.49:0/2522487019 conn(0x7f96f4102c40 msgr2=0x7f96f4105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.118+0000 7f96fc800700 1 -- 172.21.15.49:0/2522487019 shutdown_connections 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.118+0000 7f96fc800700 1 -- 172.21.15.49:0/2522487019 wait complete. 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.119+0000 7f96fc800700 1 Processor -- start 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.119+0000 7f96fc800700 1 -- start start 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.119+0000 7f96fc800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.119+0000 7f96fc800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f419b890 con 0x7f96f4107620 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.120+0000 7f96f9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.120+0000 7f96f9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56858/0 (socket says 172.21.15.49:56858) 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.120+0000 7f96f9e00700 1 -- 172.21.15.49:0/2605054989 learned_addr learned my addr 172.21.15.49:0/2605054989 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:33.369 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.120+0000 7f96f9e00700 1 -- 172.21.15.49:0/2605054989 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e8009550 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.120+0000 7f96f9e00700 1 --2- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f419b350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96e8011740 tx=0x7f96e8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.121+0000 7f96eea00700 1 -- 172.21.15.49:0/2605054989 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96e80039e0 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.121+0000 7f96eea00700 1 -- 172.21.15.49:0/2605054989 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96e8003b40 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.121+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f419ba90 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.121+0000 7f96eea00700 1 -- 172.21.15.49:0/2605054989 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96e8024740 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.121+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f419bf30 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.122+0000 7f96eea00700 1 -- 172.21.15.49:0/2605054989 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f96e801a430 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.122+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f410ba00 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.123+0000 7f96eea00700 1 --2- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96dc038f90 0x7f96dc03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.123+0000 7f96eea00700 1 -- 172.21.15.49:0/2605054989 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f96e804d300 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.123+0000 7f96f9400700 1 --2- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96dc038f90 0x7f96dc03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.124+0000 7f96f9400700 1 --2- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96dc038f90 0x7f96dc03b450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f96e4006fd0 tx=0x7f96e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.128+0000 7f96eea00700 1 -- 172.21.15.49:0/2605054989 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96e8010380 con 0x7f96f4107620 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.370+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f96f40630c0 con 0x7f96dc038f90 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.632+0000 7f96eea00700 1 -- 172.21.15.49:0/2605054989 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f96f40630c0 con 0x7f96dc038f90 2024-02-26T06:12:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.635+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96dc038f90 msgr2=0x7f96dc03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.635+0000 7f96fc800700 1 --2- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96dc038f90 0x7f96dc03b450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f96e4006fd0 tx=0x7f96e4006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.635+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 msgr2=0x7f96f419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.635+0000 7f96fc800700 1 --2- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f419b350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96e8011740 tx=0x7f96e8011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.635+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 shutdown_connections 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.635+0000 7f96fc800700 1 --2- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96dc038f90 0x7f96dc03b450 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.635+0000 7f96fc800700 1 --2- 172.21.15.49:0/2605054989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f4107620 0x7f96f419b350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.635+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 >> 172.21.15.49:0/2605054989 conn(0x7f96f4102c40 msgr2=0x7f96f41036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.636+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 shutdown_connections 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:32.636+0000 7f96fc800700 1 -- 172.21.15.49:0/2605054989 wait complete. 2024-02-26T06:12:33.371 INFO:teuthology.orchestra.run.smithi049.stdout:Fetching dashboard port number... 2024-02-26T06:12:33.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:33 smithi049 ceph-mon[31696]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:12:33.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:33.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:34.898 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:34 smithi049 ceph-mon[31696]: mgrmap e13: x(active, since 6s) 2024-02-26T06:12:34.898 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:34 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/521685375' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-26T06:12:35.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 8443 2024-02-26T06:12:35.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.142+0000 7ff814a00700 1 Processor -- start 2024-02-26T06:12:35.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.142+0000 7ff814a00700 1 -- start start 2024-02-26T06:12:35.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.143+0000 7ff814a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff810106330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:35.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.143+0000 7ff814a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81007ae50 con 0x7ff810105f10 2024-02-26T06:12:35.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.143+0000 7ff80de00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff810106330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.143+0000 7ff80de00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff810106330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56862/0 (socket says 172.21.15.49:56862) 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.143+0000 7ff80de00700 1 -- 172.21.15.49:0/866037067 learned_addr learned my addr 172.21.15.49:0/866037067 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.144+0000 7ff80de00700 1 -- 172.21.15.49:0/866037067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810106870 con 0x7ff810105f10 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.144+0000 7ff80de00700 1 --2- 172.21.15.49:0/866037067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff810106330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8000098a0 tx=0x7ff800009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1df33b201fcc147 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.145+0000 7ff80ca00700 1 -- 172.21.15.49:0/866037067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff800004160 con 0x7ff810105f10 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.145+0000 7ff80ca00700 1 -- 172.21.15.49:0/866037067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8000042c0 con 0x7ff810105f10 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.145+0000 7ff80ca00700 1 -- 172.21.15.49:0/866037067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff800011a80 con 0x7ff810105f10 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.145+0000 7ff814a00700 1 -- 172.21.15.49:0/866037067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 msgr2=0x7ff810106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.146+0000 7ff814a00700 1 --2- 172.21.15.49:0/866037067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff810106330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8000098a0 tx=0x7ff800009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.146+0000 7ff814a00700 1 -- 172.21.15.49:0/866037067 shutdown_connections 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.146+0000 7ff814a00700 1 --2- 172.21.15.49:0/866037067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff810106330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.146+0000 7ff814a00700 1 -- 172.21.15.49:0/866037067 >> 172.21.15.49:0/866037067 conn(0x7ff810101b90 msgr2=0x7ff810103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.146+0000 7ff814a00700 1 -- 172.21.15.49:0/866037067 shutdown_connections 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.146+0000 7ff814a00700 1 -- 172.21.15.49:0/866037067 wait complete. 2024-02-26T06:12:35.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.147+0000 7ff814a00700 1 Processor -- start 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.147+0000 7ff814a00700 1 -- start start 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.148+0000 7ff814a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff81019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.148+0000 7ff814a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81019dbd0 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.148+0000 7ff80de00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff81019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.148+0000 7ff80de00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff81019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56876/0 (socket says 172.21.15.49:56876) 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.148+0000 7ff80de00700 1 -- 172.21.15.49:0/521685375 learned_addr learned my addr 172.21.15.49:0/521685375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.148+0000 7ff80de00700 1 -- 172.21.15.49:0/521685375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800009550 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.149+0000 7ff80de00700 1 --2- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff81019d690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff800009130 tx=0x7ff800011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.149+0000 7ff806a00700 1 -- 172.21.15.49:0/521685375 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff800003aa0 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.149+0000 7ff806a00700 1 -- 172.21.15.49:0/521685375 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff800003c00 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.149+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81019ddd0 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.149+0000 7ff806a00700 1 -- 172.21.15.49:0/521685375 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8000248a0 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.149+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81019e980 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.151+0000 7ff806a00700 1 -- 172.21.15.49:0/521685375 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7ff80001ac10 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.151+0000 7ff806a00700 1 --2- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff7f4038fd0 0x7ff7f403b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.151+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8100632a0 con 0x7ff810105f10 2024-02-26T06:12:35.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.151+0000 7ff806a00700 1 -- 172.21.15.49:0/521685375 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff80004d360 con 0x7ff810105f10 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.152+0000 7ff80d400700 1 --2- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff7f4038fd0 0x7ff7f403b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.152+0000 7ff80d400700 1 --2- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff7f4038fd0 0x7ff7f403b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc006fd0 tx=0x7ff7fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.157+0000 7ff806a00700 1 -- 172.21.15.49:0/521685375 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff80002d3f0 con 0x7ff810105f10 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.395+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7ff810107f00 con 0x7ff810105f10 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.396+0000 7ff806a00700 1 -- 172.21.15.49:0/521685375 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7ff800018350 con 0x7ff810105f10 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.399+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff7f4038fd0 msgr2=0x7ff7f403b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.399+0000 7ff814a00700 1 --2- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff7f4038fd0 0x7ff7f403b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc006fd0 tx=0x7ff7fc006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.399+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 msgr2=0x7ff81019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.399+0000 7ff814a00700 1 --2- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff81019d690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff800009130 tx=0x7ff800011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.400+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 shutdown_connections 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.400+0000 7ff814a00700 1 --2- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff7f4038fd0 0x7ff7f403b490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.400+0000 7ff814a00700 1 --2- 172.21.15.49:0/521685375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff810105f10 0x7ff81019d690 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.400+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 >> 172.21.15.49:0/521685375 conn(0x7ff810101b90 msgr2=0x7ff810194380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.400+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 shutdown_connections 2024-02-26T06:12:35.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:34.400+0000 7ff814a00700 1 -- 172.21.15.49:0/521685375 wait complete. 2024-02-26T06:12:35.257 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T06:12:35.257 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-02-26T06:12:35.281 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T06:12:35.281 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-02-26T06:12:35.281 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-02-26T06:12:35.281 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-26T06:12:35.285 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph Dashboard is now available at: 2024-02-26T06:12:35.285 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:35.285 INFO:teuthology.orchestra.run.smithi049.stdout: URL: https://smithi049.front.sepia.ceph.com:8443/ 2024-02-26T06:12:35.285 INFO:teuthology.orchestra.run.smithi049.stdout: User: admin 2024-02-26T06:12:35.285 INFO:teuthology.orchestra.run.smithi049.stdout: Password: b800lnl0nc 2024-02-26T06:12:35.286 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:35.286 INFO:teuthology.orchestra.run.smithi049.stdout:Saving cluster configuration to /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config directory 2024-02-26T06:12:35.287 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling autotune for osd_memory_target 2024-02-26T06:12:37.338 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.369+0000 7f587c800700 1 Processor -- start 2024-02-26T06:12:37.338 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.369+0000 7f587c800700 1 -- start start 2024-02-26T06:12:37.338 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.369+0000 7f587c800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:37.338 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.369+0000 7f587c800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587407b820 con 0x7f587407ce30 2024-02-26T06:12:37.338 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.370+0000 7f5879e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:37.338 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.370+0000 7f5879e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587407b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56888/0 (socket says 172.21.15.49:56888) 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.370+0000 7f5879e00700 1 -- 172.21.15.49:0/1016311555 learned_addr learned my addr 172.21.15.49:0/1016311555 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.370+0000 7f5879e00700 1 -- 172.21.15.49:0/1016311555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587407b960 con 0x7f587407ce30 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.370+0000 7f5879e00700 1 --2- 172.21.15.49:0/1016311555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587407b2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58680098a0 tx=0x7f5868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539c06c982946f1b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.371+0000 7f5878a00700 1 -- 172.21.15.49:0/1016311555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5868004160 con 0x7f587407ce30 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.371+0000 7f5878a00700 1 -- 172.21.15.49:0/1016311555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58680042c0 con 0x7f587407ce30 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.371+0000 7f5878a00700 1 -- 172.21.15.49:0/1016311555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5868011a80 con 0x7f587407ce30 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.371+0000 7f587c800700 1 -- 172.21.15.49:0/1016311555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 msgr2=0x7f587407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.371+0000 7f587c800700 1 --2- 172.21.15.49:0/1016311555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587407b2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58680098a0 tx=0x7f5868009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.371+0000 7f587c800700 1 -- 172.21.15.49:0/1016311555 shutdown_connections 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.371+0000 7f587c800700 1 --2- 172.21.15.49:0/1016311555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587407b2e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.371+0000 7f587c800700 1 -- 172.21.15.49:0/1016311555 >> 172.21.15.49:0/1016311555 conn(0x7f5874102d00 msgr2=0x7f5874105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.372+0000 7f587c800700 1 -- 172.21.15.49:0/1016311555 shutdown_connections 2024-02-26T06:12:37.339 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.372+0000 7f587c800700 1 -- 172.21.15.49:0/1016311555 wait complete. 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.373+0000 7f587c800700 1 Processor -- start 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.373+0000 7f587c800700 1 -- start start 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.373+0000 7f587c800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.373+0000 7f587c800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587419dad0 con 0x7f587407ce30 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.373+0000 7f5879e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.373+0000 7f5879e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56894/0 (socket says 172.21.15.49:56894) 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.373+0000 7f5879e00700 1 -- 172.21.15.49:0/929169578 learned_addr learned my addr 172.21.15.49:0/929169578 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.374+0000 7f5879e00700 1 -- 172.21.15.49:0/929169578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868009550 con 0x7f587407ce30 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.374+0000 7f5879e00700 1 --2- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587419d590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5868009130 tx=0x7f5868011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.374+0000 7f586ea00700 1 -- 172.21.15.49:0/929169578 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5868003aa0 con 0x7f587407ce30 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.375+0000 7f586ea00700 1 -- 172.21.15.49:0/929169578 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5868003c00 con 0x7f587407ce30 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.375+0000 7f586ea00700 1 -- 172.21.15.49:0/929169578 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58680248a0 con 0x7f587407ce30 2024-02-26T06:12:37.340 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.375+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587419dcd0 con 0x7f587407ce30 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.375+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587419e170 con 0x7f587407ce30 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.376+0000 7f586ea00700 1 -- 172.21.15.49:0/929169578 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f586801ac10 con 0x7f587407ce30 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.376+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587410dac0 con 0x7f587407ce30 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.376+0000 7f586ea00700 1 --2- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f585c039030 0x7f585c03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.376+0000 7f586ea00700 1 -- 172.21.15.49:0/929169578 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f586804d3c0 con 0x7f587407ce30 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.377+0000 7f5879400700 1 --2- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f585c039030 0x7f585c03b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.378+0000 7f5879400700 1 --2- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f585c039030 0x7f585c03b4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5864006fd0 tx=0x7f5864006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.382+0000 7f586ea00700 1 -- 172.21.15.49:0/929169578 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5868024a60 con 0x7f587407ce30 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.612+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f58741a0a50 con 0x7f587407ce30 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.612+0000 7f586ea00700 1 -- 172.21.15.49:0/929169578 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f5868018350 con 0x7f587407ce30 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f585c039030 msgr2=0x7f585c03b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 --2- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f585c039030 0x7f585c03b4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5864006fd0 tx=0x7f5864006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 msgr2=0x7f587419d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 --2- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587419d590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5868009130 tx=0x7f5868011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 shutdown_connections 2024-02-26T06:12:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 --2- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f585c039030 0x7f585c03b4f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:37.342 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 --2- 172.21.15.49:0/929169578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f587407ce30 0x7f587419d590 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:37.342 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 >> 172.21.15.49:0/929169578 conn(0x7f5874102d00 msgr2=0x7f5874103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:37.342 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.615+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 shutdown_connections 2024-02-26T06:12:37.342 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:36.616+0000 7f587c800700 1 -- 172.21.15.49:0/929169578 wait complete. 2024-02-26T06:12:39.244 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.171+0000 7faa45400700 1 Processor -- start 2024-02-26T06:12:39.245 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.171+0000 7faa45400700 1 -- start start 2024-02-26T06:12:39.245 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.172+0000 7faa45400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa4007dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.172+0000 7faa45400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4007e270 con 0x7faa4007d880 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.172+0000 7faa3e800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa4007dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.172+0000 7faa3e800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa4007dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56902/0 (socket says 172.21.15.49:56902) 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.172+0000 7faa3e800700 1 -- 172.21.15.49:0/2346941327 learned_addr learned my addr 172.21.15.49:0/2346941327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.173+0000 7faa3e800700 1 -- 172.21.15.49:0/2346941327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4007ea80 con 0x7faa4007d880 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.173+0000 7faa3e800700 1 --2- 172.21.15.49:0/2346941327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa4007dca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa300098a0 tx=0x7faa30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaf937faf3d9da0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.173+0000 7faa3de00700 1 -- 172.21.15.49:0/2346941327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa30004160 con 0x7faa4007d880 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.173+0000 7faa3de00700 1 -- 172.21.15.49:0/2346941327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa300042c0 con 0x7faa4007d880 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.173+0000 7faa3de00700 1 -- 172.21.15.49:0/2346941327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa30011a80 con 0x7faa4007d880 2024-02-26T06:12:39.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.174+0000 7faa45400700 1 -- 172.21.15.49:0/2346941327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 msgr2=0x7faa4007dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.174+0000 7faa45400700 1 --2- 172.21.15.49:0/2346941327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa4007dca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa300098a0 tx=0x7faa30009bb0 comp rx=0 tx=0).stop 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.174+0000 7faa45400700 1 -- 172.21.15.49:0/2346941327 shutdown_connections 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.174+0000 7faa45400700 1 --2- 172.21.15.49:0/2346941327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa4007dca0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.174+0000 7faa45400700 1 -- 172.21.15.49:0/2346941327 >> 172.21.15.49:0/2346941327 conn(0x7faa4007ce30 msgr2=0x7faa401074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.174+0000 7faa45400700 1 -- 172.21.15.49:0/2346941327 shutdown_connections 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.174+0000 7faa45400700 1 -- 172.21.15.49:0/2346941327 wait complete. 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.175+0000 7faa45400700 1 Processor -- start 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.175+0000 7faa45400700 1 -- start start 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.176+0000 7faa45400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa401a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.176+0000 7faa45400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401a1e10 con 0x7faa4007d880 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.176+0000 7faa3e800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa401a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.176+0000 7faa3e800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa401a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56914/0 (socket says 172.21.15.49:56914) 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.176+0000 7faa3e800700 1 -- 172.21.15.49:0/3298933514 learned_addr learned my addr 172.21.15.49:0/3298933514 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:39.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.177+0000 7faa3e800700 1 -- 172.21.15.49:0/3298933514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa30009550 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.177+0000 7faa3e800700 1 --2- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa401a18d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa30011740 tx=0x7faa30011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.177+0000 7faa37400700 1 -- 172.21.15.49:0/3298933514 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa30003b20 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.177+0000 7faa37400700 1 -- 172.21.15.49:0/3298933514 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa3001a910 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.177+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa401a2010 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.177+0000 7faa37400700 1 -- 172.21.15.49:0/3298933514 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa300248c0 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.177+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa401a24b0 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.179+0000 7faa37400700 1 -- 172.21.15.49:0/3298933514 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7faa3001aa80 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.179+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4007fca0 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.179+0000 7faa37400700 1 --2- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa24039030 0x7faa2403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.179+0000 7faa37400700 1 -- 172.21.15.49:0/3298933514 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faa3004d530 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.179+0000 7faa37e00700 1 --2- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa24039030 0x7faa2403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.180+0000 7faa37e00700 1 --2- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa24039030 0x7faa2403b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa28006fd0 tx=0x7faa28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.184+0000 7faa37400700 1 -- 172.21.15.49:0/3298933514 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa3001adf0 con 0x7faa4007d880 2024-02-26T06:12:39.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.534+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7faa4019afa0 con 0x7faa4007d880 2024-02-26T06:12:39.252 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.540+0000 7faa37400700 1 -- 172.21.15.49:0/3298933514 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7faa3002cc80 con 0x7faa4007d880 2024-02-26T06:12:39.252 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa24039030 msgr2=0x7faa2403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:39.253 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 --2- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa24039030 0x7faa2403b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa28006fd0 tx=0x7faa28006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:39.253 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 msgr2=0x7faa401a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:39.253 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 --2- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa401a18d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa30011740 tx=0x7faa30011b00 comp rx=0 tx=0).stop 2024-02-26T06:12:39.253 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 shutdown_connections 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 --2- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa24039030 0x7faa2403b4f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 --2- 172.21.15.49:0/3298933514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa4007d880 0x7faa401a18d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 >> 172.21.15.49:0/3298933514 conn(0x7faa4007ce30 msgr2=0x7faa40107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 shutdown_connections 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-02-26T06:12:38.543+0000 7faa45400700 1 -- 172.21.15.49:0/3298933514 wait complete. 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:Or, if you are only running a single cluster on this host: 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: ceph telemetry on 2024-02-26T06:12:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:39.255 INFO:teuthology.orchestra.run.smithi049.stdout:For more information see: 2024-02-26T06:12:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-26T06:12:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:39.255 INFO:teuthology.orchestra.run.smithi049.stdout:Bootstrap complete. 2024-02-26T06:12:39.382 INFO:tasks.cephadm:Fetching config... 2024-02-26T06:12:39.383 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:12:39.383 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-26T06:12:39.432 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-26T06:12:39.432 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:12:39.432 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-26T06:12:39.475 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-26T06:12:39.475 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:12:39.475 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-26T06:12:39.612 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-26T06:12:39.612 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:12:39.612 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-26T06:12:39.712 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-26T06:12:39.712 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI6LUyyf7R5ACDWK/pSEdOTtIjYTo69zD8crQYolIeoAU+ad1SaSmQ+ij0SSeZ9nhsgMa3fwNWFfuzwMWigdnsNyI48ZCPgQOKHBzOOiRzSZp8EOllfUFWgc7Ip1tHr1RxVivjG+YW30Wk60iSIzTyg6jWChsd3/QPEDhk+YvxpX4O79/1HDdvMd63AyYo6106tJlxjc8zuhkRhOa/K6yPJGv1dp21+T0m10EUdlzK2PbaeunZRU9Bawlfl6V1aJSrtXoHfSty2X43HrNg26WdqtI4qAiKGd8K4KGUGq4bos6iB68s0kzmAnNnce69zNX7lrGpOSn1N6RgzvdlmXvYq2XYtKDqkf3pnzu4JlKQBvmBIb65y2TmVlMc9tNvZVuZLHmwoY1+TmmOlz2x623tGKUal2fr0khpr/VVSkx83rQLDD4FRdtfyhLzT0PsTg/E2QNtYRGl4QPjs4qWqD8oHNcE/isMM6lm2//tW7YLnLWSRQsN7GBl+fCe6hdZyc8= ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T06:12:39.788 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:39 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3298933514' entity='client.admin' 2024-02-26T06:12:39.843 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI6LUyyf7R5ACDWK/pSEdOTtIjYTo69zD8crQYolIeoAU+ad1SaSmQ+ij0SSeZ9nhsgMa3fwNWFfuzwMWigdnsNyI48ZCPgQOKHBzOOiRzSZp8EOllfUFWgc7Ip1tHr1RxVivjG+YW30Wk60iSIzTyg6jWChsd3/QPEDhk+YvxpX4O79/1HDdvMd63AyYo6106tJlxjc8zuhkRhOa/K6yPJGv1dp21+T0m10EUdlzK2PbaeunZRU9Bawlfl6V1aJSrtXoHfSty2X43HrNg26WdqtI4qAiKGd8K4KGUGq4bos6iB68s0kzmAnNnce69zNX7lrGpOSn1N6RgzvdlmXvYq2XYtKDqkf3pnzu4JlKQBvmBIb65y2TmVlMc9tNvZVuZLHmwoY1+TmmOlz2x623tGKUal2fr0khpr/VVSkx83rQLDD4FRdtfyhLzT0PsTg/E2QNtYRGl4QPjs4qWqD8oHNcE/isMM6lm2//tW7YLnLWSRQsN7GBl+fCe6hdZyc8= ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:12:39.915 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI6LUyyf7R5ACDWK/pSEdOTtIjYTo69zD8crQYolIeoAU+ad1SaSmQ+ij0SSeZ9nhsgMa3fwNWFfuzwMWigdnsNyI48ZCPgQOKHBzOOiRzSZp8EOllfUFWgc7Ip1tHr1RxVivjG+YW30Wk60iSIzTyg6jWChsd3/QPEDhk+YvxpX4O79/1HDdvMd63AyYo6106tJlxjc8zuhkRhOa/K6yPJGv1dp21+T0m10EUdlzK2PbaeunZRU9Bawlfl6V1aJSrtXoHfSty2X43HrNg26WdqtI4qAiKGd8K4KGUGq4bos6iB68s0kzmAnNnce69zNX7lrGpOSn1N6RgzvdlmXvYq2XYtKDqkf3pnzu4JlKQBvmBIb65y2TmVlMc9tNvZVuZLHmwoY1+TmmOlz2x623tGKUal2fr0khpr/VVSkx83rQLDD4FRdtfyhLzT0PsTg/E2QNtYRGl4QPjs4qWqD8oHNcE/isMM6lm2//tW7YLnLWSRQsN7GBl+fCe6hdZyc8= ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T06:12:39.962 INFO:teuthology.orchestra.run.smithi080.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI6LUyyf7R5ACDWK/pSEdOTtIjYTo69zD8crQYolIeoAU+ad1SaSmQ+ij0SSeZ9nhsgMa3fwNWFfuzwMWigdnsNyI48ZCPgQOKHBzOOiRzSZp8EOllfUFWgc7Ip1tHr1RxVivjG+YW30Wk60iSIzTyg6jWChsd3/QPEDhk+YvxpX4O79/1HDdvMd63AyYo6106tJlxjc8zuhkRhOa/K6yPJGv1dp21+T0m10EUdlzK2PbaeunZRU9Bawlfl6V1aJSrtXoHfSty2X43HrNg26WdqtI4qAiKGd8K4KGUGq4bos6iB68s0kzmAnNnce69zNX7lrGpOSn1N6RgzvdlmXvYq2XYtKDqkf3pnzu4JlKQBvmBIb65y2TmVlMc9tNvZVuZLHmwoY1+TmmOlz2x623tGKUal2fr0khpr/VVSkx83rQLDD4FRdtfyhLzT0PsTg/E2QNtYRGl4QPjs4qWqD8oHNcE/isMM6lm2//tW7YLnLWSRQsN7GBl+fCe6hdZyc8= ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:12:39.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI6LUyyf7R5ACDWK/pSEdOTtIjYTo69zD8crQYolIeoAU+ad1SaSmQ+ij0SSeZ9nhsgMa3fwNWFfuzwMWigdnsNyI48ZCPgQOKHBzOOiRzSZp8EOllfUFWgc7Ip1tHr1RxVivjG+YW30Wk60iSIzTyg6jWChsd3/QPEDhk+YvxpX4O79/1HDdvMd63AyYo6106tJlxjc8zuhkRhOa/K6yPJGv1dp21+T0m10EUdlzK2PbaeunZRU9Bawlfl6V1aJSrtXoHfSty2X43HrNg26WdqtI4qAiKGd8K4KGUGq4bos6iB68s0kzmAnNnce69zNX7lrGpOSn1N6RgzvdlmXvYq2XYtKDqkf3pnzu4JlKQBvmBIb65y2TmVlMc9tNvZVuZLHmwoY1+TmmOlz2x623tGKUal2fr0khpr/VVSkx83rQLDD4FRdtfyhLzT0PsTg/E2QNtYRGl4QPjs4qWqD8oHNcE/isMM6lm2//tW7YLnLWSRQsN7GBl+fCe6hdZyc8= ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T06:12:40.022 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI6LUyyf7R5ACDWK/pSEdOTtIjYTo69zD8crQYolIeoAU+ad1SaSmQ+ij0SSeZ9nhsgMa3fwNWFfuzwMWigdnsNyI48ZCPgQOKHBzOOiRzSZp8EOllfUFWgc7Ip1tHr1RxVivjG+YW30Wk60iSIzTyg6jWChsd3/QPEDhk+YvxpX4O79/1HDdvMd63AyYo6106tJlxjc8zuhkRhOa/K6yPJGv1dp21+T0m10EUdlzK2PbaeunZRU9Bawlfl6V1aJSrtXoHfSty2X43HrNg26WdqtI4qAiKGd8K4KGUGq4bos6iB68s0kzmAnNnce69zNX7lrGpOSn1N6RgzvdlmXvYq2XYtKDqkf3pnzu4JlKQBvmBIb65y2TmVlMc9tNvZVuZLHmwoY1+TmmOlz2x623tGKUal2fr0khpr/VVSkx83rQLDD4FRdtfyhLzT0PsTg/E2QNtYRGl4QPjs4qWqD8oHNcE/isMM6lm2//tW7YLnLWSRQsN7GBl+fCe6hdZyc8= ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:12:40.037 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-26T06:12:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:12:43.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.097+0000 7f11a5600700 1 -- 172.21.15.49:0/2695717423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 msgr2=0x7f11a0104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.097+0000 7f11a5600700 1 --2- 172.21.15.49:0/2695717423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 0x7f11a0104520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1190009910 tx=0x7f1190009c20 comp rx=0 tx=0).stop 2024-02-26T06:12:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.097+0000 7f11a5600700 1 -- 172.21.15.49:0/2695717423 shutdown_connections 2024-02-26T06:12:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.097+0000 7f11a5600700 1 --2- 172.21.15.49:0/2695717423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 0x7f11a0104520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.097+0000 7f11a5600700 1 -- 172.21.15.49:0/2695717423 >> 172.21.15.49:0/2695717423 conn(0x7f11a00fbb40 msgr2=0x7f11a00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.098+0000 7f11a5600700 1 -- 172.21.15.49:0/2695717423 shutdown_connections 2024-02-26T06:12:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.098+0000 7f11a5600700 1 -- 172.21.15.49:0/2695717423 wait complete. 2024-02-26T06:12:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.099+0000 7f11a5600700 1 Processor -- start 2024-02-26T06:12:43.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.099+0000 7f11a5600700 1 -- start start 2024-02-26T06:12:43.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.100+0000 7f11a5600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 0x7f11a0198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:43.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.100+0000 7f11a5600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a01992f0 con 0x7f11a0102130 2024-02-26T06:12:43.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.100+0000 7f119ea00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 0x7f11a0198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:43.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.100+0000 7f119ea00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 0x7f11a0198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33716/0 (socket says 172.21.15.49:33716) 2024-02-26T06:12:43.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.100+0000 7f119ea00700 1 -- 172.21.15.49:0/1472881108 learned_addr learned my addr 172.21.15.49:0/1472881108 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:43.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.101+0000 7f119ea00700 1 -- 172.21.15.49:0/1472881108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11900095f0 con 0x7f11a0102130 2024-02-26T06:12:43.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.101+0000 7f119ea00700 1 --2- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 0x7f11a0198db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1190004f40 tx=0x7f1190005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:43.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.101+0000 7f1197400700 1 -- 172.21.15.49:0/1472881108 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f119001d070 con 0x7f11a0102130 2024-02-26T06:12:43.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.101+0000 7f1197400700 1 -- 172.21.15.49:0/1472881108 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119000be40 con 0x7f11a0102130 2024-02-26T06:12:43.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.101+0000 7f1197400700 1 -- 172.21.15.49:0/1472881108 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f119000f460 con 0x7f11a0102130 2024-02-26T06:12:43.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.101+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a01994f0 con 0x7f11a0102130 2024-02-26T06:12:43.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.101+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a0199990 con 0x7f11a0102130 2024-02-26T06:12:43.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.103+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a0192900 con 0x7f11a0102130 2024-02-26T06:12:43.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.103+0000 7f1197400700 1 -- 172.21.15.49:0/1472881108 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f11900226c0 con 0x7f11a0102130 2024-02-26T06:12:43.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.103+0000 7f1197400700 1 --2- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1184038fe0 0x7f118403b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:43.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.103+0000 7f1197400700 1 -- 172.21.15.49:0/1472881108 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f119004d390 con 0x7f11a0102130 2024-02-26T06:12:43.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.104+0000 7f1197e00700 1 --2- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1184038fe0 0x7f118403b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:43.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.104+0000 7f1197e00700 1 --2- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1184038fe0 0x7f118403b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1188006fd0 tx=0x7f1188006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:43.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.109+0000 7f1197400700 1 -- 172.21.15.49:0/1472881108 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f119002baa0 con 0x7f11a0102130 2024-02-26T06:12:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.331+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f11a00632a0 con 0x7f11a0102130 2024-02-26T06:12:43.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.337+0000 7f1197400700 1 -- 172.21.15.49:0/1472881108 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1190027020 con 0x7f11a0102130 2024-02-26T06:12:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.340+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1184038fe0 msgr2=0x7f118403b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.340+0000 7f11a5600700 1 --2- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1184038fe0 0x7f118403b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1188006fd0 tx=0x7f1188006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.340+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 msgr2=0x7f11a0198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.340+0000 7f11a5600700 1 --2- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 0x7f11a0198db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1190004f40 tx=0x7f1190005020 comp rx=0 tx=0).stop 2024-02-26T06:12:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.340+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 shutdown_connections 2024-02-26T06:12:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.340+0000 7f11a5600700 1 --2- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1184038fe0 0x7f118403b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.340+0000 7f11a5600700 1 --2- 172.21.15.49:0/1472881108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a0102130 0x7f11a0198db0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.340+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 >> 172.21.15.49:0/1472881108 conn(0x7f11a00fbb40 msgr2=0x7f11a00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.341+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 shutdown_connections 2024-02-26T06:12:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:43.341+0000 7f11a5600700 1 -- 172.21.15.49:0/1472881108 wait complete. 2024-02-26T06:12:43.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:43.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:43.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:12:43.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:43.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:12:43.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:43.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1472881108' entity='client.admin' 2024-02-26T06:12:43.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:12:43.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:44.125 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-26T06:12:44.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-26T06:12:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:12:46.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.697+0000 7f8c86c00700 1 -- 172.21.15.49:0/1521944749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 msgr2=0x7f8c801082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:46.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.697+0000 7f8c86c00700 1 --2- 172.21.15.49:0/1521944749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 0x7f8c801082a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009910 tx=0x7f8c70009c20 comp rx=0 tx=0).stop 2024-02-26T06:12:46.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.697+0000 7f8c86c00700 1 -- 172.21.15.49:0/1521944749 shutdown_connections 2024-02-26T06:12:46.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.697+0000 7f8c86c00700 1 --2- 172.21.15.49:0/1521944749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 0x7f8c801082a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:46.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.697+0000 7f8c86c00700 1 -- 172.21.15.49:0/1521944749 >> 172.21.15.49:0/1521944749 conn(0x7f8c80076d60 msgr2=0x7f8c800791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:46.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.697+0000 7f8c86c00700 1 -- 172.21.15.49:0/1521944749 shutdown_connections 2024-02-26T06:12:46.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.698+0000 7f8c86c00700 1 -- 172.21.15.49:0/1521944749 wait complete. 2024-02-26T06:12:46.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.699+0000 7f8c86c00700 1 Processor -- start 2024-02-26T06:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.699+0000 7f8c86c00700 1 -- start start 2024-02-26T06:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.699+0000 7f8c86c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 0x7f8c8019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.699+0000 7f8c86c00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8019d6f0 con 0x7f8c80107e80 2024-02-26T06:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.699+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 0x7f8c8019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.700+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 0x7f8c8019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33730/0 (socket says 172.21.15.49:33730) 2024-02-26T06:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.700+0000 7f8c7fe00700 1 -- 172.21.15.49:0/3978663291 learned_addr learned my addr 172.21.15.49:0/3978663291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.700+0000 7f8c7fe00700 1 -- 172.21.15.49:0/3978663291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700095f0 con 0x7f8c80107e80 2024-02-26T06:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.700+0000 7f8c7fe00700 1 --2- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 0x7f8c8019d1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c70005950 tx=0x7f8c70004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.701+0000 7f8c7d600700 1 -- 172.21.15.49:0/3978663291 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c7001d070 con 0x7f8c80107e80 2024-02-26T06:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.701+0000 7f8c7d600700 1 -- 172.21.15.49:0/3978663291 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c70022470 con 0x7f8c80107e80 2024-02-26T06:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.701+0000 7f8c7d600700 1 -- 172.21.15.49:0/3978663291 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c7000f5f0 con 0x7f8c80107e80 2024-02-26T06:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.701+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8019d8f0 con 0x7f8c80107e80 2024-02-26T06:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.701+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8019dd90 con 0x7f8c80107e80 2024-02-26T06:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.702+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c80196ad0 con 0x7f8c80107e80 2024-02-26T06:12:46.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.702+0000 7f8c7d600700 1 -- 172.21.15.49:0/3978663291 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f8c700225e0 con 0x7f8c80107e80 2024-02-26T06:12:46.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.703+0000 7f8c7d600700 1 --2- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8c64038f80 0x7f8c6403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:46.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.703+0000 7f8c7d600700 1 -- 172.21.15.49:0/3978663291 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8c7004d2b0 con 0x7f8c80107e80 2024-02-26T06:12:46.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.703+0000 7f8c77e00700 1 --2- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8c64038f80 0x7f8c6403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:46.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.704+0000 7f8c77e00700 1 --2- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8c64038f80 0x7f8c6403b440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8c68006fd0 tx=0x7f8c68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:46.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.708+0000 7f8c7d600700 1 -- 172.21.15.49:0/3978663291 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c70020550 con 0x7f8c80107e80 2024-02-26T06:12:46.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.938+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c8002d7d0 con 0x7f8c64038f80 2024-02-26T06:12:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.946+0000 7f8c7d600700 1 -- 172.21.15.49:0/3978663291 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8c8002d7d0 con 0x7f8c64038f80 2024-02-26T06:12:46.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8c64038f80 msgr2=0x7f8c6403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:46.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 --2- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8c64038f80 0x7f8c6403b440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8c68006fd0 tx=0x7f8c68006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:46.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 msgr2=0x7f8c8019d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:46.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 --2- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 0x7f8c8019d1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c70005950 tx=0x7f8c70004ff0 comp rx=0 tx=0).stop 2024-02-26T06:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 shutdown_connections 2024-02-26T06:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 --2- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8c64038f80 0x7f8c6403b440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 --2- 172.21.15.49:0/3978663291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c80107e80 0x7f8c8019d1b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 >> 172.21.15.49:0/3978663291 conn(0x7f8c80076d60 msgr2=0x7f8c80077770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 shutdown_connections 2024-02-26T06:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:46.949+0000 7f8c86c00700 1 -- 172.21.15.49:0/3978663291 wait complete. 2024-02-26T06:12:47.692 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi080 2024-02-26T06:12:47.692 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:12:47.692 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/etc/ceph/ceph.conf 2024-02-26T06:12:47.710 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:12:47.710 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:12:47.767 INFO:tasks.cephadm:Adding host smithi080 to orchestrator... 2024-02-26T06:12:47.768 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch host add smithi080 2024-02-26T06:12:48.217 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:12:48.364 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:47 smithi049 ceph-mon[31696]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:12:48.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:48.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:12:48.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:12:48.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:12:48.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:47 smithi049 ceph-mon[31696]: Updating smithi049:/etc/ceph/ceph.conf 2024-02-26T06:12:49.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:48 smithi049 ceph-mon[31696]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:49.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:48 smithi049 ceph-mon[31696]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:12:50.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:49 smithi049 ceph-mon[31696]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:12:50.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:49 smithi049 ceph-mon[31696]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:12:50.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:50.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:50.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.243+0000 7f576c400700 1 -- 172.21.15.49:0/3460037428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 msgr2=0x7f5764074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.243+0000 7f576c400700 1 --2- 172.21.15.49:0/3460037428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 0x7f5764074420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f575800b0a0 tx=0x7f575800b3b0 comp rx=0 tx=0).stop 2024-02-26T06:12:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.243+0000 7f576c400700 1 -- 172.21.15.49:0/3460037428 shutdown_connections 2024-02-26T06:12:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.243+0000 7f576c400700 1 --2- 172.21.15.49:0/3460037428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 0x7f5764074420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.243+0000 7f576c400700 1 -- 172.21.15.49:0/3460037428 >> 172.21.15.49:0/3460037428 conn(0x7f57640fd170 msgr2=0x7f57640ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.243+0000 7f576c400700 1 -- 172.21.15.49:0/3460037428 shutdown_connections 2024-02-26T06:12:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.243+0000 7f576c400700 1 -- 172.21.15.49:0/3460037428 wait complete. 2024-02-26T06:12:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.244+0000 7f576c400700 1 Processor -- start 2024-02-26T06:12:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.244+0000 7f576c400700 1 -- start start 2024-02-26T06:12:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.244+0000 7f576c400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 0x7f5764199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.244+0000 7f576c400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5764199740 con 0x7f5764074040 2024-02-26T06:12:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.245+0000 7f5769a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 0x7f5764199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.245+0000 7f5769a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 0x7f5764199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55212/0 (socket says 172.21.15.49:55212) 2024-02-26T06:12:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.245+0000 7f5769a00700 1 -- 172.21.15.49:0/599481965 learned_addr learned my addr 172.21.15.49:0/599481965 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.245+0000 7f5769a00700 1 -- 172.21.15.49:0/599481965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5758009d00 con 0x7f5764074040 2024-02-26T06:12:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.245+0000 7f5769a00700 1 --2- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 0x7f5764199200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5758000f80 tx=0x7f57580075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:50.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.246+0000 7f5762000700 1 -- 172.21.15.49:0/599481965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f575800e040 con 0x7f5764074040 2024-02-26T06:12:50.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.246+0000 7f5762000700 1 -- 172.21.15.49:0/599481965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5758007bc0 con 0x7f5764074040 2024-02-26T06:12:50.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.246+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5764199940 con 0x7f5764074040 2024-02-26T06:12:50.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.246+0000 7f5762000700 1 -- 172.21.15.49:0/599481965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f575801d5d0 con 0x7f5764074040 2024-02-26T06:12:50.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.246+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5764199d60 con 0x7f5764074040 2024-02-26T06:12:50.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.247+0000 7f5762000700 1 -- 172.21.15.49:0/599481965 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f5758004040 con 0x7f5764074040 2024-02-26T06:12:50.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.247+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57640632a0 con 0x7f5764074040 2024-02-26T06:12:50.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.248+0000 7f5762000700 1 --2- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f574c039020 0x7f574c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:50.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.248+0000 7f5762000700 1 -- 172.21.15.49:0/599481965 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f575804d250 con 0x7f5764074040 2024-02-26T06:12:50.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.248+0000 7f5769000700 1 --2- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f574c039020 0x7f574c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:50.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.249+0000 7f5769000700 1 --2- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f574c039020 0x7f574c03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5750006fd0 tx=0x7f5750006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:50.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.253+0000 7f5762000700 1 -- 172.21.15.49:0/599481965 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f575801c5c0 con 0x7f5764074040 2024-02-26T06:12:50.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:50.481+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi080", "target": ["mon-mgr", ""]}) v1 -- 0x7f576410de10 con 0x7f574c039020 2024-02-26T06:12:51.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:50 smithi049 ceph-mon[31696]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:52.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:51 smithi049 ceph-mon[31696]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi080", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:12:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:52 smithi049 ceph-mon[31696]: Deploying cephadm binary to smithi080 2024-02-26T06:12:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:52 smithi049 ceph-mon[31696]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:55.362 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi080' with addr '172.21.15.80' 2024-02-26T06:12:55.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.360+0000 7f5762000700 1 -- 172.21.15.49:0/599481965 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f576410de10 con 0x7f574c039020 2024-02-26T06:12:55.362 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:54 smithi049 ceph-mon[31696]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:55.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f574c039020 msgr2=0x7f574c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:55.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 --2- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f574c039020 0x7f574c03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5750006fd0 tx=0x7f5750006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:55.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 msgr2=0x7f5764199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 --2- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 0x7f5764199200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5758000f80 tx=0x7f57580075c0 comp rx=0 tx=0).stop 2024-02-26T06:12:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 shutdown_connections 2024-02-26T06:12:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 --2- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f574c039020 0x7f574c03b4e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 --2- 172.21.15.49:0/599481965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5764074040 0x7f5764199200 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 >> 172.21.15.49:0/599481965 conn(0x7f57640fd170 msgr2=0x7f5764105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 shutdown_connections 2024-02-26T06:12:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:55.365+0000 7f576c400700 1 -- 172.21.15.49:0/599481965 wait complete. 2024-02-26T06:12:56.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-26T06:12:56.430 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:56 smithi049 ceph-mon[31696]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:56.430 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:56.430 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:56 smithi049 ceph-mon[31696]: Added host smithi080 2024-02-26T06:12:56.430 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:12:56.430 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:56.894 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:12:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:57 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:58.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:58 smithi049 ceph-mon[31696]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:12:58.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:12:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:12:58.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.915+0000 7fee5ea00700 1 -- 172.21.15.49:0/2377126668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 msgr2=0x7fee58109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:58.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.915+0000 7fee5ea00700 1 --2- 172.21.15.49:0/2377126668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 0x7fee58109a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009910 tx=0x7fee4c009c20 comp rx=0 tx=0).stop 2024-02-26T06:12:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.915+0000 7fee5ea00700 1 -- 172.21.15.49:0/2377126668 shutdown_connections 2024-02-26T06:12:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.915+0000 7fee5ea00700 1 --2- 172.21.15.49:0/2377126668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 0x7fee58109a80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.915+0000 7fee5ea00700 1 -- 172.21.15.49:0/2377126668 >> 172.21.15.49:0/2377126668 conn(0x7fee580ff170 msgr2=0x7fee58101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.916+0000 7fee5ea00700 1 -- 172.21.15.49:0/2377126668 shutdown_connections 2024-02-26T06:12:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.916+0000 7fee5ea00700 1 -- 172.21.15.49:0/2377126668 wait complete. 2024-02-26T06:12:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.917+0000 7fee5ea00700 1 Processor -- start 2024-02-26T06:12:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.917+0000 7fee5ea00700 1 -- start start 2024-02-26T06:12:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.917+0000 7fee5ea00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 0x7fee58199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.917+0000 7fee5ea00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee581995c0 con 0x7fee581096a0 2024-02-26T06:12:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.917+0000 7fee5d600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 0x7fee58199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.917+0000 7fee5d600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 0x7fee58199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55238/0 (socket says 172.21.15.49:55238) 2024-02-26T06:12:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.917+0000 7fee5d600700 1 -- 172.21.15.49:0/1300794873 learned_addr learned my addr 172.21.15.49:0/1300794873 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:12:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.918+0000 7fee5d600700 1 -- 172.21.15.49:0/1300794873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0095f0 con 0x7fee581096a0 2024-02-26T06:12:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.918+0000 7fee5d600700 1 --2- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 0x7fee58199080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee4c004f40 tx=0x7fee4c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.919+0000 7fee56000700 1 -- 172.21.15.49:0/1300794873 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4c01d070 con 0x7fee581096a0 2024-02-26T06:12:58.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.919+0000 7fee56000700 1 -- 172.21.15.49:0/1300794873 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee4c00bbd0 con 0x7fee581096a0 2024-02-26T06:12:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.919+0000 7fee56000700 1 -- 172.21.15.49:0/1300794873 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4c003900 con 0x7fee581096a0 2024-02-26T06:12:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.919+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee58199820 con 0x7fee581096a0 2024-02-26T06:12:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.919+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee58199ca0 con 0x7fee581096a0 2024-02-26T06:12:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.920+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee58104de0 con 0x7fee581096a0 2024-02-26T06:12:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.920+0000 7fee56000700 1 -- 172.21.15.49:0/1300794873 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fee4c00bd40 con 0x7fee581096a0 2024-02-26T06:12:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.921+0000 7fee56000700 1 --2- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee40038f80 0x7fee4003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:12:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.921+0000 7fee56000700 1 -- 172.21.15.49:0/1300794873 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee4c04cdc0 con 0x7fee581096a0 2024-02-26T06:12:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.921+0000 7fee5cc00700 1 --2- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee40038f80 0x7fee4003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:12:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.922+0000 7fee5cc00700 1 --2- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee40038f80 0x7fee4003b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fee44006fd0 tx=0x7fee44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:12:58.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:58.929+0000 7fee56000700 1 -- 172.21.15.49:0/1300794873 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee4c02ade0 con 0x7fee581096a0 2024-02-26T06:12:59.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.161+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee5810dbf0 con 0x7fee40038f80 2024-02-26T06:12:59.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.165+0000 7fee56000700 1 -- 172.21.15.49:0/1300794873 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fee5810dbf0 con 0x7fee40038f80 2024-02-26T06:12:59.165 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:12:59.165 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.80", "hostname": "smithi080", "labels": [], "status": ""}] 2024-02-26T06:12:59.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.167+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee40038f80 msgr2=0x7fee4003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:59.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.167+0000 7fee5ea00700 1 --2- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee40038f80 0x7fee4003b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fee44006fd0 tx=0x7fee44006e40 comp rx=0 tx=0).stop 2024-02-26T06:12:59.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.167+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 msgr2=0x7fee58199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:12:59.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.167+0000 7fee5ea00700 1 --2- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 0x7fee58199080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee4c004f40 tx=0x7fee4c005020 comp rx=0 tx=0).stop 2024-02-26T06:12:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.168+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 shutdown_connections 2024-02-26T06:12:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.168+0000 7fee5ea00700 1 --2- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee40038f80 0x7fee4003b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.168+0000 7fee5ea00700 1 --2- 172.21.15.49:0/1300794873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee581096a0 0x7fee58199080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:12:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.168+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 >> 172.21.15.49:0/1300794873 conn(0x7fee580ff170 msgr2=0x7fee5810b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:12:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.168+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 shutdown_connections 2024-02-26T06:12:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:12:59.168+0000 7fee5ea00700 1 -- 172.21.15.49:0/1300794873 wait complete. 2024-02-26T06:12:59.846 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi115 2024-02-26T06:12:59.846 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:12:59.846 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.conf 2024-02-26T06:12:59.864 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:12:59.864 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:12:59.922 INFO:tasks.cephadm:Adding host smithi115 to orchestrator... 2024-02-26T06:12:59.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch host add smithi115 2024-02-26T06:13:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:13:00.672 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:00 smithi049 ceph-mon[31696]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:13:00.672 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:00 smithi049 ceph-mon[31696]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:02.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.403+0000 7f0eb6800700 1 -- 172.21.15.49:0/3308526868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 msgr2=0x7f0eb0102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:02.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.403+0000 7f0eb6800700 1 --2- 172.21.15.49:0/3308526868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 0x7f0eb0102be0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009910 tx=0x7f0ea0009c20 comp rx=0 tx=0).stop 2024-02-26T06:13:02.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.403+0000 7f0eb6800700 1 -- 172.21.15.49:0/3308526868 shutdown_connections 2024-02-26T06:13:02.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.403+0000 7f0eb6800700 1 --2- 172.21.15.49:0/3308526868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 0x7f0eb0102be0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:02.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.403+0000 7f0eb6800700 1 -- 172.21.15.49:0/3308526868 >> 172.21.15.49:0/3308526868 conn(0x7f0eb00771b0 msgr2=0x7f0eb00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:02.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.404+0000 7f0eb6800700 1 -- 172.21.15.49:0/3308526868 shutdown_connections 2024-02-26T06:13:02.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.404+0000 7f0eb6800700 1 -- 172.21.15.49:0/3308526868 wait complete. 2024-02-26T06:13:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.405+0000 7f0eb6800700 1 Processor -- start 2024-02-26T06:13:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.405+0000 7f0eb6800700 1 -- start start 2024-02-26T06:13:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.405+0000 7f0eb6800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 0x7f0eb0100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.406+0000 7f0eb6800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0102350 con 0x7f0eb0102800 2024-02-26T06:13:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.406+0000 7f0eaf400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 0x7f0eb0100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.406+0000 7f0eaf400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 0x7f0eb0100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55816/0 (socket says 172.21.15.49:55816) 2024-02-26T06:13:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.406+0000 7f0eaf400700 1 -- 172.21.15.49:0/3237524147 learned_addr learned my addr 172.21.15.49:0/3237524147 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:13:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.406+0000 7f0eaf400700 1 -- 172.21.15.49:0/3237524147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea00095f0 con 0x7f0eb0102800 2024-02-26T06:13:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.407+0000 7f0eaf400700 1 --2- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 0x7f0eb0100460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0006010 tx=0x7f0ea0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.407+0000 7f0ea7e00700 1 -- 172.21.15.49:0/3237524147 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ea001d070 con 0x7f0eb0102800 2024-02-26T06:13:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.407+0000 7f0ea7e00700 1 -- 172.21.15.49:0/3237524147 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ea000bc30 con 0x7f0eb0102800 2024-02-26T06:13:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.407+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb01009a0 con 0x7f0eb0102800 2024-02-26T06:13:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.407+0000 7f0ea7e00700 1 -- 172.21.15.49:0/3237524147 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ea00039a0 con 0x7f0eb0102800 2024-02-26T06:13:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.407+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb0100dc0 con 0x7f0eb0102800 2024-02-26T06:13:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.408+0000 7f0ea7e00700 1 -- 172.21.15.49:0/3237524147 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f0ea0004390 con 0x7f0eb0102800 2024-02-26T06:13:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.408+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb00632a0 con 0x7f0eb0102800 2024-02-26T06:13:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.409+0000 7f0ea7e00700 1 --2- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0e94038fe0 0x7f0e9403b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.409+0000 7f0ea7e00700 1 -- 172.21.15.49:0/3237524147 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0ea004ce50 con 0x7f0eb0102800 2024-02-26T06:13:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.409+0000 7f0eaea00700 1 --2- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0e94038fe0 0x7f0e9403b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.410+0000 7f0eaea00700 1 --2- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0e94038fe0 0x7f0e9403b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c006fd0 tx=0x7f0e9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:02.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.418+0000 7f0ea7e00700 1 -- 172.21.15.49:0/3237524147 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ea002ae20 con 0x7f0eb0102800 2024-02-26T06:13:02.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:02.650+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eb01010e0 con 0x7f0e94038fe0 2024-02-26T06:13:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi080", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:13:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:02.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: Updating smithi080:/etc/ceph/ceph.conf 2024-02-26T06:13:02.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:02 smithi049 ceph-mon[31696]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:03 smithi049 ceph-mon[31696]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:13:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:03 smithi049 ceph-mon[31696]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:13:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:03 smithi049 ceph-mon[31696]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:05.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:04 smithi049 ceph-mon[31696]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:05.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:04 smithi049 ceph-mon[31696]: Deploying cephadm binary to smithi115 2024-02-26T06:13:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:06 smithi049 ceph-mon[31696]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.610+0000 7f0ea7e00700 1 -- 172.21.15.49:0/3237524147 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0eb01010e0 con 0x7f0e94038fe0 2024-02-26T06:13:07.612 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi115' with addr '172.21.15.115' 2024-02-26T06:13:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.614+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0e94038fe0 msgr2=0x7f0e9403b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.614+0000 7f0eb6800700 1 --2- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0e94038fe0 0x7f0e9403b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c006fd0 tx=0x7f0e9c006e40 comp rx=0 tx=0).stop 2024-02-26T06:13:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.614+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 msgr2=0x7f0eb0100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.614+0000 7f0eb6800700 1 --2- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 0x7f0eb0100460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0006010 tx=0x7f0ea0004ff0 comp rx=0 tx=0).stop 2024-02-26T06:13:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.615+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 shutdown_connections 2024-02-26T06:13:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.615+0000 7f0eb6800700 1 --2- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0e94038fe0 0x7f0e9403b4a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.615+0000 7f0eb6800700 1 --2- 172.21.15.49:0/3237524147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0eb0102800 0x7f0eb0100460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.615+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 >> 172.21.15.49:0/3237524147 conn(0x7f0eb00771b0 msgr2=0x7f0eb01035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.615+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 shutdown_connections 2024-02-26T06:13:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:07.615+0000 7f0eb6800700 1 -- 172.21.15.49:0/3237524147 wait complete. 2024-02-26T06:13:08.370 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-26T06:13:08.827 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:13:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:08 smithi049 ceph-mon[31696]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:08 smithi049 ceph-mon[31696]: Added host smithi115 2024-02-26T06:13:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:09.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.865+0000 7f9c4c600700 1 -- 172.21.15.49:0/1625920298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 msgr2=0x7f9c44102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.865+0000 7f9c4c600700 1 --2- 172.21.15.49:0/1625920298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 0x7f9c44102be0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009910 tx=0x7f9c38009c20 comp rx=0 tx=0).stop 2024-02-26T06:13:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.865+0000 7f9c4c600700 1 -- 172.21.15.49:0/1625920298 shutdown_connections 2024-02-26T06:13:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.865+0000 7f9c4c600700 1 --2- 172.21.15.49:0/1625920298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 0x7f9c44102be0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.865+0000 7f9c4c600700 1 -- 172.21.15.49:0/1625920298 >> 172.21.15.49:0/1625920298 conn(0x7f9c440771b0 msgr2=0x7f9c440fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:10.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.865+0000 7f9c4c600700 1 -- 172.21.15.49:0/1625920298 shutdown_connections 2024-02-26T06:13:10.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.865+0000 7f9c4c600700 1 -- 172.21.15.49:0/1625920298 wait complete. 2024-02-26T06:13:10.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.866+0000 7f9c4c600700 1 Processor -- start 2024-02-26T06:13:10.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.866+0000 7f9c4c600700 1 -- start start 2024-02-26T06:13:10.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.867+0000 7f9c4c600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 0x7f9c44199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:10.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.867+0000 7f9c4c600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c441997c0 con 0x7f9c44102800 2024-02-26T06:13:10.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.867+0000 7f9c49c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 0x7f9c44199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:10.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.867+0000 7f9c49c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 0x7f9c44199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40840/0 (socket says 172.21.15.49:40840) 2024-02-26T06:13:10.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.867+0000 7f9c49c00700 1 -- 172.21.15.49:0/118669805 learned_addr learned my addr 172.21.15.49:0/118669805 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:13:10.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.868+0000 7f9c49c00700 1 -- 172.21.15.49:0/118669805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c380095f0 con 0x7f9c44102800 2024-02-26T06:13:10.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.868+0000 7f9c49c00700 1 --2- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 0x7f9c44199280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9c38006010 tx=0x7f9c38004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:10.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.868+0000 7f9c42000700 1 -- 172.21.15.49:0/118669805 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c3801d070 con 0x7f9c44102800 2024-02-26T06:13:10.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.868+0000 7f9c42000700 1 -- 172.21.15.49:0/118669805 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c3800bc30 con 0x7f9c44102800 2024-02-26T06:13:10.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.868+0000 7f9c42000700 1 -- 172.21.15.49:0/118669805 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c380039a0 con 0x7f9c44102800 2024-02-26T06:13:10.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.868+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c441999c0 con 0x7f9c44102800 2024-02-26T06:13:10.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.869+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c44199de0 con 0x7f9c44102800 2024-02-26T06:13:10.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.870+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c440632a0 con 0x7f9c44102800 2024-02-26T06:13:10.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.870+0000 7f9c42000700 1 -- 172.21.15.49:0/118669805 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f9c38004390 con 0x7f9c44102800 2024-02-26T06:13:10.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.870+0000 7f9c42000700 1 --2- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c2c038fe0 0x7f9c2c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:10.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.870+0000 7f9c42000700 1 -- 172.21.15.49:0/118669805 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9c3804ce50 con 0x7f9c44102800 2024-02-26T06:13:10.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.870+0000 7f9c49200700 1 --2- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c2c038fe0 0x7f9c2c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:10.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.871+0000 7f9c49200700 1 --2- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c2c038fe0 0x7f9c2c03b4a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9c30006fd0 tx=0x7f9c30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:10.876+0000 7f9c42000700 1 -- 172.21.15.49:0/118669805 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c3802ae20 con 0x7f9c44102800 2024-02-26T06:13:10.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:10 smithi049 ceph-mon[31696]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:10.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:11.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.110+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c4410bae0 con 0x7f9c2c038fe0 2024-02-26T06:13:11.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.112+0000 7f9c42000700 1 -- 172.21.15.49:0/118669805 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f9c4410bae0 con 0x7f9c2c038fe0 2024-02-26T06:13:11.112 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:13:11.113 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.80", "hostname": "smithi080", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}] 2024-02-26T06:13:11.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.114+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c2c038fe0 msgr2=0x7f9c2c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.114+0000 7f9c4c600700 1 --2- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c2c038fe0 0x7f9c2c03b4a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9c30006fd0 tx=0x7f9c30006e40 comp rx=0 tx=0).stop 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.114+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 msgr2=0x7f9c44199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.114+0000 7f9c4c600700 1 --2- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 0x7f9c44199280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9c38006010 tx=0x7f9c38004ff0 comp rx=0 tx=0).stop 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.115+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 shutdown_connections 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.115+0000 7f9c4c600700 1 --2- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c2c038fe0 0x7f9c2c03b4a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.115+0000 7f9c4c600700 1 --2- 172.21.15.49:0/118669805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c44102800 0x7f9c44199280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.115+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 >> 172.21.15.49:0/118669805 conn(0x7f9c440771b0 msgr2=0x7f9c441079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.115+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 shutdown_connections 2024-02-26T06:13:11.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:11.115+0000 7f9c4c600700 1 -- 172.21.15.49:0/118669805 wait complete. 2024-02-26T06:13:11.922 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-26T06:13:11.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd crush tunables default 2024-02-26T06:13:12.403 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:13:12.811 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:12 smithi049 ceph-mon[31696]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:13:12.812 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:12 smithi049 ceph-mon[31696]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:14.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.433+0000 7f28a3c00700 1 -- 172.21.15.49:0/3715618593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 msgr2=0x7f289c1007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:14.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.433+0000 7f28a3c00700 1 --2- 172.21.15.49:0/3715618593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 0x7f289c1007d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f288c009910 tx=0x7f288c009c20 comp rx=0 tx=0).stop 2024-02-26T06:13:14.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.433+0000 7f28a3c00700 1 -- 172.21.15.49:0/3715618593 shutdown_connections 2024-02-26T06:13:14.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.433+0000 7f28a3c00700 1 --2- 172.21.15.49:0/3715618593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 0x7f289c1007d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:14.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.433+0000 7f28a3c00700 1 -- 172.21.15.49:0/3715618593 >> 172.21.15.49:0/3715618593 conn(0x7f289c0771b0 msgr2=0x7f289c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:14.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.433+0000 7f28a3c00700 1 -- 172.21.15.49:0/3715618593 shutdown_connections 2024-02-26T06:13:14.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.433+0000 7f28a3c00700 1 -- 172.21.15.49:0/3715618593 wait complete. 2024-02-26T06:13:14.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.435+0000 7f28a3c00700 1 Processor -- start 2024-02-26T06:13:14.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.435+0000 7f28a3c00700 1 -- start start 2024-02-26T06:13:14.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.435+0000 7f28a3c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 0x7f289c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:14.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.435+0000 7f28a3c00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c199700 con 0x7f289c1003f0 2024-02-26T06:13:14.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.436+0000 7f28a1200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 0x7f289c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:14.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.436+0000 7f28a1200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 0x7f289c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40860/0 (socket says 172.21.15.49:40860) 2024-02-26T06:13:14.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.436+0000 7f28a1200700 1 -- 172.21.15.49:0/1682641920 learned_addr learned my addr 172.21.15.49:0/1682641920 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:13:14.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.436+0000 7f28a1200700 1 -- 172.21.15.49:0/1682641920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0095f0 con 0x7f289c1003f0 2024-02-26T06:13:14.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.436+0000 7f28a1200700 1 --2- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 0x7f289c1991c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f288c006010 tx=0x7f288c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:14.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.437+0000 7f289a000700 1 -- 172.21.15.49:0/1682641920 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f288c01d070 con 0x7f289c1003f0 2024-02-26T06:13:14.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.437+0000 7f289a000700 1 -- 172.21.15.49:0/1682641920 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f288c00bc30 con 0x7f289c1003f0 2024-02-26T06:13:14.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.437+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c199900 con 0x7f289c1003f0 2024-02-26T06:13:14.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.437+0000 7f289a000700 1 -- 172.21.15.49:0/1682641920 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f288c003930 con 0x7f289c1003f0 2024-02-26T06:13:14.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.437+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c199d20 con 0x7f289c1003f0 2024-02-26T06:13:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.438+0000 7f289a000700 1 -- 172.21.15.49:0/1682641920 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f288c003ae0 con 0x7f289c1003f0 2024-02-26T06:13:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.438+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c0632a0 con 0x7f289c1003f0 2024-02-26T06:13:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.439+0000 7f289a000700 1 --2- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2880039030 0x7f288003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.439+0000 7f289a000700 1 -- 172.21.15.49:0/1682641920 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f288c04cde0 con 0x7f289c1003f0 2024-02-26T06:13:14.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.439+0000 7f2893e00700 1 --2- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2880039030 0x7f288003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:14.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.440+0000 7f2893e00700 1 --2- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2880039030 0x7f288003b4f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2884006fd0 tx=0x7f2884006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:14.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.444+0000 7f289a000700 1 -- 172.21.15.49:0/1682641920 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f288c00bda0 con 0x7f289c1003f0 2024-02-26T06:13:14.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:14.677+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f289c10dd10 con 0x7f289c1003f0 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: Updating smithi115:/etc/ceph/ceph.conf 2024-02-26T06:13:14.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:14 smithi049 ceph-mon[31696]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:15.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.549+0000 7f289a000700 1 -- 172.21.15.49:0/1682641920 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f288c026020 con 0x7f289c1003f0 2024-02-26T06:13:15.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.553+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2880039030 msgr2=0x7f288003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:15.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.553+0000 7f28a3c00700 1 --2- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2880039030 0x7f288003b4f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2884006fd0 tx=0x7f2884006e40 comp rx=0 tx=0).stop 2024-02-26T06:13:15.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.553+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 msgr2=0x7f289c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:15.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.553+0000 7f28a3c00700 1 --2- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 0x7f289c1991c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f288c006010 tx=0x7f288c004ff0 comp rx=0 tx=0).stop 2024-02-26T06:13:15.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.553+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 shutdown_connections 2024-02-26T06:13:15.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.553+0000 7f28a3c00700 1 --2- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2880039030 0x7f288003b4f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:15.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.553+0000 7f28a3c00700 1 --2- 172.21.15.49:0/1682641920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f289c1003f0 0x7f289c1991c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:15.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.553+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 >> 172.21.15.49:0/1682641920 conn(0x7f289c0771b0 msgr2=0x7f289c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:15.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.554+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 shutdown_connections 2024-02-26T06:13:15.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:15.554+0000 7f28a3c00700 1 -- 172.21.15.49:0/1682641920 wait complete. 2024-02-26T06:13:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-02-26T06:13:15.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:15 smithi049 ceph-mon[31696]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:13:15.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:15 smithi049 ceph-mon[31696]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:13:15.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:15 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1682641920' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-26T06:13:15.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:15 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:15.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:15 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:15.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:15 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:16.434 INFO:tasks.cephadm:Adding mon.a on smithi049 2024-02-26T06:13:16.434 INFO:tasks.cephadm:Adding mon.b on smithi080 2024-02-26T06:13:16.434 INFO:tasks.cephadm:Adding mon.c on smithi115 2024-02-26T06:13:16.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch apply mon '3;smithi049:172.21.15.49=a;smithi080:172.21.15.80=b;smithi115:172.21.15.115=c' 2024-02-26T06:13:16.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:16 smithi049 ceph-mon[31696]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:16.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:16 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1682641920' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-26T06:13:16.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:16 smithi049 ceph-mon[31696]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:16.669 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.482+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2477201425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 msgr2=0x7f4c54102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.482+0000 7f4c5a56d700 1 --2- 172.21.15.115:0/2477201425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 0x7f4c54102be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009910 tx=0x7f4c44009c20 comp rx=0 tx=0).stop 2024-02-26T06:13:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.483+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2477201425 shutdown_connections 2024-02-26T06:13:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.483+0000 7f4c5a56d700 1 --2- 172.21.15.115:0/2477201425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 0x7f4c54102be0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.483+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2477201425 >> 172.21.15.115:0/2477201425 conn(0x7f4c540771b0 msgr2=0x7f4c540fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.483+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2477201425 shutdown_connections 2024-02-26T06:13:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.483+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2477201425 wait complete. 2024-02-26T06:13:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.483+0000 7f4c5a56d700 1 Processor -- start 2024-02-26T06:13:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.483+0000 7f4c5a56d700 1 -- start start 2024-02-26T06:13:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.484+0000 7f4c5a56d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 0x7f4c54100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.484+0000 7f4c5a56d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54102350 con 0x7f4c54102800 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.484+0000 7f4c53fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 0x7f4c54100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.484+0000 7f4c53fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 0x7f4c54100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.115:57600/0 (socket says 172.21.15.115:57600) 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.484+0000 7f4c53fff700 1 -- 172.21.15.115:0/2937563920 learned_addr learned my addr 172.21.15.115:0/2937563920 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.485+0000 7f4c53fff700 1 -- 172.21.15.115:0/2937563920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440095f0 con 0x7f4c54102800 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.485+0000 7f4c53fff700 1 --2- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 0x7f4c54100460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c44006010 tx=0x7f4c44004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.485+0000 7f4c517fa700 1 -- 172.21.15.115:0/2937563920 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c4401d070 con 0x7f4c54102800 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.485+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c541009a0 con 0x7f4c54102800 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.485+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c54100dc0 con 0x7f4c54102800 2024-02-26T06:13:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.485+0000 7f4c517fa700 1 -- 172.21.15.115:0/2937563920 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c4400bc30 con 0x7f4c54102800 2024-02-26T06:13:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.485+0000 7f4c517fa700 1 -- 172.21.15.115:0/2937563920 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c440038a0 con 0x7f4c54102800 2024-02-26T06:13:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.486+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c40005330 con 0x7f4c54102800 2024-02-26T06:13:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.487+0000 7f4c517fa700 1 -- 172.21.15.115:0/2937563920 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f4c44004390 con 0x7f4c54102800 2024-02-26T06:13:18.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.487+0000 7f4c517fa700 1 --2- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4c3c038fe0 0x7f4c3c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:18.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.487+0000 7f4c517fa700 1 -- 172.21.15.115:0/2937563920 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4c4404cf60 con 0x7f4c54102800 2024-02-26T06:13:18.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.488+0000 7f4c537fe700 1 --2- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4c3c038fe0 0x7f4c3c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:18.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.489+0000 7f4c537fe700 1 --2- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4c3c038fe0 0x7f4c3c03b4a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4c48006fd0 tx=0x7f4c48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:18.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.503+0000 7f4c517fa700 1 -- 172.21.15.115:0/2937563920 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c4400b750 con 0x7f4c54102800 2024-02-26T06:13:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.708+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi080:172.21.15.80=b;smithi115:172.21.15.115=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c40000c80 con 0x7f4c3c038fe0 2024-02-26T06:13:18.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.724+0000 7f4c517fa700 1 -- 172.21.15.115:0/2937563920 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4c40000c80 con 0x7f4c3c038fe0 2024-02-26T06:13:18.724 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mon update... 2024-02-26T06:13:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4c3c038fe0 msgr2=0x7f4c3c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 --2- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4c3c038fe0 0x7f4c3c03b4a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4c48006fd0 tx=0x7f4c48006e40 comp rx=0 tx=0).stop 2024-02-26T06:13:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 msgr2=0x7f4c54100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 --2- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 0x7f4c54100460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c44006010 tx=0x7f4c44004ff0 comp rx=0 tx=0).stop 2024-02-26T06:13:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 shutdown_connections 2024-02-26T06:13:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 --2- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4c3c038fe0 0x7f4c3c03b4a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 --2- 172.21.15.115:0/2937563920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c54102800 0x7f4c54100460 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:18.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 >> 172.21.15.115:0/2937563920 conn(0x7f4c540771b0 msgr2=0x7f4c541035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:18.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 shutdown_connections 2024-02-26T06:13:18.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:18.725+0000 7f4c5a56d700 1 -- 172.21.15.115:0/2937563920 wait complete. 2024-02-26T06:13:18.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:18 smithi049 ceph-mon[31696]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:19.310 DEBUG:teuthology.orchestra.run.smithi080:mon.b> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.b.service 2024-02-26T06:13:19.313 DEBUG:teuthology.orchestra.run.smithi115:mon.c> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.c.service 2024-02-26T06:13:19.315 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T06:13:19.315 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T06:13:20.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi080:172.21.15.80=b;smithi115:172.21.15.115=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:20.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi080:172.21.15.80=b;smithi115:172.21.15.115=c;count:3 2024-02-26T06:13:20.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:20.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:20.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:20.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:20.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:20.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:20.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:20.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:19 smithi049 ceph-mon[31696]: Deploying daemon mon.c on smithi115 2024-02-26T06:13:20.913 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:21.071 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:20 smithi049 ceph-mon[31696]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:22.500 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:22 smithi115 systemd[1]: Starting Ceph mon.c for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:13:23.063 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:22 smithi115 podman[36379]: 2024-02-26 06:13:22.878592785 +0000 UTC m=+0.289404815 container create a8521d9b0855132a294f2ef2d4529930c8be9de7f53a9419468735cdd638fc69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-26T06:13:23.064 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:22 smithi115 podman[36379]: 2024-02-26 06:13:22.787610035 +0000 UTC m=+0.198422089 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:13:23.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.163+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3594199845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 msgr2=0x7fb964102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:23.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.163+0000 7fb96bc1b700 1 --2- 172.21.15.115:0/3594199845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 0x7fb964102c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00b0a0 tx=0x7fb95c00b3b0 comp rx=0 tx=0).stop 2024-02-26T06:13:23.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.164+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3594199845 shutdown_connections 2024-02-26T06:13:23.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.164+0000 7fb96bc1b700 1 --2- 172.21.15.115:0/3594199845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 0x7fb964102c50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:23.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.164+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3594199845 >> 172.21.15.115:0/3594199845 conn(0x7fb9640fc040 msgr2=0x7fb9640fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:23.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.164+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3594199845 shutdown_connections 2024-02-26T06:13:23.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.164+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3594199845 wait complete. 2024-02-26T06:13:23.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.164+0000 7fb96bc1b700 1 Processor -- start 2024-02-26T06:13:23.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.164+0000 7fb96bc1b700 1 -- start start 2024-02-26T06:13:23.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.165+0000 7fb96bc1b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 0x7fb9641004d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:23.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.165+0000 7fb96bc1b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964102370 con 0x7fb964102870 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.165+0000 7fb9699b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 0x7fb9641004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.165+0000 7fb9699b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 0x7fb9641004d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.115:57626/0 (socket says 172.21.15.115:57626) 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.165+0000 7fb9699b7700 1 -- 172.21.15.115:0/3442212159 learned_addr learned my addr 172.21.15.115:0/3442212159 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.166+0000 7fb9699b7700 1 -- 172.21.15.115:0/3442212159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c009d00 con 0x7fb964102870 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.166+0000 7fb9699b7700 1 --2- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 0x7fb9641004d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000f80 tx=0x7fb95c0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.166+0000 7fb962ffd700 1 -- 172.21.15.115:0/3442212159 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb95c00e040 con 0x7fb964102870 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.166+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb964100a10 con 0x7fb964102870 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.166+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb964100e30 con 0x7fb964102870 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.167+0000 7fb962ffd700 1 -- 172.21.15.115:0/3442212159 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb95c007bc0 con 0x7fb964102870 2024-02-26T06:13:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.167+0000 7fb962ffd700 1 -- 172.21.15.115:0/3442212159 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb95c0049a0 con 0x7fb964102870 2024-02-26T06:13:23.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.167+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb950005330 con 0x7fb964102870 2024-02-26T06:13:23.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.168+0000 7fb962ffd700 1 -- 172.21.15.115:0/3442212159 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fb95c004040 con 0x7fb964102870 2024-02-26T06:13:23.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.168+0000 7fb962ffd700 1 --2- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb94c038fd0 0x7fb94c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:23.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.168+0000 7fb962ffd700 1 -- 172.21.15.115:0/3442212159 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb95c04d050 con 0x7fb964102870 2024-02-26T06:13:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:22 smithi049 ceph-mon[31696]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.169+0000 7fb9691b6700 1 --2- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb94c038fd0 0x7fb94c03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:23.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.170+0000 7fb9691b6700 1 --2- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb94c038fd0 0x7fb94c03b490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb958006fd0 tx=0x7fb958006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:23.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.184+0000 7fb962ffd700 1 -- 172.21.15.115:0/3442212159 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb95c01ae20 con 0x7fb964102870 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 podman[36379]: 2024-02-26 06:13:23.248300666 +0000 UTC m=+0.659112687 container init a8521d9b0855132a294f2ef2d4529930c8be9de7f53a9419468735cdd638fc69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-c, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 podman[36379]: 2024-02-26 06:13:23.251339769 +0000 UTC m=+0.662151802 container start a8521d9b0855132a294f2ef2d4529930c8be9de7f53a9419468735cdd638fc69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True) 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: set uid:gid to 167:167 (ceph:ceph) 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev), process ceph-mon, pid 2 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pidfile_write: ignore empty --pid-file 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: load: jerasure load: lrc 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: RocksDB version: 7.9.2 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Git sha 0 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Compile date 2024-02-16 06:18:30 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: DB SUMMARY 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: DB Session ID: C4QWXXNLHKKWY9IXFXQ1 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: CURRENT file: CURRENT 2024-02-26T06:13:23.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: IDENTITY file: IDENTITY 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.error_if_exists: 0 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.create_if_missing: 0 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.paranoid_checks: 1 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.env: 0x55f5f0ebaae0 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.fs: PosixFileSystem 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.info_log: 0x55f5f265f8c0 2024-02-26T06:13:23.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_file_opening_threads: 16 2024-02-26T06:13:23.317 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.statistics: (nil) 2024-02-26T06:13:23.317 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.use_fsync: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_log_file_size: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.keep_log_file_num: 1000 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.recycle_log_file_num: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.allow_fallocate: 1 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.allow_mmap_reads: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.allow_mmap_writes: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.use_direct_reads: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.create_missing_column_families: 0 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.db_log_dir: 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.wal_dir: 2024-02-26T06:13:23.318 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.advise_random_on_open: 1 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.db_write_buffer_size: 0 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.write_buffer_manager: 0x55f5f2633400 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.rate_limiter: (nil) 2024-02-26T06:13:23.319 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.wal_recovery_mode: 2 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.enable_thread_tracking: 0 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.enable_pipelined_write: 0 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.unordered_write: 0 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.row_cache: None 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.wal_filter: None 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.allow_ingest_behind: 0 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.two_write_queues: 0 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.manual_wal_flush: 0 2024-02-26T06:13:23.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.wal_compression: 0 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.atomic_flush: 0 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.log_readahead_size: 0 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.best_efforts_recovery: 0 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.allow_data_in_errors: 0 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.db_host_id: __hostname__ 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_background_jobs: 2 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_background_compactions: -1 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_subcompactions: 1 2024-02-26T06:13:23.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_total_wal_size: 0 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_open_files: -1 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bytes_per_sync: 0 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_readahead_size: 0 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_background_flushes: -1 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Compression algorithms supported: 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-26T06:13:23.322 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: kZSTD supported: 0 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: kXpressCompression supported: 0 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: kLZ4HCCompression supported: 1 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: kZlibCompression supported: 1 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: kSnappyCompression supported: 1 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: kLZ4Compression supported: 1 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: kBZip2Compression supported: 0 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.merge_operator: 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_filter: None 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_filter_factory: None 2024-02-26T06:13:23.323 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.sst_partitioner_factory: None 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f5f265f9c0) 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_type: 0 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: data_block_index_type: 0 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_shortening: 1 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: checksum: 4 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: no_block_cache: 0 2024-02-26T06:13:23.324 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache: 0x55f5f2663090 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_options: 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: capacity : 536870912 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: num_shard_bits : 4 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: strict_capacity_limit : 0 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_compressed: (nil) 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: persistent_cache: (nil) 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_size: 4096 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_size_deviation: 10 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_restart_interval: 16 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_block_restart_interval: 1 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: metadata_block_size: 4096 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: partition_filters: 0 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: use_delta_encoding: 1 2024-02-26T06:13:23.325 INFO:journalctl@ceph.mon.c.smithi115.stdout: filter_policy: bloomfilter 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: whole_key_filtering: 1 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: verify_compression: 0 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: format_version: 5 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: enable_index_compression: 1 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_align: 0 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: max_auto_readahead_size: 262144 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: prepopulate_block_cache: 0 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: initial_auto_readahead_size: 8192 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout: num_file_reads_for_auto_readahead: 2 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.write_buffer_size: 33554432 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_write_buffer_number: 2 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression: NoCompression 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression: Disabled 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.prefix_extractor: nullptr 2024-02-26T06:13:23.326 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.num_levels: 7 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-26T06:13:23.327 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.level: 32767 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.strategy: 0 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.enabled: false 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.target_file_size_base: 67108864 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-26T06:13:23.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.arena_block_size: 1048576 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.disable_auto_compactions: 0 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-26T06:13:23.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.table_properties_collectors: 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.inplace_update_support: 0 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-26T06:13:23.330 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.bloom_locality: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.max_successive_merges: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.paranoid_file_checks: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.force_consistency_checks: 1 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.report_bg_io_stats: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.ttl: 2592000 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.enable_blob_files: false 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.min_blob_size: 0 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.blob_file_size: 268435456 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-26T06:13:23.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.blob_file_starting_level: 0 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 609cb776-b773-4cb9-b270-1678d289cd15 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928003281466, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928003282031, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708928003, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "609cb776-b773-4cb9-b270-1678d289cd15", "db_session_id": "C4QWXXNLHKKWY9IXFXQ1", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928003282135, "job": 1, "event": "recovery_finished"} 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f5f2754000 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: DB pointer 0x55f5f273e000 2024-02-26T06:13:23.332 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** DB Stats ** 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T06:13:23.333 INFO:journalctl@ceph.mon.c.smithi115.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-26T06:13:23.334 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: Block cache BinnedLRUCache@0x55f5f2663090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: using public_addr v2:172.21.15.115:0/0 -> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: starting mon.c rank -1 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(???) e0 preinit fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).mds e1 new map 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: e1 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T06:13:23.335 INFO:journalctl@ceph.mon.c.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout: legacy client fscid: -1 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout: No filesystems configured 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Deploying cephadm binary to smithi080 2024-02-26T06:13:23.336 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Added host smithi080 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi080", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Updating smithi080:/etc/ceph/ceph.conf 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Deploying cephadm binary to smithi115 2024-02-26T06:13:23.338 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Added host smithi115 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.339 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Updating smithi115:/etc/ceph/ceph.conf 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1682641920' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1682641920' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi080:172.21.15.80=b;smithi115:172.21.15.115=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi080:172.21.15.80=b;smithi115:172.21.15.115=c;count:3 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: Deploying daemon mon.c on smithi115 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 ceph-mon[36426]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-26T06:13:23.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.319+0000 7fb962ffd700 1 -- 172.21.15.115:0/3442212159 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb95c01ae20 con 0x7fb964102870 2024-02-26T06:13:23.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:23.480+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb9500055e0 con 0x7fb964102870 2024-02-26T06:13:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 bash[36379]: a8521d9b0855132a294f2ef2d4529930c8be9de7f53a9419468735cdd638fc69 2024-02-26T06:13:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:23 smithi115 systemd[1]: Started Ceph mon.c for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:13:28.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.356+0000 7fb962ffd700 1 -- 172.21.15.115:0/3442212159 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fb95c017030 con 0x7fb964102870 2024-02-26T06:13:28.357 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:13:28.357 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":2,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","modified":"2024-02-26T06:13:23.318326Z","created":"2024-02-26T06:11:04.727500Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb94c038fd0 msgr2=0x7fb94c03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 --2- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb94c038fd0 0x7fb94c03b490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb958006fd0 tx=0x7fb958006e40 comp rx=0 tx=0).stop 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 msgr2=0x7fb9641004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 --2- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 0x7fb9641004d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000f80 tx=0x7fb95c0075c0 comp rx=0 tx=0).stop 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 shutdown_connections 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 --2- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb94c038fd0 0x7fb94c03b490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 --2- 172.21.15.115:0/3442212159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb964102870 0x7fb9641004d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 >> 172.21.15.115:0/3442212159 conn(0x7fb9640fc040 msgr2=0x7fb964103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 shutdown_connections 2024-02-26T06:13:28.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:28.358+0000 7fb96bc1b700 1 -- 172.21.15.115:0/3442212159 wait complete. 2024-02-26T06:13:28.360 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 2 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: mon.a calling monitor election 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/3442212159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: mon.c calling monitor election 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: monmap e2: 2 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: fsmap 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: mgrmap e13: x(active, since 61s) 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: overall HEALTH_OK 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:28.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:28.675 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: mon.a calling monitor election 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/3442212159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: mon.c calling monitor election 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:13:28.676 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: monmap e2: 2 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:13:28.677 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: fsmap 2024-02-26T06:13:28.677 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:28.677 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: mgrmap e13: x(active, since 61s) 2024-02-26T06:13:28.677 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: overall HEALTH_OK 2024-02-26T06:13:28.677 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:28.677 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:28.677 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:28.677 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:29.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:29.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:29 smithi049 ceph-mon[31696]: Deploying daemon mon.b on smithi080 2024-02-26T06:13:29.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:29.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:29.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:29 smithi115 ceph-mon[36426]: Deploying daemon mon.b on smithi080 2024-02-26T06:13:29.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:29.949 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T06:13:29.950 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T06:13:30.191 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:13:30.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:30 smithi115 ceph-mon[36426]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:30.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:30 smithi049 ceph-mon[31696]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:30.669 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:13:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:13:30.319+0000 7fc9e8a00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-02-26T06:13:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 -- 172.21.15.115:0/2684561625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 msgr2=0x7fe3c4107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 --2- 172.21.15.115:0/2684561625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 0x7fe3c4107b30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00b0a0 tx=0x7fe3bc00b3b0 comp rx=0 tx=0).stop 2024-02-26T06:13:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 -- 172.21.15.115:0/2684561625 shutdown_connections 2024-02-26T06:13:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 --2- 172.21.15.115:0/2684561625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 0x7fe3c4107b30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 -- 172.21.15.115:0/2684561625 >> 172.21.15.115:0/2684561625 conn(0x7fe3c40fd240 msgr2=0x7fe3c40ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 -- 172.21.15.115:0/2684561625 shutdown_connections 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 -- 172.21.15.115:0/2684561625 wait complete. 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 Processor -- start 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.724+0000 7fe3cc08c700 1 -- start start 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3cc08c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 0x7fe3c4073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3cc08c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 0x7fe3c4074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3cc08c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c4075e60 con 0x7fe3c4107750 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3cc08c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c41a3460 con 0x7fe3c40744b0 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3c9627700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 0x7fe3c4074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3c9627700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 0x7fe3c4074930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56924/0 (socket says 172.21.15.115:56924) 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3c9627700 1 -- 172.21.15.115:0/593708652 learned_addr learned my addr 172.21.15.115:0/593708652 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:13:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3c9627700 1 -- 172.21.15.115:0/593708652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 msgr2=0x7fe3c4074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-26T06:13:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3c9627700 1 -- 172.21.15.115:0/593708652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 msgr2=0x7fe3c4074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-26T06:13:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3c9627700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 0x7fe3c4074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T06:13:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3c9627700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 0x7fe3c4074930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:13:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.725+0000 7fe3c9e28700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 0x7fe3c4073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.726+0000 7fe3c9e28700 1 -- 172.21.15.115:0/593708652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 msgr2=0x7fe3c4074930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:13:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.726+0000 7fe3c9e28700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 0x7fe3c4074930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.726+0000 7fe3c9e28700 1 -- 172.21.15.115:0/593708652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc009d00 con 0x7fe3c4107750 2024-02-26T06:13:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.726+0000 7fe3c9e28700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 0x7fe3c4073f70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc000f80 tx=0x7fe3bc00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.726+0000 7fe3c2ffd700 1 -- 172.21.15.115:0/593708652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe3bc00e040 con 0x7fe3c4107750 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.726+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c41a35a0 con 0x7fe3c4107750 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.727+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c41a3ac0 con 0x7fe3c4107750 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.727+0000 7fe3c2ffd700 1 -- 172.21.15.115:0/593708652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3bc009450 con 0x7fe3c4107750 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.727+0000 7fe3c2ffd700 1 -- 172.21.15.115:0/593708652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe3bc004870 con 0x7fe3c4107750 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.727+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0005330 con 0x7fe3c4107750 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.728+0000 7fe3c2ffd700 1 -- 172.21.15.115:0/593708652 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fe3bc019030 con 0x7fe3c4107750 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.728+0000 7fe3c2ffd700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ac038d20 0x7fe3ac03b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.728+0000 7fe3c2ffd700 1 -- 172.21.15.115:0/593708652 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe3bc04d090 con 0x7fe3c4107750 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.729+0000 7fe3c9627700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ac038d20 0x7fe3ac03b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.730+0000 7fe3c9627700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ac038d20 0x7fe3ac03b1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009180 tx=0x7fe3b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:31.745+0000 7fe3c2ffd700 1 -- 172.21.15.115:0/593708652 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3bc011980 con 0x7fe3c4107750 2024-02-26T06:13:32.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.019+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe3b00058b0 con 0x7fe3c4107750 2024-02-26T06:13:32.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.020+0000 7fe3c2ffd700 1 -- 172.21.15.115:0/593708652 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fe3bc011390 con 0x7fe3c4107750 2024-02-26T06:13:32.020 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:13:32.020 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":2,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","modified":"2024-02-26T06:13:23.318326Z","created":"2024-02-26T06:11:04.727500Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T06:13:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.022+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ac038d20 msgr2=0x7fe3ac03b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.022+0000 7fe3cc08c700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ac038d20 0x7fe3ac03b1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009180 tx=0x7fe3b8008040 comp rx=0 tx=0).stop 2024-02-26T06:13:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.022+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 msgr2=0x7fe3c4073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.022+0000 7fe3cc08c700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 0x7fe3c4073f70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc000f80 tx=0x7fe3bc00bd40 comp rx=0 tx=0).stop 2024-02-26T06:13:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.022+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 shutdown_connections 2024-02-26T06:13:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.022+0000 7fe3cc08c700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ac038d20 0x7fe3ac03b1e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.022+0000 7fe3cc08c700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3c4107750 0x7fe3c4073f70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.022+0000 7fe3cc08c700 1 --2- 172.21.15.115:0/593708652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c40744b0 0x7fe3c4074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.023+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 >> 172.21.15.115:0/593708652 conn(0x7fe3c40fd240 msgr2=0x7fe3c4103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.023+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 shutdown_connections 2024-02-26T06:13:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:32.023+0000 7fe3cc08c700 1 -- 172.21.15.115:0/593708652 wait complete. 2024-02-26T06:13:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 2 2024-02-26T06:13:32.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:32 smithi080 systemd[1]: Started Ceph mon.b for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:13:33.579 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T06:13:33.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T06:13:33.820 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:13:37.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: mon.c calling monitor election 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: mon.a calling monitor election 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: fsmap 2024-02-26T06:13:37.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:37.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: mgrmap e13: x(active, since 70s) 2024-02-26T06:13:37.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T06:13:37.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T06:13:37.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T06:13:37.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: mon.b (rank 2) addr [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] is down (out of quorum) 2024-02-26T06:13:37.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:37.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:37.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:37.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: mon.c calling monitor election 2024-02-26T06:13:37.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:13:37.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: mon.a calling monitor election 2024-02-26T06:13:37.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:37.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: fsmap 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: mgrmap e13: x(active, since 70s) 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T06:13:37.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T06:13:37.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: mon.b (rank 2) addr [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] is down (out of quorum) 2024-02-26T06:13:37.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:37.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:37.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/1914062875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b00035c0 msgr2=0x7ff9b0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 --2- 172.21.15.115:0/1914062875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b00035c0 0x7ff9b0005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4069f30 tx=0x7ff9c000feb0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/1914062875 shutdown_connections 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 --2- 172.21.15.115:0/1914062875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b00035c0 0x7ff9b0005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/1914062875 >> 172.21.15.115:0/1914062875 conn(0x7ff9c40fc180 msgr2=0x7ff9c40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/1914062875 shutdown_connections 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/1914062875 wait complete. 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 Processor -- start 2024-02-26T06:13:38.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.444+0000 7ff9cb9d0700 1 -- start start 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9cb9d0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b00035c0 0x7ff9c4100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9cb9d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c4100ad0 0x7ff9c4100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9cb9d0700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c41a3440 0x7ff9c41a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9cb9d0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a5e30 con 0x7ff9c4100ad0 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9cb9d0700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a5fa0 con 0x7ff9c41a3440 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9cb9d0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a6160 con 0x7ff9b00035c0 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b00035c0 0x7ff9c4100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b00035c0 0x7ff9c4100590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40438/0 (socket says 172.21.15.115:40438) 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 -- 172.21.15.115:0/3313690581 learned_addr learned my addr 172.21.15.115:0/3313690581 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 -- 172.21.15.115:0/3313690581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c41a3440 msgr2=0x7ff9c41a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c41a3440 0x7ff9c41a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 -- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c4100ad0 msgr2=0x7ff9c4100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c4100ad0 0x7ff9c4100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 -- 172.21.15.115:0/3313690581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c000f900 con 0x7ff9b00035c0 2024-02-26T06:13:38.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9c976c700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b00035c0 0x7ff9c4100590 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000f800 tx=0x7ff9c0004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:38.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9ba7fc700 1 -- 172.21.15.115:0/3313690581 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0013960 con 0x7ff9b00035c0 2024-02-26T06:13:38.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9ba7fc700 1 -- 172.21.15.115:0/3313690581 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9c0011470 con 0x7ff9b00035c0 2024-02-26T06:13:38.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.445+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c41a63e0 con 0x7ff9b00035c0 2024-02-26T06:13:38.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.446+0000 7ff9ba7fc700 1 -- 172.21.15.115:0/3313690581 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0024660 con 0x7ff9b00035c0 2024-02-26T06:13:38.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.446+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c41a6930 con 0x7ff9b00035c0 2024-02-26T06:13:38.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.446+0000 7ff9ba7fc700 1 -- 172.21.15.115:0/3313690581 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7ff9c00247c0 con 0x7ff9b00035c0 2024-02-26T06:13:38.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.447+0000 7ff9ba7fc700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a8039410 0x7ff9a803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:38.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.447+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c40632a0 con 0x7ff9b00035c0 2024-02-26T06:13:38.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.447+0000 7ff9ba7fc700 1 -- 172.21.15.115:0/3313690581 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff9c004e7c0 con 0x7ff9b00035c0 2024-02-26T06:13:38.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.447+0000 7ff9c8f6b700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a8039410 0x7ff9a803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:38.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.448+0000 7ff9c8f6b700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a8039410 0x7ff9a803b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40098e0 tx=0x7ff9b4006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:38.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.452+0000 7ff9ba7fc700 1 -- 172.21.15.115:0/3313690581 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9c0035350 con 0x7ff9b00035c0 2024-02-26T06:13:38.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.736+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff9c41097f0 con 0x7ff9b00035c0 2024-02-26T06:13:38.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:38 smithi115 ceph-mon[36426]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:38.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:38.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:38.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:38.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.736+0000 7ff9ba7fc700 1 -- 172.21.15.115:0/3313690581 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1279 (secure 0 0 0) 0x7ff9c0028020 con 0x7ff9b00035c0 2024-02-26T06:13:38.737 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-02-26T06:13:38.737 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":3,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","modified":"2024-02-26T06:13:32.390857Z","created":"2024-02-26T06:11:04.727500Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:3300","nonce":0},{"type":"v1","addr":"172.21.15.80:6789","nonce":0}]},"addr":"172.21.15.80:6789/0","public_addr":"172.21.15.80:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.738+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a8039410 msgr2=0x7ff9a803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.738+0000 7ff9cb9d0700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a8039410 0x7ff9a803b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40098e0 tx=0x7ff9b4006dd0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.738+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b00035c0 msgr2=0x7ff9c4100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.738+0000 7ff9cb9d0700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b00035c0 0x7ff9c4100590 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000f800 tx=0x7ff9c0004820 comp rx=0 tx=0).stop 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.739+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 shutdown_connections 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.739+0000 7ff9cb9d0700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a8039410 0x7ff9a803b8d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.739+0000 7ff9cb9d0700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c41a3440 0x7ff9c41a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.739+0000 7ff9cb9d0700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b00035c0 0x7ff9c4100590 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.739+0000 7ff9cb9d0700 1 --2- 172.21.15.115:0/3313690581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c4100ad0 0x7ff9c4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.739+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 >> 172.21.15.115:0/3313690581 conn(0x7ff9c40fc180 msgr2=0x7ff9c40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.739+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 shutdown_connections 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:38.739+0000 7ff9cb9d0700 1 -- 172.21.15.115:0/3313690581 wait complete. 2024-02-26T06:13:38.739 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 3 2024-02-26T06:13:38.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:38 smithi049 ceph-mon[31696]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:38.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:38.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:38.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:39.768 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-26T06:13:39.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph config generate-minimal-conf 2024-02-26T06:13:39.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: mon.c calling monitor election 2024-02-26T06:13:39.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:13:39.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: mon.a calling monitor election 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: fsmap 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:39.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: mgrmap e13: x(active, since 70s) 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: mon.b (rank 2) addr [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] is down (out of quorum) 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:39.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:40.217 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: mon.b calling monitor election 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: Updating smithi049:/etc/ceph/ceph.conf 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: Updating smithi080:/etc/ceph/ceph.conf 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: Updating smithi115:/etc/ceph/ceph.conf 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: mon.b calling monitor election 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: mon.c calling monitor election 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: mon.a calling monitor election 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: fsmap 2024-02-26T06:13:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: mgrmap e13: x(active, since 72s) 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: Cluster is now healthy 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: overall HEALTH_OK 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:40 smithi049 ceph-mon[31696]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:40.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: mon.b calling monitor election 2024-02-26T06:13:40.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: Updating smithi049:/etc/ceph/ceph.conf 2024-02-26T06:13:40.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: Updating smithi080:/etc/ceph/ceph.conf 2024-02-26T06:13:40.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: Updating smithi115:/etc/ceph/ceph.conf 2024-02-26T06:13:40.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: mon.b calling monitor election 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: mon.c calling monitor election 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: mon.a calling monitor election 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: fsmap 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: mgrmap e13: x(active, since 72s) 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: Cluster is now healthy 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: overall HEALTH_OK 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:40.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.582 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.582 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.582 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.582 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:40 smithi080 ceph-mon[36206]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:40.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: mon.b calling monitor election 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: Updating smithi049:/etc/ceph/ceph.conf 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: Updating smithi080:/etc/ceph/ceph.conf 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: Updating smithi115:/etc/ceph/ceph.conf 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: mon.b calling monitor election 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: mon.c calling monitor election 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: mon.a calling monitor election 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T06:13:40.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:13:40.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: fsmap 2024-02-26T06:13:40.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:13:40.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: mgrmap e13: x(active, since 72s) 2024-02-26T06:13:40.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T06:13:40.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: Cluster is now healthy 2024-02-26T06:13:40.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: overall HEALTH_OK 2024-02-26T06:13:40.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:40.593 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:40.593 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.593 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.593 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.593 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:40.593 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:40 smithi115 ceph-mon[36426]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:13:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T06:13:41.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:41.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:41.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:41.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:41 smithi115 ceph-mon[36426]: Reconfiguring daemon mon.a on smithi049 2024-02-26T06:13:41.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:41.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:41 smithi049 ceph-mon[31696]: Reconfiguring daemon mon.a on smithi049 2024-02-26T06:13:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:13:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:41.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:41 smithi080 ceph-mon[36206]: Reconfiguring daemon mon.a on smithi049 2024-02-26T06:13:42.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:42 smithi115 ceph-mon[36426]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:42.378 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:42 smithi049 ceph-mon[31696]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:42.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:42 smithi080 ceph-mon[36206]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:42.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.447+0000 7f2feb800700 1 -- 172.21.15.49:0/3284908250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe4102800 msgr2=0x7f2fe4102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:42.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.447+0000 7f2feb800700 1 --2- 172.21.15.49:0/3284908250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe4102800 0x7f2fe4102be0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009910 tx=0x7f2fd8009c20 comp rx=0 tx=0).stop 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.447+0000 7f2feb800700 1 -- 172.21.15.49:0/3284908250 shutdown_connections 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.447+0000 7f2feb800700 1 --2- 172.21.15.49:0/3284908250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe4102800 0x7f2fe4102be0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.447+0000 7f2feb800700 1 -- 172.21.15.49:0/3284908250 >> 172.21.15.49:0/3284908250 conn(0x7f2fe40771b0 msgr2=0x7f2fe40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.447+0000 7f2feb800700 1 -- 172.21.15.49:0/3284908250 shutdown_connections 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.447+0000 7f2feb800700 1 -- 172.21.15.49:0/3284908250 wait complete. 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.449+0000 7f2feb800700 1 Processor -- start 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.449+0000 7f2feb800700 1 -- start start 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2feb800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe4102800 0x7f2fe4100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2feb800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe41009a0 0x7f2fe4100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2feb800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe41a3420 0x7f2fe41a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2feb800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe41a5e10 con 0x7f2fe41a3420 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2feb800700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe41a5f80 con 0x7f2fe41009a0 2024-02-26T06:13:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2feb800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe41a6140 con 0x7f2fe4102800 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2fe8e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe4102800 0x7f2fe4100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2fe8e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe4102800 0x7f2fe4100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:36128/0 (socket says 172.21.15.49:36128) 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2fe8e00700 1 -- 172.21.15.49:0/2289903766 learned_addr learned my addr 172.21.15.49:0/2289903766 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2fe3e00700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe41009a0 0x7f2fe4100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.450+0000 7f2fe9800700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe41a3420 0x7f2fe41a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe3e00700 1 -- 172.21.15.49:0/2289903766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe4102800 msgr2=0x7f2fe4100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe3e00700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe4102800 0x7f2fe4100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe3e00700 1 -- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe41a3420 msgr2=0x7f2fe41a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe3e00700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe41a3420 0x7f2fe41a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe3e00700 1 -- 172.21.15.49:0/2289903766 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd80095f0 con 0x7f2fe41009a0 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe9800700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe41a3420 0x7f2fe41a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe3e00700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe41009a0 0x7f2fe4100e20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000b550 tx=0x7f2fd000b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe1600700 1 -- 172.21.15.49:0/2289903766 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0004230 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe1600700 1 -- 172.21.15.49:0/2289903766 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fd0004390 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2fe1600700 1 -- 172.21.15.49:0/2289903766 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0011a90 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe41a6420 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.451+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe41a6970 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.452+0000 7f2fe1600700 1 -- 172.21.15.49:0/2289903766 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f2fd0011c30 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.453+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe4069940 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.453+0000 7f2fe1600700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2fcc039410 0x7f2fcc03b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.453+0000 7f2fe1600700 1 -- 172.21.15.49:0/2289903766 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2fd004b520 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.453+0000 7f2fe8e00700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2fcc039410 0x7f2fcc03b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.454+0000 7f2fe8e00700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2fcc039410 0x7f2fcc03b8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8006010 tx=0x7f2fd800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.458+0000 7f2fe1600700 1 -- 172.21.15.49:0/2289903766 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2fd00214d0 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.685+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2fe410bae0 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.686+0000 7f2fe1600700 1 -- 172.21.15.49:0/2289903766 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f2fd000d1c0 con 0x7f2fe41009a0 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2fcc039410 msgr2=0x7f2fcc03b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2fcc039410 0x7f2fcc03b8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8006010 tx=0x7f2fd800b540 comp rx=0 tx=0).stop 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe41009a0 msgr2=0x7f2fe4100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe41009a0 0x7f2fe4100e20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000b550 tx=0x7f2fd000b860 comp rx=0 tx=0).stop 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 shutdown_connections 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2fcc039410 0x7f2fcc03b8d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe41a3420 0x7f2fe41a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe4102800 0x7f2fe4100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 --2- 172.21.15.49:0/2289903766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe41009a0 0x7f2fe4100e20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.689+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 >> 172.21.15.49:0/2289903766 conn(0x7f2fe40771b0 msgr2=0x7f2fe41036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.690+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 shutdown_connections 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:42.690+0000 7f2feb800700 1 -- 172.21.15.49:0/2289903766 wait complete. 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stdout:# minimal ceph.conf for a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-02-26T06:13:42.973 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:13:42.974 INFO:teuthology.orchestra.run.smithi049.stdout: mon_host = [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] 2024-02-26T06:13:43.586 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.586 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:43 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2289903766' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:43.586 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.586 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:43 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:43 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2289903766' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:43.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:43 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:43 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:43 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:43 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2289903766' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:43.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:43 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:43 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:43.921 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-26T06:13:43.921 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:13:43.921 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T06:13:44.021 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:13:44.021 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:13:44.127 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:13:44.127 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T06:13:44.156 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:13:44.156 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:13:44.223 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:13:44.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T06:13:44.259 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:13:44.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:13:44.329 INFO:tasks.cephadm:Adding mgr.x on smithi049 2024-02-26T06:13:44.329 INFO:tasks.cephadm:Adding mgr.y on smithi080 2024-02-26T06:13:44.329 INFO:tasks.cephadm:Adding mgr.z on smithi115 2024-02-26T06:13:44.330 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch apply mgr '3;smithi049=x;smithi080=y;smithi115=z' 2024-02-26T06:13:44.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:44 smithi049 ceph-mon[31696]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:44.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:44 smithi049 ceph-mon[31696]: Reconfiguring mon.b (monmap changed)... 2024-02-26T06:13:44.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:44 smithi080 ceph-mon[36206]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:44 smithi080 ceph-mon[36206]: Reconfiguring mon.b (monmap changed)... 2024-02-26T06:13:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:44 smithi080 ceph-mon[36206]: Reconfiguring daemon mon.b on smithi080 2024-02-26T06:13:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:44 smithi115 ceph-mon[36426]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:44 smithi115 ceph-mon[36426]: Reconfiguring mon.b (monmap changed)... 2024-02-26T06:13:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:44 smithi115 ceph-mon[36426]: Reconfiguring daemon mon.b on smithi080 2024-02-26T06:13:44.599 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:13:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:44 smithi049 ceph-mon[31696]: Reconfiguring daemon mon.b on smithi080 2024-02-26T06:13:46.161 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:46.161 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:46.161 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:45 smithi115 ceph-mon[36426]: Reconfiguring mon.c (monmap changed)... 2024-02-26T06:13:46.161 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:46.161 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:46.161 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:46.161 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:45 smithi115 ceph-mon[36426]: Reconfiguring daemon mon.c on smithi115 2024-02-26T06:13:46.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:46.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:46.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:45 smithi049 ceph-mon[31696]: Reconfiguring mon.c (monmap changed)... 2024-02-26T06:13:46.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:46.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:46.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:45 smithi049 ceph-mon[31696]: Reconfiguring daemon mon.c on smithi115 2024-02-26T06:13:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.263+0000 7ff08da97700 1 -- 172.21.15.115:0/3588553422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0881006d0 msgr2=0x7ff088100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.263+0000 7ff08da97700 1 --2- 172.21.15.115:0/3588553422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0881006d0 0x7ff088100af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b0a0 tx=0x7ff08000b3b0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.263+0000 7ff08da97700 1 -- 172.21.15.115:0/3588553422 shutdown_connections 2024-02-26T06:13:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.263+0000 7ff08da97700 1 --2- 172.21.15.115:0/3588553422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0881006d0 0x7ff088100af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.263+0000 7ff08da97700 1 -- 172.21.15.115:0/3588553422 >> 172.21.15.115:0/3588553422 conn(0x7ff0880fbc50 msgr2=0x7ff0880fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.263+0000 7ff08da97700 1 -- 172.21.15.115:0/3588553422 shutdown_connections 2024-02-26T06:13:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.263+0000 7ff08da97700 1 -- 172.21.15.115:0/3588553422 wait complete. 2024-02-26T06:13:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff08da97700 1 Processor -- start 2024-02-26T06:13:46.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff08da97700 1 -- start start 2024-02-26T06:13:46.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff08da97700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0881006d0 0x7ff088196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:46.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff08da97700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff088197320 0x7ff08819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:46.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff08da97700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0881977a0 0x7ff08819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:46.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff08da97700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08819f0c0 con 0x7ff088197320 2024-02-26T06:13:46.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff08da97700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08819f230 con 0x7ff0881977a0 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff08da97700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08819f3f0 con 0x7ff0881006d0 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff0877fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0881006d0 0x7ff088196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff0877fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0881006d0 0x7ff088196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40440/0 (socket says 172.21.15.115:40440) 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.264+0000 7ff0877fe700 1 -- 172.21.15.115:0/1184808472 learned_addr learned my addr 172.21.15.115:0/1184808472 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff0877fe700 1 -- 172.21.15.115:0/1184808472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0881977a0 msgr2=0x7ff08819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff0877fe700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0881977a0 0x7ff08819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff0877fe700 1 -- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff088197320 msgr2=0x7ff08819c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff0877fe700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff088197320 0x7ff08819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff0877fe700 1 -- 172.21.15.115:0/1184808472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080009d00 con 0x7ff0881006d0 2024-02-26T06:13:46.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff0877fe700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0881006d0 0x7ff088196de0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff080015040 tx=0x7ff080009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:46.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff084ff9700 1 -- 172.21.15.115:0/1184808472 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000e070 con 0x7ff0881006d0 2024-02-26T06:13:46.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08819f670 con 0x7ff0881006d0 2024-02-26T06:13:46.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff084ff9700 1 -- 172.21.15.115:0/1184808472 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff080003e70 con 0x7ff0881006d0 2024-02-26T06:13:46.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08819fc40 con 0x7ff0881006d0 2024-02-26T06:13:46.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.265+0000 7ff084ff9700 1 -- 172.21.15.115:0/1184808472 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08001dbe0 con 0x7ff0881006d0 2024-02-26T06:13:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.266+0000 7ff084ff9700 1 -- 172.21.15.115:0/1184808472 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7ff080012430 con 0x7ff0881006d0 2024-02-26T06:13:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.266+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff088190880 con 0x7ff0881006d0 2024-02-26T06:13:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.266+0000 7ff084ff9700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff074039410 0x7ff07403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.266+0000 7ff084ff9700 1 -- 172.21.15.115:0/1184808472 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff08004cb80 con 0x7ff0881006d0 2024-02-26T06:13:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.267+0000 7ff086ffd700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff074039410 0x7ff07403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.267+0000 7ff086ffd700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff074039410 0x7ff07403b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0700098e0 tx=0x7ff070006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:46.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.271+0000 7ff084ff9700 1 -- 172.21.15.115:0/1184808472 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0800117a0 con 0x7ff0881006d0 2024-02-26T06:13:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:45 smithi080 ceph-mon[36206]: Reconfiguring mon.c (monmap changed)... 2024-02-26T06:13:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:13:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:13:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:45 smithi080 ceph-mon[36206]: Reconfiguring daemon mon.c on smithi115 2024-02-26T06:13:46.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.478+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi080=y;smithi115=z", "target": ["mon-mgr", ""]}) v1 -- 0x7ff08802d7d0 con 0x7ff074039410 2024-02-26T06:13:46.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.491+0000 7ff084ff9700 1 -- 172.21.15.115:0/1184808472 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff08802d7d0 con 0x7ff074039410 2024-02-26T06:13:46.491 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mgr update... 2024-02-26T06:13:46.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff074039410 msgr2=0x7ff07403b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:46.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff074039410 0x7ff07403b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0700098e0 tx=0x7ff070006dd0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0881006d0 msgr2=0x7ff088196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:46.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0881006d0 0x7ff088196de0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff080015040 tx=0x7ff080009580 comp rx=0 tx=0).stop 2024-02-26T06:13:46.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 shutdown_connections 2024-02-26T06:13:46.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff074039410 0x7ff07403b8d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0881977a0 0x7ff08819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0881006d0 0x7ff088196de0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 --2- 172.21.15.115:0/1184808472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff088197320 0x7ff08819c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:46.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 >> 172.21.15.115:0/1184808472 conn(0x7ff0880fbc50 msgr2=0x7ff08806a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:46.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 shutdown_connections 2024-02-26T06:13:46.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:13:46.493+0000 7ff08da97700 1 -- 172.21.15.115:0/1184808472 wait complete. 2024-02-26T06:13:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:46 smithi115 ceph-mon[36426]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:47.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:46 smithi049 ceph-mon[31696]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:47.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:47.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:46 smithi080 ceph-mon[36206]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:47.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:47.627 DEBUG:teuthology.orchestra.run.smithi080:mgr.y> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.y.service 2024-02-26T06:13:47.630 DEBUG:teuthology.orchestra.run.smithi115:mgr.z> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.z.service 2024-02-26T06:13:47.632 INFO:tasks.cephadm:Deploying OSDs... 2024-02-26T06:13:47.632 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:13:47.632 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T06:13:47.692 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T06:13:47.692 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-02-26T06:13:47.745 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T06:13:47.745 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:47.745 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-02-26T06:13:47.746 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:47.746 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:47.746 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-02-26 06:12:40.923584029 +0000 2024-02-26T06:13:47.746 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-02-26 06:04:15.798503205 +0000 2024-02-26T06:13:47.746 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-02-26 06:04:15.798503205 +0000 2024-02-26T06:13:47.746 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-02-26 06:04:15.798503205 +0000 2024-02-26T06:13:47.746 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T06:13:47.849 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-02-26T06:13:47.849 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-02-26T06:13:47.849 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000281116 s, 1.8 MB/s 2024-02-26T06:13:47.853 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T06:13:47.912 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-02-26T06:13:47.962 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T06:13:47.963 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:47.963 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-02-26T06:13:47.963 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:47.963 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:47.963 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-02-26 06:12:41.026581242 +0000 2024-02-26T06:13:47.963 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-02-26 06:04:16.576482145 +0000 2024-02-26T06:13:47.963 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-02-26 06:04:16.576482145 +0000 2024-02-26T06:13:47.963 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-02-26 06:04:16.576482145 +0000 2024-02-26T06:13:47.963 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T06:13:48.058 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-02-26T06:13:48.058 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-02-26T06:13:48.058 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000265479 s, 1.9 MB/s 2024-02-26T06:13:48.063 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T06:13:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi080=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: Saving service mgr spec with placement smithi049=x;smithi080=y;smithi115=z;count:3 2024-02-26T06:13:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:48.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:13:48.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:48.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:48.124 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi080=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: Saving service mgr spec with placement smithi049=x;smithi080=y;smithi115=z;count:3 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:13:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:48.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:48.178 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi080=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: Saving service mgr spec with placement smithi049=x;smithi080=y;smithi115=z;count:3 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:48.179 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:48.180 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T06:13:48.180 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:48.181 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-02-26T06:13:48.181 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:48.181 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:48.181 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-02-26 06:12:41.088579565 +0000 2024-02-26T06:13:48.181 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-02-26 06:04:17.204465146 +0000 2024-02-26T06:13:48.181 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-02-26 06:04:17.204465146 +0000 2024-02-26T06:13:48.181 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-02-26 06:04:17.204465146 +0000 2024-02-26T06:13:48.181 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T06:13:48.319 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-02-26T06:13:48.319 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-02-26T06:13:48.319 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000279037 s, 1.8 MB/s 2024-02-26T06:13:48.324 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T06:13:48.384 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-02-26T06:13:48.439 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T06:13:48.439 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:48.440 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-02-26T06:13:48.440 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:48.440 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:48.440 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-02-26 06:12:41.144578050 +0000 2024-02-26T06:13:48.440 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-02-26 06:04:17.815448607 +0000 2024-02-26T06:13:48.440 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-02-26 06:04:17.815448607 +0000 2024-02-26T06:13:48.440 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-02-26 06:04:17.815448607 +0000 2024-02-26T06:13:48.440 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T06:13:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-02-26T06:13:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-02-26T06:13:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000280374 s, 1.8 MB/s 2024-02-26T06:13:48.550 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T06:13:48.611 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-02-26T06:13:48.611 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T06:13:48.630 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T06:13:48.630 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_1 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-02-26 06:13:31.839543616 +0000 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-02-26 06:03:25.018490536 +0000 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-02-26 06:03:25.018490536 +0000 2024-02-26T06:13:48.688 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-02-26 06:03:25.018490536 +0000 2024-02-26T06:13:48.689 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T06:13:48.758 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-02-26T06:13:48.758 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-02-26T06:13:48.759 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000302092 s, 1.7 MB/s 2024-02-26T06:13:48.760 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T06:13:48.819 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_2 2024-02-26T06:13:48.877 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T06:13:48.877 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:48.878 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-02-26T06:13:48.878 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:48.878 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:48.878 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-02-26 06:13:31.840543593 +0000 2024-02-26T06:13:48.878 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-02-26 06:03:25.396482006 +0000 2024-02-26T06:13:48.878 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-02-26 06:03:25.396482006 +0000 2024-02-26T06:13:48.878 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-02-26 06:03:25.396482006 +0000 2024-02-26T06:13:48.878 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T06:13:48.947 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-02-26T06:13:48.947 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-02-26T06:13:48.947 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000222123 s, 2.3 MB/s 2024-02-26T06:13:48.949 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T06:13:48.964 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:48 smithi115 ceph-mon[36426]: Deploying daemon mgr.z on smithi115 2024-02-26T06:13:48.964 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:48 smithi115 ceph-mon[36426]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:49.008 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_3 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-02-26 06:13:31.840543593 +0000 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-02-26 06:03:25.748474063 +0000 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-02-26 06:03:25.748474063 +0000 2024-02-26T06:13:49.066 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-02-26 06:03:25.748474063 +0000 2024-02-26T06:13:49.066 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T06:13:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:48 smithi080 ceph-mon[36206]: Deploying daemon mgr.z on smithi115 2024-02-26T06:13:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:48 smithi080 ceph-mon[36206]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:49.109 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-02-26T06:13:49.109 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-02-26T06:13:49.109 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000257843 s, 2.0 MB/s 2024-02-26T06:13:49.110 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T06:13:49.168 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_4 2024-02-26T06:13:49.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:48 smithi049 ceph-mon[31696]: Deploying daemon mgr.z on smithi115 2024-02-26T06:13:49.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:48 smithi049 ceph-mon[31696]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-02-26 06:13:31.840543593 +0000 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-02-26 06:03:26.081466549 +0000 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-02-26 06:03:26.081466549 +0000 2024-02-26T06:13:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-02-26 06:03:26.081466549 +0000 2024-02-26T06:13:49.225 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T06:13:49.293 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-02-26T06:13:49.293 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-02-26T06:13:49.293 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.0002101 s, 2.4 MB/s 2024-02-26T06:13:49.294 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T06:13:49.340 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:49 smithi115 systemd[1]: Starting Ceph mgr.z for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:13:49.351 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-02-26T06:13:49.351 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T06:13:49.367 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T06:13:49.367 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-02-26T06:13:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T06:13:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:49.422 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-02-26T06:13:49.422 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:49.422 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:49.423 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-02-26 06:13:49.126345076 +0000 2024-02-26T06:13:49.423 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-02-26 06:02:27.561976196 +0000 2024-02-26T06:13:49.423 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-02-26 06:02:27.561976196 +0000 2024-02-26T06:13:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-02-26 06:02:27.561976196 +0000 2024-02-26T06:13:49.423 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T06:13:49.490 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-02-26T06:13:49.490 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-02-26T06:13:49.490 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000218551 s, 2.3 MB/s 2024-02-26T06:13:49.492 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T06:13:49.547 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-02-26T06:13:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T06:13:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:49.601 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-02-26T06:13:49.602 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:49.602 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:49.602 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-02-26 06:13:49.126345076 +0000 2024-02-26T06:13:49.602 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-02-26 06:02:27.902969422 +0000 2024-02-26T06:13:49.602 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-02-26 06:02:27.902969422 +0000 2024-02-26T06:13:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-02-26 06:02:27.902969422 +0000 2024-02-26T06:13:49.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T06:13:49.665 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:49 smithi115 podman[38074]: 2024-02-26 06:13:49.30646736 +0000 UTC m=+0.017368116 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:13:49.665 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:49 smithi115 podman[38074]: 2024-02-26 06:13:49.425904574 +0000 UTC m=+0.136805327 container create 4250a23f67416e92957b7323b3fede75dc114e4e70439d9fcc0a1762d98afd40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:13:49.665 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:49 smithi115 podman[38074]: 2024-02-26 06:13:49.644936518 +0000 UTC m=+0.355837284 container init 4250a23f67416e92957b7323b3fede75dc114e4e70439d9fcc0a1762d98afd40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:13:49.665 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:49 smithi115 podman[38074]: 2024-02-26 06:13:49.647562377 +0000 UTC m=+0.358463124 container start 4250a23f67416e92957b7323b3fede75dc114e4e70439d9fcc0a1762d98afd40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS) 2024-02-26T06:13:49.667 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-02-26T06:13:49.667 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-02-26T06:13:49.668 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000187679 s, 2.7 MB/s 2024-02-26T06:13:49.669 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T06:13:49.724 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-02-26T06:13:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T06:13:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:49.779 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-02-26T06:13:49.779 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:49.779 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:49.779 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-02-26 06:13:49.127345056 +0000 2024-02-26T06:13:49.780 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-02-26 06:02:28.251962490 +0000 2024-02-26T06:13:49.780 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-02-26 06:02:28.251962490 +0000 2024-02-26T06:13:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-02-26 06:02:28.251962490 +0000 2024-02-26T06:13:49.780 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T06:13:49.846 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-02-26T06:13:49.846 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-02-26T06:13:49.846 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000175283 s, 2.9 MB/s 2024-02-26T06:13:49.848 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T06:13:49.903 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-02-26 06:13:49.127345056 +0000 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-02-26 06:02:28.614955280 +0000 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-02-26 06:02:28.614955280 +0000 2024-02-26T06:13:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-02-26 06:02:28.614955280 +0000 2024-02-26T06:13:49.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T06:13:50.022 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:49 smithi115 ceph-mgr[38183]: -- 172.21.15.115:0/471044250 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a2146701a0 con 0x55a214646800 2024-02-26T06:13:50.022 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:49 smithi115 bash[38074]: 4250a23f67416e92957b7323b3fede75dc114e4e70439d9fcc0a1762d98afd40 2024-02-26T06:13:50.022 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:49 smithi115 systemd[1]: Started Ceph mgr.z for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:13:50.024 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-02-26T06:13:50.024 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-02-26T06:13:50.024 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000188144 s, 2.7 MB/s 2024-02-26T06:13:50.025 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T06:13:50.081 INFO:tasks.cephadm:Deploying osd.0 on smithi049 with /dev/vg_nvme/lv_4... 2024-02-26T06:13:50.082 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T06:13:50.297 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:50.205+0000 7f0b7f7ab200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:13:50.297 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:50.297+0000 7f0b7f7ab200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:13:50.550 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:13:50.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:50.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:50.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:50.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:50.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:50.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:13:50.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:50.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:50.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:50 smithi049 ceph-mon[31696]: Deploying daemon mgr.y on smithi080 2024-02-26T06:13:51.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:51.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:51.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:51.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:51.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:51.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:13:51.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:51.082 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:51.082 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:50 smithi080 ceph-mon[36206]: Deploying daemon mgr.y on smithi080 2024-02-26T06:13:51.092 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:50.926+0000 7f0b7f7ab200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:13:51.092 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:51.017+0000 7f0b7f7ab200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:13:51.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:51.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:51.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:51.093 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:51.093 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:51.093 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:13:51.093 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:51.093 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:51.093 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:50 smithi115 ceph-mon[36426]: Deploying daemon mgr.y on smithi080 2024-02-26T06:13:51.597 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:51.156+0000 7f0b7f7ab200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:13:51.598 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:51.249+0000 7f0b7f7ab200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:13:51.598 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:51.330+0000 7f0b7f7ab200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:13:51.598 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:51.589+0000 7f0b7f7ab200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:13:51.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:51.697+0000 7f0b7f7ab200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:13:51.841 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:51.780+0000 7f0b7f7ab200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:13:52.074 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:51 smithi080 systemd[1]: Starting Ceph mgr.y for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:13:52.325 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 podman[37577]: 2024-02-26 06:13:51.97758101 +0000 UTC m=+0.017694848 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:13:52.325 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 podman[37577]: 2024-02-26 06:13:52.078730007 +0000 UTC m=+0.118843830 container create b645b34ca027021dc4e8392c9cf6a7e4033766d4e1eef6e811c034caf0221f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-26T06:13:52.325 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 podman[37577]: 2024-02-26 06:13:52.264344781 +0000 UTC m=+0.304458605 container init b645b34ca027021dc4e8392c9cf6a7e4033766d4e1eef6e811c034caf0221f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True) 2024-02-26T06:13:52.325 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 podman[37577]: 2024-02-26 06:13:52.266797173 +0000 UTC m=+0.306910991 container start b645b34ca027021dc4e8392c9cf6a7e4033766d4e1eef6e811c034caf0221f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-26T06:13:52.326 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-mgr[37596]: -- 172.21.15.80:0/1401021363 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55928dfce1a0 con 0x55928dfa5000 2024-02-26T06:13:52.326 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 bash[37577]: b645b34ca027021dc4e8392c9cf6a7e4033766d4e1eef6e811c034caf0221f77 2024-02-26T06:13:52.326 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 systemd[1]: Started Ceph mgr.y for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:13:52.340 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:51.874+0000 7f0b7f7ab200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:13:52.340 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:52.048+0000 7f0b7f7ab200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:13:53.080 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:52.826+0000 7f703e93e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:13:53.080 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:52.924+0000 7f703e93e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:13:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-mon[36206]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:53.163 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:53.029+0000 7f0b7f7ab200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:13:53.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:52 smithi115 ceph-mon[36426]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:53.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:52 smithi049 ceph-mon[31696]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:53.293 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T06:13:53.293 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T06:13:53.293 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-02-26T06:13:53.293 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-02-26T06:13:53.293 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0364563 s, 288 MB/s 2024-02-26T06:13:53.293 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-02-26T06:13:53.510 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:53.163+0000 7f0b7f7ab200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:13:53.510 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:53.510+0000 7f0b7f7ab200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:13:53.801 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:53.604+0000 7f0b7f7ab200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:13:53.801 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:53.801+0000 7f0b7f7ab200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:13:53.830 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:53.610+0000 7f703e93e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:13:53.830 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:53.705+0000 7f703e93e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:13:54.090 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:54.001+0000 7f0b7f7ab200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:13:54.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2024-02-26T06:13:54.306 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:53.847+0000 7f703e93e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:13:54.306 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:53.945+0000 7f703e93e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:13:54.306 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:54.043+0000 7f703e93e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:13:54.306 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:54.306+0000 7f703e93e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:13:54.540 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:54.284+0000 7f0b7f7ab200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:13:54.541 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:54.541+0000 7f0b7f7ab200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:13:54.559 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:13:54.580 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:54.415+0000 7f703e93e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:13:54.580 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:54.499+0000 7f703e93e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:13:54.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:54.770+0000 7f0b7f7ab200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:13:55.024 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:54.594+0000 7f703e93e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:13:55.024 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:54.768+0000 7f703e93e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:13:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:55.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:55.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:55.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:55.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:55.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:55.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:55.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:55.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:13:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:55.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:55.558+0000 7f0b7f7ab200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:13:56.029 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:55.732+0000 7f703e93e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:13:56.029 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:55.861+0000 7f703e93e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:13:56.107 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:56 smithi049 ceph-mon[31696]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T06:13:56.107 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:56 smithi049 ceph-mon[31696]: Reconfiguring daemon mgr.x on smithi049 2024-02-26T06:13:56.282 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:56 smithi080 ceph-mon[36206]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T06:13:56.282 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:56 smithi080 ceph-mon[36206]: Reconfiguring daemon mgr.x on smithi049 2024-02-26T06:13:56.282 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:56.190+0000 7f703e93e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:13:56.282 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:56.281+0000 7f703e93e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:13:56.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:56 smithi115 ceph-mon[36426]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T06:13:56.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:56 smithi115 ceph-mon[36426]: Reconfiguring daemon mgr.x on smithi049 2024-02-26T06:13:56.340 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:56.123+0000 7f0b7f7ab200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:13:56.340 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:56.219+0000 7f0b7f7ab200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:13:56.580 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:56.474+0000 7f703e93e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:13:56.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:13:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:13:56.432+0000 7f0b7f7ab200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:13:56.939 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:56.657+0000 7f703e93e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:13:56.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.978+0000 7f3111400700 1 -- 172.21.15.49:0/3096685764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c105a40 msgr2=0x7f310c10c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:56.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.978+0000 7f3111400700 1 --2- 172.21.15.49:0/3096685764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c105a40 0x7f310c10c360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f310400b0a0 tx=0x7f310400b3b0 comp rx=0 tx=0).stop 2024-02-26T06:13:56.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.978+0000 7f3111400700 1 -- 172.21.15.49:0/3096685764 shutdown_connections 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.978+0000 7f3111400700 1 --2- 172.21.15.49:0/3096685764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c105a40 0x7f310c10c360 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.978+0000 7f3111400700 1 --2- 172.21.15.49:0/3096685764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f310c103e90 0x7f310c1042b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.978+0000 7f3111400700 1 --2- 172.21.15.49:0/3096685764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f310c105080 0x7f310c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.978+0000 7f3111400700 1 -- 172.21.15.49:0/3096685764 >> 172.21.15.49:0/3096685764 conn(0x7f310c0ff480 msgr2=0x7f310c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.979+0000 7f3111400700 1 -- 172.21.15.49:0/3096685764 shutdown_connections 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.979+0000 7f3111400700 1 -- 172.21.15.49:0/3096685764 wait complete. 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.980+0000 7f3111400700 1 Processor -- start 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.980+0000 7f3111400700 1 -- start start 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f3111400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f310c103e90 0x7f310c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f3111400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f310c105080 0x7f310c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f3111400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c105a40 0x7f310c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f3111400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1a1ad0 con 0x7f310c105a40 2024-02-26T06:13:56.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f3111400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1a1c10 con 0x7f310c103e90 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f3111400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1a1da0 con 0x7f310c105080 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310a800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f310c103e90 0x7f310c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310a800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f310c103e90 0x7f310c199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:59468/0 (socket says 172.21.15.49:59468) 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310a800700 1 -- 172.21.15.49:0/350273715 learned_addr learned my addr 172.21.15.49:0/350273715 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310b200700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c105a40 0x7f310c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f3109e00700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f310c105080 0x7f310c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310a800700 1 -- 172.21.15.49:0/350273715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f310c105080 msgr2=0x7f310c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310a800700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f310c105080 0x7f310c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310a800700 1 -- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c105a40 msgr2=0x7f310c1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310a800700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c105a40 0x7f310c1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.981+0000 7f310a800700 1 -- 172.21.15.49:0/350273715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104009d00 con 0x7f310c103e90 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.982+0000 7f310a800700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f310c103e90 0x7f310c199f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00d980 tx=0x7f30fc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.982+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc004020 con 0x7f310c103e90 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.982+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30fc009730 con 0x7f310c103e90 2024-02-26T06:13:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.982+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c1a2080 con 0x7f310c103e90 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.982+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc003bb0 con 0x7f310c103e90 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.982+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c1a2580 con 0x7f310c103e90 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.983+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f30fc003d10 con 0x7f310c103e90 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.983+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c10b200 con 0x7f310c103e90 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.984+0000 7f3103400700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f30f00391c0 0x7f30f003b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.984+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f30fc04b950 con 0x7f310c103e90 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.984+0000 7f3109e00700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f30f00391c0 0x7f30f003b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.985+0000 7f3109e00700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f30f00391c0 0x7f30f003b680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f310c103bc0 tx=0x7f30f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:13:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:56.989+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30fc01a540 con 0x7f310c103e90 2024-02-26T06:13:57.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:57.048+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7f30fc01c410 con 0x7f310c103e90 2024-02-26T06:13:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:57 smithi049 ceph-mon[31696]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:57 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:13:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:57 smithi049 ceph-mon[31696]: Standby manager daemon z started 2024-02-26T06:13:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:57 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:13:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:57 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:13:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:57 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:13:57.198 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:57 smithi080 ceph-mon[36206]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:57.198 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:57 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:13:57.198 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:57 smithi080 ceph-mon[36206]: Standby manager daemon z started 2024-02-26T06:13:57.198 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:57 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:13:57.198 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:57 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:13:57.198 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:57 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:13:57.199 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:56.938+0000 7f703e93e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:13:57.199 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:57.197+0000 7f703e93e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:13:57.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:13:57.233+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f310c02d950 con 0x7f30f00391c0 2024-02-26T06:13:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:57 smithi115 ceph-mon[36426]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:57 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:13:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:57 smithi115 ceph-mon[36426]: Standby manager daemon z started 2024-02-26T06:13:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:57 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:13:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:57 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:13:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:57 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.115:0/3381822398' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:13:57.580 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:57.433+0000 7f703e93e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:13:58.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: mgrmap e14: x(active, since 89s), standbys: z 2024-02-26T06:13:58.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:58.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:58.224+0000 7f703e93e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: mgrmap e14: x(active, since 89s), standbys: z 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:58.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:58.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:58.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: mgrmap e14: x(active, since 89s), standbys: z 2024-02-26T06:13:58.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:13:58.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:13:58.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:13:58.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:58.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:58.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:13:58.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:13:58.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:13:58.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:13:59.049 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:58.783+0000 7f703e93e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:13:59.049 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:58.879+0000 7f703e93e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:13:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:59 smithi049 ceph-mon[31696]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:13:59 smithi049 ceph-mon[31696]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:59.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:59 smithi080 ceph-mon[36206]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:59.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:13:59 smithi080 ceph-mon[36206]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:13:59.331 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:13:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:13:59.088+0000 7f703e93e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:13:59.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:59 smithi115 ceph-mon[36426]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:13:59.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:13:59 smithi115 ceph-mon[36426]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:00.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:00.071+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mon.2 v2:172.21.15.80:3300/0 8 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f30fc004180 con 0x7f310c103e90 2024-02-26T06:14:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:00 smithi049 ceph-mon[31696]: Standby manager daemon y started 2024-02-26T06:14:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:00 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:14:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:00 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:14:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:00 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:14:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:00 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:14:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:00 smithi049 ceph-mon[31696]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:00.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:00 smithi080 ceph-mon[36206]: Standby manager daemon y started 2024-02-26T06:14:00.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:00 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:14:00.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:00 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:14:00.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:00 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:14:00.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:00 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:14:00.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:00 smithi080 ceph-mon[36206]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:00.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:00 smithi115 ceph-mon[36426]: Standby manager daemon y started 2024-02-26T06:14:00.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:00 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:14:00.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:00 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:14:00.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:00 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:14:00.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:00 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.80:0/1794622853' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:14:00.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:00 smithi115 ceph-mon[36426]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:01 smithi080 ceph-mon[36206]: mgrmap e15: x(active, since 92s), standbys: y, z 2024-02-26T06:14:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:14:01.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:01 smithi115 ceph-mon[36426]: mgrmap e15: x(active, since 92s), standbys: y, z 2024-02-26T06:14:01.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:14:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:01 smithi049 ceph-mon[31696]: mgrmap e15: x(active, since 92s), standbys: y, z 2024-02-26T06:14:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:14:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:02 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2354433786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]: dispatch 2024-02-26T06:14:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:02 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]: dispatch 2024-02-26T06:14:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:02 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]': finished 2024-02-26T06:14:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:02 smithi080 ceph-mon[36206]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T06:14:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:02 smithi080 ceph-mon[36206]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:02.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:02 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/15527157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:02 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2354433786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]: dispatch 2024-02-26T06:14:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:02 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]: dispatch 2024-02-26T06:14:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:02 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]': finished 2024-02-26T06:14:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:02 smithi115 ceph-mon[36426]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T06:14:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:02 smithi115 ceph-mon[36426]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:02 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/15527157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:02 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2354433786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]: dispatch 2024-02-26T06:14:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:02 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]: dispatch 2024-02-26T06:14:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:02 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db966476-ec8d-4000-8d96-2944d08fecca"}]': finished 2024-02-26T06:14:02.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:02 smithi049 ceph-mon[31696]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T06:14:02.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:02.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:02 smithi049 ceph-mon[31696]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:02.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:02 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/15527157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:04.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:04 smithi049 ceph-mon[31696]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:04.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:04 smithi080 ceph-mon[36206]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:04.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:04 smithi115 ceph-mon[36426]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:06.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:06 smithi049 ceph-mon[31696]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:06.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:06 smithi080 ceph-mon[36206]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:06 smithi115 ceph-mon[36426]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:08.636 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:08 smithi049 ceph-mon[31696]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:08.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:08 smithi080 ceph-mon[36206]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:08.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:08 smithi115 ceph-mon[36426]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:10.642 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:10 smithi115 ceph-mon[36426]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:10.643 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T06:14:10.643 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:10 smithi049 ceph-mon[31696]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T06:14:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:10.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:10 smithi080 ceph-mon[36206]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:10.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T06:14:10.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:11.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:11 smithi049 ceph-mon[31696]: Deploying daemon osd.0 on smithi049 2024-02-26T06:14:11.654 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:11 smithi115 ceph-mon[36426]: Deploying daemon osd.0 on smithi049 2024-02-26T06:14:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:11 smithi080 ceph-mon[36206]: Deploying daemon osd.0 on smithi049 2024-02-26T06:14:12.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:12 smithi049 ceph-mon[31696]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:12 smithi080 ceph-mon[36206]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:12.833 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:12 smithi115 ceph-mon[36426]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:14.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:14 smithi049 ceph-mon[31696]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:14 smithi080 ceph-mon[36206]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:14 smithi115 ceph-mon[36426]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:16 smithi049 ceph-mon[31696]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:16.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:16 smithi080 ceph-mon[36206]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:16.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:16 smithi115 ceph-mon[36426]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:18.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:18 smithi049 ceph-mon[31696]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:18.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:18 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:18.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:18 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:18.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:18 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:18 smithi080 ceph-mon[36206]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:18 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:18 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:18 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:18.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:18 smithi115 ceph-mon[36426]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:18.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:18 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:18.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:18 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:18.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:18 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:20.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:20 smithi049 ceph-mon[31696]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:20.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:20 smithi080 ceph-mon[36206]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:20.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:20 smithi115 ceph-mon[36426]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:22 smithi080 ceph-mon[36206]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:22 smithi080 ceph-mon[36206]: from='osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:14:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:22 smithi080 ceph-mon[36206]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:14:23.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:22 smithi049 ceph-mon[31696]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:23.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:23.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:23.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:22 smithi049 ceph-mon[31696]: from='osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:14:23.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:22 smithi049 ceph-mon[31696]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:14:23.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.226+0000 7f3103400700 1 -- 172.21.15.49:0/350273715 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f310c02d950 con 0x7f30f00391c0 2024-02-26T06:14:23.228 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 0 on host 'smithi049' 2024-02-26T06:14:23.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.230+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f30f00391c0 msgr2=0x7f30f003b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f30f00391c0 0x7f30f003b680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f310c103bc0 tx=0x7f30f8009260 comp rx=0 tx=0).stop 2024-02-26T06:14:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f310c103e90 msgr2=0x7f310c199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f310c103e90 0x7f310c199f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00d980 tx=0x7f30fc00dc90 comp rx=0 tx=0).stop 2024-02-26T06:14:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 shutdown_connections 2024-02-26T06:14:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f30f00391c0 0x7f30f003b680 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c105a40 0x7f310c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f310c103e90 0x7f310c199f40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 --2- 172.21.15.49:0/350273715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f310c105080 0x7f310c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 >> 172.21.15.49:0/350273715 conn(0x7f310c0ff480 msgr2=0x7f310c108340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:14:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 shutdown_connections 2024-02-26T06:14:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:14:23.231+0000 7f3111400700 1 -- 172.21.15.49:0/350273715 wait complete. 2024-02-26T06:14:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:22 smithi115 ceph-mon[36426]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:22 smithi115 ceph-mon[36426]: from='osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:14:23.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:22 smithi115 ceph-mon[36426]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:14:24.054 DEBUG:teuthology.orchestra.run.smithi049:osd.0> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.0.service 2024-02-26T06:14:24.059 INFO:tasks.cephadm:Deploying osd.1 on smithi080 with /dev/vg_nvme/lv_4... 2024-02-26T06:14:24.059 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T06:14:24.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:23 smithi049 ceph-mon[31696]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T06:14:24.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:23 smithi049 ceph-mon[31696]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T06:14:24.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:24.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:23 smithi049 ceph-mon[31696]: from='osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:14:24.289 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:23 smithi049 ceph-mon[31696]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:14:24.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:24.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:24.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:24.304 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:14:24.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:23 smithi080 ceph-mon[36206]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T06:14:24.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:23 smithi080 ceph-mon[36206]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T06:14:24.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:24.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:23 smithi080 ceph-mon[36206]: from='osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:14:24.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:23 smithi080 ceph-mon[36206]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:14:24.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:24.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:24.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:24.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:23 smithi115 ceph-mon[36426]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T06:14:24.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:23 smithi115 ceph-mon[36426]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T06:14:24.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:24.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:23 smithi115 ceph-mon[36426]: from='osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:14:24.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:23 smithi115 ceph-mon[36426]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:14:24.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:24.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:24.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:25.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:24 smithi049 ceph-mon[31696]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:25.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:24 smithi049 ceph-mon[31696]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:14:25.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:24 smithi049 ceph-mon[31696]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T06:14:25.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:24 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:25.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:24 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:25.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:24 smithi080 ceph-mon[36206]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:25.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:24 smithi080 ceph-mon[36206]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:14:25.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:24 smithi080 ceph-mon[36206]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T06:14:25.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:24 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:25.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:24 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:24 smithi115 ceph-mon[36426]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:14:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:24 smithi115 ceph-mon[36426]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:14:25.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:24 smithi115 ceph-mon[36426]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T06:14:25.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:24 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:25.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:24 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:25.986 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T06:14:25.987 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T06:14:25.987 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-02-26T06:14:25.987 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-02-26T06:14:25.987 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126616 s, 828 MB/s 2024-02-26T06:14:25.987 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-02-26T06:14:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:25 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:14:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:25 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:14:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:25 smithi080 ceph-mon[36206]: osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] boot 2024-02-26T06:14:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:25 smithi080 ceph-mon[36206]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T06:14:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:25 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:26.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:25 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:14:26.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:25 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:14:26.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:25 smithi115 ceph-mon[36426]: osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] boot 2024-02-26T06:14:26.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:25 smithi115 ceph-mon[36426]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T06:14:26.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:25 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:26.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:25 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:14:26.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:25 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:14:26.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:25 smithi049 ceph-mon[31696]: osd.0 [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] boot 2024-02-26T06:14:26.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:25 smithi049 ceph-mon[31696]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T06:14:26.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:25 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:14:26.717 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi080:vg_nvme/lv_4 2024-02-26T06:14:26.948 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:14:27.287 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:27 smithi080 ceph-mon[36206]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:27.287 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:27 smithi080 ceph-mon[36206]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T06:14:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:27 smithi115 ceph-mon[36426]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:27 smithi115 ceph-mon[36426]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T06:14:27.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:27 smithi049 ceph-mon[31696]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:27 smithi049 ceph-mon[31696]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 -- 172.21.15.80:0/1957008511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c100b90 msgr2=0x7f7f3c100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 --2- 172.21.15.80:0/1957008511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c100b90 0x7f7f3c100fb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7f380099c0 tx=0x7f7f38009cd0 comp rx=0 tx=0).stop 2024-02-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 -- 172.21.15.80:0/1957008511 shutdown_connections 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 --2- 172.21.15.80:0/1957008511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3c102730 0x7f7f3c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 --2- 172.21.15.80:0/1957008511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c100b90 0x7f7f3c100fb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 --2- 172.21.15.80:0/1957008511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c101d90 0x7f7f3c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 -- 172.21.15.80:0/1957008511 >> 172.21.15.80:0/1957008511 conn(0x7f7f3c0fc130 msgr2=0x7f7f3c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 -- 172.21.15.80:0/1957008511 shutdown_connections 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.412+0000 7f7f4374d700 1 -- 172.21.15.80:0/1957008511 wait complete. 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f4374d700 1 Processor -- start 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f4374d700 1 -- start start 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f4374d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3c100b90 0x7f7f3c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f4374d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c101d90 0x7f7f3c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f4374d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c102730 0x7f7f3c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f4374d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c19f950 con 0x7f7f3c102730 2024-02-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f4374d700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c19fa90 con 0x7f7f3c101d90 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f4374d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c19fc20 con 0x7f7f3c100b90 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f40ce8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c101d90 0x7f7f3c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f40ce8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c101d90 0x7f7f3c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52118/0 (socket says 172.21.15.80:52118) 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f40ce8700 1 -- 172.21.15.80:0/2487854220 learned_addr learned my addr 172.21.15.80:0/2487854220 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f414e9700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3c100b90 0x7f7f3c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f40ce8700 1 -- 172.21.15.80:0/2487854220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3c100b90 msgr2=0x7f7f3c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f40ce8700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3c100b90 0x7f7f3c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f40ce8700 1 -- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c102730 msgr2=0x7f7f3c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f40ce8700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c102730 0x7f7f3c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f40ce8700 1 -- 172.21.15.80:0/2487854220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380095f0 con 0x7f7f3c101d90 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.413+0000 7f7f414e9700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3c100b90 0x7f7f3c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.414+0000 7f7f40ce8700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c101d90 0x7f7f3c198300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00b530 tx=0x7f7f2c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.414+0000 7f7f327fc700 1 -- 172.21.15.80:0/2487854220 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c011830 con 0x7f7f3c101d90 2024-02-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.414+0000 7f7f327fc700 1 -- 172.21.15.80:0/2487854220 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c011e70 con 0x7f7f3c101d90 2024-02-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.414+0000 7f7f327fc700 1 -- 172.21.15.80:0/2487854220 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c00f2e0 con 0x7f7f3c101d90 2024-02-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.414+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c19ff00 con 0x7f7f3c101d90 2024-02-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.414+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c1a0440 con 0x7f7f3c101d90 2024-02-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.415+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c068960 con 0x7f7f3c101d90 2024-02-26T06:14:28.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.416+0000 7f7f327fc700 1 -- 172.21.15.80:0/2487854220 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f7f2c0105a0 con 0x7f7f3c101d90 2024-02-26T06:14:28.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.416+0000 7f7f327fc700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7f280a2140 0x7f7f280a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:14:28.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.417+0000 7f7f327fc700 1 -- 172.21.15.80:0/2487854220 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f7f2c0cc020 con 0x7f7f3c101d90 2024-02-26T06:14:28.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.417+0000 7f7f414e9700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7f280a2140 0x7f7f280a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:14:28.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.418+0000 7f7f414e9700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7f280a2140 0x7f7f280a4600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800b580 tx=0x7f7f38009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:14:28.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.420+0000 7f7f327fc700 1 -- 172.21.15.80:0/2487854220 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f2c096a60 con 0x7f7f3c101d90 2024-02-26T06:14:28.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:28.623+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f3c0fe2f0 con 0x7f7f280a2140 2024-02-26T06:14:29.267 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:29.268 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:14:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:14:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:14:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:29.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:14:29.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:14:29.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:29.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:14:29.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:29.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:29.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:29.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:30.269 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:30 smithi080 ceph-mon[36206]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:14:30.269 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:30 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi049 2024-02-26T06:14:30.269 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:30 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi049 to 17091M 2024-02-26T06:14:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:30 smithi115 ceph-mon[36426]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:14:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:30 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi049 2024-02-26T06:14:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:30 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi049 to 17091M 2024-02-26T06:14:30.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:30 smithi049 ceph-mon[31696]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:14:30.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:30 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi049 2024-02-26T06:14:30.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:30 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi049 to 17091M 2024-02-26T06:14:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:31 smithi080 ceph-mon[36206]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:31.338 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:31 smithi049 ceph-mon[31696]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:31 smithi115 ceph-mon[36426]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:32 smithi080 ceph-mon[36206]: from='client.? 172.21.15.80:0/1169201049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]: dispatch 2024-02-26T06:14:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:32 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]: dispatch 2024-02-26T06:14:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:32 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]': finished 2024-02-26T06:14:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:32 smithi080 ceph-mon[36206]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T06:14:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:32 smithi080 ceph-mon[36206]: from='client.? 172.21.15.80:0/502902105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:32 smithi115 ceph-mon[36426]: from='client.? 172.21.15.80:0/1169201049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]: dispatch 2024-02-26T06:14:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:32 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]: dispatch 2024-02-26T06:14:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:32 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]': finished 2024-02-26T06:14:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:32 smithi115 ceph-mon[36426]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T06:14:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:32 smithi115 ceph-mon[36426]: from='client.? 172.21.15.80:0/502902105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:32 smithi049 ceph-mon[31696]: from='client.? 172.21.15.80:0/1169201049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]: dispatch 2024-02-26T06:14:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:32 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]: dispatch 2024-02-26T06:14:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:32 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cbb4214-1d89-4598-834c-d567d0d63d04"}]': finished 2024-02-26T06:14:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:32 smithi049 ceph-mon[31696]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T06:14:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:32 smithi049 ceph-mon[31696]: from='client.? 172.21.15.80:0/502902105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:33 smithi080 ceph-mon[36206]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:33 smithi115 ceph-mon[36426]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:33.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:33 smithi049 ceph-mon[31696]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:35.133 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:35 smithi080 ceph-mon[36206]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:35 smithi115 ceph-mon[36426]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:35.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:35 smithi049 ceph-mon[31696]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:37.242 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:37 smithi080 ceph-mon[36206]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:37 smithi115 ceph-mon[36426]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:37 smithi049 ceph-mon[31696]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:39.133 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:39 smithi080 ceph-mon[36206]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:39.133 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T06:14:39.133 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:39 smithi115 ceph-mon[36426]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:39 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T06:14:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:39 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:39 smithi049 ceph-mon[31696]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:39 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T06:14:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:39 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:40.265 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:40 smithi080 ceph-mon[36206]: Deploying daemon osd.1 on smithi080 2024-02-26T06:14:40.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:40 smithi115 ceph-mon[36426]: Deploying daemon osd.1 on smithi080 2024-02-26T06:14:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:40 smithi049 ceph-mon[31696]: Deploying daemon osd.1 on smithi080 2024-02-26T06:14:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:41 smithi080 ceph-mon[36206]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:41 smithi115 ceph-mon[36426]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:41.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:41 smithi049 ceph-mon[31696]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:43.264 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:43 smithi080 ceph-mon[36206]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:43.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:43 smithi115 ceph-mon[36426]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:43.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:43 smithi049 ceph-mon[31696]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:44 smithi080 ceph-mon[36206]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:44 smithi115 ceph-mon[36426]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:44 smithi049 ceph-mon[31696]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:46 smithi080 ceph-mon[36206]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:46.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:46 smithi115 ceph-mon[36426]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:46.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:46.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:46.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:46.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:46.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:46.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:46 smithi049 ceph-mon[31696]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:46.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:46.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:46.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:46.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:46.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:47.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.013+0000 7f7f327fc700 1 -- 172.21.15.80:0/2487854220 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7f3c0fe2f0 con 0x7f7f280a2140 2024-02-26T06:14:47.015 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 1 on host 'smithi080' 2024-02-26T06:14:47.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.015+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7f280a2140 msgr2=0x7f7f280a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:47.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.015+0000 7f7f4374d700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7f280a2140 0x7f7f280a4600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800b580 tx=0x7f7f38009530 comp rx=0 tx=0).stop 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.015+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c101d90 msgr2=0x7f7f3c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.015+0000 7f7f4374d700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c101d90 0x7f7f3c198300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00b530 tx=0x7f7f2c00b8f0 comp rx=0 tx=0).stop 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.016+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 shutdown_connections 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.016+0000 7f7f4374d700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7f280a2140 0x7f7f280a4600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.016+0000 7f7f4374d700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c102730 0x7f7f3c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.016+0000 7f7f4374d700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3c100b90 0x7f7f3c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.016+0000 7f7f4374d700 1 --2- 172.21.15.80:0/2487854220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f3c101d90 0x7f7f3c198300 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.016+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 >> 172.21.15.80:0/2487854220 conn(0x7f7f3c0fc130 msgr2=0x7f7f3c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.016+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 shutdown_connections 2024-02-26T06:14:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:14:47.016+0000 7f7f4374d700 1 -- 172.21.15.80:0/2487854220 wait complete. 2024-02-26T06:14:47.616 DEBUG:teuthology.orchestra.run.smithi080:osd.1> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.1.service 2024-02-26T06:14:47.618 INFO:tasks.cephadm:Deploying osd.2 on smithi115 with /dev/vg_nvme/lv_4... 2024-02-26T06:14:47.618 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T06:14:47.858 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:14:48.258 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:48.258 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:48.258 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:48.258 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:47 smithi115 ceph-mon[36426]: from='osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:14:48.259 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:47 smithi115 ceph-mon[36426]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:14:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:47 smithi080 ceph-mon[36206]: from='osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:14:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:47 smithi080 ceph-mon[36206]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:14:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:14:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:47 smithi049 ceph-mon[31696]: from='osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:14:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:47 smithi049 ceph-mon[31696]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:14:49.043 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:48 smithi080 ceph-mon[36206]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:49.043 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:48 smithi080 ceph-mon[36206]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T06:14:49.043 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:48 smithi080 ceph-mon[36206]: from='osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:14:49.043 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:48 smithi080 ceph-mon[36206]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T06:14:49.043 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:48 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:49.043 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:48 smithi080 ceph-mon[36206]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:14:49.130 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:48 smithi115 ceph-mon[36426]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:49.130 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:48 smithi115 ceph-mon[36426]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T06:14:49.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:48 smithi115 ceph-mon[36426]: from='osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:14:49.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:48 smithi115 ceph-mon[36426]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T06:14:49.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:48 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:49.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:48 smithi115 ceph-mon[36426]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:14:49.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:48 smithi049 ceph-mon[31696]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:49.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:48 smithi049 ceph-mon[31696]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T06:14:49.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:48 smithi049 ceph-mon[31696]: from='osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:14:49.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:48 smithi049 ceph-mon[31696]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T06:14:49.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:48 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:49.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:48 smithi049 ceph-mon[31696]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:14:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T06:14:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T06:14:49.689 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-02-26T06:14:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-02-26T06:14:49.689 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121459 s, 863 MB/s 2024-02-26T06:14:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-02-26T06:14:49.830 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:14:49 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1[40786]: 2024-02-26T06:14:49.463+0000 7f6381d5b700 -1 osd.1 0 waiting for initial osdmap 2024-02-26T06:14:49.830 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:14:49 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1[40786]: 2024-02-26T06:14:49.467+0000 7f6379b46700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:14:50.242 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:49 smithi115 ceph-mon[36426]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:14:50.242 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:49 smithi115 ceph-mon[36426]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T06:14:50.243 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:50.243 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:50.243 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:49 smithi115 ceph-mon[36426]: from='osd.1 ' entity='osd.1' 2024-02-26T06:14:50.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-02-26T06:14:50.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:49 smithi080 ceph-mon[36206]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:14:50.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:49 smithi080 ceph-mon[36206]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T06:14:50.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:50.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:50.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:49 smithi080 ceph-mon[36206]: from='osd.1 ' entity='osd.1' 2024-02-26T06:14:50.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:49 smithi049 ceph-mon[31696]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:14:50.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:49 smithi049 ceph-mon[31696]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T06:14:50.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:50.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:50.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:49 smithi049 ceph-mon[31696]: from='osd.1 ' entity='osd.1' 2024-02-26T06:14:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:14:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:14:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:14:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] boot 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] boot 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.322 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:51.323 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:51.323 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:14:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:14:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:14:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: osd.1 [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] boot 2024-02-26T06:14:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T06:14:51.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:14:51.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:14:51.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:51.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:51.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:14:51.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:14:52.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- 172.21.15.115:0/3985502372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f381021d0 msgr2=0x7f3f381025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:52.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3985502372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f381021d0 0x7f3f381025f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009990 tx=0x7f3f34009ca0 comp rx=0 tx=0).stop 2024-02-26T06:14:52.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- 172.21.15.115:0/3985502372 shutdown_connections 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3985502372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f38100700 0x7f3f38100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3985502372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f381021d0 0x7f3f381025f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3985502372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f380ffd60 0x7f3f381001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- 172.21.15.115:0/3985502372 >> 172.21.15.115:0/3985502372 conn(0x7f3f380fb900 msgr2=0x7f3f380fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- 172.21.15.115:0/3985502372 shutdown_connections 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- 172.21.15.115:0/3985502372 wait complete. 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 Processor -- start 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- start start 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f380ffd60 0x7f3f38199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f38100700 0x7f3f3819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f381021d0 0x7f3f381a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:14:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f381a1a10 con 0x7f3f381021d0 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f381a1b50 con 0x7f3f38100700 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.030+0000 7f3f3f720700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f381a1ce0 con 0x7f3f380ffd60 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f380ffd60 0x7f3f38199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f380ffd60 0x7f3f38199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36066/0 (socket says 172.21.15.115:36066) 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 -- 172.21.15.115:0/3371655296 learned_addr learned my addr 172.21.15.115:0/3371655296 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 -- 172.21.15.115:0/3371655296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f38100700 msgr2=0x7f3f3819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f38100700 0x7f3f3819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 -- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f381021d0 msgr2=0x7f3f381a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f381021d0 0x7f3f381a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 -- 172.21.15.115:0/3371655296 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340095f0 con 0x7f3f380ffd60 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3d4bc700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f380ffd60 0x7f3f38199e80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f34000c00 tx=0x7f3f3400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:14:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f2e7fc700 1 -- 172.21.15.115:0/3371655296 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3401d070 con 0x7f3f380ffd60 2024-02-26T06:14:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f2e7fc700 1 -- 172.21.15.115:0/3371655296 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f34005510 con 0x7f3f380ffd60 2024-02-26T06:14:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f381a1f60 con 0x7f3f380ffd60 2024-02-26T06:14:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f2e7fc700 1 -- 172.21.15.115:0/3371655296 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34004bd0 con 0x7f3f380ffd60 2024-02-26T06:14:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.031+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f381a2310 con 0x7f3f380ffd60 2024-02-26T06:14:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.032+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f38068970 con 0x7f3f380ffd60 2024-02-26T06:14:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.033+0000 7f3f2e7fc700 1 -- 172.21.15.115:0/3371655296 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f3f34004d30 con 0x7f3f380ffd60 2024-02-26T06:14:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.034+0000 7f3f2e7fc700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3f240a20c0 0x7f3f240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:14:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.034+0000 7f3f2e7fc700 1 -- 172.21.15.115:0/3371655296 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f3f340cd410 con 0x7f3f380ffd60 2024-02-26T06:14:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.034+0000 7f3f3ccbb700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3f240a20c0 0x7f3f240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:14:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.035+0000 7f3f3ccbb700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3f240a20c0 0x7f3f240a4580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f280097b0 tx=0x7f3f28006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:14:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.038+0000 7f3f2e7fc700 1 -- 172.21.15.115:0/3371655296 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f34097cc0 con 0x7f3f380ffd60 2024-02-26T06:14:52.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:14:52.236+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f3802dab0 con 0x7f3f240a20c0 2024-02-26T06:14:52.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:52 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi080 2024-02-26T06:14:52.237 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:52 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi080 to 17080M 2024-02-26T06:14:52.287 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:52 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi080 2024-02-26T06:14:52.299 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:52 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi080 to 17080M 2024-02-26T06:14:52.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:52 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi080 2024-02-26T06:14:52.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:52 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi080 to 17080M 2024-02-26T06:14:53.120 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:53 smithi115 ceph-mon[36426]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:53.120 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:53 smithi115 ceph-mon[36426]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T06:14:53.120 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:53 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:14:53.120 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:53 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:14:53.120 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:53 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:53.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:53 smithi080 ceph-mon[36206]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:53.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:53 smithi080 ceph-mon[36206]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T06:14:53.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:53 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:14:53.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:53 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:14:53.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:53 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:53 smithi049 ceph-mon[31696]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:53 smithi049 ceph-mon[31696]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T06:14:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:53 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:14:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:53 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:14:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:53 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:14:54.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:54 smithi080 ceph-mon[36206]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:14:54.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:54 smithi115 ceph-mon[36426]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:14:54.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:54 smithi049 ceph-mon[31696]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:14:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:55 smithi080 ceph-mon[36206]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:55 smithi080 ceph-mon[36206]: from='client.? 172.21.15.115:0/2062670308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]: dispatch 2024-02-26T06:14:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:55 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]: dispatch 2024-02-26T06:14:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:55 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]': finished 2024-02-26T06:14:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:55 smithi080 ceph-mon[36206]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T06:14:55.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:14:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:55 smithi115 ceph-mon[36426]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:55 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/2062670308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]: dispatch 2024-02-26T06:14:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:55 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]: dispatch 2024-02-26T06:14:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:55 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]': finished 2024-02-26T06:14:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:55 smithi115 ceph-mon[36426]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T06:14:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:14:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:55 smithi049 ceph-mon[31696]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:55 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/2062670308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]: dispatch 2024-02-26T06:14:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:55 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]: dispatch 2024-02-26T06:14:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:55 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d416d235-0756-4c38-835b-3d8a8e04d2cc"}]': finished 2024-02-26T06:14:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:55 smithi049 ceph-mon[31696]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T06:14:55.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:14:56.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:56 smithi080 ceph-mon[36206]: from='client.? 172.21.15.115:0/2183970141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:56.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:56 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/2183970141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:56.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:56 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/2183970141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:14:57.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:57 smithi080 ceph-mon[36206]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:57 smithi115 ceph-mon[36426]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:57.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:57 smithi049 ceph-mon[31696]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:59.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:14:59 smithi080 ceph-mon[36206]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:59.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:14:59 smithi115 ceph-mon[36426]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:14:59.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:14:59 smithi049 ceph-mon[31696]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:01 smithi115 ceph-mon[36426]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:01 smithi080 ceph-mon[36206]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:01 smithi049 ceph-mon[31696]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T06:15:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T06:15:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T06:15:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:03.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:03 smithi080 ceph-mon[36206]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:03.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:03 smithi080 ceph-mon[36206]: Deploying daemon osd.2 on smithi115 2024-02-26T06:15:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:03 smithi115 ceph-mon[36426]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:03 smithi115 ceph-mon[36426]: Deploying daemon osd.2 on smithi115 2024-02-26T06:15:03.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:03 smithi049 ceph-mon[31696]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:03.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:03 smithi049 ceph-mon[31696]: Deploying daemon osd.2 on smithi115 2024-02-26T06:15:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:05 smithi080 ceph-mon[36206]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:05.331 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:05 smithi115 ceph-mon[36426]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:05.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:05 smithi049 ceph-mon[31696]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:07 smithi080 ceph-mon[36206]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:07 smithi115 ceph-mon[36426]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:07 smithi049 ceph-mon[31696]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:08.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:08 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:08.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:08 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:08.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:08 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:08.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:08 smithi115 ceph-mon[36426]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:08.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:08 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:08.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:08 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:08.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:08 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:08.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:08 smithi080 ceph-mon[36206]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:08.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:08.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:08.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:08.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:08 smithi049 ceph-mon[31696]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:10.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.420+0000 7f3f2e7fc700 1 -- 172.21.15.115:0/3371655296 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3f3802dab0 con 0x7f3f240a20c0 2024-02-26T06:15:10.421 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 2 on host 'smithi115' 2024-02-26T06:15:10.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3f240a20c0 msgr2=0x7f3f240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:10.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3f240a20c0 0x7f3f240a4580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f280097b0 tx=0x7f3f28006d20 comp rx=0 tx=0).stop 2024-02-26T06:15:10.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f380ffd60 msgr2=0x7f3f38199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:10.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f380ffd60 0x7f3f38199e80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f34000c00 tx=0x7f3f3400f740 comp rx=0 tx=0).stop 2024-02-26T06:15:10.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 shutdown_connections 2024-02-26T06:15:10.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3f240a20c0 0x7f3f240a4580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:10.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f381021d0 0x7f3f381a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:10.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f380ffd60 0x7f3f38199e80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:10.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 --2- 172.21.15.115:0/3371655296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f38100700 0x7f3f3819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:10.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 >> 172.21.15.115:0/3371655296 conn(0x7f3f380fb900 msgr2=0x7f3f380fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:15:10.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 shutdown_connections 2024-02-26T06:15:10.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:15:10.422+0000 7f3f3f720700 1 -- 172.21.15.115:0/3371655296 wait complete. 2024-02-26T06:15:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:10 smithi049 ceph-mon[31696]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:10.723 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:10 smithi115 ceph-mon[36426]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:10.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:10 smithi080 ceph-mon[36206]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:10.993 DEBUG:teuthology.orchestra.run.smithi115:osd.2> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.2.service 2024-02-26T06:15:10.995 INFO:tasks.cephadm:Deploying osd.3 on smithi049 with /dev/vg_nvme/lv_3... 2024-02-26T06:15:10.995 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T06:15:11.296 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2[41579]: 2024-02-26T06:15:11.081+0000 7f386f72c7c0 -1 osd.2 0 log_to_monitors true 2024-02-26T06:15:11.466 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:15:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:11 smithi115 ceph-mon[36426]: from='osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:15:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:11 smithi115 ceph-mon[36426]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:15:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:11 smithi049 ceph-mon[31696]: from='osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:15:11.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:11 smithi049 ceph-mon[31696]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:15:11.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:11 smithi080 ceph-mon[36206]: from='osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:15:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:11 smithi080 ceph-mon[36206]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:15:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:12.590 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:12 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2[41579]: 2024-02-26T06:15:12.426+0000 7f3864d91700 -1 osd.2 0 waiting for initial osdmap 2024-02-26T06:15:12.590 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:12 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2[41579]: 2024-02-26T06:15:12.433+0000 7f3860b84700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:15:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:12 smithi115 ceph-mon[36426]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:12 smithi115 ceph-mon[36426]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T06:15:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:12 smithi115 ceph-mon[36426]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T06:15:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:12 smithi115 ceph-mon[36426]: from='osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:15:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:12 smithi115 ceph-mon[36426]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:15:12.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:12 smithi049 ceph-mon[31696]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:12.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:12 smithi049 ceph-mon[31696]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T06:15:12.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:12 smithi049 ceph-mon[31696]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T06:15:12.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:12.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:12 smithi049 ceph-mon[31696]: from='osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:15:12.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:12 smithi049 ceph-mon[31696]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:15:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:12 smithi080 ceph-mon[36206]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:12 smithi080 ceph-mon[36206]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T06:15:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:12 smithi080 ceph-mon[36206]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T06:15:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:12 smithi080 ceph-mon[36206]: from='osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:15:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:12 smithi080 ceph-mon[36206]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:15:13.548 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:13 smithi049 ceph-mon[31696]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:15:13.548 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:13 smithi049 ceph-mon[31696]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T06:15:13.548 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:13.548 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:13.751 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:13 smithi115 ceph-mon[36426]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:15:13.752 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:13 smithi115 ceph-mon[36426]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T06:15:13.752 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:13.752 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:13.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:13 smithi080 ceph-mon[36206]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:15:13.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:13 smithi080 ceph-mon[36206]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T06:15:13.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:13.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:14.455 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T06:15:14.455 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T06:15:14.456 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-02-26T06:15:14.456 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-02-26T06:15:14.456 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0396788 s, 264 MB/s 2024-02-26T06:15:14.456 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-02-26T06:15:14.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:14 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:15:14.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:14 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:15:14.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:14 smithi049 ceph-mon[31696]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:14.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:14.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:14 smithi049 ceph-mon[31696]: osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] boot 2024-02-26T06:15:14.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:14 smithi049 ceph-mon[31696]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T06:15:14.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:14 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:15:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:14 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:15:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:14 smithi080 ceph-mon[36206]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:14 smithi080 ceph-mon[36206]: osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] boot 2024-02-26T06:15:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:14 smithi080 ceph-mon[36206]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T06:15:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:14 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:15:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:14 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:15:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:14 smithi115 ceph-mon[36426]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:15:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:14 smithi115 ceph-mon[36426]: osd.2 [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] boot 2024-02-26T06:15:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:14 smithi115 ceph-mon[36426]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T06:15:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:15:15.305 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2024-02-26T06:15:15.765 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:15:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T06:15:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi115 2024-02-26T06:15:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi115 to 17080M 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:16.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:16 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi115 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi115 to 17080M 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:16.594 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:16.595 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.595 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:16.595 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:16.595 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:16.595 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:16 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi115 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi115 to 17080M 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:16.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:16 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:17.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:17 smithi049 ceph-mon[31696]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:17.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:15:17.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T06:15:17.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:17 smithi049 ceph-mon[31696]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T06:15:17.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:15:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:17 smithi080 ceph-mon[36206]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:15:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T06:15:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:17 smithi080 ceph-mon[36206]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T06:15:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:15:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:17 smithi115 ceph-mon[36426]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:15:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T06:15:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:17 smithi115 ceph-mon[36426]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T06:15:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:15:17.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 -- 172.21.15.49:0/1518592718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4540746d0 msgr2=0x7ff454074b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:17.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 --2- 172.21.15.49:0/1518592718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4540746d0 0x7ff454074b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff440009990 tx=0x7ff440009ca0 comp rx=0 tx=0).stop 2024-02-26T06:15:17.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 -- 172.21.15.49:0/1518592718 shutdown_connections 2024-02-26T06:15:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 --2- 172.21.15.49:0/1518592718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff454109ab0 0x7ff45410bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 --2- 172.21.15.49:0/1518592718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff454075ca0 0x7ff454074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 --2- 172.21.15.49:0/1518592718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4540746d0 0x7ff454074b50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 -- 172.21.15.49:0/1518592718 >> 172.21.15.49:0/1518592718 conn(0x7ff4540fd180 msgr2=0x7ff4540ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:15:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 -- 172.21.15.49:0/1518592718 shutdown_connections 2024-02-26T06:15:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.948+0000 7ff453200700 1 -- 172.21.15.49:0/1518592718 wait complete. 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.949+0000 7ff453200700 1 Processor -- start 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.949+0000 7ff453200700 1 -- start start 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff453200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4540746d0 0x7ff454199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff453200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454075ca0 0x7ff45419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff453200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff454109ab0 0x7ff4541a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff453200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541a18f0 con 0x7ff454075ca0 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff453200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541a1a30 con 0x7ff4540746d0 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff453200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541a1bf0 con 0x7ff454109ab0 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4540746d0 0x7ff454199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:15:17.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454075ca0 0x7ff45419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4540746d0 0x7ff454199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60666/0 (socket says 172.21.15.49:60666) 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451e00700 1 -- 172.21.15.49:0/3079039093 learned_addr learned my addr 172.21.15.49:0/3079039093 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff452800700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff454109ab0 0x7ff4541a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451e00700 1 -- 172.21.15.49:0/3079039093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff454109ab0 msgr2=0x7ff4541a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451e00700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff454109ab0 0x7ff4541a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451e00700 1 -- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454075ca0 msgr2=0x7ff45419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451e00700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454075ca0 0x7ff45419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff451e00700 1 -- 172.21.15.49:0/3079039093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4400095f0 con 0x7ff4540746d0 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.950+0000 7ff452800700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff454109ab0 0x7ff4541a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.951+0000 7ff451e00700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4540746d0 0x7ff454199df0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff44400e970 tx=0x7ff44400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.951+0000 7ff44aa00700 1 -- 172.21.15.49:0/3079039093 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444009790 con 0x7ff4540746d0 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.951+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4541a1ed0 con 0x7ff4540746d0 2024-02-26T06:15:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.951+0000 7ff44aa00700 1 -- 172.21.15.49:0/3079039093 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44400eec0 con 0x7ff4540746d0 2024-02-26T06:15:17.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.951+0000 7ff44aa00700 1 -- 172.21.15.49:0/3079039093 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444005690 con 0x7ff4540746d0 2024-02-26T06:15:17.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.951+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4541a22f0 con 0x7ff4540746d0 2024-02-26T06:15:17.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.952+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff454108870 con 0x7ff4540746d0 2024-02-26T06:15:17.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.954+0000 7ff44aa00700 1 -- 172.21.15.49:0/3079039093 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7ff44400cbd0 con 0x7ff4540746d0 2024-02-26T06:15:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.956+0000 7ff44aa00700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4380a2140 0x7ff4380a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:15:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.956+0000 7ff44aa00700 1 -- 172.21.15.49:0/3079039093 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7ff444014070 con 0x7ff4540746d0 2024-02-26T06:15:17.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.956+0000 7ff451400700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4380a2140 0x7ff4380a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:15:17.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.957+0000 7ff451400700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4380a2140 0x7ff4380a4600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b6f0 tx=0x7ff440005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:15:17.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:17.958+0000 7ff44aa00700 1 -- 172.21.15.49:0/3079039093 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff444097060 con 0x7ff4540746d0 2024-02-26T06:15:18.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:18.192+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff45402d8d0 con 0x7ff4380a2140 2024-02-26T06:15:18.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:18 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:15:18.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:18 smithi049 ceph-mon[31696]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T06:15:18.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:18 smithi049 ceph-mon[31696]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:18.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:18 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:15:18.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:18 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:15:18.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:18 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:18.420 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:15:18 smithi049 sudo[48121]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:15:18.420 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:15:18 smithi049 sudo[48121]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:18.420 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:15:18 smithi049 sudo[48121]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:18.420 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:15:18 smithi049 sudo[48121]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:18.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:18 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:15:18.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:18 smithi080 ceph-mon[36206]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T06:15:18.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:18 smithi080 ceph-mon[36206]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:18.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:18 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:15:18.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:18 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:15:18.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:18 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:18.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:18 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:15:18.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:18 smithi115 ceph-mon[36426]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T06:15:18.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:18 smithi115 ceph-mon[36426]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:18.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:18 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:15:18.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:18 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:15:18.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:18 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:19.169 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:15:18 smithi049 sudo[48218]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:15:19.169 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:15:18 smithi049 sudo[48218]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:19.169 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:15:18 smithi049 sudo[48218]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:19.169 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:15:18 smithi049 sudo[48218]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:19.451 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:19 smithi080 ceph-mon[36206]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:15:19.451 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:19 smithi080 ceph-mon[36206]: osdmap e22: 3 total, 3 up, 3 in 2024-02-26T06:15:19.451 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:19.451 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:15:19 smithi080 sudo[44917]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:15:19.451 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:15:19 smithi080 sudo[44917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:19.451 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:15:19 smithi080 sudo[44917]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:19.451 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:15:19 smithi080 sudo[44917]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:19.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:19 smithi049 ceph-mon[31696]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:15:19.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:19 smithi049 ceph-mon[31696]: osdmap e22: 3 total, 3 up, 3 in 2024-02-26T06:15:19.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:19 smithi115 ceph-mon[36426]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:15:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:19 smithi115 ceph-mon[36426]: osdmap e22: 3 total, 3 up, 3 in 2024-02-26T06:15:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:19.830 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:15:19 smithi080 sudo[44920]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:15:19.830 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:15:19 smithi080 sudo[44920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:19.830 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:15:19 smithi080 sudo[44920]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:19.830 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:15:19 smithi080 sudo[44920]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:20.047 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:19 smithi115 sudo[45362]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:15:20.047 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:19 smithi115 sudo[45362]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:20.047 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:19 smithi115 sudo[45362]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:20.047 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:19 smithi115 sudo[45362]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:20.340 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:20 smithi115 sudo[45365]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:15:20.340 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:20 smithi115 sudo[45365]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:20.340 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:20 smithi115 sudo[45365]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:20.340 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:15:20 smithi115 sudo[45365]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:21.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:20 smithi049 sudo[48291]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:15:21.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:20 smithi049 sudo[48291]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:21.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:20 smithi049 sudo[48291]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:21.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:20 smithi049 sudo[48291]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:21.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:21.671+0000 7ff44aa00700 1 -- 172.21.15.49:0/3079039093 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff444097430 con 0x7ff4540746d0 2024-02-26T06:15:21.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:21 smithi049 sudo[48315]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:15:21.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:21 smithi049 sudo[48315]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:21.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:21 smithi049 sudo[48315]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:21.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:21 smithi049 sudo[48315]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:21.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:21 smithi049 ceph-mon[31696]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:21.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:21 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:21.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:21 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:21.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:21 smithi049 ceph-mon[31696]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:21.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:21 smithi115 ceph-mon[36426]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:21.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:21 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:21.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:21 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:21.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:21 smithi115 ceph-mon[36426]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:21.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:21 smithi115 sudo[45368]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:15:21.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:21 smithi115 sudo[45368]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:21.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:21 smithi115 sudo[45368]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:21.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:21 smithi115 sudo[45368]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:21 smithi080 ceph-mon[36206]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:21 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:21 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:21 smithi080 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:22.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 sudo[45371]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:15:22.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 sudo[45371]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:22.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 sudo[45371]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:22.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 sudo[45371]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:22.659 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 sudo[44923]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:15:22.659 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 sudo[44923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:22.659 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 sudo[44923]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:22.659 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 sudo[44923]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 sudo[44926]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:15:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 sudo[44926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:15:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 sudo[44926]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 sudo[44926]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3184438833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "674ab7f1-6ee1-4e0d-9e10-42770a30c5f8"}]: dispatch 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:23.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3184438833' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "674ab7f1-6ee1-4e0d-9e10-42770a30c5f8"}]': finished 2024-02-26T06:15:23.082 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T06:15:23.082 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:23.082 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.082 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.082 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:22 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3184438833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "674ab7f1-6ee1-4e0d-9e10-42770a30c5f8"}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3184438833' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "674ab7f1-6ee1-4e0d-9e10-42770a30c5f8"}]': finished 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:22 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3184438833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "674ab7f1-6ee1-4e0d-9e10-42770a30c5f8"}]: dispatch 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3184438833' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "674ab7f1-6ee1-4e0d-9e10-42770a30c5f8"}]': finished 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:15:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:22 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:15:24.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:23 smithi080 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:24.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:23 smithi080 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:24.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:23 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1039379977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:15:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:23 smithi115 ceph-mon[36426]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:23 smithi115 ceph-mon[36426]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:23 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1039379977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:15:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:23 smithi049 ceph-mon[31696]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:15:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:23 smithi049 ceph-mon[31696]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:15:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:23 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1039379977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:15:25.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:24 smithi080 ceph-mon[36206]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:25.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:24 smithi115 ceph-mon[36426]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:24 smithi049 ceph-mon[31696]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:27.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:26 smithi080 ceph-mon[36206]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:27.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:26 smithi115 ceph-mon[36426]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:27.145 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:26 smithi049 ceph-mon[31696]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:28 smithi049 ceph-mon[31696]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-26T06:15:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:28 smithi080 ceph-mon[36206]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-26T06:15:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:28 smithi115 ceph-mon[36426]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-26T06:15:30.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:30 smithi049 ceph-mon[31696]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-26T06:15:31.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:30 smithi080 ceph-mon[36206]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-26T06:15:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:30 smithi115 ceph-mon[36426]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-26T06:15:32.083 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:31 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T06:15:32.084 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:31 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:32.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:31 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T06:15:32.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:31 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:32.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T06:15:32.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:32 smithi080 ceph-mon[36206]: Deploying daemon osd.3 on smithi049 2024-02-26T06:15:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:32 smithi080 ceph-mon[36206]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-26T06:15:33.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:32 smithi115 ceph-mon[36426]: Deploying daemon osd.3 on smithi049 2024-02-26T06:15:33.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:32 smithi115 ceph-mon[36426]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-26T06:15:33.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:32 smithi049 ceph-mon[31696]: Deploying daemon osd.3 on smithi049 2024-02-26T06:15:33.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:32 smithi049 ceph-mon[31696]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-26T06:15:35.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:34 smithi080 ceph-mon[36206]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 917 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-26T06:15:35.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:34 smithi115 ceph-mon[36426]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 917 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-26T06:15:35.093 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:34 smithi049 ceph-mon[31696]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 917 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-26T06:15:36.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:36 smithi049 ceph-mon[31696]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:15:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:36 smithi080 ceph-mon[36206]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:15:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:36 smithi115 ceph-mon[36426]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:15:38.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:38 smithi049 ceph-mon[31696]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:15:39.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:38 smithi080 ceph-mon[36206]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:15:39.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:38 smithi115 ceph-mon[36426]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:15:41.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:40 smithi049 ceph-mon[31696]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:41.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:41.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:41.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:40 smithi080 ceph-mon[36206]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:40 smithi115 ceph-mon[36426]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:43.232 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:42 smithi049 ceph-mon[31696]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:43.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:42 smithi080 ceph-mon[36206]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:43.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:42 smithi115 ceph-mon[36426]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:44 smithi049 ceph-mon[31696]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:44.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:44.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:44 smithi049 ceph-mon[31696]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:15:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:44 smithi080 ceph-mon[36206]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:44 smithi080 ceph-mon[36206]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:15:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:44 smithi115 ceph-mon[36426]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:44 smithi115 ceph-mon[36426]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:15:44.889 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 3 on host 'smithi049' 2024-02-26T06:15:44.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.887+0000 7ff44aa00700 1 -- 172.21.15.49:0/3079039093 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff45402d8d0 con 0x7ff4380a2140 2024-02-26T06:15:44.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4380a2140 msgr2=0x7ff4380a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:44.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4380a2140 0x7ff4380a4600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b6f0 tx=0x7ff440005fb0 comp rx=0 tx=0).stop 2024-02-26T06:15:44.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4540746d0 msgr2=0x7ff454199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:44.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4540746d0 0x7ff454199df0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff44400e970 tx=0x7ff44400ec80 comp rx=0 tx=0).stop 2024-02-26T06:15:44.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 shutdown_connections 2024-02-26T06:15:44.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4380a2140 0x7ff4380a4600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:44.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff454109ab0 0x7ff4541a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:44.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4540746d0 0x7ff454199df0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:44.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 --2- 172.21.15.49:0/3079039093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454075ca0 0x7ff45419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:44.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 >> 172.21.15.49:0/3079039093 conn(0x7ff4540fd180 msgr2=0x7ff4541059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:15:44.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.890+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 shutdown_connections 2024-02-26T06:15:44.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:15:44.891+0000 7ff453200700 1 -- 172.21.15.49:0/3079039093 wait complete. 2024-02-26T06:15:45.685 DEBUG:teuthology.orchestra.run.smithi049:osd.3> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.3.service 2024-02-26T06:15:45.691 INFO:tasks.cephadm:Deploying osd.4 on smithi080 with /dev/vg_nvme/lv_3... 2024-02-26T06:15:45.691 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T06:15:45.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:45.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:45.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:45.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-mon[31696]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T06:15:45.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-mon[31696]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T06:15:45.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-mon[31696]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:15:45.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:45.934 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:15:46.056 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:46.057 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:46.057 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:46.057 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:45 smithi080 ceph-mon[36206]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T06:15:46.057 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:45 smithi080 ceph-mon[36206]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T06:15:46.057 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:45 smithi080 ceph-mon[36206]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:15:46.057 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:46.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:46.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:46.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:15:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:45 smithi115 ceph-mon[36426]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T06:15:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:45 smithi115 ceph-mon[36426]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T06:15:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:45 smithi115 ceph-mon[36426]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:15:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:46.419 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:15:45.976+0000 7f70b4600700 -1 osd.3 0 waiting for initial osdmap 2024-02-26T06:15:46.420 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:15:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:15:45.986+0000 7f70af400700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:15:47.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:46 smithi049 ceph-mon[31696]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:47.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:46 smithi049 ceph-mon[31696]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:15:47.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:46 smithi049 ceph-mon[31696]: osdmap e25: 4 total, 3 up, 4 in 2024-02-26T06:15:47.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:47.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:47.219 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:46 smithi080 ceph-mon[36206]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:47.219 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:46 smithi080 ceph-mon[36206]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:15:47.219 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:46 smithi080 ceph-mon[36206]: osdmap e25: 4 total, 3 up, 4 in 2024-02-26T06:15:47.219 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:47.219 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:46 smithi115 ceph-mon[36426]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:15:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:46 smithi115 ceph-mon[36426]: from='osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:15:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:46 smithi115 ceph-mon[36426]: osdmap e25: 4 total, 3 up, 4 in 2024-02-26T06:15:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:47.781 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T06:15:47.781 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T06:15:47.781 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-02-26T06:15:47.781 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-02-26T06:15:47.781 INFO:teuthology.orchestra.run.smithi080.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0136932 s, 766 MB/s 2024-02-26T06:15:47.781 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-02-26T06:15:48.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:47 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:15:48.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:47 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:15:48.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:47 smithi049 ceph-mon[31696]: osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] boot 2024-02-26T06:15:48.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:47 smithi049 ceph-mon[31696]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T06:15:48.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:48.299 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:47 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:15:48.299 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:47 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:15:48.299 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:47 smithi080 ceph-mon[36206]: osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] boot 2024-02-26T06:15:48.299 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:47 smithi080 ceph-mon[36206]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T06:15:48.299 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:48.325 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi080:vg_nvme/lv_3 2024-02-26T06:15:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:47 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:15:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:47 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:15:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:47 smithi115 ceph-mon[36426]: osd.3 [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] boot 2024-02-26T06:15:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:47 smithi115 ceph-mon[36426]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T06:15:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:15:48.564 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:15:49.095 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:48 smithi049 ceph-mon[31696]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:15:49.095 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:48 smithi049 ceph-mon[31696]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T06:15:49.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:48 smithi080 ceph-mon[36206]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:15:49.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:48 smithi080 ceph-mon[36206]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T06:15:49.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:48 smithi115 ceph-mon[36426]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:15:49.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:48 smithi115 ceph-mon[36426]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T06:15:50.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:49 smithi080 ceph-mon[36206]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T06:15:50.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:49 smithi080 ceph-mon[36206]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 -- 172.21.15.80:0/1066379445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708102730 msgr2=0x7fd708104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 --2- 172.21.15.80:0/1066379445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708102730 0x7fd708104b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009990 tx=0x7fd6f8009ca0 comp rx=0 tx=0).stop 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 -- 172.21.15.80:0/1066379445 shutdown_connections 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 --2- 172.21.15.80:0/1066379445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708107990 0x7fd708109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 --2- 172.21.15.80:0/1066379445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708102730 0x7fd708104b20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 --2- 172.21.15.80:0/1066379445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd708105060 0x7fd708107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 -- 172.21.15.80:0/1066379445 >> 172.21.15.80:0/1066379445 conn(0x7fd7080fc140 msgr2=0x7fd7080fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 -- 172.21.15.80:0/1066379445 shutdown_connections 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.211+0000 7fd70e408700 1 -- 172.21.15.80:0/1066379445 wait complete. 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70e408700 1 Processor -- start 2024-02-26T06:15:50.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70e408700 1 -- start start 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70e408700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708102730 0x7fd708197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70e408700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd708105060 0x7fd708198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70e408700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708107990 0x7fd70819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70e408700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70819f870 con 0x7fd708102730 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70e408700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70819f9b0 con 0x7fd708107990 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70e408700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70819fb70 con 0x7fd708105060 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70c9a5700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708107990 0x7fd70819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70c9a5700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708107990 0x7fd70819f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38798/0 (socket says 172.21.15.80:38798) 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70c9a5700 1 -- 172.21.15.80:0/3934508859 learned_addr learned my addr 172.21.15.80:0/3934508859 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd7077fe700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd708105060 0x7fd708198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70c9a5700 1 -- 172.21.15.80:0/3934508859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd708105060 msgr2=0x7fd708198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70c9a5700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd708105060 0x7fd708198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70c9a5700 1 -- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708102730 msgr2=0x7fd708197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70c9a5700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708102730 0x7fd708197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:15:50.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.212+0000 7fd70c9a5700 1 -- 172.21.15.80:0/3934508859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80095f0 con 0x7fd708107990 2024-02-26T06:15:50.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.213+0000 7fd70c9a5700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708107990 0x7fd70819f2a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd70000b9e0 tx=0x7fd70000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:15:50.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.213+0000 7fd7057fa700 1 -- 172.21.15.80:0/3934508859 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000c9b0 con 0x7fd708107990 2024-02-26T06:15:50.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.213+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70819fe50 con 0x7fd708107990 2024-02-26T06:15:50.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.213+0000 7fd7057fa700 1 -- 172.21.15.80:0/3934508859 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd700014440 con 0x7fd708107990 2024-02-26T06:15:50.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.213+0000 7fd7057fa700 1 -- 172.21.15.80:0/3934508859 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7000125a0 con 0x7fd708107990 2024-02-26T06:15:50.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.213+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7081a0270 con 0x7fd708107990 2024-02-26T06:15:50.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.214+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7080fdcc0 con 0x7fd708107990 2024-02-26T06:15:50.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.215+0000 7fd7057fa700 1 -- 172.21.15.80:0/3934508859 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd7000145b0 con 0x7fd708107990 2024-02-26T06:15:50.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.215+0000 7fd7057fa700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd6f00a2150 0x7fd6f00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:15:50.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.216+0000 7fd7057fa700 1 -- 172.21.15.80:0/3934508859 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3669+0+0 (secure 0 0 0) 0x7fd700006e40 con 0x7fd708107990 2024-02-26T06:15:50.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.216+0000 7fd707fff700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd6f00a2150 0x7fd6f00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:15:50.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.217+0000 7fd707fff700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd6f00a2150 0x7fd6f00a4610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80098e0 tx=0x7fd6f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:15:50.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.219+0000 7fd7057fa700 1 -- 172.21.15.80:0/3934508859 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7000967a0 con 0x7fd708107990 2024-02-26T06:15:50.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:49 smithi115 ceph-mon[36426]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T06:15:50.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:49 smithi115 ceph-mon[36426]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T06:15:50.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:15:50.419+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd70802d950 con 0x7fd6f00a2150 2024-02-26T06:15:50.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:49 smithi049 ceph-mon[31696]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T06:15:50.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:49 smithi049 ceph-mon[31696]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T06:15:51.296 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:50 smithi080 ceph-mon[36206]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:15:51.296 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:15:51.296 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:15:51.296 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:50 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:51.296 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:50 smithi080 ceph-mon[36206]: osdmap e30: 4 total, 4 up, 4 in 2024-02-26T06:15:51.321 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:50 smithi049 ceph-mon[31696]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:15:51.321 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:15:51.321 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:15:51.321 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:50 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:51.321 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:50 smithi049 ceph-mon[31696]: osdmap e30: 4 total, 4 up, 4 in 2024-02-26T06:15:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:50 smithi115 ceph-mon[36426]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:15:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:15:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:15:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:50 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:50 smithi115 ceph-mon[36426]: osdmap e30: 4 total, 4 up, 4 in 2024-02-26T06:15:52.062 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:15:52.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:52.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:52.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:52.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:52.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:51 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.285 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:15:52.286 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.286 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.286 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:52.286 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:52.286 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.286 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:52.286 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:52.286 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:51 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:15:52.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:52.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:15:52.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:52.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:15:52.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:15:52.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:51 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:15:53.302 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:52 smithi080 ceph-mon[36206]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:15:53.302 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:52 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi049 2024-02-26T06:15:53.302 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:52 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi049 to 8545M 2024-02-26T06:15:53.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:52 smithi115 ceph-mon[36426]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:15:53.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:52 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi049 2024-02-26T06:15:53.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:52 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi049 to 8545M 2024-02-26T06:15:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:52 smithi049 ceph-mon[31696]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:15:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:52 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi049 2024-02-26T06:15:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:52 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi049 to 8545M 2024-02-26T06:15:54.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:53 smithi080 ceph-mon[36206]: from='client.? 172.21.15.80:0/4044671531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]: dispatch 2024-02-26T06:15:54.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:53 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]: dispatch 2024-02-26T06:15:54.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:53 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]': finished 2024-02-26T06:15:54.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:53 smithi080 ceph-mon[36206]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T06:15:54.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:53 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:15:54.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:53 smithi080 ceph-mon[36206]: from='client.? 172.21.15.80:0/1501943963' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:15:54.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:53 smithi115 ceph-mon[36426]: from='client.? 172.21.15.80:0/4044671531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]: dispatch 2024-02-26T06:15:54.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:53 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]: dispatch 2024-02-26T06:15:54.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:53 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]': finished 2024-02-26T06:15:54.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:53 smithi115 ceph-mon[36426]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T06:15:54.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:53 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:15:54.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:53 smithi115 ceph-mon[36426]: from='client.? 172.21.15.80:0/1501943963' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:15:54.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:53 smithi049 ceph-mon[31696]: from='client.? 172.21.15.80:0/4044671531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]: dispatch 2024-02-26T06:15:54.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:53 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]: dispatch 2024-02-26T06:15:54.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:53 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41dcb99d-6d5c-4ee1-a461-06944881881c"}]': finished 2024-02-26T06:15:54.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:53 smithi049 ceph-mon[31696]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T06:15:54.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:53 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:15:54.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:53 smithi049 ceph-mon[31696]: from='client.? 172.21.15.80:0/1501943963' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:15:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:54 smithi080 ceph-mon[36206]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-02-26T06:15:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:54 smithi115 ceph-mon[36426]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-02-26T06:15:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:54 smithi049 ceph-mon[31696]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-02-26T06:15:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:57 smithi080 ceph-mon[36206]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:15:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:57 smithi115 ceph-mon[36426]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:15:57.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:57 smithi049 ceph-mon[31696]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:15:59.263 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:15:59 smithi080 ceph-mon[36206]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:15:59.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:15:59 smithi115 ceph-mon[36426]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:15:59.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:15:59 smithi049 ceph-mon[31696]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:16:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:01 smithi080 ceph-mon[36206]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:16:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T06:16:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:01.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:01 smithi115 ceph-mon[36426]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:16:01.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T06:16:01.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:01 smithi049 ceph-mon[31696]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:16:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T06:16:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:02 smithi080 ceph-mon[36206]: Deploying daemon osd.4 on smithi080 2024-02-26T06:16:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:02 smithi115 ceph-mon[36426]: Deploying daemon osd.4 on smithi080 2024-02-26T06:16:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:02 smithi049 ceph-mon[31696]: Deploying daemon osd.4 on smithi080 2024-02-26T06:16:03.312 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:03 smithi080 ceph-mon[36206]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:03.528 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:03 smithi049 ceph-mon[31696]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:03.529 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:03 smithi115 ceph-mon[36426]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:05 smithi080 ceph-mon[36206]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:05 smithi115 ceph-mon[36426]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:05.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:05 smithi049 ceph-mon[31696]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:06 smithi080 ceph-mon[36206]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:06 smithi115 ceph-mon[36426]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:06 smithi049 ceph-mon[31696]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:09.169 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:08 smithi080 ceph-mon[36206]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:09.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.245+0000 7fd7057fa700 1 -- 172.21.15.80:0/3934508859 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd70802d950 con 0x7fd6f00a2150 2024-02-26T06:16:09.246 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 4 on host 'smithi080' 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd6f00a2150 msgr2=0x7fd6f00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd6f00a2150 0x7fd6f00a4610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80098e0 tx=0x7fd6f8005fb0 comp rx=0 tx=0).stop 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708107990 msgr2=0x7fd70819f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708107990 0x7fd70819f2a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd70000b9e0 tx=0x7fd70000bda0 comp rx=0 tx=0).stop 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 shutdown_connections 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd6f00a2150 0x7fd6f00a4610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd708107990 0x7fd70819f2a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd708102730 0x7fd708197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 --2- 172.21.15.80:0/3934508859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd708105060 0x7fd708198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 >> 172.21.15.80:0/3934508859 conn(0x7fd7080fc140 msgr2=0x7fd708103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 shutdown_connections 2024-02-26T06:16:09.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:16:09.248+0000 7fd70e408700 1 -- 172.21.15.80:0/3934508859 wait complete. 2024-02-26T06:16:09.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:08 smithi115 ceph-mon[36426]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:09.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:08 smithi049 ceph-mon[31696]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:09.944 DEBUG:teuthology.orchestra.run.smithi080:osd.4> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.4.service 2024-02-26T06:16:09.946 INFO:tasks.cephadm:Deploying osd.5 on smithi115 with /dev/vg_nvme/lv_3... 2024-02-26T06:16:09.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T06:16:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:16:10.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:10 smithi115 ceph-mon[36426]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:10.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:10 smithi115 ceph-mon[36426]: from='osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:16:10.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:10 smithi115 ceph-mon[36426]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:16:10.476 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.476 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.476 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:10 smithi080 ceph-mon[36206]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:10.476 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.476 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.476 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:10 smithi080 ceph-mon[36206]: from='osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:16:10.476 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:10 smithi080 ceph-mon[36206]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:16:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:10 smithi049 ceph-mon[31696]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:10.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:10 smithi049 ceph-mon[31696]: from='osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:16:10.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:10 smithi049 ceph-mon[31696]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:16:11.695 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:11 smithi115 ceph-mon[36426]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T06:16:11.695 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:11 smithi115 ceph-mon[36426]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T06:16:11.695 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:11.695 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:11 smithi115 ceph-mon[36426]: from='osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:16:11.695 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:11 smithi115 ceph-mon[36426]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:16:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:11 smithi080 ceph-mon[36206]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T06:16:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:11 smithi080 ceph-mon[36206]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T06:16:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:11 smithi080 ceph-mon[36206]: from='osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:16:11.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:11 smithi080 ceph-mon[36206]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:16:11.831 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:16:11 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:16:11.580+0000 7fbdf256a700 -1 osd.4 0 waiting for initial osdmap 2024-02-26T06:16:11.831 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:16:11 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:16:11.588+0000 7fbdee35d700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:16:11.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:11 smithi049 ceph-mon[31696]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T06:16:11.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:11 smithi049 ceph-mon[31696]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T06:16:11.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:11.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:11 smithi049 ceph-mon[31696]: from='osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:16:11.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:11 smithi049 ceph-mon[31696]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:16:12.297 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T06:16:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T06:16:12.298 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-02-26T06:16:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-02-26T06:16:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122969 s, 853 MB/s 2024-02-26T06:16:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-02-26T06:16:12.810 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:12 smithi115 ceph-mon[36426]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:12.810 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:12 smithi115 ceph-mon[36426]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:16:12.810 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:12 smithi115 ceph-mon[36426]: osdmap e33: 5 total, 4 up, 5 in 2024-02-26T06:16:12.810 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:12.810 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:12 smithi080 ceph-mon[36206]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:12 smithi080 ceph-mon[36206]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:16:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:12 smithi080 ceph-mon[36206]: osdmap e33: 5 total, 4 up, 5 in 2024-02-26T06:16:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:12.836 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-02-26T06:16:12.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:12 smithi049 ceph-mon[31696]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:12.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:12 smithi049 ceph-mon[31696]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:16:12.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:12 smithi049 ceph-mon[31696]: osdmap e33: 5 total, 4 up, 5 in 2024-02-26T06:16:12.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:12.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:13.081 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:16:13.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:16:13.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] boot 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi080 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi080 to 8540M 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:13.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:13.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:13.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] boot 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi080 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:13.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi080 to 8540M 2024-02-26T06:16:13.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:13.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:13.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:13.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:13.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:13.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:13.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:14.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:16:14.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:16:14.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:14.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: osd.4 [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] boot 2024-02-26T06:16:14.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T06:16:14.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi080 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi080 to 8540M 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:14.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:14.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 -- 172.21.15.115:0/2921239526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8102ea0 msgr2=0x7f1bb8103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:14.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/2921239526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8102ea0 0x7f1bb8103300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009990 tx=0x7f1bac009ca0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 -- 172.21.15.115:0/2921239526 shutdown_connections 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/2921239526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bb8103840 0x7f1bb810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/2921239526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1bb8101ca0 0x7f1bb81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/2921239526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8102ea0 0x7f1bb8103300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 -- 172.21.15.115:0/2921239526 >> 172.21.15.115:0/2921239526 conn(0x7f1bb80fd280 msgr2=0x7f1bb80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 -- 172.21.15.115:0/2921239526 shutdown_connections 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.640+0000 7f1bbdc12700 1 -- 172.21.15.115:0/2921239526 wait complete. 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bbdc12700 1 Processor -- start 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bbdc12700 1 -- start start 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bbdc12700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8101ca0 0x7f1bb8197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bbdc12700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bb8102ea0 0x7f1bb81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bbdc12700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1bb8103840 0x7f1bb819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bbdc12700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb819f830 con 0x7f1bb8102ea0 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bbdc12700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb819f970 con 0x7f1bb8103840 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bbdc12700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb819fb30 con 0x7f1bb8101ca0 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8101ca0 0x7f1bb8197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8101ca0 0x7f1bb8197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57130/0 (socket says 172.21.15.115:57130) 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bb77fe700 1 -- 172.21.15.115:0/1971425030 learned_addr learned my addr 172.21.15.115:0/1971425030 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.641+0000 7f1bb7fff700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1bb8103840 0x7f1bb819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb77fe700 1 -- 172.21.15.115:0/1971425030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1bb8103840 msgr2=0x7f1bb819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb77fe700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1bb8103840 0x7f1bb819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb77fe700 1 -- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bb8102ea0 msgr2=0x7f1bb81981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb77fe700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bb8102ea0 0x7f1bb81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb77fe700 1 -- 172.21.15.115:0/1971425030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac0095f0 con 0x7f1bb8101ca0 2024-02-26T06:16:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb77fe700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8101ca0 0x7f1bb8197cb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c970 tx=0x7f1ba800cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb4ff9700 1 -- 172.21.15.115:0/1971425030 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8004220 con 0x7f1bb8101ca0 2024-02-26T06:16:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb4ff9700 1 -- 172.21.15.115:0/1971425030 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ba8007d80 con 0x7f1bb8101ca0 2024-02-26T06:16:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb819fe10 con 0x7f1bb8101ca0 2024-02-26T06:16:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bb4ff9700 1 -- 172.21.15.115:0/1971425030 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8003da0 con 0x7f1bb8101ca0 2024-02-26T06:16:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.642+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb81a0300 con 0x7f1bb8101ca0 2024-02-26T06:16:14.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.643+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb8068930 con 0x7f1bb8101ca0 2024-02-26T06:16:14.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.644+0000 7f1bb4ff9700 1 -- 172.21.15.115:0/1971425030 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1ba80078a0 con 0x7f1bb8101ca0 2024-02-26T06:16:14.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.644+0000 7f1bb4ff9700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1ba00a2150 0x7f1ba00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:14.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.645+0000 7f1bb4ff9700 1 -- 172.21.15.115:0/1971425030 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f1ba80ccbd0 con 0x7f1bb8101ca0 2024-02-26T06:16:14.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.645+0000 7f1bb6ffd700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1ba00a2150 0x7f1ba00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:14.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.646+0000 7f1bb6ffd700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1ba00a2150 0x7f1ba00a4610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1bac003800 tx=0x7f1bac009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:14.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.648+0000 7f1bb4ff9700 1 -- 172.21.15.115:0/1971425030 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ba8096c10 con 0x7f1bb8101ca0 2024-02-26T06:16:14.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:14.852+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bb802d950 con 0x7f1ba00a2150 2024-02-26T06:16:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:14 smithi080 ceph-mon[36206]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:14 smithi080 ceph-mon[36206]: osdmap e35: 5 total, 5 up, 5 in 2024-02-26T06:16:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:16:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:16:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:14 smithi115 ceph-mon[36426]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:14 smithi115 ceph-mon[36426]: osdmap e35: 5 total, 5 up, 5 in 2024-02-26T06:16:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:16:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:16:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:14 smithi049 ceph-mon[31696]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-02-26T06:16:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:14 smithi049 ceph-mon[31696]: osdmap e35: 5 total, 5 up, 5 in 2024-02-26T06:16:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:16:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:16:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:16.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:15 smithi115 ceph-mon[36426]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:16.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:15 smithi080 ceph-mon[36206]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:16.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:15 smithi049 ceph-mon[31696]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:17.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:16 smithi115 ceph-mon[36426]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:17.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:16 smithi080 ceph-mon[36206]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:17.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:16 smithi049 ceph-mon[31696]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:18.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:17 smithi080 ceph-mon[36206]: from='client.? 172.21.15.115:0/3023041676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]: dispatch 2024-02-26T06:16:18.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:17 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]: dispatch 2024-02-26T06:16:18.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:17 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]': finished 2024-02-26T06:16:18.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:17 smithi080 ceph-mon[36206]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T06:16:18.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:18.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:17 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/3023041676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]: dispatch 2024-02-26T06:16:18.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:17 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]: dispatch 2024-02-26T06:16:18.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:17 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]': finished 2024-02-26T06:16:18.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:17 smithi115 ceph-mon[36426]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T06:16:18.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:18.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:17 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/3023041676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]: dispatch 2024-02-26T06:16:18.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:17 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]: dispatch 2024-02-26T06:16:18.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:17 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53d273fb-162f-4ca7-b995-1387266dacb6"}]': finished 2024-02-26T06:16:18.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:17 smithi049 ceph-mon[31696]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T06:16:18.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:19.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:18 smithi080 ceph-mon[36206]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:19.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:18 smithi080 ceph-mon[36206]: from='client.? 172.21.15.115:0/3450759573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:16:19.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:18 smithi115 ceph-mon[36426]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:19.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:18 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/3450759573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:16:19.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:18 smithi049 ceph-mon[31696]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:19.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:18 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/3450759573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:16:21.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:20 smithi080 ceph-mon[36206]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:21.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:20 smithi115 ceph-mon[36426]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:21.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:20 smithi049 ceph-mon[31696]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:23.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:22 smithi080 ceph-mon[36206]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:22 smithi115 ceph-mon[36426]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:23.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:22 smithi049 ceph-mon[31696]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:25.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:24 smithi080 ceph-mon[36206]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:25.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:24 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T06:16:25.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:24 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:24 smithi115 ceph-mon[36426]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:24 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T06:16:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:24 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:25.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:24 smithi049 ceph-mon[31696]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:25.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:24 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T06:16:25.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:24 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:26.100 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:26 smithi115 ceph-mon[36426]: Deploying daemon osd.5 on smithi115 2024-02-26T06:16:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:26 smithi080 ceph-mon[36206]: Deploying daemon osd.5 on smithi115 2024-02-26T06:16:26.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:26 smithi049 ceph-mon[31696]: Deploying daemon osd.5 on smithi115 2024-02-26T06:16:27.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:27 smithi080 ceph-mon[36206]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:27 smithi115 ceph-mon[36426]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:27 smithi049 ceph-mon[31696]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:29.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:29 smithi080 ceph-mon[36206]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:29 smithi115 ceph-mon[36426]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:29.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:29 smithi049 ceph-mon[31696]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:31 smithi115 ceph-mon[36426]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:31 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:31 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:31.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:31 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:31 smithi080 ceph-mon[36206]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:31 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:31 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:31 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:31 smithi049 ceph-mon[31696]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:33.316 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:33 smithi115 ceph-mon[36426]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:33 smithi080 ceph-mon[36206]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:33.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:33 smithi049 ceph-mon[31696]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.341+0000 7f1bb4ff9700 1 -- 172.21.15.115:0/1971425030 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1bb802d950 con 0x7f1ba00a2150 2024-02-26T06:16:34.349 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 5 on host 'smithi115' 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1ba00a2150 msgr2=0x7f1ba00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1ba00a2150 0x7f1ba00a4610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1bac003800 tx=0x7f1bac009530 comp rx=0 tx=0).stop 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8101ca0 msgr2=0x7f1bb8197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8101ca0 0x7f1bb8197cb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c970 tx=0x7f1ba800cd30 comp rx=0 tx=0).stop 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 shutdown_connections 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1ba00a2150 0x7f1ba00a4610 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1bb8103840 0x7f1bb819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bb8101ca0 0x7f1bb8197cb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 --2- 172.21.15.115:0/1971425030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bb8102ea0 0x7f1bb81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.343+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 >> 172.21.15.115:0/1971425030 conn(0x7f1bb80fd280 msgr2=0x7f1bb8106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:34.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.344+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 shutdown_connections 2024-02-26T06:16:34.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:16:34.344+0000 7f1bbdc12700 1 -- 172.21.15.115:0/1971425030 wait complete. 2024-02-26T06:16:34.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:16:34.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:34 smithi115 ceph-mon[36426]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:16:34.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:16:34.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:34 smithi080 ceph-mon[36206]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:16:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:16:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:34 smithi049 ceph-mon[31696]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:16:35.009 DEBUG:teuthology.orchestra.run.smithi115:osd.5> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.5.service 2024-02-26T06:16:35.012 INFO:tasks.cephadm:Deploying osd.6 on smithi049 with /dev/vg_nvme/lv_2... 2024-02-26T06:16:35.013 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T06:16:35.500 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:16:36.568 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:36 smithi115 ceph-mon[36426]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T06:16:36.568 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:36 smithi115 ceph-mon[36426]: from='osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:16:36.568 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:36 smithi115 ceph-mon[36426]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T06:16:36.568 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:36.568 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:36 smithi115 ceph-mon[36426]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:16:36.568 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:36 smithi115 ceph-mon[36426]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:36.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:36 smithi080 ceph-mon[36206]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T06:16:36.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:36 smithi080 ceph-mon[36206]: from='osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:16:36.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:36 smithi080 ceph-mon[36206]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T06:16:36.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:36.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:36 smithi080 ceph-mon[36206]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:16:36.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:36 smithi080 ceph-mon[36206]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:36 smithi049 ceph-mon[31696]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T06:16:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:36 smithi049 ceph-mon[31696]: from='osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:16:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:36 smithi049 ceph-mon[31696]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T06:16:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:36 smithi049 ceph-mon[31696]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:16:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:36 smithi049 ceph-mon[31696]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:16:36.840 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:16:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:16:36.745+0000 7fcabb2b8700 -1 osd.5 0 waiting for initial osdmap 2024-02-26T06:16:36.841 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:16:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:16:36.754+0000 7fcab30a3700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:16:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:37 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:16:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:37 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:16:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:37 smithi049 ceph-mon[31696]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:16:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:37 smithi049 ceph-mon[31696]: osdmap e38: 6 total, 5 up, 6 in 2024-02-26T06:16:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:37 smithi049 ceph-mon[31696]: from='osd.5 ' entity='osd.5' 2024-02-26T06:16:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:37 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:16:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:37 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:16:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:37 smithi080 ceph-mon[36206]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:16:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:37 smithi080 ceph-mon[36206]: osdmap e38: 6 total, 5 up, 6 in 2024-02-26T06:16:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:37 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:37.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:37 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:37.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:37 smithi080 ceph-mon[36206]: from='osd.5 ' entity='osd.5' 2024-02-26T06:16:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:37 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:16:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:37 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:16:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:37 smithi115 ceph-mon[36426]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:16:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:37 smithi115 ceph-mon[36426]: osdmap e38: 6 total, 5 up, 6 in 2024-02-26T06:16:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:37.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:37 smithi115 ceph-mon[36426]: from='osd.5 ' entity='osd.5' 2024-02-26T06:16:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] boot 2024-02-26T06:16:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T06:16:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:38 smithi080 ceph-mon[36206]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T06:16:38.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] boot 2024-02-26T06:16:38.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T06:16:38.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:38 smithi115 ceph-mon[36426]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T06:16:38.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: osd.5 [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] boot 2024-02-26T06:16:38.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T06:16:38.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:16:38.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:38 smithi049 ceph-mon[31696]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T06:16:38.712 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T06:16:38.712 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T06:16:38.712 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-02-26T06:16:38.712 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-02-26T06:16:38.712 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0322997 s, 325 MB/s 2024-02-26T06:16:38.712 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-02-26T06:16:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:39 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi115 2024-02-26T06:16:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:39 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi115 to 8540M 2024-02-26T06:16:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:39 smithi080 ceph-mon[36206]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T06:16:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:39 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi115 2024-02-26T06:16:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:39 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi115 to 8540M 2024-02-26T06:16:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:39 smithi115 ceph-mon[36426]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T06:16:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:39 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi115 2024-02-26T06:16:39.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:39 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi115 to 8540M 2024-02-26T06:16:39.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:39 smithi049 ceph-mon[31696]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T06:16:39.746 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2024-02-26T06:16:40.200 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:16:40.870 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:40 smithi049 ceph-mon[31696]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:40.870 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:40 smithi049 ceph-mon[31696]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T06:16:41.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:40 smithi080 ceph-mon[36206]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:41.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:40 smithi080 ceph-mon[36206]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T06:16:41.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:40 smithi115 ceph-mon[36426]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:41.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:40 smithi115 ceph-mon[36426]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T06:16:42.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:41 smithi049 ceph-mon[31696]: osdmap e43: 6 total, 6 up, 6 in 2024-02-26T06:16:42.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:41 smithi080 ceph-mon[36206]: osdmap e43: 6 total, 6 up, 6 in 2024-02-26T06:16:42.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:41 smithi115 ceph-mon[36426]: osdmap e43: 6 total, 6 up, 6 in 2024-02-26T06:16:42.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.543+0000 7f323c400700 1 -- 172.21.15.49:0/1688168602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3234105850 msgr2=0x7f3234105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:42.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.543+0000 7f323c400700 1 --2- 172.21.15.49:0/1688168602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3234105850 0x7f3234105cd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f322c00b0a0 tx=0x7f322c00b3b0 comp rx=0 tx=0).stop 2024-02-26T06:16:42.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.543+0000 7f323c400700 1 -- 172.21.15.49:0/1688168602 shutdown_connections 2024-02-26T06:16:42.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.543+0000 7f323c400700 1 --2- 172.21.15.49:0/1688168602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3234105850 0x7f3234105cd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:42.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.543+0000 7f323c400700 1 --2- 172.21.15.49:0/1688168602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234100510 0x7f3234102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:42.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.543+0000 7f323c400700 1 --2- 172.21.15.49:0/1688168602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3234102e80 0x7f3234105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:42.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.543+0000 7f323c400700 1 -- 172.21.15.49:0/1688168602 >> 172.21.15.49:0/1688168602 conn(0x7f32340fc130 msgr2=0x7f32340fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:42.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.543+0000 7f323c400700 1 -- 172.21.15.49:0/1688168602 shutdown_connections 2024-02-26T06:16:42.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.544+0000 7f323c400700 1 -- 172.21.15.49:0/1688168602 wait complete. 2024-02-26T06:16:42.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.545+0000 7f323c400700 1 Processor -- start 2024-02-26T06:16:42.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.545+0000 7f323c400700 1 -- start start 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f323c400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3234100510 0x7f323419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f323c400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3234102e80 0x7f323419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f323c400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234105850 0x7f32341a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f323c400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32341a5eb0 con 0x7f3234105850 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f323c400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32341a5ff0 con 0x7f3234102e80 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f323c400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32341a6180 con 0x7f3234100510 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f3239a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3234100510 0x7f323419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f3239a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3234100510 0x7f323419e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39974/0 (socket says 172.21.15.49:39974) 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f3239a00700 1 -- 172.21.15.49:0/1544339782 learned_addr learned my addr 172.21.15.49:0/1544339782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:16:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f323a400700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234105850 0x7f32341a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.546+0000 7f3239000700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3234102e80 0x7f323419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3239a00700 1 -- 172.21.15.49:0/1544339782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3234102e80 msgr2=0x7f323419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3239a00700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3234102e80 0x7f323419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3239a00700 1 -- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234105850 msgr2=0x7f32341a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3239a00700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234105850 0x7f32341a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3239a00700 1 -- 172.21.15.49:0/1544339782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220009520 con 0x7f3234100510 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3239a00700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3234100510 0x7f323419e320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f322800e9d0 tx=0x7f322800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3232000700 1 -- 172.21.15.49:0/1544339782 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228009790 con 0x7f3234100510 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3232000700 1 -- 172.21.15.49:0/1544339782 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f322800ee70 con 0x7f3234100510 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c009d00 con 0x7f3234100510 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f3232000700 1 -- 172.21.15.49:0/1544339782 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228018680 con 0x7f3234100510 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.547+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32341a67d0 con 0x7f3234100510 2024-02-26T06:16:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.548+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32340fdcb0 con 0x7f3234100510 2024-02-26T06:16:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.550+0000 7f3232000700 1 -- 172.21.15.49:0/1544339782 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f322800cbd0 con 0x7f3234100510 2024-02-26T06:16:42.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.552+0000 7f3232000700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f321c0a2150 0x7f321c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:42.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.552+0000 7f3232000700 1 -- 172.21.15.49:0/1544339782 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f3228014070 con 0x7f3234100510 2024-02-26T06:16:42.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.552+0000 7f3239000700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f321c0a2150 0x7f321c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:42.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.553+0000 7f3239000700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f321c0a2150 0x7f321c0a4610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3220009df0 tx=0x7f3220009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:42.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.556+0000 7f3232000700 1 -- 172.21.15.49:0/1544339782 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3228052d20 con 0x7f3234100510 2024-02-26T06:16:42.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:16:42.795+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f323402d8d0 con 0x7f321c0a2150 2024-02-26T06:16:43.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:42 smithi080 ceph-mon[36206]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:16:43.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:42 smithi115 ceph-mon[36426]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:16:43.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:42 smithi049 ceph-mon[31696]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:16:44.042 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:43 smithi049 ceph-mon[31696]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:44.042 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:16:44.042 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:16:44.042 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:43 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:44.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:43 smithi080 ceph-mon[36206]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:44.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:43 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:16:44.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:43 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:16:44.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:43 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:44.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:43 smithi115 ceph-mon[36426]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:44.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:43 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:16:44.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:43 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:16:44.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:43 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:44.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:44 smithi049 ceph-mon[31696]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-02-26T06:16:45.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:44 smithi080 ceph-mon[36206]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-02-26T06:16:45.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:44 smithi115 ceph-mon[36426]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-02-26T06:16:47.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:46 smithi080 ceph-mon[36206]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-02-26T06:16:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:46 smithi115 ceph-mon[36426]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-02-26T06:16:47.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:46 smithi049 ceph-mon[31696]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-02-26T06:16:47.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:47 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3884580747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]: dispatch 2024-02-26T06:16:47.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:47 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]: dispatch 2024-02-26T06:16:47.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:47 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]': finished 2024-02-26T06:16:47.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:47 smithi049 ceph-mon[31696]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T06:16:47.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:47 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:16:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:47 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3884580747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]: dispatch 2024-02-26T06:16:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:47 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]: dispatch 2024-02-26T06:16:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:47 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]': finished 2024-02-26T06:16:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:47 smithi080 ceph-mon[36206]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T06:16:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:47 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:16:48.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:47 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3884580747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]: dispatch 2024-02-26T06:16:48.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:47 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]: dispatch 2024-02-26T06:16:48.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:47 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "570e6f42-9220-477e-b01f-0a4169640324"}]': finished 2024-02-26T06:16:48.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:47 smithi115 ceph-mon[36426]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T06:16:48.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:47 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:16:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:48 smithi080 ceph-mon[36206]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:16:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:48 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1128612490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:16:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:48 smithi115 ceph-mon[36426]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:16:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:48 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1128612490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:16:49.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:48 smithi049 ceph-mon[31696]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:16:49.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:48 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1128612490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:16:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:50 smithi080 ceph-mon[36206]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:16:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:50 smithi115 ceph-mon[36426]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:16:51.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:50 smithi049 ceph-mon[31696]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:16:53.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:52 smithi049 ceph-mon[31696]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:52 smithi080 ceph-mon[36206]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:53.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:52 smithi115 ceph-mon[36426]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:54.850 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:54 smithi049 ceph-mon[31696]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:54 smithi080 ceph-mon[36206]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:54 smithi115 ceph-mon[36426]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:56 smithi080 ceph-mon[36206]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T06:16:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:56 smithi115 ceph-mon[36426]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T06:16:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:56 smithi049 ceph-mon[31696]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T06:16:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:58.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:57 smithi080 ceph-mon[36206]: Deploying daemon osd.6 on smithi049 2024-02-26T06:16:58.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:57 smithi115 ceph-mon[36426]: Deploying daemon osd.6 on smithi049 2024-02-26T06:16:58.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:57 smithi049 ceph-mon[31696]: Deploying daemon osd.6 on smithi049 2024-02-26T06:16:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:16:58 smithi080 ceph-mon[36206]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:16:58 smithi115 ceph-mon[36426]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:16:59.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:16:58 smithi049 ceph-mon[31696]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:00 smithi115 ceph-mon[36426]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:00 smithi049 ceph-mon[31696]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:00 smithi080 ceph-mon[36206]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:03.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:02 smithi115 ceph-mon[36426]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:03.850 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:02 smithi080 ceph-mon[36206]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:03.850 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:02 smithi049 ceph-mon[31696]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:05.031 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:04 smithi049 ceph-mon[31696]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:04 smithi080 ceph-mon[36206]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:04 smithi115 ceph-mon[36426]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:06.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:06 smithi049 ceph-mon[31696]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:06.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:06.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:06.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:06 smithi080 ceph-mon[36206]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:07.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:06 smithi115 ceph-mon[36426]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:07.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:07.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:07.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:08.866 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:08 smithi049 ceph-mon[31696]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:09.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:08 smithi080 ceph-mon[36206]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:09.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:08 smithi115 ceph-mon[36426]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:10.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:10.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:10.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:10 smithi049 ceph-mon[31696]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:10.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:10 smithi049 ceph-mon[31696]: from='osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:17:10.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:10 smithi049 ceph-mon[31696]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:17:10.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:10.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:10.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:10 smithi080 ceph-mon[36206]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:10.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:10 smithi080 ceph-mon[36206]: from='osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:17:10.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:10 smithi080 ceph-mon[36206]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:17:10.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:10.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:10.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:10 smithi115 ceph-mon[36426]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:10.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:10 smithi115 ceph-mon[36426]: from='osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:17:10.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:10 smithi115 ceph-mon[36426]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:17:10.807 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 6 on host 'smithi049' 2024-02-26T06:17:10.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.806+0000 7f3232000700 1 -- 172.21.15.49:0/1544339782 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f323402d8d0 con 0x7f321c0a2150 2024-02-26T06:17:10.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.810+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f321c0a2150 msgr2=0x7f321c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:10.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.810+0000 7f323c400700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f321c0a2150 0x7f321c0a4610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3220009df0 tx=0x7f3220009260 comp rx=0 tx=0).stop 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3234100510 msgr2=0x7f323419e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3234100510 0x7f323419e320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f322800e9d0 tx=0x7f322800ece0 comp rx=0 tx=0).stop 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 shutdown_connections 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f321c0a2150 0x7f321c0a4610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234105850 0x7f32341a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3234100510 0x7f323419e320 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 --2- 172.21.15.49:0/1544339782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3234102e80 0x7f323419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 >> 172.21.15.49:0/1544339782 conn(0x7f32340fc130 msgr2=0x7f32341011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:10.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 shutdown_connections 2024-02-26T06:17:10.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:17:10.811+0000 7f323c400700 1 -- 172.21.15.49:0/1544339782 wait complete. 2024-02-26T06:17:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T06:17:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:17:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T06:17:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:17:11.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:11.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:11.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:11 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T06:17:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:17:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T06:17:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:17:11.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:11.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:11.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:11 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:11.592 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T06:17:11.592 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:11.593 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:11 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:11.651 DEBUG:teuthology.orchestra.run.smithi049:osd.6> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.6.service 2024-02-26T06:17:11.656 INFO:tasks.cephadm:Deploying osd.7 on smithi080 with /dev/vg_nvme/lv_2... 2024-02-26T06:17:11.656 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T06:17:11.910 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:17:12.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:17:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:17:12.025+0000 7f911c800700 -1 osd.6 0 waiting for initial osdmap 2024-02-26T06:17:12.171 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:17:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:17:12.053+0000 7f9111c00700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:17:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:12 smithi080 ceph-mon[36206]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:17:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:12 smithi080 ceph-mon[36206]: osdmap e46: 7 total, 6 up, 7 in 2024-02-26T06:17:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:12 smithi080 ceph-mon[36206]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:12 smithi080 ceph-mon[36206]: from='osd.6 ' entity='osd.6' 2024-02-26T06:17:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:12 smithi115 ceph-mon[36426]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:17:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:12 smithi115 ceph-mon[36426]: osdmap e46: 7 total, 6 up, 7 in 2024-02-26T06:17:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:12 smithi115 ceph-mon[36426]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:12 smithi115 ceph-mon[36426]: from='osd.6 ' entity='osd.6' 2024-02-26T06:17:12.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:12 smithi049 ceph-mon[31696]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:17:12.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:12 smithi049 ceph-mon[31696]: osdmap e46: 7 total, 6 up, 7 in 2024-02-26T06:17:12.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:12.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:12.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:12 smithi049 ceph-mon[31696]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:17:12.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:12 smithi049 ceph-mon[31696]: from='osd.6 ' entity='osd.6' 2024-02-26T06:17:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:13 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:17:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:13 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:17:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:13 smithi049 ceph-mon[31696]: osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] boot 2024-02-26T06:17:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:13 smithi049 ceph-mon[31696]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T06:17:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:13 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:17:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:13 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:17:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:13 smithi080 ceph-mon[36206]: osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] boot 2024-02-26T06:17:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:13 smithi080 ceph-mon[36206]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T06:17:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:13 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:17:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:13 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:17:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:13 smithi115 ceph-mon[36426]: osd.6 [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] boot 2024-02-26T06:17:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:13 smithi115 ceph-mon[36426]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T06:17:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:17:13.941 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T06:17:13.941 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T06:17:13.941 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-02-26T06:17:13.941 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-02-26T06:17:13.941 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112089 s, 935 MB/s 2024-02-26T06:17:13.941 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-02-26T06:17:14.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:14 smithi080 ceph-mon[36206]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:14.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:14 smithi115 ceph-mon[36426]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:14.650 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi080:vg_nvme/lv_2 2024-02-26T06:17:14.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:14 smithi049 ceph-mon[31696]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:14.898 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:17:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:15 smithi080 ceph-mon[36206]: osdmap e48: 7 total, 7 up, 7 in 2024-02-26T06:17:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:15 smithi115 ceph-mon[36426]: osdmap e48: 7 total, 7 up, 7 in 2024-02-26T06:17:15.606 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:15 smithi049 ceph-mon[31696]: osdmap e48: 7 total, 7 up, 7 in 2024-02-26T06:17:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 -- 172.21.15.80:0/457204264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 msgr2=0x7fab6c108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 --2- 172.21.15.80:0/457204264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 0x7fab6c108fe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab6400b0d0 tx=0x7fab6400b3e0 comp rx=0 tx=0).stop 2024-02-26T06:17:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 -- 172.21.15.80:0/457204264 shutdown_connections 2024-02-26T06:17:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 --2- 172.21.15.80:0/457204264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 0x7fab6c108fe0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 --2- 172.21.15.80:0/457204264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab6c100b90 0x7fab6c100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 --2- 172.21.15.80:0/457204264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab6c101d90 0x7fab6c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 -- 172.21.15.80:0/457204264 >> 172.21.15.80:0/457204264 conn(0x7fab6c0fc170 msgr2=0x7fab6c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 -- 172.21.15.80:0/457204264 shutdown_connections 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.549+0000 7fab7116a700 1 -- 172.21.15.80:0/457204264 wait complete. 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.550+0000 7fab7116a700 1 Processor -- start 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.550+0000 7fab7116a700 1 -- start start 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.550+0000 7fab7116a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab6c100b90 0x7fab6c06af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.550+0000 7fab7116a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab6c101d90 0x7fab6c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.550+0000 7fab7116a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 0x7fab6c1a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.550+0000 7fab7116a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1a1ae0 con 0x7fab6c101d90 2024-02-26T06:17:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.550+0000 7fab7116a700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1a1c50 con 0x7fab6c102730 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.550+0000 7fab7116a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1a1e10 con 0x7fab6c100b90 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 0x7fab6c1a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 0x7fab6c1a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39428/0 (socket says 172.21.15.80:39428) 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 -- 172.21.15.80:0/3024118972 learned_addr learned my addr 172.21.15.80:0/3024118972 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6ad9d700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab6c100b90 0x7fab6c06af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 -- 172.21.15.80:0/3024118972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab6c100b90 msgr2=0x7fab6c06af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab6c100b90 0x7fab6c06af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 -- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab6c101d90 msgr2=0x7fab6c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab6c101d90 0x7fab6c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 -- 172.21.15.80:0/3024118972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab64009d00 con 0x7fab6c102730 2024-02-26T06:17:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.551+0000 7fab6b59e700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 0x7fab6c1a1430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab64007c90 tx=0x7fab64009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.552+0000 7fab53fff700 1 -- 172.21.15.80:0/3024118972 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400e040 con 0x7fab6c102730 2024-02-26T06:17:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.552+0000 7fab53fff700 1 -- 172.21.15.80:0/3024118972 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab6401dd70 con 0x7fab6c102730 2024-02-26T06:17:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.552+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c1a2090 con 0x7fab6c102730 2024-02-26T06:17:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.552+0000 7fab53fff700 1 -- 172.21.15.80:0/3024118972 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6401be60 con 0x7fab6c102730 2024-02-26T06:17:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.552+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c1a25d0 con 0x7fab6c102730 2024-02-26T06:17:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.553+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c107f50 con 0x7fab6c102730 2024-02-26T06:17:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.554+0000 7fab53fff700 1 -- 172.21.15.80:0/3024118972 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fab6401b4e0 con 0x7fab6c102730 2024-02-26T06:17:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.554+0000 7fab53fff700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fab540a21d0 0x7fab540a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.555+0000 7fab53fff700 1 -- 172.21.15.80:0/3024118972 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fab640cec10 con 0x7fab6c102730 2024-02-26T06:17:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.555+0000 7fab6ad9d700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fab540a21d0 0x7fab540a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.556+0000 7fab6ad9d700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fab540a21d0 0x7fab540a4690 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab6c101bf0 tx=0x7fab5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:16.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.559+0000 7fab53fff700 1 -- 172.21.15.80:0/3024118972 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab64098980 con 0x7fab6c102730 2024-02-26T06:17:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:16 smithi080 ceph-mon[36206]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:16.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:16 smithi115 ceph-mon[36426]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:16.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:16 smithi049 ceph-mon[31696]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:16.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:16.760+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fab6c02d870 con 0x7fab540a21d0 2024-02-26T06:17:17.666 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi049 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi049 to 5697M 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:17:17.667 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:17:17.668 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:17 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:17.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi049 2024-02-26T06:17:17.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi049 to 5697M 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:17:17.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:17 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:17.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi049 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi049 to 5697M 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:17:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:17 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:18.738 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:18 smithi080 ceph-mon[36206]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:18.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:18 smithi049 ceph-mon[31696]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:19.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:18 smithi115 ceph-mon[36426]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:19 smithi049 ceph-mon[31696]: from='client.? 172.21.15.80:0/2504924420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]: dispatch 2024-02-26T06:17:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:19 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]: dispatch 2024-02-26T06:17:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:19 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]': finished 2024-02-26T06:17:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:19 smithi049 ceph-mon[31696]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T06:17:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:19.893 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:19 smithi080 ceph-mon[36206]: from='client.? 172.21.15.80:0/2504924420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]: dispatch 2024-02-26T06:17:19.893 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:19 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]: dispatch 2024-02-26T06:17:19.893 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:19 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]': finished 2024-02-26T06:17:19.893 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:19 smithi080 ceph-mon[36206]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T06:17:19.893 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:20.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:19 smithi115 ceph-mon[36426]: from='client.? 172.21.15.80:0/2504924420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]: dispatch 2024-02-26T06:17:20.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:19 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]: dispatch 2024-02-26T06:17:20.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:19 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f"}]': finished 2024-02-26T06:17:20.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:19 smithi115 ceph-mon[36426]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T06:17:20.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:20.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:20 smithi049 ceph-mon[31696]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:20.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:20 smithi049 ceph-mon[31696]: from='client.? 172.21.15.80:0/445440408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:17:21.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:20 smithi080 ceph-mon[36206]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:21.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:20 smithi080 ceph-mon[36206]: from='client.? 172.21.15.80:0/445440408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:17:21.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:20 smithi115 ceph-mon[36426]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:21.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:20 smithi115 ceph-mon[36426]: from='client.? 172.21.15.80:0/445440408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:17:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:22 smithi049 ceph-mon[31696]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:22 smithi080 ceph-mon[36206]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:22 smithi115 ceph-mon[36426]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:24.733 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:24 smithi080 ceph-mon[36206]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:24.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:24 smithi049 ceph-mon[31696]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:25.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:24 smithi115 ceph-mon[36426]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:26.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:26 smithi080 ceph-mon[36206]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:26.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:26 smithi049 ceph-mon[31696]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:27.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:26 smithi115 ceph-mon[36426]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:27.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T06:17:27.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:27.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:27 smithi049 ceph-mon[31696]: Deploying daemon osd.7 on smithi080 2024-02-26T06:17:28.051 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T06:17:28.051 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:28.052 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:27 smithi080 ceph-mon[36206]: Deploying daemon osd.7 on smithi080 2024-02-26T06:17:28.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T06:17:28.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:28.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:27 smithi115 ceph-mon[36426]: Deploying daemon osd.7 on smithi080 2024-02-26T06:17:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:28 smithi049 ceph-mon[31696]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:29.003 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:28 smithi080 ceph-mon[36206]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:28 smithi115 ceph-mon[36426]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:30.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:30 smithi049 ceph-mon[31696]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:31.049 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:30 smithi080 ceph-mon[36206]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:31.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:30 smithi115 ceph-mon[36426]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:32.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:32 smithi049 ceph-mon[31696]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:32.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:32.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:32.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:32.966 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:32 smithi080 ceph-mon[36206]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:32.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:32.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:32.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:33.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:32 smithi115 ceph-mon[36426]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:33.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:33.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:33.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:34.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:34 smithi049 ceph-mon[31696]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:35.025 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:34 smithi080 ceph-mon[36206]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:35.102 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:34 smithi115 ceph-mon[36426]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:36.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.206+0000 7fab53fff700 1 -- 172.21.15.80:0/3024118972 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fab6c02d870 con 0x7fab540a21d0 2024-02-26T06:17:36.206 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 7 on host 'smithi080' 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fab540a21d0 msgr2=0x7fab540a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fab540a21d0 0x7fab540a4690 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab6c101bf0 tx=0x7fab5c00b410 comp rx=0 tx=0).stop 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 msgr2=0x7fab6c1a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 0x7fab6c1a1430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab64007c90 tx=0x7fab64009380 comp rx=0 tx=0).stop 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 shutdown_connections 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fab540a21d0 0x7fab540a4690 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab6c102730 0x7fab6c1a1430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab6c100b90 0x7fab6c06af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 --2- 172.21.15.80:0/3024118972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab6c101d90 0x7fab6c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 >> 172.21.15.80:0/3024118972 conn(0x7fab6c0fc170 msgr2=0x7fab6c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:36.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 shutdown_connections 2024-02-26T06:17:36.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:17:36.208+0000 7fab7116a700 1 -- 172.21.15.80:0/3024118972 wait complete. 2024-02-26T06:17:36.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:36.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:36.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.528 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:36.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:17:36.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:17:36.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.529 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:36.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:36.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:36.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:36.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:17:36.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:17:36.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:36.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:36.861 DEBUG:teuthology.orchestra.run.smithi080:osd.7> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.7.service 2024-02-26T06:17:36.863 INFO:tasks.cephadm:Deploying osd.8 on smithi115 with /dev/vg_nvme/lv_2... 2024-02-26T06:17:36.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T06:17:37.106 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:17:37.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:37 smithi115 ceph-mon[36426]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T06:17:37.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:37 smithi115 ceph-mon[36426]: from='osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:17:37.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:37 smithi115 ceph-mon[36426]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T06:17:37.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:37.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:37 smithi115 ceph-mon[36426]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:17:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:37 smithi080 ceph-mon[36206]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T06:17:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:37 smithi080 ceph-mon[36206]: from='osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:17:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:37 smithi080 ceph-mon[36206]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T06:17:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:37 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:37 smithi080 ceph-mon[36206]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:17:37.581 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:17:37 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:17:37.354+0000 7f75c2c03700 -1 osd.7 0 waiting for initial osdmap 2024-02-26T06:17:37.581 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:17:37 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:17:37.363+0000 7f75be9f6700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:17:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:37 smithi049 ceph-mon[31696]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T06:17:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:37 smithi049 ceph-mon[31696]: from='osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:17:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:37 smithi049 ceph-mon[31696]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T06:17:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:37 smithi049 ceph-mon[31696]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:17:38.565 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:38 smithi115 ceph-mon[36426]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:17:38.566 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:38 smithi115 ceph-mon[36426]: osdmap e51: 8 total, 7 up, 8 in 2024-02-26T06:17:38.566 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:38.566 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:38.566 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:38 smithi115 ceph-mon[36426]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:38.566 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:38 smithi115 ceph-mon[36426]: osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] boot 2024-02-26T06:17:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:38 smithi080 ceph-mon[36206]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:17:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:38 smithi080 ceph-mon[36206]: osdmap e51: 8 total, 7 up, 8 in 2024-02-26T06:17:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:38 smithi080 ceph-mon[36206]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:38 smithi080 ceph-mon[36206]: osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] boot 2024-02-26T06:17:38.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:38 smithi049 ceph-mon[31696]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:17:38.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:38 smithi049 ceph-mon[31696]: osdmap e51: 8 total, 7 up, 8 in 2024-02-26T06:17:38.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:38.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:38.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:38 smithi049 ceph-mon[31696]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:17:38.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:38 smithi049 ceph-mon[31696]: osd.7 [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] boot 2024-02-26T06:17:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T06:17:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T06:17:39.147 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-02-26T06:17:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-02-26T06:17:39.147 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121812 s, 861 MB/s 2024-02-26T06:17:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-02-26T06:17:39.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:39 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:17:39.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:39 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:17:39.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:39 smithi115 ceph-mon[36426]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T06:17:39.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:39 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:39.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:39 smithi115 ceph-mon[36426]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T06:17:39.461 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:39 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:17:39.461 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:39 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:17:39.461 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:39 smithi080 ceph-mon[36206]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T06:17:39.461 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:39.461 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:39 smithi080 ceph-mon[36206]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T06:17:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:39 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:17:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:39 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:17:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:39 smithi049 ceph-mon[31696]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T06:17:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:39 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:17:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:39 smithi049 ceph-mon[31696]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T06:17:39.889 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-02-26T06:17:40.136 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi080 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi080 to 5693M 2024-02-26T06:17:41.135 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.136 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:41.136 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:41.136 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.136 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:40 smithi115 ceph-mon[36426]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi080 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi080 to 5693M 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:41.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:40 smithi049 ceph-mon[31696]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T06:17:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi080 2024-02-26T06:17:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi080 to 5693M 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:41.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:40 smithi080 ceph-mon[36206]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.647+0000 7f56c6e11700 1 -- 172.21.15.115:0/3555986319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0100b90 msgr2=0x7f56c0100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.647+0000 7f56c6e11700 1 --2- 172.21.15.115:0/3555986319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0100b90 0x7f56c0100fb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009990 tx=0x7f56b4009ca0 comp rx=0 tx=0).stop 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 -- 172.21.15.115:0/3555986319 shutdown_connections 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 --2- 172.21.15.115:0/3555986319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0102730 0x7f56c0108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 --2- 172.21.15.115:0/3555986319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0100b90 0x7f56c0100fb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 --2- 172.21.15.115:0/3555986319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56c0101d90 0x7f56c01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 -- 172.21.15.115:0/3555986319 >> 172.21.15.115:0/3555986319 conn(0x7f56c00fc130 msgr2=0x7f56c00fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 -- 172.21.15.115:0/3555986319 shutdown_connections 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 -- 172.21.15.115:0/3555986319 wait complete. 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 Processor -- start 2024-02-26T06:17:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 -- start start 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0100b90 0x7f56c0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56c0101d90 0x7f56c0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0102730 0x7f56c019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019f950 con 0x7f56c0101d90 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019fa90 con 0x7f56c0100b90 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c6e11700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019fc20 con 0x7f56c0102730 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c53ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0102730 0x7f56c019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.648+0000 7f56c53ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0102730 0x7f56c019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47760/0 (socket says 172.21.15.115:47760) 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c53ae700 1 -- 172.21.15.115:0/1827072646 learned_addr learned my addr 172.21.15.115:0/1827072646 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c4bad700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0100b90 0x7f56c0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c53ae700 1 -- 172.21.15.115:0/1827072646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0100b90 msgr2=0x7f56c0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c53ae700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0100b90 0x7f56c0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:41.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c53ae700 1 -- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56c0101d90 msgr2=0x7f56c0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c53ae700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56c0101d90 0x7f56c0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c53ae700 1 -- 172.21.15.115:0/1827072646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40095f0 con 0x7f56c0102730 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c4bad700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0100b90 0x7f56c0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c53ae700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0102730 0x7f56c019f380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b9a0 tx=0x7f56b800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56bdffb700 1 -- 172.21.15.115:0/1827072646 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800c9b0 con 0x7f56c0102730 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56bdffb700 1 -- 172.21.15.115:0/1827072646 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56b8014440 con 0x7f56c0102730 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c019ff00 con 0x7f56c0102730 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56bdffb700 1 -- 172.21.15.115:0/1827072646 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80075d0 con 0x7f56c0102730 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.649+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c01a03f0 con 0x7f56c0102730 2024-02-26T06:17:41.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.650+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c0068960 con 0x7f56c0102730 2024-02-26T06:17:41.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.651+0000 7f56bdffb700 1 -- 172.21.15.115:0/1827072646 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f56b8014920 con 0x7f56c0102730 2024-02-26T06:17:41.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.652+0000 7f56bdffb700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56b00a2150 0x7f56b00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:41.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.652+0000 7f56bdffb700 1 -- 172.21.15.115:0/1827072646 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f56b8006e40 con 0x7f56c0102730 2024-02-26T06:17:41.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.652+0000 7f56c4bad700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56b00a2150 0x7f56b00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:41.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.653+0000 7f56c4bad700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56b00a2150 0x7f56b00a4610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56b4003930 tx=0x7f56b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:41.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.656+0000 7f56bdffb700 1 -- 172.21.15.115:0/1827072646 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56b80963d0 con 0x7f56c0102730 2024-02-26T06:17:41.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:17:41.857+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f56c00fe380 con 0x7f56b00a2150 2024-02-26T06:17:42.625 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:42 smithi115 ceph-mon[36426]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:42.625 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:42 smithi115 ceph-mon[36426]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T06:17:42.625 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:42 smithi115 ceph-mon[36426]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:42.626 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:42 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:17:42.626 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:42 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:17:42.626 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:42 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:42.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:42 smithi049 ceph-mon[31696]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:42.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:42 smithi049 ceph-mon[31696]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T06:17:42.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:42 smithi049 ceph-mon[31696]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:42.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:42 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:17:42.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:42 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:17:42.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:42 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:42 smithi080 ceph-mon[36206]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:42 smithi080 ceph-mon[36206]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T06:17:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:42 smithi080 ceph-mon[36206]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:42 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:17:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:42 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:17:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:42 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:43.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:43 smithi115 ceph-mon[36426]: osdmap e56: 8 total, 8 up, 8 in 2024-02-26T06:17:43.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:43 smithi049 ceph-mon[31696]: osdmap e56: 8 total, 8 up, 8 in 2024-02-26T06:17:43.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:43 smithi080 ceph-mon[36206]: osdmap e56: 8 total, 8 up, 8 in 2024-02-26T06:17:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:44 smithi049 ceph-mon[31696]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:44 smithi080 ceph-mon[36206]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:44.830 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:44 smithi115 ceph-mon[36426]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:45 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/1454481646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]: dispatch 2024-02-26T06:17:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:45 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]: dispatch 2024-02-26T06:17:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:45 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]': finished 2024-02-26T06:17:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:45 smithi049 ceph-mon[31696]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T06:17:45.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:17:45.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:45 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/3850722271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:17:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:45 smithi080 ceph-mon[36206]: from='client.? 172.21.15.115:0/1454481646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]: dispatch 2024-02-26T06:17:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:45 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]: dispatch 2024-02-26T06:17:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:45 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]': finished 2024-02-26T06:17:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:45 smithi080 ceph-mon[36206]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T06:17:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:17:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:45 smithi080 ceph-mon[36206]: from='client.? 172.21.15.115:0/3850722271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:17:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:45 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/1454481646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]: dispatch 2024-02-26T06:17:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:45 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]: dispatch 2024-02-26T06:17:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:45 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c86f9fc9-9e22-4686-83ef-2a7b3b0652cc"}]': finished 2024-02-26T06:17:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:45 smithi115 ceph-mon[36426]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T06:17:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:17:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:45 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/3850722271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:17:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:46 smithi080 ceph-mon[36206]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:46 smithi115 ceph-mon[36426]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:46 smithi049 ceph-mon[31696]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:48.820 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:48 smithi115 ceph-mon[36426]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:48.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:48 smithi080 ceph-mon[36206]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:48.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:48 smithi049 ceph-mon[31696]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:50.736 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:50 smithi115 ceph-mon[36426]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:50.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:50 smithi080 ceph-mon[36206]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:50.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:50 smithi049 ceph-mon[31696]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:52.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:52 smithi049 ceph-mon[31696]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:52.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T06:17:52.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:52.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:52 smithi049 ceph-mon[31696]: Deploying daemon osd.8 on smithi115 2024-02-26T06:17:52.768 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:52 smithi115 ceph-mon[36426]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:52.768 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T06:17:52.768 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:52.768 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:52 smithi115 ceph-mon[36426]: Deploying daemon osd.8 on smithi115 2024-02-26T06:17:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:52 smithi080 ceph-mon[36206]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T06:17:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:52 smithi080 ceph-mon[36206]: Deploying daemon osd.8 on smithi115 2024-02-26T06:17:54.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:54 smithi049 ceph-mon[31696]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:54.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:54 smithi080 ceph-mon[36206]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:54 smithi115 ceph-mon[36426]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:56.790 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:56 smithi115 ceph-mon[36426]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:56.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:56 smithi080 ceph-mon[36206]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:56.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:56 smithi049 ceph-mon[31696]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:58 smithi080 ceph-mon[36206]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:17:58 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:58 smithi115 ceph-mon[36426]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:17:58 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:58 smithi049 ceph-mon[31696]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:17:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:17:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:17:58 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:00.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:00 smithi115 ceph-mon[36426]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:00 smithi080 ceph-mon[36206]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:00 smithi049 ceph-mon[31696]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:01.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.694+0000 7f56bdffb700 1 -- 172.21.15.115:0/1827072646 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f56c00fe380 con 0x7f56b00a2150 2024-02-26T06:18:01.694 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 8 on host 'smithi115' 2024-02-26T06:18:01.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56b00a2150 msgr2=0x7f56b00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:01.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56b00a2150 0x7f56b00a4610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56b4003930 tx=0x7f56b4005fb0 comp rx=0 tx=0).stop 2024-02-26T06:18:01.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0102730 msgr2=0x7f56c019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:01.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0102730 0x7f56c019f380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b9a0 tx=0x7f56b800bd60 comp rx=0 tx=0).stop 2024-02-26T06:18:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 shutdown_connections 2024-02-26T06:18:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56b00a2150 0x7f56b00a4610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c0102730 0x7f56c019f380 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0100b90 0x7f56c0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 --2- 172.21.15.115:0/1827072646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56c0101d90 0x7f56c0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 >> 172.21.15.115:0/1827072646 conn(0x7f56c00fc130 msgr2=0x7f56c00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.696+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 shutdown_connections 2024-02-26T06:18:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:18:01.697+0000 7f56c6e11700 1 -- 172.21.15.115:0/1827072646 wait complete. 2024-02-26T06:18:02.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:02.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:02.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:18:02.016 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:18:02.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.017 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:18:02.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:18:02.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:02.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:02.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:02.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:18:02.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:18:02.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:02.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:02.336 DEBUG:teuthology.orchestra.run.smithi115:osd.8> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.8.service 2024-02-26T06:18:02.338 INFO:tasks.cephadm:Deploying osd.9 on smithi049 with /dev/vg_nvme/lv_1... 2024-02-26T06:18:02.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T06:18:02.832 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:18:02.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:02 smithi115 ceph-mon[36426]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:02.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:02 smithi115 ceph-mon[36426]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T06:18:02.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:02 smithi115 ceph-mon[36426]: from='osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:18:02.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:02 smithi115 ceph-mon[36426]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T06:18:02.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:02.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:02 smithi115 ceph-mon[36426]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:18:02.916 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:18:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:18:02.796+0000 7fd4b637d700 -1 osd.8 0 waiting for initial osdmap 2024-02-26T06:18:02.916 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:18:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:18:02.806+0000 7fd4b2170700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:18:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:02 smithi080 ceph-mon[36206]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:02 smithi080 ceph-mon[36206]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T06:18:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:02 smithi080 ceph-mon[36206]: from='osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:18:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:02 smithi080 ceph-mon[36206]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T06:18:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:02 smithi080 ceph-mon[36206]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:18:03.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:02 smithi049 ceph-mon[31696]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:03.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:02 smithi049 ceph-mon[31696]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T06:18:03.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:02 smithi049 ceph-mon[31696]: from='osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:18:03.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:02 smithi049 ceph-mon[31696]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T06:18:03.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:03.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:02 smithi049 ceph-mon[31696]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:18:04.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:03 smithi080 ceph-mon[36206]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:18:04.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:03 smithi080 ceph-mon[36206]: osdmap e59: 9 total, 8 up, 9 in 2024-02-26T06:18:04.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:03 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:04.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:03 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:04.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:03 smithi115 ceph-mon[36426]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:18:04.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:03 smithi115 ceph-mon[36426]: osdmap e59: 9 total, 8 up, 9 in 2024-02-26T06:18:04.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:03 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:04.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:03 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:03 smithi049 ceph-mon[31696]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:18:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:03 smithi049 ceph-mon[31696]: osdmap e59: 9 total, 8 up, 9 in 2024-02-26T06:18:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:05.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:04 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:18:05.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:04 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:18:05.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:04 smithi049 ceph-mon[31696]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:05.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:04 smithi049 ceph-mon[31696]: osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] boot 2024-02-26T06:18:05.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:04 smithi049 ceph-mon[31696]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T06:18:05.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:05.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:04 smithi049 ceph-mon[31696]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T06:18:05.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:04 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:18:05.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:04 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:18:05.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:04 smithi115 ceph-mon[36426]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:05.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:04 smithi115 ceph-mon[36426]: osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] boot 2024-02-26T06:18:05.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:04 smithi115 ceph-mon[36426]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T06:18:05.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:05.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:04 smithi115 ceph-mon[36426]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T06:18:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:04 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:18:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:04 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:18:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:04 smithi080 ceph-mon[36206]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:18:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:04 smithi080 ceph-mon[36206]: osd.8 [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] boot 2024-02-26T06:18:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:04 smithi080 ceph-mon[36206]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T06:18:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:18:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:04 smithi080 ceph-mon[36206]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T06:18:06.088 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T06:18:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T06:18:06.089 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-02-26T06:18:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-02-26T06:18:06.089 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.038798 s, 270 MB/s 2024-02-26T06:18:06.089 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-02-26T06:18:06.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:06.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi115 2024-02-26T06:18:06.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi115 to 5693M 2024-02-26T06:18:06.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:06.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:06.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:06 smithi049 ceph-mon[31696]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi115 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi115 to 5693M 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:06.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:06.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:06 smithi080 ceph-mon[36206]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T06:18:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi115 2024-02-26T06:18:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:06.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi115 to 5693M 2024-02-26T06:18:06.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:06.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:06.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:06.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:06 smithi115 ceph-mon[36426]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T06:18:06.962 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_1 2024-02-26T06:18:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:18:07.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:07 smithi049 ceph-mon[31696]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T06:18:08.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:07 smithi080 ceph-mon[36206]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T06:18:08.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:07 smithi115 ceph-mon[36426]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T06:18:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:08 smithi049 ceph-mon[31696]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:08 smithi049 ceph-mon[31696]: osdmap e64: 9 total, 9 up, 9 in 2024-02-26T06:18:09.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:08 smithi080 ceph-mon[36206]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:09.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:08 smithi080 ceph-mon[36206]: osdmap e64: 9 total, 9 up, 9 in 2024-02-26T06:18:09.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:08 smithi115 ceph-mon[36426]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:09.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:08 smithi115 ceph-mon[36426]: osdmap e64: 9 total, 9 up, 9 in 2024-02-26T06:18:09.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 -- 172.21.15.49:0/984438130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 msgr2=0x7f9e40107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:09.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 --2- 172.21.15.49:0/984438130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 0x7f9e40107380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e300098f0 tx=0x7f9e30009c00 comp rx=0 tx=0).stop 2024-02-26T06:18:09.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 -- 172.21.15.49:0/984438130 shutdown_connections 2024-02-26T06:18:09.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 --2- 172.21.15.49:0/984438130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e401078c0 0x7f9e40109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:09.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 --2- 172.21.15.49:0/984438130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e40102660 0x7f9e40104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:09.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 --2- 172.21.15.49:0/984438130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 0x7f9e40107380 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:09.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 -- 172.21.15.49:0/984438130 >> 172.21.15.49:0/984438130 conn(0x7f9e400fc090 msgr2=0x7f9e400fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:09.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 -- 172.21.15.49:0/984438130 shutdown_connections 2024-02-26T06:18:09.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.940+0000 7f9e47200700 1 -- 172.21.15.49:0/984438130 wait complete. 2024-02-26T06:18:09.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.941+0000 7f9e47200700 1 Processor -- start 2024-02-26T06:18:09.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e47200700 1 -- start start 2024-02-26T06:18:09.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e47200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e40102660 0x7f9e40197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:09.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e47200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 0x7f9e401980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:09.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e47200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e401078c0 0x7f9e4019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e47200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4019f730 con 0x7f9e40104f90 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e47200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4019f870 con 0x7f9e40102660 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e47200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4019fa00 con 0x7f9e401078c0 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e45400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 0x7f9e401980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e45e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e40102660 0x7f9e40197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e45400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 0x7f9e401980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53034/0 (socket says 172.21.15.49:53034) 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e45400700 1 -- 172.21.15.49:0/1560573117 learned_addr learned my addr 172.21.15.49:0/1560573117 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.942+0000 7f9e46800700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e401078c0 0x7f9e4019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.943+0000 7f9e45e00700 1 -- 172.21.15.49:0/1560573117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e401078c0 msgr2=0x7f9e4019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.943+0000 7f9e45e00700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e401078c0 0x7f9e4019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.943+0000 7f9e45e00700 1 -- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 msgr2=0x7f9e401980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.943+0000 7f9e45e00700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 0x7f9e401980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:09.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.943+0000 7f9e45e00700 1 -- 172.21.15.49:0/1560573117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c009d00 con 0x7f9e40102660 2024-02-26T06:18:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.943+0000 7f9e45400700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 0x7f9e401980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:18:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.943+0000 7f9e45e00700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e40102660 0x7f9e40197ba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400e9f0 tx=0x7f9e3400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.943+0000 7f9e3aa00700 1 -- 172.21.15.49:0/1560573117 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34004220 con 0x7f9e40102660 2024-02-26T06:18:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.944+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e30009520 con 0x7f9e40102660 2024-02-26T06:18:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.944+0000 7f9e3aa00700 1 -- 172.21.15.49:0/1560573117 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e34004d10 con 0x7f9e40102660 2024-02-26T06:18:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.944+0000 7f9e3aa00700 1 -- 172.21.15.49:0/1560573117 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34005020 con 0x7f9e40102660 2024-02-26T06:18:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.944+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e401a0050 con 0x7f9e40102660 2024-02-26T06:18:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.945+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e400fdc10 con 0x7f9e40102660 2024-02-26T06:18:09.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.947+0000 7f9e3aa00700 1 -- 172.21.15.49:0/1560573117 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9e34004750 con 0x7f9e40102660 2024-02-26T06:18:09.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.948+0000 7f9e3aa00700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9e280a2150 0x7f9e280a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:09.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.948+0000 7f9e3aa00700 1 -- 172.21.15.49:0/1560573117 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f9e34014070 con 0x7f9e40102660 2024-02-26T06:18:09.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.948+0000 7f9e45400700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9e280a2150 0x7f9e280a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:09.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.949+0000 7f9e45400700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9e280a2150 0x7f9e280a4610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000fb00 tx=0x7f9e30019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:09.952+0000 7f9e3aa00700 1 -- 172.21.15.49:0/1560573117 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e34096ac0 con 0x7f9e40102660 2024-02-26T06:18:10.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:10.193+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e4002d800 con 0x7f9e280a2150 2024-02-26T06:18:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:10 smithi080 ceph-mon[36206]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-02-26T06:18:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:10 smithi115 ceph-mon[36426]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-02-26T06:18:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:11.140 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:10 smithi049 ceph-mon[31696]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-02-26T06:18:11.140 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:11.140 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:11.141 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:12.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:11 smithi115 ceph-mon[36426]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:12.129 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:11 smithi049 ceph-mon[31696]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:12.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:11 smithi080 ceph-mon[36206]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:13.081 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:12 smithi049 ceph-mon[31696]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:18:13.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:12 smithi115 ceph-mon[36426]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:18:13.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:12 smithi080 ceph-mon[36206]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:18:15.162 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:14 smithi049 ceph-mon[31696]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:18:15.162 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:14 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/576395332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438322dd-a5d2-4d7a-b7f6-1de27c32fa94"}]: dispatch 2024-02-26T06:18:15.162 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:14 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/576395332' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "438322dd-a5d2-4d7a-b7f6-1de27c32fa94"}]': finished 2024-02-26T06:18:15.162 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:14 smithi049 ceph-mon[31696]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T06:18:15.162 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:14 smithi080 ceph-mon[36206]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:18:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:14 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/576395332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438322dd-a5d2-4d7a-b7f6-1de27c32fa94"}]: dispatch 2024-02-26T06:18:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:14 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/576395332' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "438322dd-a5d2-4d7a-b7f6-1de27c32fa94"}]': finished 2024-02-26T06:18:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:14 smithi080 ceph-mon[36206]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T06:18:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:14 smithi115 ceph-mon[36426]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:18:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:14 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/576395332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438322dd-a5d2-4d7a-b7f6-1de27c32fa94"}]: dispatch 2024-02-26T06:18:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:14 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/576395332' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "438322dd-a5d2-4d7a-b7f6-1de27c32fa94"}]': finished 2024-02-26T06:18:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:14 smithi115 ceph-mon[36426]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T06:18:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:16.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:15 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/717851271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:16.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:15 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/717851271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:16.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:15 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/717851271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:17.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:16 smithi049 ceph-mon[31696]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:18:17.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:16 smithi080 ceph-mon[36206]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:18:17.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:16 smithi115 ceph-mon[36426]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:18:19.112 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:18 smithi049 ceph-mon[31696]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-26T06:18:19.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:18 smithi080 ceph-mon[36206]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-26T06:18:19.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:18 smithi115 ceph-mon[36426]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-26T06:18:21.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:20 smithi049 ceph-mon[31696]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:21.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:20 smithi080 ceph-mon[36206]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:21.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:20 smithi115 ceph-mon[36426]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:22.925 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:22 smithi049 ceph-mon[31696]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:22 smithi115 ceph-mon[36426]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:23.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:22 smithi080 ceph-mon[36206]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:25.133 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:24 smithi080 ceph-mon[36206]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:25.133 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:24 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T06:18:25.133 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:24 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:24 smithi049 ceph-mon[31696]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:24 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T06:18:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:24 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:24 smithi115 ceph-mon[36426]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:24 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T06:18:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:24 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:26.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:25 smithi049 ceph-mon[31696]: Deploying daemon osd.9 on smithi049 2024-02-26T06:18:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:25 smithi080 ceph-mon[36206]: Deploying daemon osd.9 on smithi049 2024-02-26T06:18:26.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:25 smithi115 ceph-mon[36426]: Deploying daemon osd.9 on smithi049 2024-02-26T06:18:27.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:26 smithi049 ceph-mon[31696]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:27.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:26 smithi080 ceph-mon[36206]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:26 smithi115 ceph-mon[36426]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:29.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:28 smithi049 ceph-mon[31696]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:29.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:28 smithi080 ceph-mon[36206]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:28 smithi115 ceph-mon[36426]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:30 smithi049 ceph-mon[31696]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:30 smithi080 ceph-mon[36206]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:30 smithi115 ceph-mon[36426]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:33.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:32 smithi049 ceph-mon[31696]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:32 smithi080 ceph-mon[36206]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:32 smithi115 ceph-mon[36426]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:34.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:34.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:34.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:34.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:34.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:34.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:34.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:34.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:34.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:35.210 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:34 smithi049 ceph-mon[31696]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:34 smithi080 ceph-mon[36206]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:34 smithi115 ceph-mon[36426]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:36.992 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:36 smithi049 ceph-mon[31696]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:36 smithi080 ceph-mon[36206]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:36 smithi115 ceph-mon[36426]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:38.275 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:37 smithi049 ceph-mon[31696]: from='osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:18:38.275 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:37 smithi049 ceph-mon[31696]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:18:38.275 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:38.275 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:37 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:37 smithi080 ceph-mon[36206]: from='osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:18:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:37 smithi080 ceph-mon[36206]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:18:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:37 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:37 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:37 smithi115 ceph-mon[36426]: from='osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:18:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:37 smithi115 ceph-mon[36426]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:18:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:37 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:38.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.496+0000 7f9e3aa00700 1 -- 172.21.15.49:0/1560573117 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9e4002d800 con 0x7f9e280a2150 2024-02-26T06:18:38.498 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 9 on host 'smithi049' 2024-02-26T06:18:38.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9e280a2150 msgr2=0x7f9e280a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:38.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9e280a2150 0x7f9e280a4610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000fb00 tx=0x7f9e30019040 comp rx=0 tx=0).stop 2024-02-26T06:18:38.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e40102660 msgr2=0x7f9e40197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:38.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e40102660 0x7f9e40197ba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400e9f0 tx=0x7f9e3400ed00 comp rx=0 tx=0).stop 2024-02-26T06:18:38.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 shutdown_connections 2024-02-26T06:18:38.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9e280a2150 0x7f9e280a4610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:38.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e401078c0 0x7f9e4019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:38.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e40102660 0x7f9e40197ba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:38.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 --2- 172.21.15.49:0/1560573117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e40104f90 0x7f9e401980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:38.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 >> 172.21.15.49:0/1560573117 conn(0x7f9e400fc090 msgr2=0x7f9e40102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:38.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 shutdown_connections 2024-02-26T06:18:38.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:18:38.500+0000 7f9e47200700 1 -- 172.21.15.49:0/1560573117 wait complete. 2024-02-26T06:18:39.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:38 smithi049 ceph-mon[31696]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:39.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:38 smithi049 ceph-mon[31696]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T06:18:39.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:38 smithi049 ceph-mon[31696]: from='osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:18:39.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:38 smithi049 ceph-mon[31696]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T06:18:39.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:39.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:38 smithi049 ceph-mon[31696]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:18:39.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:39.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:38 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:38 smithi080 ceph-mon[36206]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:38 smithi080 ceph-mon[36206]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T06:18:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:38 smithi080 ceph-mon[36206]: from='osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:18:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:38 smithi080 ceph-mon[36206]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T06:18:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:39.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:38 smithi080 ceph-mon[36206]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:18:39.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:39.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:38 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:38 smithi115 ceph-mon[36426]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:38 smithi115 ceph-mon[36426]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T06:18:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:38 smithi115 ceph-mon[36426]: from='osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:18:39.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:38 smithi115 ceph-mon[36426]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T06:18:39.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:39.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:38 smithi115 ceph-mon[36426]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-02-26T06:18:39.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:39.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:38 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:39.347 DEBUG:teuthology.orchestra.run.smithi049:osd.9> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.9.service 2024-02-26T06:18:39.352 INFO:tasks.cephadm:Deploying osd.10 on smithi080 with /dev/vg_nvme/lv_1... 2024-02-26T06:18:39.353 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T06:18:39.596 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:18:39.919 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:18:39.661+0000 7fa7a3a00700 -1 osd.9 0 waiting for initial osdmap 2024-02-26T06:18:39.920 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:18:39.692+0000 7fa799800700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:18:40.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:39 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:18:40.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:39 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:18:40.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:39 smithi080 ceph-mon[36206]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:18:40.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:39 smithi080 ceph-mon[36206]: osdmap e67: 10 total, 9 up, 10 in 2024-02-26T06:18:40.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:40.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:39 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:40.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:39 smithi080 ceph-mon[36206]: from='osd.9 ' entity='osd.9' 2024-02-26T06:18:40.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:39 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:18:40.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:39 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:18:40.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:39 smithi115 ceph-mon[36426]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:18:40.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:39 smithi115 ceph-mon[36426]: osdmap e67: 10 total, 9 up, 10 in 2024-02-26T06:18:40.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:39 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:40.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:39 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:40.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:39 smithi115 ceph-mon[36426]: from='osd.9 ' entity='osd.9' 2024-02-26T06:18:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:18:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:18:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-mon[31696]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-02-26T06:18:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-mon[31696]: osdmap e67: 10 total, 9 up, 10 in 2024-02-26T06:18:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:40.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:40.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:39 smithi049 ceph-mon[31696]: from='osd.9 ' entity='osd.9' 2024-02-26T06:18:41.242 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:40 smithi115 ceph-mon[36426]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:41.242 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:41.242 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:40 smithi115 ceph-mon[36426]: osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] boot 2024-02-26T06:18:41.242 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:40 smithi115 ceph-mon[36426]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T06:18:41.242 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:40 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:41.307 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:40 smithi080 ceph-mon[36206]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:41.308 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:41.308 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:40 smithi080 ceph-mon[36206]: osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] boot 2024-02-26T06:18:41.308 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:40 smithi080 ceph-mon[36206]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T06:18:41.308 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:40 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:41.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:40 smithi049 ceph-mon[31696]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:18:41.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:41.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:40 smithi049 ceph-mon[31696]: osd.9 [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] boot 2024-02-26T06:18:41.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:40 smithi049 ceph-mon[31696]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T06:18:41.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:40 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:18:41.654 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T06:18:41.654 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T06:18:41.654 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-02-26T06:18:41.655 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-02-26T06:18:41.655 INFO:teuthology.orchestra.run.smithi080.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116286 s, 902 MB/s 2024-02-26T06:18:41.655 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-02-26T06:18:42.332 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi080:vg_nvme/lv_1 2024-02-26T06:18:42.575 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:18:42.940 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:42 smithi080 ceph-mon[36206]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:42.940 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:42 smithi080 ceph-mon[36206]: osdmap e69: 10 total, 10 up, 10 in 2024-02-26T06:18:43.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:42 smithi115 ceph-mon[36426]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:43.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:42 smithi115 ceph-mon[36426]: osdmap e69: 10 total, 10 up, 10 in 2024-02-26T06:18:43.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:42 smithi049 ceph-mon[31696]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:43.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:42 smithi049 ceph-mon[31696]: osdmap e69: 10 total, 10 up, 10 in 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 -- 172.21.15.80:0/3323650883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 msgr2=0x7fbbe0109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 --2- 172.21.15.80:0/3323650883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 0x7fbbe0109e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800b0a0 tx=0x7fbbd800b3b0 comp rx=0 tx=0).stop 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 -- 172.21.15.80:0/3323650883 shutdown_connections 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 --2- 172.21.15.80:0/3323650883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 0x7fbbe0109e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 --2- 172.21.15.80:0/3323650883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbe01027b0 0x7fbbe0104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 --2- 172.21.15.80:0/3323650883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbe01050e0 0x7fbbe01074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 -- 172.21.15.80:0/3323650883 >> 172.21.15.80:0/3323650883 conn(0x7fbbe00fc140 msgr2=0x7fbbe00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 -- 172.21.15.80:0/3323650883 shutdown_connections 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.279+0000 7fbbe633f700 1 -- 172.21.15.80:0/3323650883 wait complete. 2024-02-26T06:18:44.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe633f700 1 Processor -- start 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe633f700 1 -- start start 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe633f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbe01027b0 0x7fbbe0102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe633f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbe01050e0 0x7fbbe0100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe633f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 0x7fbbe0100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe633f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0101420 con 0x7fbbe01027b0 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe633f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0101590 con 0x7fbbe0107a10 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe633f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01a1fe0 con 0x7fbbe01050e0 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe48dc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 0x7fbbe0100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe48dc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 0x7fbbe0100ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39308/0 (socket says 172.21.15.80:39308) 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe48dc700 1 -- 172.21.15.80:0/1577428056 learned_addr learned my addr 172.21.15.80:0/1577428056 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-02-26T06:18:44.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbdf7fe700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbe01050e0 0x7fbbe0100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:44.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe48dc700 1 -- 172.21.15.80:0/1577428056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbe01050e0 msgr2=0x7fbbe0100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe48dc700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbe01050e0 0x7fbbe0100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe48dc700 1 -- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbe01027b0 msgr2=0x7fbbe0102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe48dc700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbe01027b0 0x7fbbe0102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.280+0000 7fbbe48dc700 1 -- 172.21.15.80:0/1577428056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009d00 con 0x7fbbe0107a10 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.281+0000 7fbbe48dc700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 0x7fbbe0100ca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80095e0 tx=0x7fbbd800ce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.281+0000 7fbbdd7fa700 1 -- 172.21.15.80:0/1577428056 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800e040 con 0x7fbbe0107a10 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.281+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe01a2260 con 0x7fbbe0107a10 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.281+0000 7fbbdd7fa700 1 -- 172.21.15.80:0/1577428056 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbd801b930 con 0x7fbbe0107a10 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.281+0000 7fbbdd7fa700 1 -- 172.21.15.80:0/1577428056 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8005710 con 0x7fbbe0107a10 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.281+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe01a2700 con 0x7fbbe0107a10 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.282+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe00fdcc0 con 0x7fbbe0107a10 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.283+0000 7fbbdd7fa700 1 -- 172.21.15.80:0/1577428056 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbbd801baa0 con 0x7fbbe0107a10 2024-02-26T06:18:44.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.283+0000 7fbbdd7fa700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbbc80a2150 0x7fbbc80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:44.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.283+0000 7fbbdd7fa700 1 -- 172.21.15.80:0/1577428056 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fbbd80cf1c0 con 0x7fbbe0107a10 2024-02-26T06:18:44.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.284+0000 7fbbdffff700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbbc80a2150 0x7fbbc80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:44.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.285+0000 7fbbdffff700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbbc80a2150 0x7fbbc80a4610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00097b0 tx=0x7fbbd0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:44.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.288+0000 7fbbdd7fa700 1 -- 172.21.15.80:0/1577428056 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbd8098a20 con 0x7fbbe0107a10 2024-02-26T06:18:44.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:18:44.492+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbe002d7d0 con 0x7fbbc80a2150 2024-02-26T06:18:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:44 smithi080 ceph-mon[36206]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:44 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:45.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:44 smithi115 ceph-mon[36426]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:45.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:45.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:45.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:44 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:45.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:44 smithi049 ceph-mon[31696]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:45.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:45.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:45.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:44 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi049 2024-02-26T06:18:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi049 to 4272M 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:46.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:46.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi049 2024-02-26T06:18:46.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi049 to 4272M 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi049 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi049 to 4272M 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:46.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:46.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:46.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:46.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:46.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:46.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:18:47.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:47 smithi080 ceph-mon[36206]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:47 smithi115 ceph-mon[36426]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:47.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:47 smithi049 ceph-mon[31696]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:48 smithi080 ceph-mon[36206]: from='client.? 172.21.15.80:0/1971432221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]: dispatch 2024-02-26T06:18:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:48 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]: dispatch 2024-02-26T06:18:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:48 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]': finished 2024-02-26T06:18:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:48 smithi080 ceph-mon[36206]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T06:18:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:48 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:18:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:48 smithi080 ceph-mon[36206]: from='client.? 172.21.15.80:0/2411640655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:48 smithi115 ceph-mon[36426]: from='client.? 172.21.15.80:0/1971432221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]: dispatch 2024-02-26T06:18:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:48 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]: dispatch 2024-02-26T06:18:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:48 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]': finished 2024-02-26T06:18:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:48 smithi115 ceph-mon[36426]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T06:18:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:48 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:18:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:48 smithi115 ceph-mon[36426]: from='client.? 172.21.15.80:0/2411640655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:48.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:48 smithi049 ceph-mon[31696]: from='client.? 172.21.15.80:0/1971432221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]: dispatch 2024-02-26T06:18:48.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:48 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]: dispatch 2024-02-26T06:18:48.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:48 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88c87856-9714-45a8-be85-a007457ae094"}]': finished 2024-02-26T06:18:48.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:48 smithi049 ceph-mon[31696]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T06:18:48.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:48 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:18:48.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:48 smithi049 ceph-mon[31696]: from='client.? 172.21.15.80:0/2411640655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:49.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:49 smithi080 ceph-mon[36206]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:49.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:49 smithi115 ceph-mon[36426]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:49.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:49 smithi049 ceph-mon[31696]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:51 smithi080 ceph-mon[36206]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:51 smithi115 ceph-mon[36426]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:51.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:51 smithi049 ceph-mon[31696]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:53.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:53 smithi115 ceph-mon[36426]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:53 smithi049 ceph-mon[31696]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:53.466 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:53 smithi080 ceph-mon[36206]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:55 smithi080 ceph-mon[36206]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T06:18:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:55 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:55 smithi115 ceph-mon[36426]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T06:18:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:55 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:55 smithi049 ceph-mon[31696]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T06:18:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:55 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:56.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:56 smithi049 ceph-mon[31696]: Deploying daemon osd.10 on smithi080 2024-02-26T06:18:56.424 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:56 smithi080 ceph-mon[36206]: Deploying daemon osd.10 on smithi080 2024-02-26T06:18:56.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:56 smithi115 ceph-mon[36426]: Deploying daemon osd.10 on smithi080 2024-02-26T06:18:57.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:57 smithi080 ceph-mon[36206]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:57.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:57 smithi049 ceph-mon[31696]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:57.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:57 smithi115 ceph-mon[36426]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:59.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:18:59 smithi049 ceph-mon[31696]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:59.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:18:59 smithi080 ceph-mon[36206]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:18:59.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:18:59 smithi115 ceph-mon[36426]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:01.191 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:01 smithi080 ceph-mon[36206]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:01.191 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:01.191 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:01.191 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:01 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:01 smithi049 ceph-mon[31696]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:01 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:01 smithi115 ceph-mon[36426]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:01 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:03.254 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:03 smithi080 ceph-mon[36206]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:03.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:03 smithi049 ceph-mon[31696]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:03.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:03 smithi115 ceph-mon[36426]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:04.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.438+0000 7fbbdd7fa700 1 -- 172.21.15.80:0/1577428056 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fbbe002d7d0 con 0x7fbbc80a2150 2024-02-26T06:19:04.438 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 10 on host 'smithi080' 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbbc80a2150 msgr2=0x7fbbc80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbbc80a2150 0x7fbbc80a4610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00097b0 tx=0x7fbbd0006d20 comp rx=0 tx=0).stop 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 msgr2=0x7fbbe0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 0x7fbbe0100ca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80095e0 tx=0x7fbbd800ce90 comp rx=0 tx=0).stop 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 shutdown_connections 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbbc80a2150 0x7fbbc80a4610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe0107a10 0x7fbbe0100ca0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbe01027b0 0x7fbbe0102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 --2- 172.21.15.80:0/1577428056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbe01050e0 0x7fbbe0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 >> 172.21.15.80:0/1577428056 conn(0x7fbbe00fc140 msgr2=0x7fbbe01031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 shutdown_connections 2024-02-26T06:19:04.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-02-26T06:19:04.440+0000 7fbbe633f700 1 -- 172.21.15.80:0/1577428056 wait complete. 2024-02-26T06:19:04.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:04.804 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:04.804 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.805 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:04.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:04.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:05.125 DEBUG:teuthology.orchestra.run.smithi080:osd.10> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.10.service 2024-02-26T06:19:05.127 INFO:tasks.cephadm:Deploying osd.11 on smithi115 with /dev/vg_nvme/lv_1... 2024-02-26T06:19:05.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T06:19:05.369 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:19:06.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:06 smithi080 ceph-mon[36206]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T06:19:06.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:06 smithi080 ceph-mon[36206]: from='osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:19:06.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:06 smithi080 ceph-mon[36206]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T06:19:06.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:06.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:06 smithi080 ceph-mon[36206]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:19:06.322 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:06 smithi080 ceph-mon[36206]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:06.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:06 smithi049 ceph-mon[31696]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T06:19:06.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:06 smithi049 ceph-mon[31696]: from='osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:19:06.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:06 smithi049 ceph-mon[31696]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T06:19:06.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:06.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:06 smithi049 ceph-mon[31696]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:19:06.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:06 smithi049 ceph-mon[31696]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:06.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:06 smithi115 ceph-mon[36426]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T06:19:06.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:06 smithi115 ceph-mon[36426]: from='osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:19:06.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:06 smithi115 ceph-mon[36426]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T06:19:06.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:06.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:06 smithi115 ceph-mon[36426]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-02-26T06:19:06.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:06 smithi115 ceph-mon[36426]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:19:06.867 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:19:06 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:19:06.591+0000 7f0d7e211700 -1 osd.10 0 waiting for initial osdmap 2024-02-26T06:19:06.868 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:19:06 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:19:06.604+0000 7f0d75ffc700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:19:07.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:07 smithi115 ceph-mon[36426]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:19:07.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:07 smithi115 ceph-mon[36426]: osdmap e72: 11 total, 10 up, 11 in 2024-02-26T06:19:07.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:07 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:07.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:07 smithi115 ceph-mon[36426]: from='osd.10 ' entity='osd.10' 2024-02-26T06:19:07.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:07 smithi115 ceph-mon[36426]: osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] boot 2024-02-26T06:19:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:07 smithi049 ceph-mon[31696]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:19:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:07 smithi049 ceph-mon[31696]: osdmap e72: 11 total, 10 up, 11 in 2024-02-26T06:19:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:07 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:07 smithi049 ceph-mon[31696]: from='osd.10 ' entity='osd.10' 2024-02-26T06:19:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:07 smithi049 ceph-mon[31696]: osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] boot 2024-02-26T06:19:07.446 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:07 smithi080 ceph-mon[36206]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-02-26T06:19:07.446 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:07 smithi080 ceph-mon[36206]: osdmap e72: 11 total, 10 up, 11 in 2024-02-26T06:19:07.446 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:07 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:07.446 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:07 smithi080 ceph-mon[36206]: from='osd.10 ' entity='osd.10' 2024-02-26T06:19:07.446 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:07 smithi080 ceph-mon[36206]: osd.10 [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] boot 2024-02-26T06:19:07.512 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T06:19:07.512 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T06:19:07.512 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-02-26T06:19:07.513 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-02-26T06:19:07.513 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123975 s, 846 MB/s 2024-02-26T06:19:07.513 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-02-26T06:19:08.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2024-02-26T06:19:08.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:08 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:19:08.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:08 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:19:08.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:08 smithi049 ceph-mon[31696]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T06:19:08.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:08.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:08 smithi049 ceph-mon[31696]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:08.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:08 smithi049 ceph-mon[31696]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T06:19:08.440 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:19:08.559 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:08 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:19:08.559 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:08 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:19:08.559 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:08 smithi115 ceph-mon[36426]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T06:19:08.559 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:08 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:08.559 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:08 smithi115 ceph-mon[36426]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:08.559 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:08 smithi115 ceph-mon[36426]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T06:19:08.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:08 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:19:08.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:08 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:19:08.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:08 smithi080 ceph-mon[36206]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T06:19:08.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:08 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:19:08.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:08 smithi080 ceph-mon[36206]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:08.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:08 smithi080 ceph-mon[36206]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T06:19:09.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi080 2024-02-26T06:19:09.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi080 to 4270M 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:09 smithi080 ceph-mon[36206]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi080 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi080 to 4270M 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.588 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:09.589 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:09.589 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.589 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:09 smithi115 ceph-mon[36426]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T06:19:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi080 2024-02-26T06:19:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi080 to 4270M 2024-02-26T06:19:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:09 smithi049 ceph-mon[31696]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T06:19:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 -- 172.21.15.115:0/3951919240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 msgr2=0x7f6b28108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/3951919240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 0x7f6b28108ff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b0a0 tx=0x7f6b2000b3b0 comp rx=0 tx=0).stop 2024-02-26T06:19:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 -- 172.21.15.115:0/3951919240 shutdown_connections 2024-02-26T06:19:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/3951919240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 0x7f6b28108ff0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/3951919240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b28100ba0 0x7f6b28100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/3951919240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b28101da0 0x7f6b28102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 -- 172.21.15.115:0/3951919240 >> 172.21.15.115:0/3951919240 conn(0x7f6b280fc140 msgr2=0x7f6b280fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 -- 172.21.15.115:0/3951919240 shutdown_connections 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.110+0000 7f6b2c87a700 1 -- 172.21.15.115:0/3951919240 wait complete. 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2c87a700 1 Processor -- start 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2c87a700 1 -- start start 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2c87a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b28100ba0 0x7f6b28196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2c87a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b28101da0 0x7f6b28197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2c87a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 0x7f6b2819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2c87a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2819e830 con 0x7f6b28101da0 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2c87a700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2819e970 con 0x7f6b28100ba0 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2c87a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2819eb30 con 0x7f6b28102740 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 0x7f6b2819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 0x7f6b2819e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55300/0 (socket says 172.21.15.115:55300) 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b26d9d700 1 -- 172.21.15.115:0/1158064252 learned_addr learned my addr 172.21.15.115:0/1158064252 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.111+0000 7f6b2659c700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b28100ba0 0x7f6b28196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b26d9d700 1 -- 172.21.15.115:0/1158064252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b28100ba0 msgr2=0x7f6b28196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b26d9d700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b28100ba0 0x7f6b28196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b26d9d700 1 -- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b28101da0 msgr2=0x7f6b28197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b26d9d700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b28101da0 0x7f6b28197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b26d9d700 1 -- 172.21.15.115:0/1158064252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b20009d00 con 0x7f6b28102740 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b26d9d700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 0x7f6b2819e260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b070 tx=0x7f6b20004e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b1f7fe700 1 -- 172.21.15.115:0/1158064252 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000e040 con 0x7f6b28102740 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b28069920 con 0x7f6b28102740 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b1f7fe700 1 -- 172.21.15.115:0/1158064252 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b20004020 con 0x7f6b28102740 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b1f7fe700 1 -- 172.21.15.115:0/1158064252 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2001a720 con 0x7f6b28102740 2024-02-26T06:19:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.112+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b28069ef0 con 0x7f6b28102740 2024-02-26T06:19:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.113+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b28068930 con 0x7f6b28102740 2024-02-26T06:19:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.114+0000 7f6b1f7fe700 1 -- 172.21.15.115:0/1158064252 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6b20004190 con 0x7f6b28102740 2024-02-26T06:19:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.114+0000 7f6b1f7fe700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6b140a2150 0x7f6b140a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.115+0000 7f6b1f7fe700 1 -- 172.21.15.115:0/1158064252 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f6b200d0ba0 con 0x7f6b28102740 2024-02-26T06:19:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.115+0000 7f6b2659c700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6b140a2150 0x7f6b140a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.116+0000 7f6b2659c700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6b140a2150 0x7f6b140a4610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b281981c0 tx=0x7f6b1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.119+0000 7f6b1f7fe700 1 -- 172.21.15.115:0/1158064252 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b2009a250 con 0x7f6b28102740 2024-02-26T06:19:10.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:10.320+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b28107ab0 con 0x7f6b140a2150 2024-02-26T06:19:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:10 smithi080 ceph-mon[36206]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:10 smithi080 ceph-mon[36206]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T06:19:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:10 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:10 smithi115 ceph-mon[36426]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:10 smithi115 ceph-mon[36426]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T06:19:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:10 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:11.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:10 smithi049 ceph-mon[31696]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:11.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:10 smithi049 ceph-mon[31696]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T06:19:11.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:11.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:11.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:10 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:12.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:11 smithi115 ceph-mon[36426]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:12.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:11 smithi115 ceph-mon[36426]: osdmap e77: 11 total, 11 up, 11 in 2024-02-26T06:19:12.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:11 smithi049 ceph-mon[31696]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:12.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:11 smithi049 ceph-mon[31696]: osdmap e77: 11 total, 11 up, 11 in 2024-02-26T06:19:12.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:11 smithi080 ceph-mon[36206]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:12.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:11 smithi080 ceph-mon[36206]: osdmap e77: 11 total, 11 up, 11 in 2024-02-26T06:19:13.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:12 smithi115 ceph-mon[36426]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:13.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:12 smithi049 ceph-mon[31696]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:13.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:12 smithi080 ceph-mon[36206]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:14.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:13 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/3589166858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]: dispatch 2024-02-26T06:19:14.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:13 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]: dispatch 2024-02-26T06:19:14.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:13 smithi049 ceph-mon[31696]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]': finished 2024-02-26T06:19:14.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:13 smithi049 ceph-mon[31696]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T06:19:14.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:13 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:14.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:13 smithi049 ceph-mon[31696]: from='client.? 172.21.15.115:0/1294175011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:14.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:13 smithi080 ceph-mon[36206]: from='client.? 172.21.15.115:0/3589166858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]: dispatch 2024-02-26T06:19:14.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:13 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]: dispatch 2024-02-26T06:19:14.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:13 smithi080 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]': finished 2024-02-26T06:19:14.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:13 smithi080 ceph-mon[36206]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T06:19:14.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:13 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:14.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:13 smithi080 ceph-mon[36206]: from='client.? 172.21.15.115:0/1294175011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:14.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:13 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/3589166858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]: dispatch 2024-02-26T06:19:14.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:13 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]: dispatch 2024-02-26T06:19:14.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:13 smithi115 ceph-mon[36426]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46f79795-b13d-4997-b0d2-d89822eedf67"}]': finished 2024-02-26T06:19:14.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:13 smithi115 ceph-mon[36426]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T06:19:14.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:13 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:14.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:13 smithi115 ceph-mon[36426]: from='client.? 172.21.15.115:0/1294175011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:15.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:14 smithi049 ceph-mon[31696]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-02-26T06:19:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:14 smithi080 ceph-mon[36206]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-02-26T06:19:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:14 smithi115 ceph-mon[36426]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-02-26T06:19:17.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:16 smithi115 ceph-mon[36426]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:19:17.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:16 smithi049 ceph-mon[31696]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:19:17.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:16 smithi080 ceph-mon[36206]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:19:19.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:18 smithi115 ceph-mon[36426]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:19:19.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:18 smithi049 ceph-mon[31696]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:19:19.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:18 smithi080 ceph-mon[36206]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:19:21.130 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:20 smithi115 ceph-mon[36426]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:19:21.130 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:20 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T06:19:21.130 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:20 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:21.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:20 smithi049 ceph-mon[31696]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:19:21.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:20 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T06:19:21.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:20 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:21.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:20 smithi080 ceph-mon[36206]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-26T06:19:21.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:20 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T06:19:21.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:20 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:21.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:21 smithi115 ceph-mon[36426]: Deploying daemon osd.11 on smithi115 2024-02-26T06:19:22.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:21 smithi049 ceph-mon[31696]: Deploying daemon osd.11 on smithi115 2024-02-26T06:19:22.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:21 smithi080 ceph-mon[36206]: Deploying daemon osd.11 on smithi115 2024-02-26T06:19:23.159 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:22 smithi115 ceph-mon[36426]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:19:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:22 smithi049 ceph-mon[31696]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:19:23.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:22 smithi080 ceph-mon[36206]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:19:25.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:24 smithi115 ceph-mon[36426]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-02-26T06:19:25.132 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:24 smithi080 ceph-mon[36206]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-02-26T06:19:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:24 smithi049 ceph-mon[31696]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-02-26T06:19:27.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:27 smithi080 ceph-mon[36206]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:27.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:27.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:27.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:27 smithi115 ceph-mon[36426]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:27 smithi049 ceph-mon[31696]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:29.281 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:29 smithi115 ceph-mon[36426]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:29.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:29 smithi080 ceph-mon[36206]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:29.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:29 smithi049 ceph-mon[31696]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:29.605 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 11 on host 'smithi115' 2024-02-26T06:19:29.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.605+0000 7f6b1f7fe700 1 -- 172.21.15.115:0/1158064252 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f6b28107ab0 con 0x7f6b140a2150 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6b140a2150 msgr2=0x7f6b140a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6b140a2150 0x7f6b140a4610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b281981c0 tx=0x7f6b1800b410 comp rx=0 tx=0).stop 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 msgr2=0x7f6b2819e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 0x7f6b2819e260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b070 tx=0x7f6b20004e20 comp rx=0 tx=0).stop 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 shutdown_connections 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6b140a2150 0x7f6b140a4610 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b28102740 0x7f6b2819e260 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b28100ba0 0x7f6b28196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 --2- 172.21.15.115:0/1158064252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b28101da0 0x7f6b28197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 >> 172.21.15.115:0/1158064252 conn(0x7f6b280fc140 msgr2=0x7f6b281050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 shutdown_connections 2024-02-26T06:19:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:29.608+0000 7f6b2c87a700 1 -- 172.21.15.115:0/1158064252 wait complete. 2024-02-26T06:19:30.283 DEBUG:teuthology.orchestra.run.smithi115:osd.11> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.11.service 2024-02-26T06:19:30.285 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-26T06:19:30.286 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd stat -f json 2024-02-26T06:19:30.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:30 smithi049 ceph-mon[31696]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:30.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:30 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:30 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:30 smithi049 ceph-mon[31696]: from='osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:19:30.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:30 smithi049 ceph-mon[31696]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:19:30.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:30 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:30 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.782 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:19:30.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:30 smithi080 ceph-mon[36206]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:30.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:30 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:30 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:30 smithi080 ceph-mon[36206]: from='osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:19:30.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:30 smithi080 ceph-mon[36206]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:19:30.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:30 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:30 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:30 smithi115 ceph-mon[36426]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:30 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:30 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:30 smithi115 ceph-mon[36426]: from='osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:19:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:30 smithi115 ceph-mon[36426]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:19:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:30 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:30 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:31.587 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:31 smithi049 ceph-mon[31696]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T06:19:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:31 smithi115 ceph-mon[36426]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T06:19:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:31 smithi115 ceph-mon[36426]: from='osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:19:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:31 smithi115 ceph-mon[36426]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T06:19:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:31 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:31 smithi115 ceph-mon[36426]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:19:31.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:31 smithi049 ceph-mon[31696]: from='osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:19:31.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:31 smithi049 ceph-mon[31696]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T06:19:31.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:31.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:31 smithi049 ceph-mon[31696]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:19:32.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:31 smithi080 ceph-mon[36206]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T06:19:32.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:31 smithi080 ceph-mon[36206]: from='osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:19:32.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:31 smithi080 ceph-mon[36206]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T06:19:32.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:31 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:32.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:31 smithi080 ceph-mon[36206]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-02-26T06:19:32.286 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:19:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:19:32.029+0000 7fca28fbd700 -1 osd.11 0 waiting for initial osdmap 2024-02-26T06:19:32.286 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:19:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:19:32.044+0000 7fca20da8700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:19:32.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:32 smithi115 ceph-mon[36426]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:32.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:32 smithi115 ceph-mon[36426]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:19:32.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:32 smithi115 ceph-mon[36426]: osdmap e80: 12 total, 11 up, 12 in 2024-02-26T06:19:32.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:32.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:32 smithi115 ceph-mon[36426]: from='osd.11 ' entity='osd.11' 2024-02-26T06:19:32.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:32 smithi049 ceph-mon[31696]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:32.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:32 smithi049 ceph-mon[31696]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:19:32.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:32 smithi049 ceph-mon[31696]: osdmap e80: 12 total, 11 up, 12 in 2024-02-26T06:19:32.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:32.893 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:32 smithi049 ceph-mon[31696]: from='osd.11 ' entity='osd.11' 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.073+0000 7ff9bfe00700 1 -- 172.21.15.49:0/3067947314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 msgr2=0x7ff9c0105450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.073+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/3067947314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 0x7ff9c0105450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009920 tx=0x7ff9ac009c30 comp rx=0 tx=0).stop 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.074+0000 7ff9bfe00700 1 -- 172.21.15.49:0/3067947314 shutdown_connections 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.074+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/3067947314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c0105990 0x7ff9c010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.074+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/3067947314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c0103df0 0x7ff9c0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.074+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/3067947314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 0x7ff9c0105450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.074+0000 7ff9bfe00700 1 -- 172.21.15.49:0/3067947314 >> 172.21.15.49:0/3067947314 conn(0x7ff9c00ff390 msgr2=0x7ff9c01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.074+0000 7ff9bfe00700 1 -- 172.21.15.49:0/3067947314 shutdown_connections 2024-02-26T06:19:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.074+0000 7ff9bfe00700 1 -- 172.21.15.49:0/3067947314 wait complete. 2024-02-26T06:19:33.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.075+0000 7ff9bfe00700 1 Processor -- start 2024-02-26T06:19:33.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.075+0000 7ff9bfe00700 1 -- start start 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bfe00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c0103df0 0x7ff9c0199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bfe00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 0x7ff9c019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bfe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c0105990 0x7ff9c01a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bfe00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01a19d0 con 0x7ff9c0104ff0 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bfe00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01a1b10 con 0x7ff9c0103df0 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bfe00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01a1ca0 con 0x7ff9c0105990 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bea00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c0103df0 0x7ff9c0199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9be000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 0x7ff9c019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bea00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c0103df0 0x7ff9c0199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:56538/0 (socket says 172.21.15.49:56538) 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bea00700 1 -- 172.21.15.49:0/4166827812 learned_addr learned my addr 172.21.15.49:0/4166827812 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:33.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.076+0000 7ff9bf400700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c0105990 0x7ff9c01a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:33.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9be000700 1 -- 172.21.15.49:0/4166827812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c0105990 msgr2=0x7ff9c01a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:33.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9be000700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c0105990 0x7ff9c01a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9be000700 1 -- 172.21.15.49:0/4166827812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c0103df0 msgr2=0x7ff9c0199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9be000700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c0103df0 0x7ff9c0199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9be000700 1 -- 172.21.15.49:0/4166827812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b8009d00 con 0x7ff9c0104ff0 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9be000700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 0x7ff9c019a380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000c00 tx=0x7ff9ac005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9c4a00700 1 -- 172.21.15.49:0/4166827812 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac01c070 con 0x7ff9c0104ff0 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9c4a00700 1 -- 172.21.15.49:0/4166827812 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9ac0055d0 con 0x7ff9c0104ff0 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac009520 con 0x7ff9c0104ff0 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.077+0000 7ff9c4a00700 1 -- 172.21.15.49:0/4166827812 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac003c00 con 0x7ff9c0104ff0 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.078+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c01a21a0 con 0x7ff9c0104ff0 2024-02-26T06:19:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.079+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c010b230 con 0x7ff9c0104ff0 2024-02-26T06:19:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:32 smithi080 ceph-mon[36206]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:32 smithi080 ceph-mon[36206]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-02-26T06:19:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:32 smithi080 ceph-mon[36206]: osdmap e80: 12 total, 11 up, 12 in 2024-02-26T06:19:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:32 smithi080 ceph-mon[36206]: from='osd.11 ' entity='osd.11' 2024-02-26T06:19:33.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.081+0000 7ff9c4a00700 1 -- 172.21.15.49:0/4166827812 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff9ac00b560 con 0x7ff9c0104ff0 2024-02-26T06:19:33.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.082+0000 7ff9c4a00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a40a21d0 0x7ff9a40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:33.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.082+0000 7ff9c4a00700 1 -- 172.21.15.49:0/4166827812 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9ac0ce390 con 0x7ff9c0104ff0 2024-02-26T06:19:33.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.083+0000 7ff9bea00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a40a21d0 0x7ff9a40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:33.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.083+0000 7ff9bea00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a40a21d0 0x7ff9a40a4690 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005950 tx=0x7ff9b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:33.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.087+0000 7ff9c4a00700 1 -- 172.21.15.49:0/4166827812 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9ac0977f0 con 0x7ff9c0104ff0 2024-02-26T06:19:33.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.311+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff9c002d760 con 0x7ff9c0104ff0 2024-02-26T06:19:33.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.312+0000 7ff9c4a00700 1 -- 172.21.15.49:0/4166827812 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff9ac09aeb0 con 0x7ff9c0104ff0 2024-02-26T06:19:33.312 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a40a21d0 msgr2=0x7ff9a40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a40a21d0 0x7ff9a40a4690 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005950 tx=0x7ff9b0009260 comp rx=0 tx=0).stop 2024-02-26T06:19:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 msgr2=0x7ff9c019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 0x7ff9c019a380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000c00 tx=0x7ff9ac005250 comp rx=0 tx=0).stop 2024-02-26T06:19:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 shutdown_connections 2024-02-26T06:19:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff9a40a21d0 0x7ff9a40a4690 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c0105990 0x7ff9c01a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c0103df0 0x7ff9c0199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 --2- 172.21.15.49:0/4166827812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9c0104ff0 0x7ff9c019a380 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.315+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 >> 172.21.15.49:0/4166827812 conn(0x7ff9c00ff390 msgr2=0x7ff9c0108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.316+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 shutdown_connections 2024-02-26T06:19:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:33.316+0000 7ff9bfe00700 1 -- 172.21.15.49:0/4166827812 wait complete. 2024-02-26T06:19:33.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:33 smithi115 ceph-mon[36426]: purged_snaps scrub starts 2024-02-26T06:19:33.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:33 smithi115 ceph-mon[36426]: purged_snaps scrub ok 2024-02-26T06:19:33.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:33.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:33 smithi115 ceph-mon[36426]: osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] boot 2024-02-26T06:19:33.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:33 smithi115 ceph-mon[36426]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T06:19:33.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:33.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:33 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4166827812' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:19:33.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:33 smithi049 ceph-mon[31696]: purged_snaps scrub starts 2024-02-26T06:19:33.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:33 smithi049 ceph-mon[31696]: purged_snaps scrub ok 2024-02-26T06:19:33.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:33.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:33 smithi049 ceph-mon[31696]: osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] boot 2024-02-26T06:19:33.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:33 smithi049 ceph-mon[31696]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T06:19:33.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:33.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:33 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4166827812' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:19:34.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:33 smithi080 ceph-mon[36206]: purged_snaps scrub starts 2024-02-26T06:19:34.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:33 smithi080 ceph-mon[36206]: purged_snaps scrub ok 2024-02-26T06:19:34.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:34.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:33 smithi080 ceph-mon[36206]: osd.11 [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] boot 2024-02-26T06:19:34.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:33 smithi080 ceph-mon[36206]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T06:19:34.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:19:34.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:33 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4166827812' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:19:34.194 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1708928373,"num_in_osds":12,"osd_in_since":1708928353,"num_remapped_pgs":0} 2024-02-26T06:19:34.195 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T06:19:34.642 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:19:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: Detected new or changed devices on smithi115 2024-02-26T06:19:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi115 to 4270M 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:35.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:35 smithi080 ceph-mon[36206]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T06:19:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: Detected new or changed devices on smithi115 2024-02-26T06:19:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi115 to 4270M 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:35 smithi115 ceph-mon[36426]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T06:19:35.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:19:35.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: Detected new or changed devices on smithi115 2024-02-26T06:19:35.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi115 to 4270M 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:35.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:35.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:35.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:35 smithi049 ceph-mon[31696]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T06:19:36.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:36.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:36.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:36 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:36.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:36.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:36.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:36 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:36.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:36.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:36.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:36 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 -- 172.21.15.49:0/1278305427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60069920 msgr2=0x7fbc60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 --2- 172.21.15.49:0/1278305427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60069920 0x7fbc60069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009990 tx=0x7fbc48009ca0 comp rx=0 tx=0).stop 2024-02-26T06:19:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 -- 172.21.15.49:0/1278305427 shutdown_connections 2024-02-26T06:19:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 --2- 172.21.15.49:0/1278305427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc6006a2c0 0x7fbc6010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 --2- 172.21.15.49:0/1278305427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc60107cc0 0x7fbc601080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 --2- 172.21.15.49:0/1278305427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60069920 0x7fbc60069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 -- 172.21.15.49:0/1278305427 >> 172.21.15.49:0/1278305427 conn(0x7fbc60077aa0 msgr2=0x7fbc60077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 -- 172.21.15.49:0/1278305427 shutdown_connections 2024-02-26T06:19:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.875+0000 7fbc66000700 1 -- 172.21.15.49:0/1278305427 wait complete. 2024-02-26T06:19:36.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.877+0000 7fbc66000700 1 Processor -- start 2024-02-26T06:19:36.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.878+0000 7fbc66000700 1 -- start start 2024-02-26T06:19:36.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.878+0000 7fbc66000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc60069920 0x7fbc60198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:36.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.878+0000 7fbc66000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc6006a2c0 0x7fbc60198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.878+0000 7fbc66000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60107cc0 0x7fbc6019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.878+0000 7fbc66000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6019d0f0 con 0x7fbc60069920 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.878+0000 7fbc66000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6019d260 con 0x7fbc60107cc0 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.878+0000 7fbc66000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6019d420 con 0x7fbc6006a2c0 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.878+0000 7fbc5f400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc60069920 0x7fbc60198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5f400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc60069920 0x7fbc60198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46768/0 (socket says 172.21.15.49:46768) 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5f400700 1 -- 172.21.15.49:0/3169559824 learned_addr learned my addr 172.21.15.49:0/3169559824 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc56a00700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc6006a2c0 0x7fbc60198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:36.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5fe00700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60107cc0 0x7fbc6019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5fe00700 1 -- 172.21.15.49:0/3169559824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc6006a2c0 msgr2=0x7fbc60198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5fe00700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc6006a2c0 0x7fbc60198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5fe00700 1 -- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc60069920 msgr2=0x7fbc60198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:36.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5fe00700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc60069920 0x7fbc60198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5fe00700 1 -- 172.21.15.49:0/3169559824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc480095f0 con 0x7fbc60107cc0 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5f400700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc60069920 0x7fbc60198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.879+0000 7fbc5fe00700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60107cc0 0x7fbc6019cae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800efc0 tx=0x7fbc5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.880+0000 7fbc5cc00700 1 -- 172.21.15.49:0/3169559824 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58018070 con 0x7fbc60107cc0 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.880+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6019d6a0 con 0x7fbc60107cc0 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.880+0000 7fbc5cc00700 1 -- 172.21.15.49:0/3169559824 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc5800f040 con 0x7fbc60107cc0 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.880+0000 7fbc5cc00700 1 -- 172.21.15.49:0/3169559824 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800cca0 con 0x7fbc60107cc0 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.880+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc601a4f60 con 0x7fbc60107cc0 2024-02-26T06:19:36.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.881+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6010a1e0 con 0x7fbc60107cc0 2024-02-26T06:19:36.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.883+0000 7fbc5cc00700 1 -- 172.21.15.49:0/3169559824 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbc58004ad0 con 0x7fbc60107cc0 2024-02-26T06:19:36.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.885+0000 7fbc5cc00700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbc440a21d0 0x7fbc440a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:36.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.885+0000 7fbc5cc00700 1 -- 172.21.15.49:0/3169559824 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbc580cd3a0 con 0x7fbc60107cc0 2024-02-26T06:19:36.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.885+0000 7fbc5f400700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbc440a21d0 0x7fbc440a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:36.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.886+0000 7fbc5f400700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbc440a21d0 0x7fbc440a4690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbc50011550 tx=0x7fbc50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:36.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:36.889+0000 7fbc5cc00700 1 -- 172.21.15.49:0/3169559824 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc58096800 con 0x7fbc60107cc0 2024-02-26T06:19:37.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.115+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc60199570 con 0x7fbc60107cc0 2024-02-26T06:19:37.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.116+0000 7fbc5cc00700 1 -- 172.21.15.49:0/3169559824 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18443 (secure 0 0 0) 0x7fbc5800ce60 con 0x7fbc60107cc0 2024-02-26T06:19:37.117 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:19:34.587856+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.119+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbc440a21d0 msgr2=0x7fbc440a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.119+0000 7fbc66000700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbc440a21d0 0x7fbc440a4690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbc50011550 tx=0x7fbc50009260 comp rx=0 tx=0).stop 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.119+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60107cc0 msgr2=0x7fbc6019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.119+0000 7fbc66000700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60107cc0 0x7fbc6019cae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800efc0 tx=0x7fbc5800c3a0 comp rx=0 tx=0).stop 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.120+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 shutdown_connections 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.120+0000 7fbc66000700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbc440a21d0 0x7fbc440a4690 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.120+0000 7fbc66000700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc60107cc0 0x7fbc6019cae0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.120+0000 7fbc66000700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc60069920 0x7fbc60198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.120+0000 7fbc66000700 1 --2- 172.21.15.49:0/3169559824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc6006a2c0 0x7fbc60198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.120+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 >> 172.21.15.49:0/3169559824 conn(0x7fbc60077aa0 msgr2=0x7fbc600ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.120+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 shutdown_connections 2024-02-26T06:19:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:37.120+0000 7fbc66000700 1 -- 172.21.15.49:0/3169559824 wait complete. 2024-02-26T06:19:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:37 smithi080 ceph-mon[36206]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:37 smithi115 ceph-mon[36426]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:37 smithi049 ceph-mon[31696]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:37.974 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:19:37.974 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-26T06:19:38.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:38 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3169559824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:19:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:38 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3169559824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:19:38.457 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:19:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:38 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3169559824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:19:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:39 smithi115 ceph-mon[36426]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:39.760 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:39 smithi080 ceph-mon[36206]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:39.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:39 smithi049 ceph-mon[31696]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:40.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 -- 172.21.15.49:0/670812666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb806a410 msgr2=0x7f3bb806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:40.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 --2- 172.21.15.49:0/670812666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb806a410 0x7f3bb806a7f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009990 tx=0x7f3bac009ca0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 -- 172.21.15.49:0/670812666 shutdown_connections 2024-02-26T06:19:40.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 --2- 172.21.15.49:0/670812666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb8106f80 0x7f3bb810eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 --2- 172.21.15.49:0/670812666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb806a410 0x7f3bb806a7f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 --2- 172.21.15.49:0/670812666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb806adc0 0x7f3bb81068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 -- 172.21.15.49:0/670812666 >> 172.21.15.49:0/670812666 conn(0x7f3bb8076bc0 msgr2=0x7f3bb8076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:40.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 -- 172.21.15.49:0/670812666 shutdown_connections 2024-02-26T06:19:40.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.722+0000 7f3bc0600700 1 -- 172.21.15.49:0/670812666 wait complete. 2024-02-26T06:19:40.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.723+0000 7f3bc0600700 1 Processor -- start 2024-02-26T06:19:40.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bc0600700 1 -- start start 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bc0600700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb806a410 0x7f3bb819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bc0600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb806adc0 0x7f3bb819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bc0600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb8106f80 0x7f3bb819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bc0600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb819f240 con 0x7f3bb8106f80 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bc0600700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb819f3b0 con 0x7f3bb806a410 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bc0600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb819f570 con 0x7f3bb806adc0 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bbdc00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb806a410 0x7f3bb819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bbe600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb8106f80 0x7f3bb819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bbdc00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb806a410 0x7f3bb819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45280/0 (socket says 172.21.15.49:45280) 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.724+0000 7f3bbdc00700 1 -- 172.21.15.49:0/4015437715 learned_addr learned my addr 172.21.15.49:0/4015437715 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.725+0000 7f3bbd200700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb806adc0 0x7f3bb819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.725+0000 7f3bbdc00700 1 -- 172.21.15.49:0/4015437715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb806adc0 msgr2=0x7f3bb819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.725+0000 7f3bbdc00700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb806adc0 0x7f3bb819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.725+0000 7f3bbdc00700 1 -- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb8106f80 msgr2=0x7f3bb819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.725+0000 7f3bbdc00700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb8106f80 0x7f3bb819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.725+0000 7f3bbdc00700 1 -- 172.21.15.49:0/4015437715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0095f0 con 0x7f3bb806a410 2024-02-26T06:19:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.725+0000 7f3bbdc00700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb806a410 0x7f3bb819a3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3bac005910 tx=0x7f3bac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.726+0000 7f3bb6000700 1 -- 172.21.15.49:0/4015437715 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac01d070 con 0x7f3bb806a410 2024-02-26T06:19:40.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.726+0000 7f3bb6000700 1 -- 172.21.15.49:0/4015437715 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bac00b610 con 0x7f3bb806a410 2024-02-26T06:19:40.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.726+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb819f7f0 con 0x7f3bb806a410 2024-02-26T06:19:40.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.726+0000 7f3bb6000700 1 -- 172.21.15.49:0/4015437715 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0173f0 con 0x7f3bb806a410 2024-02-26T06:19:40.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.726+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb81a70b0 con 0x7f3bb806a410 2024-02-26T06:19:40.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.727+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb81001b0 con 0x7f3bb806a410 2024-02-26T06:19:40.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.729+0000 7f3bb6000700 1 -- 172.21.15.49:0/4015437715 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3bac0225e0 con 0x7f3bb806a410 2024-02-26T06:19:40.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.730+0000 7f3bb6000700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3ba00a2150 0x7f3ba00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:40.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.731+0000 7f3bb6000700 1 -- 172.21.15.49:0/4015437715 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3bac0cea70 con 0x7f3bb806a410 2024-02-26T06:19:40.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.731+0000 7f3bbd200700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3ba00a2150 0x7f3ba00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:40.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.731+0000 7f3bbd200700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3ba00a2150 0x7f3ba00a4610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8108370 tx=0x7f3ba4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:40.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.733+0000 7f3bb6000700 1 -- 172.21.15.49:0/4015437715 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3bac097ed0 con 0x7f3bb806a410 2024-02-26T06:19:40.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.960+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bb819b6c0 con 0x7f3bb806a410 2024-02-26T06:19:40.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.961+0000 7f3bb6000700 1 -- 172.21.15.49:0/4015437715 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3bac09b590 con 0x7f3bb806a410 2024-02-26T06:19:40.961 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:19:40.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.964+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3ba00a2150 msgr2=0x7f3ba00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.964+0000 7f3bc0600700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3ba00a2150 0x7f3ba00a4610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8108370 tx=0x7f3ba4009260 comp rx=0 tx=0).stop 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.964+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb806a410 msgr2=0x7f3bb819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.964+0000 7f3bc0600700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb806a410 0x7f3bb819a3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3bac005910 tx=0x7f3bac004730 comp rx=0 tx=0).stop 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.965+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 shutdown_connections 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.965+0000 7f3bc0600700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3ba00a2150 0x7f3ba00a4610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.965+0000 7f3bc0600700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb8106f80 0x7f3bb819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.965+0000 7f3bc0600700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb806a410 0x7f3bb819a3b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.965+0000 7f3bc0600700 1 --2- 172.21.15.49:0/4015437715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb806adc0 0x7f3bb819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.965+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 >> 172.21.15.49:0/4015437715 conn(0x7f3bb8076bc0 msgr2=0x7f3bb810d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.965+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 shutdown_connections 2024-02-26T06:19:40.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:40.965+0000 7f3bc0600700 1 -- 172.21.15.49:0/4015437715 wait complete. 2024-02-26T06:19:41.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:41 smithi049 ceph-mon[31696]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:41.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:41 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4015437715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:19:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:41 smithi080 ceph-mon[36206]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:41 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4015437715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:19:41.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:41 smithi115 ceph-mon[36426]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:41.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:41 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4015437715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:19:41.966 INFO:tasks.cephadm:Adding mds.a on smithi049 2024-02-26T06:19:41.966 INFO:tasks.cephadm:Adding mds.d on smithi049 2024-02-26T06:19:41.966 INFO:tasks.cephadm:Adding mds.g on smithi049 2024-02-26T06:19:41.967 INFO:tasks.cephadm:Adding mds.j on smithi049 2024-02-26T06:19:41.967 INFO:tasks.cephadm:Adding mds.b on smithi080 2024-02-26T06:19:41.967 INFO:tasks.cephadm:Adding mds.e on smithi080 2024-02-26T06:19:41.967 INFO:tasks.cephadm:Adding mds.h on smithi080 2024-02-26T06:19:41.967 INFO:tasks.cephadm:Adding mds.k on smithi080 2024-02-26T06:19:41.967 INFO:tasks.cephadm:Adding mds.c on smithi115 2024-02-26T06:19:41.968 INFO:tasks.cephadm:Adding mds.f on smithi115 2024-02-26T06:19:41.968 INFO:tasks.cephadm:Adding mds.i on smithi115 2024-02-26T06:19:41.968 INFO:tasks.cephadm:Adding mds.l on smithi115 2024-02-26T06:19:41.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph orch apply mds all '12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l' 2024-02-26T06:19:42.230 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:19:43.390 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:43 smithi115 ceph-mon[36426]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:43.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:43 smithi049 ceph-mon[31696]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:43.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:43 smithi080 ceph-mon[36206]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:44.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 -- 172.21.15.115:0/2434143244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714102770 msgr2=0x7f5714109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:44.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 --2- 172.21.15.115:0/2434143244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714102770 0x7f5714109020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f570c00b0a0 tx=0x7f570c00b3b0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 -- 172.21.15.115:0/2434143244 shutdown_connections 2024-02-26T06:19:44.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 --2- 172.21.15.115:0/2434143244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714102770 0x7f5714109020 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 --2- 172.21.15.115:0/2434143244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5714100bd0 0x7f5714100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 --2- 172.21.15.115:0/2434143244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5714101dd0 0x7f5714102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 -- 172.21.15.115:0/2434143244 >> 172.21.15.115:0/2434143244 conn(0x7f57140fc170 msgr2=0x7f57140fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:44.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 -- 172.21.15.115:0/2434143244 shutdown_connections 2024-02-26T06:19:44.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.041+0000 7f5718cd5700 1 -- 172.21.15.115:0/2434143244 wait complete. 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.042+0000 7f5718cd5700 1 Processor -- start 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.042+0000 7f5718cd5700 1 -- start start 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.042+0000 7f5718cd5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714100bd0 0x7f571406aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.042+0000 7f5718cd5700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5714101dd0 0x7f571419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.042+0000 7f5718cd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5714102770 0x7f571419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.042+0000 7f5718cd5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571419f800 con 0x7f5714102770 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.042+0000 7f5718cd5700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571419f970 con 0x7f5714101dd0 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.042+0000 7f5718cd5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571419fb30 con 0x7f5714100bd0 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714100bd0 0x7f571406aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714100bd0 0x7f571406aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53006/0 (socket says 172.21.15.115:53006) 2024-02-26T06:19:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f5711d9b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5714101dd0 0x7f571419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 -- 172.21.15.115:0/3999143107 learned_addr learned my addr 172.21.15.115:0/3999143107 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 -- 172.21.15.115:0/3999143107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5714101dd0 msgr2=0x7f571419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5714101dd0 0x7f571419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 -- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5714102770 msgr2=0x7f571419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5714102770 0x7f571419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 -- 172.21.15.115:0/3999143107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c009d00 con 0x7f5714100bd0 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f571259c700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714100bd0 0x7f571406aed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f570400b500 tx=0x7f570400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f57037fe700 1 -- 172.21.15.115:0/3999143107 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704011830 con 0x7f5714100bd0 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f57037fe700 1 -- 172.21.15.115:0/3999143107 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5704011e70 con 0x7f5714100bd0 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571419fe10 con 0x7f5714100bd0 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f57037fe700 1 -- 172.21.15.115:0/3999143107 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570400f2e0 con 0x7f5714100bd0 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.043+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57141a02a0 con 0x7f5714100bd0 2024-02-26T06:19:44.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.044+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5714107ec0 con 0x7f5714100bd0 2024-02-26T06:19:44.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.046+0000 7f57037fe700 1 -- 172.21.15.115:0/3999143107 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f57040105f0 con 0x7f5714100bd0 2024-02-26T06:19:44.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.047+0000 7f57037fe700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56fc0a2150 0x7f56fc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:44.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.047+0000 7f57037fe700 1 -- 172.21.15.115:0/3999143107 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f57040cd6a0 con 0x7f5714100bd0 2024-02-26T06:19:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.047+0000 7f5711d9b700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56fc0a2150 0x7f56fc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:44.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.048+0000 7f5711d9b700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56fc0a2150 0x7f56fc0a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f570800ac50 tx=0x7f570800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:44.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.051+0000 7f57037fe700 1 -- 172.21.15.115:0/3999143107 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5704096b10 con 0x7f5714100bd0 2024-02-26T06:19:44.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.251+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f571402da90 con 0x7f56fc0a2150 2024-02-26T06:19:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.267+0000 7f57037fe700 1 -- 172.21.15.115:0/3999143107 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f571402da90 con 0x7f56fc0a2150 2024-02-26T06:19:44.268 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mds.all update... 2024-02-26T06:19:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56fc0a2150 msgr2=0x7f56fc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56fc0a2150 0x7f56fc0a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f570800ac50 tx=0x7f570800a380 comp rx=0 tx=0).stop 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714100bd0 msgr2=0x7f571406aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714100bd0 0x7f571406aed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f570400b500 tx=0x7f570400b810 comp rx=0 tx=0).stop 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 shutdown_connections 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f56fc0a2150 0x7f56fc0a4610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5714102770 0x7f571419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5714100bd0 0x7f571406aed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 --2- 172.21.15.115:0/3999143107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5714101dd0 0x7f571419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 >> 172.21.15.115:0/3999143107 conn(0x7f57140fc170 msgr2=0x7f5714105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 shutdown_connections 2024-02-26T06:19:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-02-26T06:19:44.270+0000 7f5718cd5700 1 -- 172.21.15.115:0/3999143107 wait complete. 2024-02-26T06:19:44.979 DEBUG:teuthology.orchestra.run.smithi049:mds.a> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.a.service 2024-02-26T06:19:44.984 DEBUG:teuthology.orchestra.run.smithi049:mds.d> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.d.service 2024-02-26T06:19:45.029 DEBUG:teuthology.orchestra.run.smithi049:mds.g> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.g.service 2024-02-26T06:19:45.075 DEBUG:teuthology.orchestra.run.smithi049:mds.j> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.j.service 2024-02-26T06:19:45.121 DEBUG:teuthology.orchestra.run.smithi080:mds.b> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.b.service 2024-02-26T06:19:45.124 DEBUG:teuthology.orchestra.run.smithi080:mds.e> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.e.service 2024-02-26T06:19:45.168 DEBUG:teuthology.orchestra.run.smithi080:mds.h> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.h.service 2024-02-26T06:19:45.212 DEBUG:teuthology.orchestra.run.smithi080:mds.k> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.k.service 2024-02-26T06:19:45.255 DEBUG:teuthology.orchestra.run.smithi115:mds.c> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.c.service 2024-02-26T06:19:45.258 DEBUG:teuthology.orchestra.run.smithi115:mds.f> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.f.service 2024-02-26T06:19:45.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:45 smithi115 ceph-mon[36426]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:45.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:45.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:45 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:45.283 DEBUG:teuthology.orchestra.run.smithi115:mds.i> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.i.service 2024-02-26T06:19:45.325 DEBUG:teuthology.orchestra.run.smithi115:mds.l> sudo journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.l.service 2024-02-26T06:19:45.368 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-26T06:19:45.369 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:19:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:45 smithi049 ceph-mon[31696]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:45 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:45 smithi080 ceph-mon[36206]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:45 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:45.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 -- 172.21.15.49:0/1454633039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81089b0 msgr2=0x7f6af8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:45.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 --2- 172.21.15.49:0/1454633039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81089b0 0x7f6af8108e30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009900 tx=0x7f6ae402bc30 comp rx=0 tx=0).stop 2024-02-26T06:19:45.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 -- 172.21.15.49:0/1454633039 shutdown_connections 2024-02-26T06:19:45.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 --2- 172.21.15.49:0/1454633039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6af8109370 0x7f6af810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 --2- 172.21.15.49:0/1454633039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81089b0 0x7f6af8108e30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 --2- 172.21.15.49:0/1454633039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af81077c0 0x7f6af8107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 -- 172.21.15.49:0/1454633039 >> 172.21.15.49:0/1454633039 conn(0x7f6af8076790 msgr2=0x7f6af8078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:45.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 -- 172.21.15.49:0/1454633039 shutdown_connections 2024-02-26T06:19:45.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.634+0000 7f6affa00640 1 -- 172.21.15.49:0/1454633039 wait complete. 2024-02-26T06:19:45.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.635+0000 7f6affa00640 1 Processor -- start 2024-02-26T06:19:45.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.635+0000 7f6affa00640 1 -- start start 2024-02-26T06:19:45.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.635+0000 7f6affa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81077c0 0x7f6af819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:45.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.635+0000 7f6affa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af81089b0 0x7f6af819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:45.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.635+0000 7f6affa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6af8109370 0x7f6af81a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.635+0000 7f6affa00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81a5320 con 0x7f6af81089b0 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.635+0000 7f6affa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81a5490 con 0x7f6af8109370 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.635+0000 7f6affa00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81a5650 con 0x7f6af81077c0 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6af7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af81089b0 0x7f6af819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6afd000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81077c0 0x7f6af819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6af7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af81089b0 0x7f6af819dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35900/0 (socket says 172.21.15.49:35900) 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6af7e00640 1 -- 172.21.15.49:0/1664419029 learned_addr learned my addr 172.21.15.49:0/1664419029 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6afda00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6af8109370 0x7f6af81a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6afd000640 1 -- 172.21.15.49:0/1664419029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6af8109370 msgr2=0x7f6af81a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6afd000640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6af8109370 0x7f6af81a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6afd000640 1 -- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af81089b0 msgr2=0x7f6af819dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6afd000640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af81089b0 0x7f6af819dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6afd000640 1 -- 172.21.15.49:0/1664419029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec009490 con 0x7f6af81077c0 2024-02-26T06:19:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6af7e00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af81089b0 0x7f6af819dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:19:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6afd000640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81077c0 0x7f6af819d830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e9c0 tx=0x7f6aec00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.636+0000 7f6af5600640 1 -- 172.21.15.49:0/1664419029 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00cd80 con 0x7f6af81077c0 2024-02-26T06:19:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.637+0000 7f6af5600640 1 -- 172.21.15.49:0/1664419029 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6aec010910 con 0x7f6af81077c0 2024-02-26T06:19:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.637+0000 7f6af5600640 1 -- 172.21.15.49:0/1664419029 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec018900 con 0x7f6af81077c0 2024-02-26T06:19:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.637+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae4009560 con 0x7f6af81077c0 2024-02-26T06:19:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.637+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af81a5c60 con 0x7f6af81077c0 2024-02-26T06:19:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.638+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af810eae0 con 0x7f6af81077c0 2024-02-26T06:19:45.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.640+0000 7f6af5600640 1 -- 172.21.15.49:0/1664419029 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6aec010a80 con 0x7f6af81077c0 2024-02-26T06:19:45.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.641+0000 7f6af5600640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6ae00a2140 0x7f6ae00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:45.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.641+0000 7f6af5600640 1 -- 172.21.15.49:0/1664419029 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6aec014070 con 0x7f6af81077c0 2024-02-26T06:19:45.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.641+0000 7f6af7e00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6ae00a2140 0x7f6ae00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:45.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.642+0000 7f6af7e00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6ae00a2140 0x7f6ae00a4600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6af819ed40 tx=0x7f6ae402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:45.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.645+0000 7f6af5600640 1 -- 172.21.15.49:0/1664419029 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6aec0970b0 con 0x7f6af81077c0 2024-02-26T06:19:45.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.849+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6af802ce70 con 0x7f6af81077c0 2024-02-26T06:19:45.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.851+0000 7f6af5600640 1 -- 172.21.15.49:0/1664419029 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18443 (secure 0 0 0) 0x7f6aec09a770 con 0x7f6af81077c0 2024-02-26T06:19:45.852 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:45.852 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:19:34.587856+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6ae00a2140 msgr2=0x7f6ae00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6ae00a2140 0x7f6ae00a4600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6af819ed40 tx=0x7f6ae402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81077c0 msgr2=0x7f6af819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81077c0 0x7f6af819d830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e9c0 tx=0x7f6aec00ee80 comp rx=0 tx=0).stop 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 shutdown_connections 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6ae00a2140 0x7f6ae00a4600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6af8109370 0x7f6af81a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af81089b0 0x7f6af819dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 --2- 172.21.15.49:0/1664419029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af81077c0 0x7f6af819d830 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 >> 172.21.15.49:0/1664419029 conn(0x7f6af8076790 msgr2=0x7f6af810bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 shutdown_connections 2024-02-26T06:19:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:45.854+0000 7f6affa00640 1 -- 172.21.15.49:0/1664419029 wait complete. 2024-02-26T06:19:45.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:19:45.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:19:46.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.117+0000 7f0067200640 1 -- 172.21.15.49:0/2042551643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0068100c00 msgr2=0x7f0068101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.117+0000 7f0067200640 1 --2- 172.21.15.49:0/2042551643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0068100c00 0x7f0068101080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0060009be0 tx=0x7f006002bef0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.117+0000 7f0067200640 1 -- 172.21.15.49:0/2042551643 shutdown_connections 2024-02-26T06:19:46.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.117+0000 7f0067200640 1 --2- 172.21.15.49:0/2042551643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0068100c00 0x7f0068101080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.117+0000 7f0067200640 1 --2- 172.21.15.49:0/2042551643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100260 0x7f00681006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.117+0000 7f0067200640 1 --2- 172.21.15.49:0/2042551643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00680ff900 0x7f00680ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.117+0000 7f0067200640 1 -- 172.21.15.49:0/2042551643 >> 172.21.15.49:0/2042551643 conn(0x7f00680fb500 msgr2=0x7f00680fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:46.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.118+0000 7f0067200640 1 -- 172.21.15.49:0/2042551643 shutdown_connections 2024-02-26T06:19:46.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.118+0000 7f0067200640 1 -- 172.21.15.49:0/2042551643 wait complete. 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.119+0000 7f0067200640 1 Processor -- start 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.119+0000 7f0067200640 1 -- start start 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.119+0000 7f0067200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00680ff900 0x7f0068199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.119+0000 7f0067200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100260 0x7f00681997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.119+0000 7f0067200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0068100c00 0x7f00681a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.119+0000 7f0067200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681a0d80 con 0x7f0068100260 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.119+0000 7f0067200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681a0ef0 con 0x7f0068100c00 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.119+0000 7f0067200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681a10b0 con 0x7f00680ff900 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100260 0x7f00681997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100260 0x7f00681997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35914/0 (socket says 172.21.15.49:35914) 2024-02-26T06:19:46.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 -- 172.21.15.49:0/4114995045 learned_addr learned my addr 172.21.15.49:0/4114995045 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:46.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0066800640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0068100c00 0x7f00681a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:46.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065e00640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00680ff900 0x7f0068199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:46.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 -- 172.21.15.49:0/4114995045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00680ff900 msgr2=0x7f0068199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00680ff900 0x7f0068199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 -- 172.21.15.49:0/4114995045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0068100c00 msgr2=0x7f00681a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0068100c00 0x7f00681a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 -- 172.21.15.49:0/4114995045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0058009560 con 0x7f0068100260 2024-02-26T06:19:46.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.120+0000 7f0065400640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100260 0x7f00681997d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f005400ea60 tx=0x7f005400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:46.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.121+0000 7f005ea00640 1 -- 172.21.15.49:0/4114995045 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400cde0 con 0x7f0068100260 2024-02-26T06:19:46.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.121+0000 7f005ea00640 1 -- 172.21.15.49:0/4114995045 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00540035f0 con 0x7f0068100260 2024-02-26T06:19:46.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.121+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0054009490 con 0x7f0068100260 2024-02-26T06:19:46.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.121+0000 7f005ea00640 1 -- 172.21.15.49:0/4114995045 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054010690 con 0x7f0068100260 2024-02-26T06:19:46.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.121+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0060009840 con 0x7f0068100260 2024-02-26T06:19:46.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.122+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006810a760 con 0x7f0068100260 2024-02-26T06:19:46.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.124+0000 7f005ea00640 1 -- 172.21.15.49:0/4114995045 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0054003af0 con 0x7f0068100260 2024-02-26T06:19:46.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.125+0000 7f005ea00640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f004c0a2140 0x7f004c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:46.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.126+0000 7f005ea00640 1 -- 172.21.15.49:0/4114995045 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0054014070 con 0x7f0068100260 2024-02-26T06:19:46.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.126+0000 7f0065e00640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f004c0a2140 0x7f004c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:46.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.126+0000 7f0065e00640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f004c0a2140 0x7f004c0a4600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0058005fd0 tx=0x7f0058009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:46.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.130+0000 7f005ea00640 1 -- 172.21.15.49:0/4114995045 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00540970c0 con 0x7f0068100260 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:46.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:46 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1664419029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:19:46.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.332+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f006802cda0 con 0x7f0068100260 2024-02-26T06:19:46.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.334+0000 7f005ea00640 1 -- 172.21.15.49:0/4114995045 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f005409a780 con 0x7f0068100260 2024-02-26T06:19:46.334 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:19:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.336+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f004c0a2140 msgr2=0x7f004c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.336+0000 7f0067200640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f004c0a2140 0x7f004c0a4600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0058005fd0 tx=0x7f0058009260 comp rx=0 tx=0).stop 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.336+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100260 msgr2=0x7f00681997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.336+0000 7f0067200640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100260 0x7f00681997d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f005400ea60 tx=0x7f005400ef20 comp rx=0 tx=0).stop 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.337+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 shutdown_connections 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.337+0000 7f0067200640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f004c0a2140 0x7f004c0a4600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.337+0000 7f0067200640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0068100c00 0x7f00681a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.337+0000 7f0067200640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100260 0x7f00681997d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.337+0000 7f0067200640 1 --2- 172.21.15.49:0/4114995045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00680ff900 0x7f0068199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.337+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 >> 172.21.15.49:0/4114995045 conn(0x7f00680fb500 msgr2=0x7f00680fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.337+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 shutdown_connections 2024-02-26T06:19:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.337+0000 7f0067200640 1 -- 172.21.15.49:0/4114995045 wait complete. 2024-02-26T06:19:46.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:19:46.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:46.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2024-02-26T06:19:46.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:46.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:46.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:46.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:46.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:46.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:46.381 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:46 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1664419029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:19:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2024-02-26T06:19:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:46.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:46.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:46.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:46.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:46.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:46 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1664419029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:19:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.584+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1268649324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981031d0 msgr2=0x7f5f981035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.584+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1268649324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981031d0 0x7f5f981035f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009900 tx=0x7f5f8802bc30 comp rx=0 tx=0).stop 2024-02-26T06:19:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.584+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1268649324 shutdown_connections 2024-02-26T06:19:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.584+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1268649324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f98104d80 0x7f5f9810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.584+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1268649324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f981043c0 0x7f5f98104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.584+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1268649324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981031d0 0x7f5f981035f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.584+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1268649324 >> 172.21.15.49:0/1268649324 conn(0x7f5f980fe760 msgr2=0x7f5f98100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.584+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1268649324 shutdown_connections 2024-02-26T06:19:46.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.585+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1268649324 wait complete. 2024-02-26T06:19:46.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.586+0000 7f5f9ca00640 1 Processor -- start 2024-02-26T06:19:46.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.586+0000 7f5f9ca00640 1 -- start start 2024-02-26T06:19:46.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.586+0000 7f5f9ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f981031d0 0x7f5f981992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:46.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.586+0000 7f5f9ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981043c0 0x7f5f98199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:46.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.586+0000 7f5f9ca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f98104d80 0x7f5f981a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:46.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.586+0000 7f5f9ca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f981a0dd0 con 0x7f5f98104d80 2024-02-26T06:19:46.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.586+0000 7f5f9ca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f981a0f40 con 0x7f5f981043c0 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.586+0000 7f5f9ca00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f981a1100 con 0x7f5f981031d0 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f981031d0 0x7f5f981992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f981031d0 0x7f5f981992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39574/0 (socket says 172.21.15.49:39574) 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 -- 172.21.15.49:0/1411922992 learned_addr learned my addr 172.21.15.49:0/1411922992 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f97200640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f98104d80 0x7f5f981a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f95e00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981043c0 0x7f5f98199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 -- 172.21.15.49:0/1411922992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981043c0 msgr2=0x7f5f98199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981043c0 0x7f5f98199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 -- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f98104d80 msgr2=0x7f5f981a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f98104d80 0x7f5f981a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 -- 172.21.15.49:0/1411922992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f88009560 con 0x7f5f981031d0 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f97200640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f98104d80 0x7f5f981a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f95e00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981043c0 0x7f5f98199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:19:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.587+0000 7f5f96800640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f981031d0 0x7f5f981992e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f88000c00 tx=0x7f5f88005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:46.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.588+0000 7f5f8f400640 1 -- 172.21.15.49:0/1411922992 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f880043d0 con 0x7f5f981031d0 2024-02-26T06:19:46.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.588+0000 7f5f8f400640 1 -- 172.21.15.49:0/1411922992 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f880052d0 con 0x7f5f981031d0 2024-02-26T06:19:46.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.588+0000 7f5f8f400640 1 -- 172.21.15.49:0/1411922992 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88003d70 con 0x7f5f981031d0 2024-02-26T06:19:46.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.588+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f981a1380 con 0x7f5f981031d0 2024-02-26T06:19:46.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.588+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f981a1890 con 0x7f5f981031d0 2024-02-26T06:19:46.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.589+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9810a490 con 0x7f5f981031d0 2024-02-26T06:19:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.591+0000 7f5f8f400640 1 -- 172.21.15.49:0/1411922992 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5f88005440 con 0x7f5f981031d0 2024-02-26T06:19:46.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.592+0000 7f5f8f400640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5f7c0a20c0 0x7f5f7c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.592+0000 7f5f8f400640 1 -- 172.21.15.49:0/1411922992 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5f880f0270 con 0x7f5f981031d0 2024-02-26T06:19:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.592+0000 7f5f95e00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5f7c0a20c0 0x7f5f7c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.593+0000 7f5f95e00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5f7c0a20c0 0x7f5f7c0a4580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f9819a7f0 tx=0x7f5f84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:46.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.594+0000 7f5f8f400640 1 -- 172.21.15.49:0/1411922992 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f880b9640 con 0x7f5f981031d0 2024-02-26T06:19:46.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.835+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f9810ec30 con 0x7f5f981031d0 2024-02-26T06:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.835+0000 7f5f8f400640 1 -- 172.21.15.49:0/1411922992 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f5f880bcd00 con 0x7f5f981031d0 2024-02-26T06:19:46.836 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-26T06:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 1 2024-02-26T06:19:46.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5f7c0a20c0 msgr2=0x7f5f7c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5f7c0a20c0 0x7f5f7c0a4580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f9819a7f0 tx=0x7f5f84005f90 comp rx=0 tx=0).stop 2024-02-26T06:19:46.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f981031d0 msgr2=0x7f5f981992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f981031d0 0x7f5f981992e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f88000c00 tx=0x7f5f88005e40 comp rx=0 tx=0).stop 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 shutdown_connections 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5f7c0a20c0 0x7f5f7c0a4580 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f981043c0 0x7f5f98199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f98104d80 0x7f5f981a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 --2- 172.21.15.49:0/1411922992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f981031d0 0x7f5f981992e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.838+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 >> 172.21.15.49:0/1411922992 conn(0x7f5f980fe760 msgr2=0x7f5f98107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.839+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 shutdown_connections 2024-02-26T06:19:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:46.839+0000 7f5f9ca00640 1 -- 172.21.15.49:0/1411922992 wait complete. 2024-02-26T06:19:47.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:47 smithi049 ceph-mon[31696]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:47.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:47 smithi049 ceph-mon[31696]: Deploying daemon mds.l on smithi115 2024-02-26T06:19:47.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:47 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4114995045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:19:47.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:47 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1411922992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:47.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:47 smithi115 ceph-mon[36426]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:47.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:47 smithi115 ceph-mon[36426]: Deploying daemon mds.l on smithi115 2024-02-26T06:19:47.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:47 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4114995045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:19:47.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:47 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1411922992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:47 smithi080 ceph-mon[36206]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:47 smithi080 ceph-mon[36206]: Deploying daemon mds.l on smithi115 2024-02-26T06:19:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:47 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4114995045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:19:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:47 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1411922992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:47.841 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:19:47 smithi115 systemd[1]: Starting Ceph mds.l for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:19:48.116 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:19:47 smithi115 podman[66601]: 2024-02-26 06:19:47.764815943 +0000 UTC m=+0.017668723 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:19:48.117 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:19:47 smithi115 podman[66601]: 2024-02-26 06:19:47.862716282 +0000 UTC m=+0.115569037 container create 8283382f724e537d1ffd4d0ee81f6a4593ea5c3ae97d1fb1b2f266f95a332d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-26T06:19:48.117 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:19:48 smithi115 podman[66601]: 2024-02-26 06:19:48.090531543 +0000 UTC m=+0.343384308 container init 8283382f724e537d1ffd4d0ee81f6a4593ea5c3ae97d1fb1b2f266f95a332d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-26T06:19:48.117 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:19:48 smithi115 podman[66601]: 2024-02-26 06:19:48.092964336 +0000 UTC m=+0.345817094 container start 8283382f724e537d1ffd4d0ee81f6a4593ea5c3ae97d1fb1b2f266f95a332d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-26T06:19:48.590 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:19:48 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l[66616]: starting mds.l at 2024-02-26T06:19:48.590 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:19:48 smithi115 bash[66601]: 8283382f724e537d1ffd4d0ee81f6a4593ea5c3ae97d1fb1b2f266f95a332d82 2024-02-26T06:19:48.590 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:19:48 smithi115 systemd[1]: Started Ceph mds.l for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:19:48.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:19:49.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 -- 172.21.15.49:0/2646712339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4041043c0 msgr2=0x7fe404104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:49.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 --2- 172.21.15.49:0/2646712339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4041043c0 0x7fe404104840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009900 tx=0x7fe3f002bc50 comp rx=0 tx=0).stop 2024-02-26T06:19:49.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 -- 172.21.15.49:0/2646712339 shutdown_connections 2024-02-26T06:19:49.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 --2- 172.21.15.49:0/2646712339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe404104d80 0x7fe40410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 --2- 172.21.15.49:0/2646712339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4041043c0 0x7fe404104840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 --2- 172.21.15.49:0/2646712339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4041031d0 0x7fe4041035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 -- 172.21.15.49:0/2646712339 >> 172.21.15.49:0/2646712339 conn(0x7fe4040fe760 msgr2=0x7fe404100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:49.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 -- 172.21.15.49:0/2646712339 shutdown_connections 2024-02-26T06:19:49.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.104+0000 7fe40a800640 1 -- 172.21.15.49:0/2646712339 wait complete. 2024-02-26T06:19:49.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.105+0000 7fe40a800640 1 Processor -- start 2024-02-26T06:19:49.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.105+0000 7fe40a800640 1 -- start start 2024-02-26T06:19:49.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe40a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4041031d0 0x7fe404199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:49.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe40a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4041043c0 0x7fe404199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:49.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe40a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe404104d80 0x7fe4041a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:49.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe40a800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4041a0d20 con 0x7fe404104d80 2024-02-26T06:19:49.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe40a800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4041a0e90 con 0x7fe4041043c0 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe40a800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4041a1050 con 0x7fe4041031d0 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe409e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe404104d80 0x7fe4041a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe408a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4041043c0 0x7fe404199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe409e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe404104d80 0x7fe4041a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35960/0 (socket says 172.21.15.49:35960) 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe409e00640 1 -- 172.21.15.49:0/287833848 learned_addr learned my addr 172.21.15.49:0/287833848 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe409400640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4041031d0 0x7fe404199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe409e00640 1 -- 172.21.15.49:0/287833848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4041031d0 msgr2=0x7fe404199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe409e00640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4041031d0 0x7fe404199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.106+0000 7fe409e00640 1 -- 172.21.15.49:0/287833848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4041043c0 msgr2=0x7fe404199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe409e00640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4041043c0 0x7fe404199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe409e00640 1 -- 172.21.15.49:0/287833848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8009490 con 0x7fe404104d80 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe409400640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4041031d0 0x7fe404199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe408a00640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4041043c0 0x7fe404199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:19:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe409e00640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe404104d80 0x7fe4041a07e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00efd0 tx=0x7fe3fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:49.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe402000640 1 -- 172.21.15.49:0/287833848 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc009b40 con 0x7fe404104d80 2024-02-26T06:19:49.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc009840 con 0x7fe404104d80 2024-02-26T06:19:49.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe402000640 1 -- 172.21.15.49:0/287833848 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc003b70 con 0x7fe404104d80 2024-02-26T06:19:49.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe402000640 1 -- 172.21.15.49:0/287833848 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0108f0 con 0x7fe404104d80 2024-02-26T06:19:49.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.107+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f0009560 con 0x7fe404104d80 2024-02-26T06:19:49.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.108+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40410a4f0 con 0x7fe404104d80 2024-02-26T06:19:49.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.111+0000 7fe402000640 1 -- 172.21.15.49:0/287833848 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe3fc003ce0 con 0x7fe404104d80 2024-02-26T06:19:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.112+0000 7fe402000640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ec0a20c0 0x7fe3ec0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:49.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.112+0000 7fe402000640 1 -- 172.21.15.49:0/287833848 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3fc014070 con 0x7fe404104d80 2024-02-26T06:19:49.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.112+0000 7fe409400640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ec0a20c0 0x7fe3ec0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:49.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.113+0000 7fe409400640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ec0a20c0 0x7fe3ec0a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe404104220 tx=0x7fe3f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:49.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.116+0000 7fe402000640 1 -- 172.21.15.49:0/287833848 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3fc0971f0 con 0x7fe404104d80 2024-02-26T06:19:49.217 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:49.217 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6832/1451697651,v1:172.21.15.115:6833/1451697651] up:boot 2024-02-26T06:19:49.217 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: fsmap 1 up:standby 2024-02-26T06:19:49.217 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:19:49.217 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.218 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.218 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.218 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:49.218 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:49.218 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:49 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:49.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.353+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe40410ec30 con 0x7fe404104d80 2024-02-26T06:19:49.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.354+0000 7fe402000640 1 -- 172.21.15.49:0/287833848 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fe3fc09a8b0 con 0x7fe404104d80 2024-02-26T06:19:49.355 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:49.355 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T06:19:49.355 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-02-26T06:19:49.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.356+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ec0a20c0 msgr2=0x7fe3ec0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:49.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.356+0000 7fe40a800640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ec0a20c0 0x7fe3ec0a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe404104220 tx=0x7fe3f8005f90 comp rx=0 tx=0).stop 2024-02-26T06:19:49.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.356+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe404104d80 msgr2=0x7fe4041a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:49.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.356+0000 7fe40a800640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe404104d80 0x7fe4041a07e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00efd0 tx=0x7fe3fc00c6a0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.356+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 shutdown_connections 2024-02-26T06:19:49.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.356+0000 7fe40a800640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3ec0a20c0 0x7fe3ec0a4580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.356+0000 7fe40a800640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4041043c0 0x7fe404199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.357+0000 7fe40a800640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe404104d80 0x7fe4041a07e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.357+0000 7fe40a800640 1 --2- 172.21.15.49:0/287833848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4041031d0 0x7fe404199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:49.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.357+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 >> 172.21.15.49:0/287833848 conn(0x7fe4040fe760 msgr2=0x7fe4041076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:49.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.357+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 shutdown_connections 2024-02-26T06:19:49.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:49.357+0000 7fe40a800640 1 -- 172.21.15.49:0/287833848 wait complete. 2024-02-26T06:19:49.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:49.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6832/1451697651,v1:172.21.15.115:6833/1451697651] up:boot 2024-02-26T06:19:49.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: fsmap 1 up:standby 2024-02-26T06:19:49.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:19:49.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:49.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:49.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:49 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:49.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:49.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6832/1451697651,v1:172.21.15.115:6833/1451697651] up:boot 2024-02-26T06:19:49.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: fsmap 1 up:standby 2024-02-26T06:19:49.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:19:49.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:49.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:49.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:49.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:49 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:50.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:50 smithi080 ceph-mon[36206]: Deploying daemon mds.k on smithi080 2024-02-26T06:19:50.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:50 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/287833848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:50.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:50 smithi049 ceph-mon[31696]: Deploying daemon mds.k on smithi080 2024-02-26T06:19:50.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:50 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/287833848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:50.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:50 smithi115 ceph-mon[36426]: Deploying daemon mds.k on smithi080 2024-02-26T06:19:50.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:50 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/287833848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:50.784 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:50 smithi080 systemd[1]: Starting Ceph mds.k for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:19:51.049 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:50 smithi080 podman[65972]: 2024-02-26 06:19:50.687709896 +0000 UTC m=+0.017445736 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:19:51.050 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:50 smithi080 podman[65972]: 2024-02-26 06:19:50.80584777 +0000 UTC m=+0.135583591 container create b45a97587864e9b2149bfc9391c0b43c639deb0a63c40dfaecc498c2bf9e31ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-26T06:19:51.050 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:51 smithi080 podman[65972]: 2024-02-26 06:19:51.050289854 +0000 UTC m=+0.380025693 container init b45a97587864e9b2149bfc9391c0b43c639deb0a63c40dfaecc498c2bf9e31ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True) 2024-02-26T06:19:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:51 smithi080 ceph-mon[36206]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:51.330 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:51 smithi080 podman[65972]: 2024-02-26 06:19:51.053222278 +0000 UTC m=+0.382958109 container start b45a97587864e9b2149bfc9391c0b43c639deb0a63c40dfaecc498c2bf9e31ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS) 2024-02-26T06:19:51.330 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:51 smithi080 ceph-mds[65992]: -- 172.21.15.80:0/1668338542 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b556af21a0 con 0x55b556ac9000 2024-02-26T06:19:51.331 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:51 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k[65988]: starting mds.k at 2024-02-26T06:19:51.331 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:51 smithi080 bash[65972]: b45a97587864e9b2149bfc9391c0b43c639deb0a63c40dfaecc498c2bf9e31ce 2024-02-26T06:19:51.331 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:19:51 smithi080 systemd[1]: Started Ceph mds.k for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:19:51.375 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:19:51.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:51 smithi049 ceph-mon[31696]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.588+0000 7fb97e800640 1 -- 172.21.15.49:0/490847079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb978100390 msgr2=0x7fb9781007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.588+0000 7fb97e800640 1 --2- 172.21.15.49:0/490847079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb978100390 0x7fb9781007f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb964009900 tx=0x7fb96402bc30 comp rx=0 tx=0).stop 2024-02-26T06:19:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.589+0000 7fb97e800640 1 -- 172.21.15.49:0/490847079 shutdown_connections 2024-02-26T06:19:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.589+0000 7fb97e800640 1 --2- 172.21.15.49:0/490847079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb978100390 0x7fb9781007f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.589+0000 7fb97e800640 1 --2- 172.21.15.49:0/490847079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978100d30 0x7fb9781011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.589+0000 7fb97e800640 1 --2- 172.21.15.49:0/490847079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9780ffa30 0x7fb9780ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.589+0000 7fb97e800640 1 -- 172.21.15.49:0/490847079 >> 172.21.15.49:0/490847079 conn(0x7fb9780fb630 msgr2=0x7fb9780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.589+0000 7fb97e800640 1 -- 172.21.15.49:0/490847079 shutdown_connections 2024-02-26T06:19:51.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.589+0000 7fb97e800640 1 -- 172.21.15.49:0/490847079 wait complete. 2024-02-26T06:19:51.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:51 smithi115 ceph-mon[36426]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.590+0000 7fb97e800640 1 Processor -- start 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.590+0000 7fb97e800640 1 -- start start 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.590+0000 7fb97e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9780ffa30 0x7fb978199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.590+0000 7fb97e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb978100390 0x7fb978199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.590+0000 7fb97e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978100d30 0x7fb9781a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.590+0000 7fb97e800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9781a0f10 con 0x7fb978100d30 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.590+0000 7fb97e800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9781a1080 con 0x7fb978100390 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.590+0000 7fb97e800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9781a1240 con 0x7fb9780ffa30 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9780ffa30 0x7fb978199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9780ffa30 0x7fb978199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34014/0 (socket says 172.21.15.49:34014) 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978100d30 0x7fb9781a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 -- 172.21.15.49:0/1594837319 learned_addr learned my addr 172.21.15.49:0/1594837319 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:51.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb976a00640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb978100390 0x7fb978199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 -- 172.21.15.49:0/1594837319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb978100390 msgr2=0x7fb978199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb978100390 0x7fb978199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 -- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978100d30 msgr2=0x7fb9781a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978100d30 0x7fb9781a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 -- 172.21.15.49:0/1594837319 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb968009490 con 0x7fb9780ffa30 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.591+0000 7fb977400640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9780ffa30 0x7fb978199420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb96800ea60 tx=0x7fb96800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.592+0000 7fb96fe00640 1 -- 172.21.15.49:0/1594837319 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96800cde0 con 0x7fb9780ffa30 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.592+0000 7fb96fe00640 1 -- 172.21.15.49:0/1594837319 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9680035f0 con 0x7fb9780ffa30 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.592+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb964009560 con 0x7fb9780ffa30 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.592+0000 7fb96fe00640 1 -- 172.21.15.49:0/1594837319 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968010690 con 0x7fb9780ffa30 2024-02-26T06:19:51.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.592+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9781a1870 con 0x7fb9780ffa30 2024-02-26T06:19:51.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.593+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97810a8c0 con 0x7fb9780ffa30 2024-02-26T06:19:51.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.595+0000 7fb96fe00640 1 -- 172.21.15.49:0/1594837319 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb968004020 con 0x7fb9780ffa30 2024-02-26T06:19:51.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.596+0000 7fb96fe00640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb95c0a20c0 0x7fb95c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.596+0000 7fb96fe00640 1 -- 172.21.15.49:0/1594837319 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb968014070 con 0x7fb9780ffa30 2024-02-26T06:19:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.596+0000 7fb976a00640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb95c0a20c0 0x7fb95c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.597+0000 7fb976a00640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb95c0a20c0 0x7fb95c0a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c00 tx=0x7fb96402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:51.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.600+0000 7fb96fe00640 1 -- 172.21.15.49:0/1594837319 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb968097110 con 0x7fb9780ffa30 2024-02-26T06:19:51.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.841+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb97810ee50 con 0x7fb9780ffa30 2024-02-26T06:19:51.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.842+0000 7fb96fe00640 1 -- 172.21.15.49:0/1594837319 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7fb96809a7d0 con 0x7fb9780ffa30 2024-02-26T06:19:51.843 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:51.843 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T06:19:51.843 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-02-26T06:19:51.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.844+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb95c0a20c0 msgr2=0x7fb95c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:51.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.844+0000 7fb97e800640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb95c0a20c0 0x7fb95c0a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c00 tx=0x7fb96402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.844+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9780ffa30 msgr2=0x7fb978199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:51.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.844+0000 7fb97e800640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9780ffa30 0x7fb978199420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb96800ea60 tx=0x7fb96800ef20 comp rx=0 tx=0).stop 2024-02-26T06:19:51.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.845+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 shutdown_connections 2024-02-26T06:19:51.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.845+0000 7fb97e800640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb95c0a20c0 0x7fb95c0a4580 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.845+0000 7fb97e800640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb978100390 0x7fb978199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.845+0000 7fb97e800640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978100d30 0x7fb9781a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.845+0000 7fb97e800640 1 --2- 172.21.15.49:0/1594837319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9780ffa30 0x7fb978199420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:51.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.845+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 >> 172.21.15.49:0/1594837319 conn(0x7fb9780fb630 msgr2=0x7fb9780fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:51.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.845+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 shutdown_connections 2024-02-26T06:19:51.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:51.845+0000 7fb97e800640 1 -- 172.21.15.49:0/1594837319 wait complete. 2024-02-26T06:19:52.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.80:6832/914291308,v1:172.21.15.80:6833/914291308] up:boot 2024-02-26T06:19:52.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: fsmap 2 up:standby 2024-02-26T06:19:52.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: Deploying daemon mds.j on smithi049 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:52.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:52 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1594837319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:52.480 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.80:6832/914291308,v1:172.21.15.80:6833/914291308] up:boot 2024-02-26T06:19:52.480 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: fsmap 2 up:standby 2024-02-26T06:19:52.480 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: Deploying daemon mds.j on smithi049 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:52.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:52 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1594837319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.80:6832/914291308,v1:172.21.15.80:6833/914291308] up:boot 2024-02-26T06:19:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: fsmap 2 up:standby 2024-02-26T06:19:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: Deploying daemon mds.j on smithi049 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:52.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:52 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1594837319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:53.864 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:19:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.095+0000 7f47d5800640 1 -- 172.21.15.49:0/313822571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0102b90 msgr2=0x7f47d0109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.095+0000 7f47d5800640 1 --2- 172.21.15.49:0/313822571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0102b90 0x7f47d0109420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009900 tx=0x7f47c802bc30 comp rx=0 tx=0).stop 2024-02-26T06:19:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.096+0000 7f47d5800640 1 -- 172.21.15.49:0/313822571 shutdown_connections 2024-02-26T06:19:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.096+0000 7f47d5800640 1 --2- 172.21.15.49:0/313822571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0102b90 0x7f47d0109420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.096+0000 7f47d5800640 1 --2- 172.21.15.49:0/313822571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d01021d0 0x7f47d0102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.096+0000 7f47d5800640 1 --2- 172.21.15.49:0/313822571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d0100fe0 0x7f47d0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.096+0000 7f47d5800640 1 -- 172.21.15.49:0/313822571 >> 172.21.15.49:0/313822571 conn(0x7f47d00fc550 msgr2=0x7f47d00fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.096+0000 7f47d5800640 1 -- 172.21.15.49:0/313822571 shutdown_connections 2024-02-26T06:19:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.096+0000 7f47d5800640 1 -- 172.21.15.49:0/313822571 wait complete. 2024-02-26T06:19:54.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.097+0000 7f47d5800640 1 Processor -- start 2024-02-26T06:19:54.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.097+0000 7f47d5800640 1 -- start start 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.097+0000 7f47d5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0100fe0 0x7f47d01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.097+0000 7f47d5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d01021d0 0x7f47d01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.097+0000 7f47d5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d0102b90 0x7f47d01a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.097+0000 7f47d5800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d01a0d90 con 0x7f47d01021d0 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.097+0000 7f47d5800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d01a0f00 con 0x7f47d0102b90 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.097+0000 7f47d5800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d01a10c0 con 0x7f47d0100fe0 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0100fe0 0x7f47d01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0100fe0 0x7f47d01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34022/0 (socket says 172.21.15.49:34022) 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 -- 172.21.15.49:0/154792330 learned_addr learned my addr 172.21.15.49:0/154792330 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cf400640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d01021d0 0x7f47d01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47d4e00640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d0102b90 0x7f47d01a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 -- 172.21.15.49:0/154792330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d0102b90 msgr2=0x7f47d01a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:54.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d0102b90 0x7f47d01a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 -- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d01021d0 msgr2=0x7f47d01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d01021d0 0x7f47d01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 -- 172.21.15.49:0/154792330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8009560 con 0x7f47d0100fe0 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.098+0000 7f47cfe00640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0100fe0 0x7f47d01992a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47c000eb70 tx=0x7f47c000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.099+0000 7f47ccc00640 1 -- 172.21.15.49:0/154792330 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0003b70 con 0x7f47d0100fe0 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.099+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d01a13a0 con 0x7f47d0100fe0 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.099+0000 7f47ccc00640 1 -- 172.21.15.49:0/154792330 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47c000ee70 con 0x7f47d0100fe0 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.099+0000 7f47ccc00640 1 -- 172.21.15.49:0/154792330 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c00107f0 con 0x7f47d0100fe0 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.099+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d01a18b0 con 0x7f47d0100fe0 2024-02-26T06:19:54.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.100+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d010c9e0 con 0x7f47d0100fe0 2024-02-26T06:19:54.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.102+0000 7f47ccc00640 1 -- 172.21.15.49:0/154792330 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f47c0004020 con 0x7f47d0100fe0 2024-02-26T06:19:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.103+0000 7f47ccc00640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47b40a2140 0x7f47b40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.104+0000 7f47ccc00640 1 -- 172.21.15.49:0/154792330 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f47c0014070 con 0x7f47d0100fe0 2024-02-26T06:19:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.104+0000 7f47cf400640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47b40a2140 0x7f47b40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:54.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.104+0000 7f47cf400640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47b40a2140 0x7f47b40a4600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47d019a7b0 tx=0x7f47b800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:54.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.108+0000 7f47ccc00640 1 -- 172.21.15.49:0/154792330 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47c00971a0 con 0x7f47d0100fe0 2024-02-26T06:19:54.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.358+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47d010cba0 con 0x7f47d0100fe0 2024-02-26T06:19:54.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.359+0000 7f47ccc00640 1 -- 172.21.15.49:0/154792330 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7f47c009a860 con 0x7f47d0100fe0 2024-02-26T06:19:54.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:54.360 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T06:19:54.360 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-02-26T06:19:54.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.361+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47b40a2140 msgr2=0x7f47b40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:54.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.361+0000 7f47d5800640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47b40a2140 0x7f47b40a4600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47d019a7b0 tx=0x7f47b800b390 comp rx=0 tx=0).stop 2024-02-26T06:19:54.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.361+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0100fe0 msgr2=0x7f47d01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.361+0000 7f47d5800640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0100fe0 0x7f47d01992a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47c000eb70 tx=0x7f47c000c6a0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.362+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 shutdown_connections 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.362+0000 7f47d5800640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47b40a2140 0x7f47b40a4600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.362+0000 7f47d5800640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d0102b90 0x7f47d01a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.362+0000 7f47d5800640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d01021d0 0x7f47d01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.362+0000 7f47d5800640 1 --2- 172.21.15.49:0/154792330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47d0100fe0 0x7f47d01992a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.362+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 >> 172.21.15.49:0/154792330 conn(0x7f47d00fc550 msgr2=0x7f47d00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.362+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 shutdown_connections 2024-02-26T06:19:54.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:54.362+0000 7f47d5800640 1 -- 172.21.15.49:0/154792330 wait complete. 2024-02-26T06:19:54.642 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:54 smithi049 ceph-mon[31696]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:54.642 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:54 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/154792330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:54.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:54 smithi080 ceph-mon[36206]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:54.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:54 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/154792330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:54 smithi115 ceph-mon[36426]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:54 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/154792330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:54.920 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:19:54 smithi049 podman[69426]: 2024-02-26 06:19:54.687926104 +0000 UTC m=+0.152098604 container create 2c863d7af9000c504a9d492dd23997034604615a448ba7633958633997a26500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-26T06:19:55.420 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:19:54 smithi049 podman[69426]: 2024-02-26 06:19:54.991714172 +0000 UTC m=+0.455886686 container init 2c863d7af9000c504a9d492dd23997034604615a448ba7633958633997a26500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-26T06:19:55.420 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:19:55 smithi049 podman[69426]: 2024-02-26 06:19:55.001647592 +0000 UTC m=+0.465820067 container start 2c863d7af9000c504a9d492dd23997034604615a448ba7633958633997a26500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:19:55.420 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:19:55 smithi049 ceph-mds[69453]: -- 172.21.15.49:0/551633618 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8a6e541a0 con 0x55a8a6e2a800 2024-02-26T06:19:55.420 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:19:55 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j[69444]: starting mds.j at 2024-02-26T06:19:55.420 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:19:55 smithi049 bash[69426]: 2c863d7af9000c504a9d492dd23997034604615a448ba7633958633997a26500 2024-02-26T06:19:55.420 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:19:55 smithi049 systemd[1]: Started Ceph mds.j for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: Deploying daemon mds.d on smithi049 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.49:6834/180447409,v1:172.21.15.49:6835/180447409] up:boot 2024-02-26T06:19:56.233 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: fsmap 3 up:standby 2024-02-26T06:19:56.234 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:56 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:19:56.385 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:19:56.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:56.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:56.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:56.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: Deploying daemon mds.d on smithi049 2024-02-26T06:19:56.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:56.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.49:6834/180447409,v1:172.21.15.49:6835/180447409] up:boot 2024-02-26T06:19:56.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: fsmap 3 up:standby 2024-02-26T06:19:56.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:56 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:19:56.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:19:56.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:19:56.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:19:56.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:56.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: Deploying daemon mds.d on smithi049 2024-02-26T06:19:56.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:56.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.49:6834/180447409,v1:172.21.15.49:6835/180447409] up:boot 2024-02-26T06:19:56.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: fsmap 3 up:standby 2024-02-26T06:19:56.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:56 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 -- 172.21.15.49:0/912468781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 msgr2=0x7fd498103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 --2- 172.21.15.49:0/912468781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 0x7fd498103e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd488009900 tx=0x7fd48802bc40 comp rx=0 tx=0).stop 2024-02-26T06:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 -- 172.21.15.49:0/912468781 shutdown_connections 2024-02-26T06:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 --2- 172.21.15.49:0/912468781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd498104380 0x7fd498106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 --2- 172.21.15.49:0/912468781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd498106cb0 0x7fd498109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 --2- 172.21.15.49:0/912468781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 0x7fd498103e40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 -- 172.21.15.49:0/912468781 >> 172.21.15.49:0/912468781 conn(0x7fd4980fb460 msgr2=0x7fd4980fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 -- 172.21.15.49:0/912468781 shutdown_connections 2024-02-26T06:19:56.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.619+0000 7fd49da00640 1 -- 172.21.15.49:0/912468781 wait complete. 2024-02-26T06:19:56.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.620+0000 7fd49da00640 1 Processor -- start 2024-02-26T06:19:56.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd49da00640 1 -- start start 2024-02-26T06:19:56.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd49da00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 0x7fd4981970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd49da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd498104380 0x7fd4981975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd49da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd498106cb0 0x7fd49819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd49da00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49819eba0 con 0x7fd498101a50 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd49da00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49819ed10 con 0x7fd498104380 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd49da00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49819eed0 con 0x7fd498106cb0 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd497e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 0x7fd4981970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd497e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 0x7fd4981970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60268/0 (socket says 172.21.15.49:60268) 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd497e00640 1 -- 172.21.15.49:0/3611938017 learned_addr learned my addr 172.21.15.49:0/3611938017 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd497400640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd498104380 0x7fd4981975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.621+0000 7fd49d000640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd498106cb0 0x7fd49819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd497e00640 1 -- 172.21.15.49:0/3611938017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd498106cb0 msgr2=0x7fd49819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd497e00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd498106cb0 0x7fd49819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd497e00640 1 -- 172.21.15.49:0/3611938017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd498104380 msgr2=0x7fd4981975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd497e00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd498104380 0x7fd4981975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd497e00640 1 -- 172.21.15.49:0/3611938017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484009490 con 0x7fd498101a50 2024-02-26T06:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd49d000640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd498106cb0 0x7fd49819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd497e00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 0x7fd4981970b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd488006010 tx=0x7fd488005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd494c00640 1 -- 172.21.15.49:0/3611938017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4880043d0 con 0x7fd498101a50 2024-02-26T06:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.622+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488009560 con 0x7fd498101a50 2024-02-26T06:19:56.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.623+0000 7fd494c00640 1 -- 172.21.15.49:0/3611938017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4880052d0 con 0x7fd498101a50 2024-02-26T06:19:56.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.623+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49819f520 con 0x7fd498101a50 2024-02-26T06:19:56.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.623+0000 7fd494c00640 1 -- 172.21.15.49:0/3611938017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488003cc0 con 0x7fd498101a50 2024-02-26T06:19:56.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.624+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4780052c0 con 0x7fd498101a50 2024-02-26T06:19:56.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.626+0000 7fd494c00640 1 -- 172.21.15.49:0/3611938017 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd488005440 con 0x7fd498101a50 2024-02-26T06:19:56.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.627+0000 7fd494c00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd47c0a20c0 0x7fd47c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:56.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.627+0000 7fd494c00640 1 -- 172.21.15.49:0/3611938017 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd4880efa10 con 0x7fd498101a50 2024-02-26T06:19:56.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.628+0000 7fd497400640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd47c0a20c0 0x7fd47c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:56.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.628+0000 7fd497400640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd47c0a20c0 0x7fd47c0a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4981985c0 tx=0x7fd484005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:56.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.632+0000 7fd494c00640 1 -- 172.21.15.49:0/3611938017 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4880b8de0 con 0x7fd498101a50 2024-02-26T06:19:56.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.873+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd478005af0 con 0x7fd498101a50 2024-02-26T06:19:56.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.875+0000 7fd494c00640 1 -- 172.21.15.49:0/3611938017 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2904 (secure 0 0 0) 0x7fd4880bc4a0 con 0x7fd498101a50 2024-02-26T06:19:56.875 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:56.876 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T06:19:56.876 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 4 2024-02-26T06:19:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd47c0a20c0 msgr2=0x7fd47c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd47c0a20c0 0x7fd47c0a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4981985c0 tx=0x7fd484005f90 comp rx=0 tx=0).stop 2024-02-26T06:19:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 msgr2=0x7fd4981970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 0x7fd4981970b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd488006010 tx=0x7fd488005e80 comp rx=0 tx=0).stop 2024-02-26T06:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 shutdown_connections 2024-02-26T06:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd47c0a20c0 0x7fd47c0a4580 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd498104380 0x7fd4981975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd498106cb0 0x7fd49819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 --2- 172.21.15.49:0/3611938017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd498101a50 0x7fd4981970b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.877+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 >> 172.21.15.49:0/3611938017 conn(0x7fd4980fb460 msgr2=0x7fd498102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.878+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 shutdown_connections 2024-02-26T06:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:56.878+0000 7fd49da00640 1 -- 172.21.15.49:0/3611938017 wait complete. 2024-02-26T06:19:57.498 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:57 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3611938017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:57.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:57 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3611938017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:57.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:57 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3611938017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:19:58.364 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:19:58 smithi049 ceph-mon[31696]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:58.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:19:58 smithi080 ceph-mon[36206]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:58.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:19:58 smithi115 ceph-mon[36426]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:19:58.669 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:58 smithi049 systemd[1]: Starting Ceph mds.d for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:19:58.899 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:19:59.070 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:58 smithi049 podman[69777]: 2024-02-26 06:19:58.65475344 +0000 UTC m=+0.040306803 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:19:59.070 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:58 smithi049 podman[69777]: 2024-02-26 06:19:58.774402481 +0000 UTC m=+0.159955870 container create 7ce57060d4c94145dc37d7e323470bdbae1af8f491278605ea32098fd383bf25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T06:19:59.070 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:59 smithi049 podman[69777]: 2024-02-26 06:19:59.069756177 +0000 UTC m=+0.455309583 container init 7ce57060d4c94145dc37d7e323470bdbae1af8f491278605ea32098fd383bf25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-26T06:19:59.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.129+0000 7fb827e00640 1 -- 172.21.15.49:0/4236587525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820102d50 msgr2=0x7fb8201095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:59.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.129+0000 7fb827e00640 1 --2- 172.21.15.49:0/4236587525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820102d50 0x7fb8201095e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb818009ca0 tx=0x7fb81802c100 comp rx=0 tx=0).stop 2024-02-26T06:19:59.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.129+0000 7fb827e00640 1 -- 172.21.15.49:0/4236587525 shutdown_connections 2024-02-26T06:19:59.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.129+0000 7fb827e00640 1 --2- 172.21.15.49:0/4236587525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820102d50 0x7fb8201095e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.129+0000 7fb827e00640 1 --2- 172.21.15.49:0/4236587525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820102390 0x7fb820102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.129+0000 7fb827e00640 1 --2- 172.21.15.49:0/4236587525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8201011a0 0x7fb8201015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.129+0000 7fb827e00640 1 -- 172.21.15.49:0/4236587525 >> 172.21.15.49:0/4236587525 conn(0x7fb8200fc730 msgr2=0x7fb8200feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:59.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.129+0000 7fb827e00640 1 -- 172.21.15.49:0/4236587525 shutdown_connections 2024-02-26T06:19:59.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.130+0000 7fb827e00640 1 -- 172.21.15.49:0/4236587525 wait complete. 2024-02-26T06:19:59.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb827e00640 1 Processor -- start 2024-02-26T06:19:59.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb827e00640 1 -- start start 2024-02-26T06:19:59.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb827e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8201011a0 0x7fb8201993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb827e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820102390 0x7fb8201998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb827e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820102d50 0x7fb8201a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb827e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a0ea0 con 0x7fb820102390 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb827e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a1010 con 0x7fb820102d50 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb827e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a11d0 con 0x7fb8201011a0 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.131+0000 7fb824a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820102390 0x7fb8201998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8201011a0 0x7fb8201993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8201011a0 0x7fb8201993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34090/0 (socket says 172.21.15.49:34090) 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820102d50 0x7fb8201a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 -- 172.21.15.49:0/1697029225 learned_addr learned my addr 172.21.15.49:0/1697029225 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 -- 172.21.15.49:0/1697029225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820102d50 msgr2=0x7fb8201a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820102d50 0x7fb8201a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 -- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820102390 msgr2=0x7fb8201998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:59.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820102390 0x7fb8201998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 -- 172.21.15.49:0/1697029225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb818009900 con 0x7fb8201011a0 2024-02-26T06:19:59.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.132+0000 7fb825400640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8201011a0 0x7fb8201993b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb81400d950 tx=0x7fb81400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:59.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.133+0000 7fb81e000640 1 -- 172.21.15.49:0/1697029225 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814009c30 con 0x7fb8201011a0 2024-02-26T06:19:59.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.133+0000 7fb81e000640 1 -- 172.21.15.49:0/1697029225 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb814009d90 con 0x7fb8201011a0 2024-02-26T06:19:59.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.133+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8201a14b0 con 0x7fb8201011a0 2024-02-26T06:19:59.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.133+0000 7fb81e000640 1 -- 172.21.15.49:0/1697029225 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81400f880 con 0x7fb8201011a0 2024-02-26T06:19:59.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.133+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8201a1a10 con 0x7fb8201011a0 2024-02-26T06:19:59.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.134+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82010cbb0 con 0x7fb8201011a0 2024-02-26T06:19:59.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.136+0000 7fb81e000640 1 -- 172.21.15.49:0/1697029225 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb81400f9e0 con 0x7fb8201011a0 2024-02-26T06:19:59.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.137+0000 7fb81e000640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8080a2140 0x7fb8080a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:59.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.138+0000 7fb81e000640 1 -- 172.21.15.49:0/1697029225 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb8140cd950 con 0x7fb8201011a0 2024-02-26T06:19:59.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.138+0000 7fb824a00640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8080a2140 0x7fb8080a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:59.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.138+0000 7fb824a00640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8080a2140 0x7fb8080a4600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb82019a8c0 tx=0x7fb80c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:59.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.142+0000 7fb81e000640 1 -- 172.21.15.49:0/1697029225 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb814096dc0 con 0x7fb8201011a0 2024-02-26T06:19:59.420 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:59 smithi049 podman[69777]: 2024-02-26 06:19:59.080872457 +0000 UTC m=+0.466425824 container start 7ce57060d4c94145dc37d7e323470bdbae1af8f491278605ea32098fd383bf25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:19:59.420 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:59 smithi049 ceph-mds[69816]: -- 172.21.15.49:0/3183423691 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b78025c1a0 con 0x55b780232c00 2024-02-26T06:19:59.420 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:59 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d[69793]: starting mds.d at 2024-02-26T06:19:59.420 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:59 smithi049 bash[69777]: 7ce57060d4c94145dc37d7e323470bdbae1af8f491278605ea32098fd383bf25 2024-02-26T06:19:59.420 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:19:59 smithi049 systemd[1]: Started Ceph mds.d for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:19:59.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.420+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8201a1c90 con 0x7fb8201011a0 2024-02-26T06:19:59.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.421+0000 7fb81e000640 1 -- 172.21.15.49:0/1697029225 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3692 (secure 0 0 0) 0x7fb81409a480 con 0x7fb8201011a0 2024-02-26T06:19:59.422 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:19:59.422 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-02-26T06:19:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 5 2024-02-26T06:19:59.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8080a2140 msgr2=0x7fb8080a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8080a2140 0x7fb8080a4600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb82019a8c0 tx=0x7fb80c005f90 comp rx=0 tx=0).stop 2024-02-26T06:19:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8201011a0 msgr2=0x7fb8201993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8201011a0 0x7fb8201993b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb81400d950 tx=0x7fb81400de10 comp rx=0 tx=0).stop 2024-02-26T06:19:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 shutdown_connections 2024-02-26T06:19:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8080a2140 0x7fb8080a4600 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820102d50 0x7fb8201a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820102390 0x7fb8201998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 --2- 172.21.15.49:0/1697029225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8201011a0 0x7fb8201993b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 >> 172.21.15.49:0/1697029225 conn(0x7fb8200fc730 msgr2=0x7fb8200733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.424+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 shutdown_connections 2024-02-26T06:19:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:19:59.425+0000 7fb827e00640 1 -- 172.21.15.49:0/1697029225 wait complete. 2024-02-26T06:20:00.269 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.49:6836/879820521,v1:172.21.15.49:6837/879820521] up:boot 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: fsmap 4 up:standby 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: Deploying daemon mds.e on smithi080 2024-02-26T06:20:00.270 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:00.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1697029225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:00.271 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:00 smithi080 ceph-mon[36206]: overall HEALTH_OK 2024-02-26T06:20:00.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.49:6836/879820521,v1:172.21.15.49:6837/879820521] up:boot 2024-02-26T06:20:00.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: fsmap 4 up:standby 2024-02-26T06:20:00.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: Deploying daemon mds.e on smithi080 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1697029225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:00.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:00 smithi115 ceph-mon[36426]: overall HEALTH_OK 2024-02-26T06:20:00.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.49:6836/879820521,v1:172.21.15.49:6837/879820521] up:boot 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: fsmap 4 up:standby 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: Deploying daemon mds.e on smithi080 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1697029225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:00.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:00 smithi049 ceph-mon[31696]: overall HEALTH_OK 2024-02-26T06:20:01.444 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 -- 172.21.15.49:0/721258987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 msgr2=0x7f51601013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 --2- 172.21.15.49:0/721258987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 0x7f51601013e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5154009900 tx=0x7f515402bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 -- 172.21.15.49:0/721258987 shutdown_connections 2024-02-26T06:20:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 --2- 172.21.15.49:0/721258987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5160102b70 0x7f5160109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 --2- 172.21.15.49:0/721258987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51601021b0 0x7f5160102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 --2- 172.21.15.49:0/721258987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 0x7f51601013e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 -- 172.21.15.49:0/721258987 >> 172.21.15.49:0/721258987 conn(0x7f51600fc510 msgr2=0x7f51600fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 -- 172.21.15.49:0/721258987 shutdown_connections 2024-02-26T06:20:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.679+0000 7f5167200640 1 -- 172.21.15.49:0/721258987 wait complete. 2024-02-26T06:20:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.680+0000 7f5167200640 1 Processor -- start 2024-02-26T06:20:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.680+0000 7f5167200640 1 -- start start 2024-02-26T06:20:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5167200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 0x7f51601992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5167200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51601021b0 0x7f5160199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5167200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5160102b70 0x7f51601a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5167200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51601a0de0 con 0x7f5160102b70 2024-02-26T06:20:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5167200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51601a0f50 con 0x7f5160100fc0 2024-02-26T06:20:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5167200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51601a1110 con 0x7f51601021b0 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5165e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 0x7f51601992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5165e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 0x7f51601992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57980/0 (socket says 172.21.15.49:57980) 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5165e00640 1 -- 172.21.15.49:0/3751627387 learned_addr learned my addr 172.21.15.49:0/3751627387 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5166800640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5160102b70 0x7f51601a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5165400640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51601021b0 0x7f5160199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5165e00640 1 -- 172.21.15.49:0/3751627387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51601021b0 msgr2=0x7f5160199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.681+0000 7f5165e00640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51601021b0 0x7f5160199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f5165e00640 1 -- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5160102b70 msgr2=0x7f51601a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f5165e00640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5160102b70 0x7f51601a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f5165e00640 1 -- 172.21.15.49:0/3751627387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154009560 con 0x7f5160100fc0 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f5165e00640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 0x7f51601992f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f51540040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f515aa00640 1 -- 172.21.15.49:0/3751627387 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51540044c0 con 0x7f5160100fc0 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f515aa00640 1 -- 172.21.15.49:0/3751627387 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51540052d0 con 0x7f5160100fc0 2024-02-26T06:20:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f515aa00640 1 -- 172.21.15.49:0/3751627387 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154003e80 con 0x7f5160100fc0 2024-02-26T06:20:01.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51601a1390 con 0x7f5160100fc0 2024-02-26T06:20:01.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.682+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51601a18f0 con 0x7f5160100fc0 2024-02-26T06:20:01.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.683+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516010c9e0 con 0x7f5160100fc0 2024-02-26T06:20:01.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.685+0000 7f515aa00640 1 -- 172.21.15.49:0/3751627387 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5154005440 con 0x7f5160100fc0 2024-02-26T06:20:01.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.686+0000 7f515aa00640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f51480a2140 0x7f51480a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:01.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.687+0000 7f515aa00640 1 -- 172.21.15.49:0/3751627387 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f51540f0060 con 0x7f5160100fc0 2024-02-26T06:20:01.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.687+0000 7f5165400640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f51480a2140 0x7f51480a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:01.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.687+0000 7f5165400640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f51480a2140 0x7f51480a4600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f516019a800 tx=0x7f5150005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:01.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.691+0000 7f515aa00640 1 -- 172.21.15.49:0/3751627387 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51540b9430 con 0x7f5160100fc0 2024-02-26T06:20:01.830 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:01 smithi080 systemd[1]: Starting Ceph mds.e for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:20:01.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.931+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f516010cbc0 con 0x7f5160100fc0 2024-02-26T06:20:01.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.932+0000 7f515aa00640 1 -- 172.21.15.49:0/3751627387 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3692 (secure 0 0 0) 0x7f51540bcaf0 con 0x7f5160100fc0 2024-02-26T06:20:01.933 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:01.933 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-02-26T06:20:01.933 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 5 2024-02-26T06:20:01.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f51480a2140 msgr2=0x7f51480a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:01.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f51480a2140 0x7f51480a4600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f516019a800 tx=0x7f5150005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:01.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 msgr2=0x7f51601992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:01.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 0x7f51601992f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f51540040c0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 shutdown_connections 2024-02-26T06:20:01.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f51480a2140 0x7f51480a4600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5160102b70 0x7f51601a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51601021b0 0x7f5160199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 --2- 172.21.15.49:0/3751627387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5160100fc0 0x7f51601992f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 >> 172.21.15.49:0/3751627387 conn(0x7f51600fc510 msgr2=0x7f51600733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 shutdown_connections 2024-02-26T06:20:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:01.934+0000 7f5167200640 1 -- 172.21.15.49:0/3751627387 wait complete. 2024-02-26T06:20:02.212 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:01 smithi080 podman[66293]: 2024-02-26 06:20:01.831909398 +0000 UTC m=+0.017580851 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:20:02.213 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:01 smithi080 podman[66293]: 2024-02-26 06:20:01.952283097 +0000 UTC m=+0.137954528 container create 24588b772ecb9464dbf3e04fc26e80da7524401881e258df9f9b3bafceb1f8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:20:02.472 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:02 smithi080 podman[66293]: 2024-02-26 06:20:02.213511931 +0000 UTC m=+0.399183370 container init 24588b772ecb9464dbf3e04fc26e80da7524401881e258df9f9b3bafceb1f8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-26T06:20:02.472 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:02 smithi080 podman[66293]: 2024-02-26 06:20:02.216081392 +0000 UTC m=+0.401752832 container start 24588b772ecb9464dbf3e04fc26e80da7524401881e258df9f9b3bafceb1f8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:20:02.472 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mds[66312]: -- 172.21.15.80:0/4170264110 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0825501a0 con 0x55d082526c00 2024-02-26T06:20:02.472 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e[66308]: starting mds.e at 2024-02-26T06:20:02.472 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:02 smithi080 bash[66293]: 24588b772ecb9464dbf3e04fc26e80da7524401881e258df9f9b3bafceb1f8a7 2024-02-26T06:20:02.472 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:20:02 smithi080 systemd[1]: Started Ceph mds.e for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:20:02.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mon[36206]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:02.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3751627387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:02.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:02.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:02.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:02.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:02 smithi115 ceph-mon[36426]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:02.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:02 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3751627387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:02.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:02.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:02.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:02.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:02 smithi049 ceph-mon[31696]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:02.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:02 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3751627387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:02.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:02.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:02.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:02.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:03.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:03 smithi115 ceph-mon[36426]: Deploying daemon mds.f on smithi115 2024-02-26T06:20:03.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:03 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.80:6834/1885591642,v1:172.21.15.80:6835/1885591642] up:boot 2024-02-26T06:20:03.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:03 smithi115 ceph-mon[36426]: fsmap 5 up:standby 2024-02-26T06:20:03.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:03 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:20:03.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:03 smithi080 ceph-mon[36206]: Deploying daemon mds.f on smithi115 2024-02-26T06:20:03.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:03 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.80:6834/1885591642,v1:172.21.15.80:6835/1885591642] up:boot 2024-02-26T06:20:03.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:03 smithi080 ceph-mon[36206]: fsmap 5 up:standby 2024-02-26T06:20:03.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:03 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:20:03.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:03 smithi049 ceph-mon[31696]: Deploying daemon mds.f on smithi115 2024-02-26T06:20:03.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:03 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.80:6834/1885591642,v1:172.21.15.80:6835/1885591642] up:boot 2024-02-26T06:20:03.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:03 smithi049 ceph-mon[31696]: fsmap 5 up:standby 2024-02-26T06:20:03.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:20:03.954 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 -- 172.21.15.49:0/4140558686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 msgr2=0x7f3b34104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4140558686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 0x7f3b34104840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009900 tx=0x7f3b2002bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 -- 172.21.15.49:0/4140558686 shutdown_connections 2024-02-26T06:20:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4140558686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b34104d80 0x7f3b3410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4140558686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 0x7f3b34104840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4140558686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b341031d0 0x7f3b341035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 -- 172.21.15.49:0/4140558686 >> 172.21.15.49:0/4140558686 conn(0x7f3b340fe760 msgr2=0x7f3b34100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 -- 172.21.15.49:0/4140558686 shutdown_connections 2024-02-26T06:20:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.195+0000 7f3b33e00640 1 -- 172.21.15.49:0/4140558686 wait complete. 2024-02-26T06:20:04.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.196+0000 7f3b33e00640 1 Processor -- start 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.196+0000 7f3b33e00640 1 -- start start 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b341031d0 0x7f3b34199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 0x7f3b34199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b34104d80 0x7f3b341a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b33e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341a0d20 con 0x7f3b34104d80 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b33e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341a0e90 con 0x7f3b341031d0 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b33e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341a1050 con 0x7f3b341043c0 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 0x7f3b34199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 0x7f3b34199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47184/0 (socket says 172.21.15.49:47184) 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32000640 1 -- 172.21.15.49:0/4107681524 learned_addr learned my addr 172.21.15.49:0/4107681524 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b33400640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b34104d80 0x7f3b341a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32a00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b341031d0 0x7f3b34199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32000640 1 -- 172.21.15.49:0/4107681524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b341031d0 msgr2=0x7f3b34199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32000640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b341031d0 0x7f3b34199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32000640 1 -- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b34104d80 msgr2=0x7f3b341a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32000640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b34104d80 0x7f3b341a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32000640 1 -- 172.21.15.49:0/4107681524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20009560 con 0x7f3b341043c0 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b33400640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b34104d80 0x7f3b341a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.197+0000 7f3b32a00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b341031d0 0x7f3b34199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.198+0000 7f3b32000640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 0x7f3b34199770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3b2002c6a0 tx=0x7f3b200040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.198+0000 7f3b2b400640 1 -- 172.21.15.49:0/4107681524 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200044c0 con 0x7f3b341043c0 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.198+0000 7f3b2b400640 1 -- 172.21.15.49:0/4107681524 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b200052d0 con 0x7f3b341043c0 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.198+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b341a12d0 con 0x7f3b341043c0 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.198+0000 7f3b2b400640 1 -- 172.21.15.49:0/4107681524 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20003e80 con 0x7f3b341043c0 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.198+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b341a1830 con 0x7f3b341043c0 2024-02-26T06:20:04.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.199+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3410a490 con 0x7f3b341043c0 2024-02-26T06:20:04.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.201+0000 7f3b2b400640 1 -- 172.21.15.49:0/4107681524 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3b20005440 con 0x7f3b341043c0 2024-02-26T06:20:04.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.202+0000 7f3b2b400640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3b180a2140 0x7f3b180a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:04.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.203+0000 7f3b2b400640 1 -- 172.21.15.49:0/4107681524 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3b200f0350 con 0x7f3b341043c0 2024-02-26T06:20:04.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.203+0000 7f3b32a00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3b180a2140 0x7f3b180a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:04.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.203+0000 7f3b32a00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3b180a2140 0x7f3b180a4600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b34104220 tx=0x7f3b24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.205+0000 7f3b2b400640 1 -- 172.21.15.49:0/4107681524 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b200b9840 con 0x7f3b341043c0 2024-02-26T06:20:04.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.446+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b3410ee80 con 0x7f3b341043c0 2024-02-26T06:20:04.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.447+0000 7f3b2b400640 1 -- 172.21.15.49:0/4107681524 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f3b20043090 con 0x7f3b341043c0 2024-02-26T06:20:04.447 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:04.447 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-26T06:20:04.447 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 6 2024-02-26T06:20:04.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.449+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3b180a2140 msgr2=0x7f3b180a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:04.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.449+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3b180a2140 0x7f3b180a4600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b34104220 tx=0x7f3b24005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.449+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 msgr2=0x7f3b34199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.449+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 0x7f3b34199770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3b2002c6a0 tx=0x7f3b200040c0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.450+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 shutdown_connections 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.450+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3b180a2140 0x7f3b180a4600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.450+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b34104d80 0x7f3b341a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.450+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b341043c0 0x7f3b34199770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.450+0000 7f3b33e00640 1 --2- 172.21.15.49:0/4107681524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b341031d0 0x7f3b34199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.450+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 >> 172.21.15.49:0/4107681524 conn(0x7f3b340fe760 msgr2=0x7f3b34107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.450+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 shutdown_connections 2024-02-26T06:20:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:04.450+0000 7f3b33e00640 1 -- 172.21.15.49:0/4107681524 wait complete. 2024-02-26T06:20:04.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:04 smithi080 ceph-mon[36206]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:04.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:04 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4107681524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:04.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:04 smithi115 ceph-mon[36426]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:04.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:04 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4107681524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:04.841 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:04 smithi115 systemd[1]: Starting Ceph mds.f for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:20:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:04 smithi049 ceph-mon[31696]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:04 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4107681524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:05.260 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:04 smithi115 podman[66912]: 2024-02-26 06:20:04.848163684 +0000 UTC m=+0.017384549 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:20:05.260 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:04 smithi115 podman[66912]: 2024-02-26 06:20:04.949512556 +0000 UTC m=+0.118733474 container create 0eae69f6ee63d22e354534cf01728dfa2dfdd2a80132dd6b043b8264bb09b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-26T06:20:05.260 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:05 smithi115 podman[66912]: 2024-02-26 06:20:05.168651255 +0000 UTC m=+0.337872107 container init 0eae69f6ee63d22e354534cf01728dfa2dfdd2a80132dd6b043b8264bb09b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, ceph=True) 2024-02-26T06:20:05.261 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:05 smithi115 podman[66912]: 2024-02-26 06:20:05.171290523 +0000 UTC m=+0.340511375 container start 0eae69f6ee63d22e354534cf01728dfa2dfdd2a80132dd6b043b8264bb09b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-26T06:20:05.261 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:05 smithi115 ceph-mds[66932]: -- 172.21.15.115:0/194799422 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb9fde81a0 con 0x55eb9fdbec00 2024-02-26T06:20:05.261 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f[66928]: starting mds.f at 2024-02-26T06:20:05.261 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:05 smithi115 bash[66912]: 0eae69f6ee63d22e354534cf01728dfa2dfdd2a80132dd6b043b8264bb09b74c 2024-02-26T06:20:05.590 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:20:05 smithi115 systemd[1]: Started Ceph mds.f for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:20:06.472 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: Deploying daemon mds.c on smithi115 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6834/2578799425,v1:172.21.15.115:6835/2578799425] up:boot 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: fsmap 6 up:standby 2024-02-26T06:20:06.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:20:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: Deploying daemon mds.c on smithi115 2024-02-26T06:20:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:06.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6834/2578799425,v1:172.21.15.115:6835/2578799425] up:boot 2024-02-26T06:20:06.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: fsmap 6 up:standby 2024-02-26T06:20:06.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:20:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: Deploying daemon mds.c on smithi115 2024-02-26T06:20:06.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:06.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6834/2578799425,v1:172.21.15.115:6835/2578799425] up:boot 2024-02-26T06:20:06.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: fsmap 6 up:standby 2024-02-26T06:20:06.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.714+0000 7f7c4f800640 1 -- 172.21.15.49:0/2893765214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c481089b0 msgr2=0x7f7c48108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.714+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2893765214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c481089b0 0x7f7c48108e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009860 tx=0x7f7c3402bb90 comp rx=0 tx=0).stop 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.715+0000 7f7c4f800640 1 -- 172.21.15.49:0/2893765214 shutdown_connections 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.715+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2893765214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c48109370 0x7f7c4810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.715+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2893765214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c481089b0 0x7f7c48108e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.715+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2893765214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c481077c0 0x7f7c48107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.715+0000 7f7c4f800640 1 -- 172.21.15.49:0/2893765214 >> 172.21.15.49:0/2893765214 conn(0x7f7c48076790 msgr2=0x7f7c48078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.715+0000 7f7c4f800640 1 -- 172.21.15.49:0/2893765214 shutdown_connections 2024-02-26T06:20:06.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.715+0000 7f7c4f800640 1 -- 172.21.15.49:0/2893765214 wait complete. 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.716+0000 7f7c4f800640 1 Processor -- start 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.716+0000 7f7c4f800640 1 -- start start 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.716+0000 7f7c4f800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c481077c0 0x7f7c4819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.716+0000 7f7c4f800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c481089b0 0x7f7c4819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.716+0000 7f7c4f800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c48109370 0x7f7c481a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.716+0000 7f7c4f800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a5320 con 0x7f7c481089b0 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.716+0000 7f7c4f800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a5490 con 0x7f7c481077c0 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.716+0000 7f7c4f800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a5650 con 0x7f7c48109370 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c47e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c481089b0 0x7f7c4819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c481077c0 0x7f7c4819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c47e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c481089b0 0x7f7c4819dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51990/0 (socket says 172.21.15.49:51990) 2024-02-26T06:20:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c481077c0 0x7f7c4819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58014/0 (socket says 172.21.15.49:58014) 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c47e00640 1 -- 172.21.15.49:0/2797687511 learned_addr learned my addr 172.21.15.49:0/2797687511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4d800640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c48109370 0x7f7c481a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4ce00640 1 -- 172.21.15.49:0/2797687511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c48109370 msgr2=0x7f7c481a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4ce00640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c48109370 0x7f7c481a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4ce00640 1 -- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c481089b0 msgr2=0x7f7c4819dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4ce00640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c481089b0 0x7f7c4819dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4ce00640 1 -- 172.21.15.49:0/2797687511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c009560 con 0x7f7c481077c0 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c47e00640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c481089b0 0x7f7c4819dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.717+0000 7f7c4ce00640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c481077c0 0x7f7c4819d830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c48108810 tx=0x7f7c3c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.718+0000 7f7c45600640 1 -- 172.21.15.49:0/2797687511 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c004440 con 0x7f7c481077c0 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.718+0000 7f7c45600640 1 -- 172.21.15.49:0/2797687511 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c00cda0 con 0x7f7c481077c0 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.718+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c34009490 con 0x7f7c481077c0 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.718+0000 7f7c45600640 1 -- 172.21.15.49:0/2797687511 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c019480 con 0x7f7c481077c0 2024-02-26T06:20:06.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.718+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c481a5d00 con 0x7f7c481077c0 2024-02-26T06:20:06.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.719+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4810eae0 con 0x7f7c481077c0 2024-02-26T06:20:06.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.721+0000 7f7c45600640 1 -- 172.21.15.49:0/2797687511 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7c3c003680 con 0x7f7c481077c0 2024-02-26T06:20:06.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.722+0000 7f7c45600640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7c300a2140 0x7f7c300a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:06.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.722+0000 7f7c45600640 1 -- 172.21.15.49:0/2797687511 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7c3c014070 con 0x7f7c481077c0 2024-02-26T06:20:06.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.722+0000 7f7c47e00640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7c300a2140 0x7f7c300a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:06.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.723+0000 7f7c47e00640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7c300a2140 0x7f7c300a4600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7c4819ed40 tx=0x7f7c34034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:06.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.726+0000 7f7c45600640 1 -- 172.21.15.49:0/2797687511 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c3c097170 con 0x7f7c481077c0 2024-02-26T06:20:06.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.969+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c481131c0 con 0x7f7c481077c0 2024-02-26T06:20:06.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.970+0000 7f7c45600640 1 -- 172.21.15.49:0/2797687511 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5277 (secure 0 0 0) 0x7f7c3c09a830 con 0x7f7c481077c0 2024-02-26T06:20:06.970 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:06.970 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-26T06:20:06.970 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-02-26T06:20:06.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7c300a2140 msgr2=0x7f7c300a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:06.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7c300a2140 0x7f7c300a4600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7c4819ed40 tx=0x7f7c34034be0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c481077c0 msgr2=0x7f7c4819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:06.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c481077c0 0x7f7c4819d830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c48108810 tx=0x7f7c3c00ee50 comp rx=0 tx=0).stop 2024-02-26T06:20:06.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 shutdown_connections 2024-02-26T06:20:06.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7c300a2140 0x7f7c300a4600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c48109370 0x7f7c481a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c481089b0 0x7f7c4819dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 --2- 172.21.15.49:0/2797687511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c481077c0 0x7f7c4819d830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:06.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.973+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 >> 172.21.15.49:0/2797687511 conn(0x7f7c48076790 msgr2=0x7f7c4810bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:06.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.974+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 shutdown_connections 2024-02-26T06:20:06.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:06.974+0000 7f7c4f800640 1 -- 172.21.15.49:0/2797687511 wait complete. 2024-02-26T06:20:07.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:07 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2797687511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:07.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:07 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2797687511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:07.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:07 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2797687511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:07.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:07 smithi115 systemd[1]: Starting Ceph mds.c for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:20:08.192 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:07 smithi115 podman[67226]: 2024-02-26 06:20:07.770888164 +0000 UTC m=+0.017589490 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:20:08.193 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:07 smithi115 podman[67226]: 2024-02-26 06:20:07.880276229 +0000 UTC m=+0.126977573 container create cd39c9578aaed51fbdcc38546d24bbfc1f1ea8ee9261a011930477c9998959f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-26T06:20:08.193 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:08 smithi115 podman[67226]: 2024-02-26 06:20:08.193453968 +0000 UTC m=+0.440155298 container init cd39c9578aaed51fbdcc38546d24bbfc1f1ea8ee9261a011930477c9998959f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-02-26T06:20:08.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:08 smithi115 ceph-mon[36426]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:08.590 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:08 smithi115 podman[67226]: 2024-02-26 06:20:08.19637732 +0000 UTC m=+0.443078651 container start cd39c9578aaed51fbdcc38546d24bbfc1f1ea8ee9261a011930477c9998959f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0) 2024-02-26T06:20:08.591 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:08 smithi115 ceph-mds[67244]: -- 172.21.15.115:0/1946475709 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560db5d381a0 con 0x560db5d0ec00 2024-02-26T06:20:08.591 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: starting mds.c at 2024-02-26T06:20:08.591 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:08 smithi115 bash[67226]: cd39c9578aaed51fbdcc38546d24bbfc1f1ea8ee9261a011930477c9998959f8 2024-02-26T06:20:08.591 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:20:08 smithi115 systemd[1]: Started Ceph mds.c for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:20:08.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:08 smithi049 ceph-mon[31696]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:08.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:08 smithi080 ceph-mon[36206]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:08.995 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 -- 172.21.15.49:0/21138099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41031d0 msgr2=0x7fc3c41035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 --2- 172.21.15.49:0/21138099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41031d0 0x7fc3c41035f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009860 tx=0x7fc3b802bb70 comp rx=0 tx=0).stop 2024-02-26T06:20:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 -- 172.21.15.49:0/21138099 shutdown_connections 2024-02-26T06:20:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 --2- 172.21.15.49:0/21138099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c4104d80 0x7fc3c410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 --2- 172.21.15.49:0/21138099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3c41043c0 0x7fc3c4104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 --2- 172.21.15.49:0/21138099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41031d0 0x7fc3c41035f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 -- 172.21.15.49:0/21138099 >> 172.21.15.49:0/21138099 conn(0x7fc3c40fe760 msgr2=0x7fc3c4100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 -- 172.21.15.49:0/21138099 shutdown_connections 2024-02-26T06:20:09.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.239+0000 7fc3cb000640 1 -- 172.21.15.49:0/21138099 wait complete. 2024-02-26T06:20:09.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.241+0000 7fc3cb000640 1 Processor -- start 2024-02-26T06:20:09.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.241+0000 7fc3cb000640 1 -- start start 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.241+0000 7fc3cb000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3c41031d0 0x7fc3c4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.241+0000 7fc3cb000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41043c0 0x7fc3c4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.241+0000 7fc3cb000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c4104d80 0x7fc3c41a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.241+0000 7fc3cb000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41a0d30 con 0x7fc3c4104d80 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.241+0000 7fc3cb000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41a0ea0 con 0x7fc3c41043c0 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.241+0000 7fc3cb000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41a1060 con 0x7fc3c41031d0 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41043c0 0x7fc3c4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41043c0 0x7fc3c4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58020/0 (socket says 172.21.15.49:58020) 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 -- 172.21.15.49:0/1131747508 learned_addr learned my addr 172.21.15.49:0/1131747508 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3ca600640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c4104d80 0x7fc3c41a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9c00640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3c41031d0 0x7fc3c4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 -- 172.21.15.49:0/1131747508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3c41031d0 msgr2=0x7fc3c4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3c41031d0 0x7fc3c4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 -- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c4104d80 msgr2=0x7fc3c41a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c4104d80 0x7fc3c41a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 -- 172.21.15.49:0/1131747508 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b8009490 con 0x7fc3c41043c0 2024-02-26T06:20:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.242+0000 7fc3c9200640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41043c0 0x7fc3c4199780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d950 tx=0x7fc3b000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:09.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.243+0000 7fc3c2000640 1 -- 172.21.15.49:0/1131747508 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0009a30 con 0x7fc3c41043c0 2024-02-26T06:20:09.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.243+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c41a1340 con 0x7fc3c41043c0 2024-02-26T06:20:09.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.243+0000 7fc3c2000640 1 -- 172.21.15.49:0/1131747508 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3b0009b90 con 0x7fc3c41043c0 2024-02-26T06:20:09.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.243+0000 7fc3c2000640 1 -- 172.21.15.49:0/1131747508 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0004a00 con 0x7fc3c41043c0 2024-02-26T06:20:09.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.243+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c41a1850 con 0x7fc3c41043c0 2024-02-26T06:20:09.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.244+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c410a4f0 con 0x7fc3c41043c0 2024-02-26T06:20:09.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.246+0000 7fc3c2000640 1 -- 172.21.15.49:0/1131747508 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc3b0009d00 con 0x7fc3c41043c0 2024-02-26T06:20:09.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.248+0000 7fc3c2000640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3ac0a20c0 0x7fc3ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:09.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.248+0000 7fc3c2000640 1 -- 172.21.15.49:0/1131747508 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc3b00cd260 con 0x7fc3c41043c0 2024-02-26T06:20:09.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.248+0000 7fc3c9c00640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3ac0a20c0 0x7fc3ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:09.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.249+0000 7fc3c9c00640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3ac0a20c0 0x7fc3ac0a4580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009830 tx=0x7fc3b8034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:09.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.251+0000 7fc3c2000640 1 -- 172.21.15.49:0/1131747508 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3b0096630 con 0x7fc3c41043c0 2024-02-26T06:20:09.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.494+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3c410ebe0 con 0x7fc3c41043c0 2024-02-26T06:20:09.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.496+0000 7fc3c2000640 1 -- 172.21.15.49:0/1131747508 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6071 (secure 0 0 0) 0x7fc3b0099cf0 con 0x7fc3c41043c0 2024-02-26T06:20:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:09.496 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-26T06:20:09.496 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-02-26T06:20:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.498+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3ac0a20c0 msgr2=0x7fc3ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.498+0000 7fc3cb000640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3ac0a20c0 0x7fc3ac0a4580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009830 tx=0x7fc3b8034c00 comp rx=0 tx=0).stop 2024-02-26T06:20:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.498+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41043c0 msgr2=0x7fc3c4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.498+0000 7fc3cb000640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41043c0 0x7fc3c4199780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d950 tx=0x7fc3b000de10 comp rx=0 tx=0).stop 2024-02-26T06:20:09.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.498+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 shutdown_connections 2024-02-26T06:20:09.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.499+0000 7fc3cb000640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3ac0a20c0 0x7fc3ac0a4580 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.499+0000 7fc3cb000640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3c41043c0 0x7fc3c4199780 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.499+0000 7fc3cb000640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c4104d80 0x7fc3c41a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.499+0000 7fc3cb000640 1 --2- 172.21.15.49:0/1131747508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3c41031d0 0x7fc3c4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:09.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.499+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 >> 172.21.15.49:0/1131747508 conn(0x7fc3c40fe760 msgr2=0x7fc3c41076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:09.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.499+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 shutdown_connections 2024-02-26T06:20:09.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:09.499+0000 7fc3cb000640 1 -- 172.21.15.49:0/1131747508 wait complete. 2024-02-26T06:20:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:boot 2024-02-26T06:20:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: fsmap 7 up:standby 2024-02-26T06:20:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:20:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:09.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:09 smithi049 ceph-mon[31696]: Deploying daemon mds.h on smithi080 2024-02-26T06:20:09.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:boot 2024-02-26T06:20:09.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: fsmap 7 up:standby 2024-02-26T06:20:09.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:20:09.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:09.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:09.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:09.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:09 smithi080 ceph-mon[36206]: Deploying daemon mds.h on smithi080 2024-02-26T06:20:09.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:boot 2024-02-26T06:20:09.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: fsmap 7 up:standby 2024-02-26T06:20:09.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:20:09.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:09.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:09.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:09.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:09.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:09 smithi115 ceph-mon[36426]: Deploying daemon mds.h on smithi080 2024-02-26T06:20:10.462 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:10 smithi080 ceph-mon[36206]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:10.462 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:10 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1131747508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:10.672 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:10 smithi049 ceph-mon[31696]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:10.672 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:10 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1131747508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:10.830 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:10 smithi080 systemd[1]: Starting Ceph mds.h for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:20:10.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:10 smithi115 ceph-mon[36426]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:10.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:10 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1131747508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:11.126 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:10 smithi080 podman[66605]: 2024-02-26 06:20:10.761454078 +0000 UTC m=+0.017689544 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:20:11.127 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:10 smithi080 podman[66605]: 2024-02-26 06:20:10.89098056 +0000 UTC m=+0.147216052 container create 8469b681a6d0c308c9d51e9cce5b2dadf0df642f3d954183a4a008a0b318312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:20:11.127 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:11 smithi080 podman[66605]: 2024-02-26 06:20:11.127085608 +0000 UTC m=+0.383321050 container init 8469b681a6d0c308c9d51e9cce5b2dadf0df642f3d954183a4a008a0b318312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:20:11.520 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:11.580 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:11 smithi080 podman[66605]: 2024-02-26 06:20:11.130046384 +0000 UTC m=+0.386281832 container start 8469b681a6d0c308c9d51e9cce5b2dadf0df642f3d954183a4a008a0b318312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214) 2024-02-26T06:20:11.580 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:11 smithi080 ceph-mds[66625]: -- 172.21.15.80:0/2564089241 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564c80ca01a0 con 0x564c80c76c00 2024-02-26T06:20:11.580 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:11 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h[66621]: starting mds.h at 2024-02-26T06:20:11.580 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:11 smithi080 bash[66605]: 8469b681a6d0c308c9d51e9cce5b2dadf0df642f3d954183a4a008a0b318312a 2024-02-26T06:20:11.580 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:20:11 smithi080 systemd[1]: Started Ceph mds.h for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:20:11.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.760+0000 7f7a9f200640 1 -- 172.21.15.49:0/1822483169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 msgr2=0x7f7aa0106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.760+0000 7f7a9f200640 1 --2- 172.21.15.49:0/1822483169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 0x7f7aa0106770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009860 tx=0x7f7a8802bb70 comp rx=0 tx=0).stop 2024-02-26T06:20:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.760+0000 7f7a9f200640 1 -- 172.21.15.49:0/1822483169 shutdown_connections 2024-02-26T06:20:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.760+0000 7f7a9f200640 1 --2- 172.21.15.49:0/1822483169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 0x7f7aa0106770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.760+0000 7f7a9f200640 1 --2- 172.21.15.49:0/1822483169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7aa0106cb0 0x7f7aa0109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.760+0000 7f7a9f200640 1 --2- 172.21.15.49:0/1822483169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0101a50 0x7f7aa0103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.760+0000 7f7a9f200640 1 -- 172.21.15.49:0/1822483169 >> 172.21.15.49:0/1822483169 conn(0x7f7aa00fb460 msgr2=0x7f7aa00fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.761+0000 7f7a9f200640 1 -- 172.21.15.49:0/1822483169 shutdown_connections 2024-02-26T06:20:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.761+0000 7f7a9f200640 1 -- 172.21.15.49:0/1822483169 wait complete. 2024-02-26T06:20:11.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9f200640 1 Processor -- start 2024-02-26T06:20:11.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9f200640 1 -- start start 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7aa0101a50 0x7f7aa01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 0x7f7aa01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0106cb0 0x7f7aa01a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01a0d90 con 0x7f7aa0106cb0 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01a0f00 con 0x7f7aa0104380 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01a10c0 con 0x7f7aa0101a50 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0106cb0 0x7f7aa01a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 0x7f7aa01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7aa0101a50 0x7f7aa01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0106cb0 0x7f7aa01a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49438/0 (socket says 172.21.15.49:49438) 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9e800640 1 -- 172.21.15.49:0/3283063740 learned_addr learned my addr 172.21.15.49:0/3283063740 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.762+0000 7f7a9d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 0x7f7aa01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58628/0 (socket says 172.21.15.49:58628) 2024-02-26T06:20:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a9d400640 1 -- 172.21.15.49:0/3283063740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7aa0101a50 msgr2=0x7f7aa01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a9d400640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7aa0101a50 0x7f7aa01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a9d400640 1 -- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0106cb0 msgr2=0x7f7aa01a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a9d400640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0106cb0 0x7f7aa01a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a9d400640 1 -- 172.21.15.49:0/3283063740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a88009490 con 0x7f7aa0104380 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a9e800640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0106cb0 0x7f7aa01a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a9d400640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 0x7f7aa01997e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009780 tx=0x7f7a88005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a96a00640 1 -- 172.21.15.49:0/3283063740 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8802c510 con 0x7f7aa0104380 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a96a00640 1 -- 172.21.15.49:0/3283063740 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a8802cb50 con 0x7f7aa0104380 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa01a1340 con 0x7f7aa0104380 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.763+0000 7f7a96a00640 1 -- 172.21.15.49:0/3283063740 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88003850 con 0x7f7aa0104380 2024-02-26T06:20:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.764+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa01a1700 con 0x7f7aa0104380 2024-02-26T06:20:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.765+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa0108800 con 0x7f7aa0104380 2024-02-26T06:20:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.766+0000 7f7a96a00640 1 -- 172.21.15.49:0/3283063740 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7a8803b040 con 0x7f7aa0104380 2024-02-26T06:20:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.768+0000 7f7a96a00640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7a840a2140 0x7f7a840a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.768+0000 7f7a96a00640 1 -- 172.21.15.49:0/3283063740 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7a880ef2e0 con 0x7f7aa0104380 2024-02-26T06:20:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.768+0000 7f7a9de00640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7a840a2140 0x7f7a840a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.769+0000 7f7a9de00640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7a840a2140 0x7f7a840a4600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005fd0 tx=0x7f7a90010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:11.772+0000 7f7a96a00640 1 -- 172.21.15.49:0/3283063740 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a880b8760 con 0x7f7aa0104380 2024-02-26T06:20:12.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.015+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aa010c9c0 con 0x7f7aa0104380 2024-02-26T06:20:12.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.016+0000 7f7a96a00640 1 -- 172.21.15.49:0/3283063740 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6862 (secure 0 0 0) 0x7f7a880bbe20 con 0x7f7aa0104380 2024-02-26T06:20:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:12.016 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-26T06:20:12.016 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 9 2024-02-26T06:20:12.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7a840a2140 msgr2=0x7f7a840a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:12.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7a840a2140 0x7f7a840a4600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005fd0 tx=0x7f7a90010040 comp rx=0 tx=0).stop 2024-02-26T06:20:12.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 msgr2=0x7f7aa01997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 0x7f7aa01997e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009780 tx=0x7f7a88005d50 comp rx=0 tx=0).stop 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 shutdown_connections 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7a840a2140 0x7f7a840a4600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa0104380 0x7f7aa01997e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0106cb0 0x7f7aa01a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 --2- 172.21.15.49:0/3283063740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7aa0101a50 0x7f7aa01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 >> 172.21.15.49:0/3283063740 conn(0x7f7aa00fb460 msgr2=0x7f7aa0101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 shutdown_connections 2024-02-26T06:20:12.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:12.019+0000 7f7a9f200640 1 -- 172.21.15.49:0/3283063740 wait complete. 2024-02-26T06:20:12.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:12.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:12.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:12.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: Deploying daemon mds.i on smithi115 2024-02-26T06:20:12.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:12.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.80:6836/2132318232,v1:172.21.15.80:6837/2132318232] up:boot 2024-02-26T06:20:12.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: fsmap 8 up:standby 2024-02-26T06:20:12.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:20:12.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:12 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3283063740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:12.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:12.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:12.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: Deploying daemon mds.i on smithi115 2024-02-26T06:20:12.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:12.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.80:6836/2132318232,v1:172.21.15.80:6837/2132318232] up:boot 2024-02-26T06:20:12.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: fsmap 8 up:standby 2024-02-26T06:20:12.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:20:12.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:12 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3283063740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: Deploying daemon mds.i on smithi115 2024-02-26T06:20:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.80:6836/2132318232,v1:172.21.15.80:6837/2132318232] up:boot 2024-02-26T06:20:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: fsmap 8 up:standby 2024-02-26T06:20:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:20:12.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:12 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3283063740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:13.825 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:13 smithi115 systemd[1]: Starting Ceph mds.i for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:20:14.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:14.090 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:13 smithi115 podman[67536]: 2024-02-26 06:20:13.728257167 +0000 UTC m=+0.017706277 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:20:14.090 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:13 smithi115 podman[67536]: 2024-02-26 06:20:13.918714998 +0000 UTC m=+0.208164088 container create c07093591ce275212ad4edef206a59bf285a4dcac29f7547fdad5e132c2344c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True) 2024-02-26T06:20:14.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.278+0000 7f438d200640 1 -- 172.21.15.49:0/2738875802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43880ff2a0 msgr2=0x7f43880ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:14.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.278+0000 7f438d200640 1 --2- 172.21.15.49:0/2738875802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43880ff2a0 0x7f43880ff6c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4378009900 tx=0x7f437802bc50 comp rx=0 tx=0).stop 2024-02-26T06:20:14.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.278+0000 7f438d200640 1 -- 172.21.15.49:0/2738875802 shutdown_connections 2024-02-26T06:20:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.278+0000 7f438d200640 1 --2- 172.21.15.49:0/2738875802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4388102270 0x7f43881026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.278+0000 7f438d200640 1 --2- 172.21.15.49:0/2738875802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388102c30 0x7f43881094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.278+0000 7f438d200640 1 --2- 172.21.15.49:0/2738875802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43880ff2a0 0x7f43880ff6c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.278+0000 7f438d200640 1 -- 172.21.15.49:0/2738875802 >> 172.21.15.49:0/2738875802 conn(0x7f43880fabd0 msgr2=0x7f43880fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.279+0000 7f438d200640 1 -- 172.21.15.49:0/2738875802 shutdown_connections 2024-02-26T06:20:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.279+0000 7f438d200640 1 -- 172.21.15.49:0/2738875802 wait complete. 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f438d200640 1 Processor -- start 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f438d200640 1 -- start start 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f438d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43880ff2a0 0x7f4388199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f438d200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388102270 0x7f4388199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f438d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4388102c30 0x7f43881a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f438d200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43881a0dc0 con 0x7f4388102c30 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f438d200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43881a0f30 con 0x7f43880ff2a0 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f438d200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43881a10f0 con 0x7f4388102270 2024-02-26T06:20:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f4387400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43880ff2a0 0x7f4388199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f4387e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4388102c30 0x7f43881a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f437fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388102270 0x7f4388199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f4387400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43880ff2a0 0x7f4388199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58650/0 (socket says 172.21.15.49:58650) 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.280+0000 7f4387400640 1 -- 172.21.15.49:0/2671288240 learned_addr learned my addr 172.21.15.49:0/2671288240 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4387400640 1 -- 172.21.15.49:0/2671288240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388102270 msgr2=0x7f4388199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4387400640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388102270 0x7f4388199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4387400640 1 -- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4388102c30 msgr2=0x7f43881a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4387400640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4388102c30 0x7f43881a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4387400640 1 -- 172.21.15.49:0/2671288240 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4378009560 con 0x7f43880ff2a0 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4387e00640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4388102c30 0x7f43881a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4387400640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43880ff2a0 0x7f4388199320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4378000c00 tx=0x7f4378004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4384c00640 1 -- 172.21.15.49:0/2671288240 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378004aa0 con 0x7f43880ff2a0 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4384c00640 1 -- 172.21.15.49:0/2671288240 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43780052d0 con 0x7f43880ff2a0 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f4384c00640 1 -- 172.21.15.49:0/2671288240 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378003e60 con 0x7f43880ff2a0 2024-02-26T06:20:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.281+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43881a1370 con 0x7f43880ff2a0 2024-02-26T06:20:14.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.282+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43881a18d0 con 0x7f43880ff2a0 2024-02-26T06:20:14.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.283+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438810c9e0 con 0x7f43880ff2a0 2024-02-26T06:20:14.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.285+0000 7f4384c00640 1 -- 172.21.15.49:0/2671288240 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4378005440 con 0x7f43880ff2a0 2024-02-26T06:20:14.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.286+0000 7f4384c00640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f436c0a2140 0x7f436c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:14.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.286+0000 7f4384c00640 1 -- 172.21.15.49:0/2671288240 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43780efaa0 con 0x7f43880ff2a0 2024-02-26T06:20:14.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.286+0000 7f437fe00640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f436c0a2140 0x7f436c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:14.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.287+0000 7f437fe00640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f436c0a2140 0x7f436c0a4600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f438819a7e0 tx=0x7f4370005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:14.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.290+0000 7f4384c00640 1 -- 172.21.15.49:0/2671288240 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43780b8e70 con 0x7f43880ff2a0 2024-02-26T06:20:14.459 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:14 smithi115 podman[67536]: 2024-02-26 06:20:14.155115897 +0000 UTC m=+0.444565003 container init c07093591ce275212ad4edef206a59bf285a4dcac29f7547fdad5e132c2344c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:20:14.459 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:14 smithi115 podman[67536]: 2024-02-26 06:20:14.158022709 +0000 UTC m=+0.447471800 container start c07093591ce275212ad4edef206a59bf285a4dcac29f7547fdad5e132c2344c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-26T06:20:14.459 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-mds[67557]: -- 172.21.15.115:0/4145272499 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ae939e1a0 con 0x561ae9374800 2024-02-26T06:20:14.459 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i[67552]: starting mds.i at 2024-02-26T06:20:14.459 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:14 smithi115 bash[67536]: c07093591ce275212ad4edef206a59bf285a4dcac29f7547fdad5e132c2344c2 2024-02-26T06:20:14.459 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:20:14 smithi115 systemd[1]: Started Ceph mds.i for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:20:14.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.540+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f438810cb70 con 0x7f43880ff2a0 2024-02-26T06:20:14.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.541+0000 7f4384c00640 1 -- 172.21.15.49:0/2671288240 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7655 (secure 0 0 0) 0x7f43780bc530 con 0x7f43880ff2a0 2024-02-26T06:20:14.541 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:20:14.541 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 10 2024-02-26T06:20:14.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.543+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f436c0a2140 msgr2=0x7f436c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:14.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.543+0000 7f438d200640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f436c0a2140 0x7f436c0a4600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f438819a7e0 tx=0x7f4370005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:14.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.543+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43880ff2a0 msgr2=0x7f4388199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:14.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.543+0000 7f438d200640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43880ff2a0 0x7f4388199320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4378000c00 tx=0x7f4378004270 comp rx=0 tx=0).stop 2024-02-26T06:20:14.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.544+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 shutdown_connections 2024-02-26T06:20:14.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.544+0000 7f438d200640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f436c0a2140 0x7f436c0a4600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.544+0000 7f438d200640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4388102c30 0x7f43881a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.544+0000 7f438d200640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388102270 0x7f4388199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.544+0000 7f438d200640 1 --2- 172.21.15.49:0/2671288240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43880ff2a0 0x7f4388199320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:14.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.544+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 >> 172.21.15.49:0/2671288240 conn(0x7f43880fabd0 msgr2=0x7f4388100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:14.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.544+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 shutdown_connections 2024-02-26T06:20:14.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:14.544+0000 7f438d200640 1 -- 172.21.15.49:0/2671288240 wait complete. 2024-02-26T06:20:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:14 smithi080 ceph-mon[36206]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:14.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:14.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:14 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-mon[36426]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:14 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:14 smithi049 ceph-mon[31696]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:14.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:14.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:14.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:14.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:14 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:15.665 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:15 smithi049 ceph-mon[31696]: Deploying daemon mds.g on smithi049 2024-02-26T06:20:15.665 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:15 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] up:boot 2024-02-26T06:20:15.666 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:15 smithi049 ceph-mon[31696]: fsmap 9 up:standby 2024-02-26T06:20:15.666 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:15 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:20:15.666 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:15 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2671288240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:15.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:15 smithi080 ceph-mon[36206]: Deploying daemon mds.g on smithi049 2024-02-26T06:20:15.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:15 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] up:boot 2024-02-26T06:20:15.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:15 smithi080 ceph-mon[36206]: fsmap 9 up:standby 2024-02-26T06:20:15.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:15 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:20:15.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:15 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2671288240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:15.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:15 smithi115 ceph-mon[36426]: Deploying daemon mds.g on smithi049 2024-02-26T06:20:15.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:15 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] up:boot 2024-02-26T06:20:15.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:15 smithi115 ceph-mon[36426]: fsmap 9 up:standby 2024-02-26T06:20:15.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:15 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:20:15.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:15 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2671288240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:16.565 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.816+0000 7efe89800640 1 -- 172.21.15.49:0/2955641194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84100fe0 msgr2=0x7efe84101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.816+0000 7efe89800640 1 --2- 172.21.15.49:0/2955641194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84100fe0 0x7efe84101400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efe74009900 tx=0x7efe7402bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.816+0000 7efe89800640 1 -- 172.21.15.49:0/2955641194 shutdown_connections 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.816+0000 7efe89800640 1 --2- 172.21.15.49:0/2955641194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe84102b90 0x7efe84109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.816+0000 7efe89800640 1 --2- 172.21.15.49:0/2955641194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe841021d0 0x7efe84102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.816+0000 7efe89800640 1 --2- 172.21.15.49:0/2955641194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84100fe0 0x7efe84101400 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.816+0000 7efe89800640 1 -- 172.21.15.49:0/2955641194 >> 172.21.15.49:0/2955641194 conn(0x7efe840fc550 msgr2=0x7efe840fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.817+0000 7efe89800640 1 -- 172.21.15.49:0/2955641194 shutdown_connections 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.817+0000 7efe89800640 1 -- 172.21.15.49:0/2955641194 wait complete. 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe89800640 1 Processor -- start 2024-02-26T06:20:16.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe89800640 1 -- start start 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe89800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe84100fe0 0x7efe84199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe89800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe841021d0 0x7efe84199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe89800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84102b90 0x7efe841a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe89800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe841a0db0 con 0x7efe84102b90 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe89800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe841a0f20 con 0x7efe84100fe0 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe89800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe841a10e0 con 0x7efe841021d0 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe88e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84102b90 0x7efe841a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe88e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84102b90 0x7efe841a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49494/0 (socket says 172.21.15.49:49494) 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe88e00640 1 -- 172.21.15.49:0/373816912 learned_addr learned my addr 172.21.15.49:0/373816912 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe83e00640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe84100fe0 0x7efe84199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.818+0000 7efe83400640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe841021d0 0x7efe84199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.819+0000 7efe88e00640 1 -- 172.21.15.49:0/373816912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe841021d0 msgr2=0x7efe84199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.819+0000 7efe88e00640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe841021d0 0x7efe84199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:16.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.819+0000 7efe88e00640 1 -- 172.21.15.49:0/373816912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe84100fe0 msgr2=0x7efe84199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.819+0000 7efe88e00640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe84100fe0 0x7efe84199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.819+0000 7efe88e00640 1 -- 172.21.15.49:0/373816912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe74009560 con 0x7efe84102b90 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.819+0000 7efe83e00640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe84100fe0 0x7efe84199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.819+0000 7efe83400640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe841021d0 0x7efe84199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.819+0000 7efe88e00640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84102b90 0x7efe841a0870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00eaa0 tx=0x7efe7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.820+0000 7efe80c00640 1 -- 172.21.15.49:0/373816912 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00cdb0 con 0x7efe84102b90 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.820+0000 7efe80c00640 1 -- 172.21.15.49:0/373816912 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe7c005020 con 0x7efe84102b90 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.820+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe841a13c0 con 0x7efe84102b90 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.820+0000 7efe80c00640 1 -- 172.21.15.49:0/373816912 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0116b0 con 0x7efe84102b90 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.820+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe841a1880 con 0x7efe84102b90 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.821+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8410ca20 con 0x7efe84102b90 2024-02-26T06:20:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.823+0000 7efe80c00640 1 -- 172.21.15.49:0/373816912 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efe7c005520 con 0x7efe84102b90 2024-02-26T06:20:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.824+0000 7efe80c00640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efe680a20c0 0x7efe680a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.825+0000 7efe80c00640 1 -- 172.21.15.49:0/373816912 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efe7c0cda40 con 0x7efe84102b90 2024-02-26T06:20:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.825+0000 7efe83e00640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efe680a20c0 0x7efe680a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.825+0000 7efe83e00640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efe680a20c0 0x7efe680a4580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe7402c240 tx=0x7efe7402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:16.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:16.829+0000 7efe80c00640 1 -- 172.21.15.49:0/373816912 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe7c096ec0 con 0x7efe84102b90 2024-02-26T06:20:16.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:16 smithi080 ceph-mon[36206]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:16.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:16 smithi115 ceph-mon[36426]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:16.863 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:16 smithi049 ceph-mon[31696]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:17.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.095+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe841a1d80 con 0x7efe84102b90 2024-02-26T06:20:17.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.096+0000 7efe80c00640 1 -- 172.21.15.49:0/373816912 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7655 (secure 0 0 0) 0x7efe7c09a580 con 0x7efe84102b90 2024-02-26T06:20:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:20:17.096 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 10 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efe680a20c0 msgr2=0x7efe680a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efe680a20c0 0x7efe680a4580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe7402c240 tx=0x7efe7402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84102b90 msgr2=0x7efe841a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84102b90 0x7efe841a0870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00eaa0 tx=0x7efe7c00ef60 comp rx=0 tx=0).stop 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 shutdown_connections 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efe680a20c0 0x7efe680a4580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe84102b90 0x7efe841a0870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe841021d0 0x7efe84199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 --2- 172.21.15.49:0/373816912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe84100fe0 0x7efe84199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 >> 172.21.15.49:0/373816912 conn(0x7efe840fc550 msgr2=0x7efe840733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 shutdown_connections 2024-02-26T06:20:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:17.099+0000 7efe89800640 1 -- 172.21.15.49:0/373816912 wait complete. 2024-02-26T06:20:17.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:17 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/373816912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:17.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:17 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/373816912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:17.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:17 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/373816912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:18.159 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:20:17 smithi049 podman[70413]: 2024-02-26 06:20:17.977981755 +0000 UTC m=+0.177885495 container create bde75ebc04e5afce76cf8c249704a381f27af90f8a783bc14ee49015da47d259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:20:18.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:18 smithi049 ceph-mon[31696]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:18.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:20:18 smithi049 podman[70413]: 2024-02-26 06:20:18.315454715 +0000 UTC m=+0.515358499 container init bde75ebc04e5afce76cf8c249704a381f27af90f8a783bc14ee49015da47d259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-26T06:20:18.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:20:18 smithi049 podman[70413]: 2024-02-26 06:20:18.325420315 +0000 UTC m=+0.525324035 container start bde75ebc04e5afce76cf8c249704a381f27af90f8a783bc14ee49015da47d259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-26T06:20:18.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:20:18 smithi049 ceph-mds[70439]: -- 172.21.15.49:0/813096265 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562e4d201080 con 0x562e4d1cb000 2024-02-26T06:20:18.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:20:18 smithi049 ceph-mds[70439]: -- 172.21.15.49:0/813096265 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562e4d1f41a0 con 0x562e4d1cb000 2024-02-26T06:20:18.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:20:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g[70433]: starting mds.g at 2024-02-26T06:20:18.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:20:18 smithi049 bash[70413]: bde75ebc04e5afce76cf8c249704a381f27af90f8a783bc14ee49015da47d259 2024-02-26T06:20:18.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:20:18 smithi049 systemd[1]: Started Ceph mds.g for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:20:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:18 smithi080 ceph-mon[36206]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:18.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:18 smithi115 ceph-mon[36426]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:19.120 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 -- 172.21.15.49:0/1998628608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc075140 msgr2=0x7ff3cc073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 --2- 172.21.15.49:0/1998628608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc075140 0x7ff3cc073590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009900 tx=0x7ff3c002bcd0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 -- 172.21.15.49:0/1998628608 shutdown_connections 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 --2- 172.21.15.49:0/1998628608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3cc109050 0x7ff3cc10b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 --2- 172.21.15.49:0/1998628608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3cc073b60 0x7ff3cc073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 --2- 172.21.15.49:0/1998628608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc075140 0x7ff3cc073590 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 -- 172.21.15.49:0/1998628608 >> 172.21.15.49:0/1998628608 conn(0x7ff3cc0fc730 msgr2=0x7ff3cc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 -- 172.21.15.49:0/1998628608 shutdown_connections 2024-02-26T06:20:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.342+0000 7ff3d4400640 1 -- 172.21.15.49:0/1998628608 wait complete. 2024-02-26T06:20:19.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.343+0000 7ff3d4400640 1 Processor -- start 2024-02-26T06:20:19.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.343+0000 7ff3d4400640 1 -- start start 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.343+0000 7ff3d4400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3cc073b60 0x7ff3cc199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.343+0000 7ff3d4400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3cc075140 0x7ff3cc199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.343+0000 7ff3d4400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc109050 0x7ff3cc1a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.343+0000 7ff3d4400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1a0f10 con 0x7ff3cc075140 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.343+0000 7ff3d4400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1a1080 con 0x7ff3cc109050 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.343+0000 7ff3d4400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1a1240 con 0x7ff3cc073b60 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d2400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc109050 0x7ff3cc1a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d2400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc109050 0x7ff3cc1a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58726/0 (socket says 172.21.15.49:58726) 2024-02-26T06:20:19.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d2400640 1 -- 172.21.15.49:0/4261449852 learned_addr learned my addr 172.21.15.49:0/4261449852 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d1a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3cc073b60 0x7ff3cc199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d1000640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3cc075140 0x7ff3cc199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d2400640 1 -- 172.21.15.49:0/4261449852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3cc073b60 msgr2=0x7ff3cc199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d2400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3cc073b60 0x7ff3cc199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d2400640 1 -- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3cc075140 msgr2=0x7ff3cc199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d2400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3cc075140 0x7ff3cc199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d2400640 1 -- 172.21.15.49:0/4261449852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c0009560 con 0x7ff3cc109050 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.344+0000 7ff3d1000640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3cc075140 0x7ff3cc199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.345+0000 7ff3d2400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc109050 0x7ff3cc1a09d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400eb10 tx=0x7ff3c400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.345+0000 7ff3ca000640 1 -- 172.21.15.49:0/4261449852 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4004b20 con 0x7ff3cc109050 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.345+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc1a1520 con 0x7ff3cc109050 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.345+0000 7ff3ca000640 1 -- 172.21.15.49:0/4261449852 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3c4004c80 con 0x7ff3cc109050 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.345+0000 7ff3ca000640 1 -- 172.21.15.49:0/4261449852 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4010650 con 0x7ff3cc109050 2024-02-26T06:20:19.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.345+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc1a1a30 con 0x7ff3cc109050 2024-02-26T06:20:19.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.347+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc107e10 con 0x7ff3cc109050 2024-02-26T06:20:19.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.348+0000 7ff3ca000640 1 -- 172.21.15.49:0/4261449852 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff3c4005020 con 0x7ff3cc109050 2024-02-26T06:20:19.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.350+0000 7ff3ca000640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3b40a20c0 0x7ff3b40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:19.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.350+0000 7ff3ca000640 1 -- 172.21.15.49:0/4261449852 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff3c4014070 con 0x7ff3cc109050 2024-02-26T06:20:19.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.350+0000 7ff3d1a00640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3b40a20c0 0x7ff3b40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:19.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.351+0000 7ff3d1a00640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3b40a20c0 0x7ff3b40a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009230 tx=0x7ff3c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:19.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.354+0000 7ff3ca000640 1 -- 172.21.15.49:0/4261449852 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff3c40972c0 con 0x7ff3cc109050 2024-02-26T06:20:19.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.617+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3cc107880 con 0x7ff3cc109050 2024-02-26T06:20:19.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.619+0000 7ff3ca000640 1 -- 172.21.15.49:0/4261449852 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8447 (secure 0 0 0) 0x7ff3c409a980 con 0x7ff3cc109050 2024-02-26T06:20:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:19.618 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:20:19.619 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 11 2024-02-26T06:20:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3b40a20c0 msgr2=0x7ff3b40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3b40a20c0 0x7ff3b40a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009230 tx=0x7ff3c0005fd0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc109050 msgr2=0x7ff3cc1a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc109050 0x7ff3cc1a09d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400eb10 tx=0x7ff3c400c2d0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 shutdown_connections 2024-02-26T06:20:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3b40a20c0 0x7ff3b40a4580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3cc109050 0x7ff3cc1a09d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3cc075140 0x7ff3cc199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 --2- 172.21.15.49:0/4261449852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3cc073b60 0x7ff3cc199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:19.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 >> 172.21.15.49:0/4261449852 conn(0x7ff3cc0fc730 msgr2=0x7ff3cc104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:19.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.621+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 shutdown_connections 2024-02-26T06:20:19.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:19.622+0000 7ff3d4400640 1 -- 172.21.15.49:0/4261449852 wait complete. 2024-02-26T06:20:19.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.49:6838/1053244563,v1:172.21.15.49:6839/1053244563] up:boot 2024-02-26T06:20:19.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: fsmap 10 up:standby 2024-02-26T06:20:19.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:20:19.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:19.770 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:19.770 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:19.770 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:19 smithi049 ceph-mon[31696]: Deploying daemon mds.a on smithi049 2024-02-26T06:20:19.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.49:6838/1053244563,v1:172.21.15.49:6839/1053244563] up:boot 2024-02-26T06:20:19.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: fsmap 10 up:standby 2024-02-26T06:20:19.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:20:19.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:19.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:19.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:19.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:19 smithi080 ceph-mon[36206]: Deploying daemon mds.a on smithi049 2024-02-26T06:20:19.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.49:6838/1053244563,v1:172.21.15.49:6839/1053244563] up:boot 2024-02-26T06:20:19.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: fsmap 10 up:standby 2024-02-26T06:20:19.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:20:19.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:19.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:19.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:19.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:19.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:19 smithi115 ceph-mon[36426]: Deploying daemon mds.a on smithi049 2024-02-26T06:20:20.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:20 smithi080 ceph-mon[36206]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:20.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:20 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4261449852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:20.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:20 smithi115 ceph-mon[36426]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:20.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:20 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4261449852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:20.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:20 smithi049 ceph-mon[31696]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:20.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:20 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4261449852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:21.639 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:21.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.912+0000 7f12ea600640 1 -- 172.21.15.49:0/204067702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4104f70 msgr2=0x7f12e410b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:21.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.912+0000 7f12ea600640 1 --2- 172.21.15.49:0/204067702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4104f70 0x7f12e410b800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009be0 tx=0x7f12dc02bef0 comp rx=0 tx=0).stop 2024-02-26T06:20:21.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.913+0000 7f12ea600640 1 -- 172.21.15.49:0/204067702 shutdown_connections 2024-02-26T06:20:21.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.913+0000 7f12ea600640 1 --2- 172.21.15.49:0/204067702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4104f70 0x7f12e410b800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:21.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.913+0000 7f12ea600640 1 --2- 172.21.15.49:0/204067702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e41045b0 0x7f12e4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:21.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.913+0000 7f12ea600640 1 --2- 172.21.15.49:0/204067702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e41033c0 0x7f12e41037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:21.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.913+0000 7f12ea600640 1 -- 172.21.15.49:0/204067702 >> 172.21.15.49:0/204067702 conn(0x7f12e40fe970 msgr2=0x7f12e4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:21.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.913+0000 7f12ea600640 1 -- 172.21.15.49:0/204067702 shutdown_connections 2024-02-26T06:20:21.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.913+0000 7f12ea600640 1 -- 172.21.15.49:0/204067702 wait complete. 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.914+0000 7f12ea600640 1 Processor -- start 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.914+0000 7f12ea600640 1 -- start start 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.914+0000 7f12ea600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e41033c0 0x7f12e4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.914+0000 7f12ea600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e41045b0 0x7f12e4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.914+0000 7f12ea600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4104f70 0x7f12e41a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.914+0000 7f12ea600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41a0ef0 con 0x7f12e41045b0 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.914+0000 7f12ea600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41a1060 con 0x7f12e4104f70 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.914+0000 7f12ea600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41a1220 con 0x7f12e41033c0 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e2a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e41045b0 0x7f12e4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e41033c0 0x7f12e4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e2a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e41045b0 0x7f12e4199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37302/0 (socket says 172.21.15.49:37302) 2024-02-26T06:20:21.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e2a00640 1 -- 172.21.15.49:0/2546224326 learned_addr learned my addr 172.21.15.49:0/2546224326 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e3e00640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4104f70 0x7f12e41a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e3400640 1 -- 172.21.15.49:0/2546224326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4104f70 msgr2=0x7f12e41a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e3400640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4104f70 0x7f12e41a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e3400640 1 -- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e41045b0 msgr2=0x7f12e4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e3400640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e41045b0 0x7f12e4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e3400640 1 -- 172.21.15.49:0/2546224326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4009560 con 0x7f12e41033c0 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e2a00640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e41045b0 0x7f12e4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.915+0000 7f12e3400640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e41033c0 0x7f12e4199400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12e4104410 tx=0x7f12d400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.916+0000 7f12dbe00640 1 -- 172.21.15.49:0/2546224326 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4004440 con 0x7f12e41033c0 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.916+0000 7f12dbe00640 1 -- 172.21.15.49:0/2546224326 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12d400cda0 con 0x7f12e41033c0 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.916+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc009840 con 0x7f12e41033c0 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.916+0000 7f12dbe00640 1 -- 172.21.15.49:0/2546224326 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4019480 con 0x7f12e41033c0 2024-02-26T06:20:21.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.916+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e41a18d0 con 0x7f12e41033c0 2024-02-26T06:20:21.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.917+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e410a680 con 0x7f12e41033c0 2024-02-26T06:20:21.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.920+0000 7f12dbe00640 1 -- 172.21.15.49:0/2546224326 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f12d4003680 con 0x7f12e41033c0 2024-02-26T06:20:21.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.921+0000 7f12dbe00640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f12c80a2140 0x7f12c80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:21.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.921+0000 7f12dbe00640 1 -- 172.21.15.49:0/2546224326 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12d4014070 con 0x7f12e41033c0 2024-02-26T06:20:21.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.921+0000 7f12e2a00640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f12c80a2140 0x7f12c80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:21.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.922+0000 7f12e2a00640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f12c80a2140 0x7f12c80a4600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12e419a910 tx=0x7f12d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:21.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:21.925+0000 7f12dbe00640 1 -- 172.21.15.49:0/2546224326 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12d4097170 con 0x7f12e41033c0 2024-02-26T06:20:22.169 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:20:21 smithi049 systemd[1]: Starting Ceph mds.a for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:20:22.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.177+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12e410edf0 con 0x7f12e41033c0 2024-02-26T06:20:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.178+0000 7f12dbe00640 1 -- 172.21.15.49:0/2546224326 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8447 (secure 0 0 0) 0x7f12d409a830 con 0x7f12e41033c0 2024-02-26T06:20:22.178 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:22.178 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:20:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 11 2024-02-26T06:20:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f12c80a2140 msgr2=0x7f12c80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f12c80a2140 0x7f12c80a4600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12e419a910 tx=0x7f12d0005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e41033c0 msgr2=0x7f12e4199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e41033c0 0x7f12e4199400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12e4104410 tx=0x7f12d400ee50 comp rx=0 tx=0).stop 2024-02-26T06:20:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 shutdown_connections 2024-02-26T06:20:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f12c80a2140 0x7f12c80a4600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:22.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4104f70 0x7f12e41a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:22.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e41045b0 0x7f12e4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:22.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 --2- 172.21.15.49:0/2546224326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e41033c0 0x7f12e4199400 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:22.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 >> 172.21.15.49:0/2546224326 conn(0x7f12e40fe970 msgr2=0x7f12e4107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:22.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 shutdown_connections 2024-02-26T06:20:22.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:22.181+0000 7f12ea600640 1 -- 172.21.15.49:0/2546224326 wait complete. 2024-02-26T06:20:22.511 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:20:22 smithi049 podman[70801]: 2024-02-26 06:20:22.086510435 +0000 UTC m=+0.033918414 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:20:22.511 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:20:22 smithi049 podman[70801]: 2024-02-26 06:20:22.198330588 +0000 UTC m=+0.145738560 container create 5b151ea5f4ef9cb79b05affb183b29091f21be7bc7c6a0c912262ddc74a7046e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:20:22.511 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:20:22 smithi049 podman[70801]: 2024-02-26 06:20:22.510869494 +0000 UTC m=+0.458277525 container init 5b151ea5f4ef9cb79b05affb183b29091f21be7bc7c6a0c912262ddc74a7046e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-26T06:20:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:22 smithi080 ceph-mon[36206]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:22 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2546224326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:22 smithi115 ceph-mon[36426]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:22 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2546224326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:22 smithi049 ceph-mon[31696]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:22 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2546224326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:22.919 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:20:22 smithi049 podman[70801]: 2024-02-26 06:20:22.521741859 +0000 UTC m=+0.469149877 container start 5b151ea5f4ef9cb79b05affb183b29091f21be7bc7c6a0c912262ddc74a7046e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-26T06:20:22.920 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:20:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a[70818]: starting mds.a at 2024-02-26T06:20:22.920 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:20:22 smithi049 bash[70801]: 5b151ea5f4ef9cb79b05affb183b29091f21be7bc7c6a0c912262ddc74a7046e 2024-02-26T06:20:22.920 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:20:22 smithi049 systemd[1]: Started Ceph mds.a for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:20:24.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: Deploying daemon mds.b on smithi080 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.49:6840/2984977662,v1:172.21.15.49:6841/2984977662] up:boot 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: fsmap 11 up:standby 2024-02-26T06:20:24.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:23 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:20:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:24.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:24.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: Deploying daemon mds.b on smithi080 2024-02-26T06:20:24.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.49:6840/2984977662,v1:172.21.15.49:6841/2984977662] up:boot 2024-02-26T06:20:24.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: fsmap 11 up:standby 2024-02-26T06:20:24.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:23 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:20:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:24.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:20:24.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:20:24.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:24.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: Deploying daemon mds.b on smithi080 2024-02-26T06:20:24.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.49:6840/2984977662,v1:172.21.15.49:6841/2984977662] up:boot 2024-02-26T06:20:24.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: fsmap 11 up:standby 2024-02-26T06:20:24.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:23 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:20:24.204 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:24.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.451+0000 7faa3f200640 1 -- 172.21.15.49:0/2786442924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40106cb0 msgr2=0x7faa40109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:24.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.451+0000 7faa3f200640 1 --2- 172.21.15.49:0/2786442924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40106cb0 0x7faa40109100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa38009900 tx=0x7faa3802bc40 comp rx=0 tx=0).stop 2024-02-26T06:20:24.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.451+0000 7faa3f200640 1 -- 172.21.15.49:0/2786442924 shutdown_connections 2024-02-26T06:20:24.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.451+0000 7faa3f200640 1 --2- 172.21.15.49:0/2786442924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa40104380 0x7faa40106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.451+0000 7faa3f200640 1 --2- 172.21.15.49:0/2786442924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40106cb0 0x7faa40109100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.451+0000 7faa3f200640 1 --2- 172.21.15.49:0/2786442924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa40101a50 0x7faa40103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.451+0000 7faa3f200640 1 -- 172.21.15.49:0/2786442924 >> 172.21.15.49:0/2786442924 conn(0x7faa400fb460 msgr2=0x7faa400fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:24.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.451+0000 7faa3f200640 1 -- 172.21.15.49:0/2786442924 shutdown_connections 2024-02-26T06:20:24.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.452+0000 7faa3f200640 1 -- 172.21.15.49:0/2786442924 wait complete. 2024-02-26T06:20:24.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.452+0000 7faa3f200640 1 Processor -- start 2024-02-26T06:20:24.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3f200640 1 -- start start 2024-02-26T06:20:24.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa40101a50 0x7faa401991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:24.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40104380 0x7faa40199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:24.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa40106cb0 0x7faa401a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:24.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401a0cc0 con 0x7faa40104380 2024-02-26T06:20:24.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401a0e30 con 0x7faa40101a50 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401a0ff0 con 0x7faa40106cb0 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40104380 0x7faa40199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40104380 0x7faa40199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37344/0 (socket says 172.21.15.49:37344) 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3d400640 1 -- 172.21.15.49:0/2887655121 learned_addr learned my addr 172.21.15.49:0/2887655121 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3de00640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa40101a50 0x7faa401991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.453+0000 7faa3e800640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa40106cb0 0x7faa401a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa3d400640 1 -- 172.21.15.49:0/2887655121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa40106cb0 msgr2=0x7faa401a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa3d400640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa40106cb0 0x7faa401a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa3d400640 1 -- 172.21.15.49:0/2887655121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa40101a50 msgr2=0x7faa401991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa3d400640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa40101a50 0x7faa401991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa3d400640 1 -- 172.21.15.49:0/2887655121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38009560 con 0x7faa40104380 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa3d400640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40104380 0x7faa40199710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa2800e9c0 tx=0x7faa2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa36a00640 1 -- 172.21.15.49:0/2887655121 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28009700 con 0x7faa40104380 2024-02-26T06:20:24.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa36a00640 1 -- 172.21.15.49:0/2887655121 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa2800cd80 con 0x7faa40104380 2024-02-26T06:20:24.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa401a12d0 con 0x7faa40104380 2024-02-26T06:20:24.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa36a00640 1 -- 172.21.15.49:0/2887655121 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28010430 con 0x7faa40104380 2024-02-26T06:20:24.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.454+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa401a17e0 con 0x7faa40104380 2024-02-26T06:20:24.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.456+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa401087a0 con 0x7faa40104380 2024-02-26T06:20:24.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.458+0000 7faa36a00640 1 -- 172.21.15.49:0/2887655121 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7faa28005000 con 0x7faa40104380 2024-02-26T06:20:24.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.459+0000 7faa36a00640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa240a20c0 0x7faa240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:24.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.459+0000 7faa36a00640 1 -- 172.21.15.49:0/2887655121 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faa28014070 con 0x7faa40104380 2024-02-26T06:20:24.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.459+0000 7faa3de00640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa240a20c0 0x7faa240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:24.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.460+0000 7faa3de00640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa240a20c0 0x7faa240a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faa30005fd0 tx=0x7faa30009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:24.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.464+0000 7faa36a00640 1 -- 172.21.15.49:0/2887655121 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa28097040 con 0x7faa40104380 2024-02-26T06:20:24.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.702+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa4010c9c0 con 0x7faa40104380 2024-02-26T06:20:24.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.704+0000 7faa36a00640 1 -- 172.21.15.49:0/2887655121 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9239 (secure 0 0 0) 0x7faa2809a700 con 0x7faa40104380 2024-02-26T06:20:24.705 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:24.705 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:20:24.705 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 12 2024-02-26T06:20:24.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.706+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa240a20c0 msgr2=0x7faa240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:24.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.706+0000 7faa3f200640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa240a20c0 0x7faa240a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faa30005fd0 tx=0x7faa30009500 comp rx=0 tx=0).stop 2024-02-26T06:20:24.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.706+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40104380 msgr2=0x7faa40199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.706+0000 7faa3f200640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40104380 0x7faa40199710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa2800e9c0 tx=0x7faa2800ee80 comp rx=0 tx=0).stop 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.707+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 shutdown_connections 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.707+0000 7faa3f200640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7faa240a20c0 0x7faa240a4580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.707+0000 7faa3f200640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa40106cb0 0x7faa401a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.707+0000 7faa3f200640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa40104380 0x7faa40199710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.707+0000 7faa3f200640 1 --2- 172.21.15.49:0/2887655121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa40101a50 0x7faa401991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.707+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 >> 172.21.15.49:0/2887655121 conn(0x7faa400fb460 msgr2=0x7faa40101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.707+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 shutdown_connections 2024-02-26T06:20:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:24.707+0000 7faa3f200640 1 -- 172.21.15.49:0/2887655121 wait complete. 2024-02-26T06:20:25.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:24 smithi080 ceph-mon[36206]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:25.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:24 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2887655121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:25.080 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:24 smithi080 systemd[1]: Starting Ceph mds.b for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:20:25.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:24 smithi115 ceph-mon[36426]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:25.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:24 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2887655121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:24 smithi049 ceph-mon[31696]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:24 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2887655121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:25.426 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:25 smithi080 podman[66911]: 2024-02-26 06:20:25.078463193 +0000 UTC m=+0.017489425 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:20:25.426 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:25 smithi080 podman[66911]: 2024-02-26 06:20:25.186900418 +0000 UTC m=+0.125926643 container create c493e60d4f9a3aeef2f62eef49dba280af207b79a7db6abac1bd2fcd83dc4cf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-26T06:20:25.426 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:25 smithi080 podman[66911]: 2024-02-26 06:20:25.423381726 +0000 UTC m=+0.362407974 container init c493e60d4f9a3aeef2f62eef49dba280af207b79a7db6abac1bd2fcd83dc4cf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-02-26T06:20:25.426 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:25 smithi080 podman[66911]: 2024-02-26 06:20:25.426035797 +0000 UTC m=+0.365062022 container start c493e60d4f9a3aeef2f62eef49dba280af207b79a7db6abac1bd2fcd83dc4cf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-26T06:20:25.830 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:25 smithi080 ceph-mds[66930]: -- 172.21.15.80:0/4167334856 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55825105a1a0 con 0x558251031000 2024-02-26T06:20:25.830 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:25 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: starting mds.b at 2024-02-26T06:20:25.830 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:25 smithi080 bash[66911]: c493e60d4f9a3aeef2f62eef49dba280af207b79a7db6abac1bd2fcd83dc4cf3 2024-02-26T06:20:25.830 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:25 smithi080 systemd[1]: Started Ceph mds.b for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:20:26.728 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:26.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] up:boot 2024-02-26T06:20:26.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: fsmap 12 up:standby 2024-02-26T06:20:26.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:20:26.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:26 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:26.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:26.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] up:boot 2024-02-26T06:20:26.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: fsmap 12 up:standby 2024-02-26T06:20:26.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:20:26.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:26 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:26.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:26.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:26.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] up:boot 2024-02-26T06:20:26.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: fsmap 12 up:standby 2024-02-26T06:20:26.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:20:26.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:26 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 -- 172.21.15.49:0/3653773394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd81045b0 msgr2=0x7fcbd8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/3653773394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd81045b0 0x7fcbd8104a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009900 tx=0x7fcbc402bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 -- 172.21.15.49:0/3653773394 shutdown_connections 2024-02-26T06:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/3653773394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd81045b0 0x7fcbd8104a30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/3653773394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd8104f70 0x7fcbd810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/3653773394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd81033c0 0x7fcbd81037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 -- 172.21.15.49:0/3653773394 >> 172.21.15.49:0/3653773394 conn(0x7fcbd80fe970 msgr2=0x7fcbd8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 -- 172.21.15.49:0/3653773394 shutdown_connections 2024-02-26T06:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.928+0000 7fcbdfe00640 1 -- 172.21.15.49:0/3653773394 wait complete. 2024-02-26T06:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.929+0000 7fcbdfe00640 1 Processor -- start 2024-02-26T06:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.929+0000 7fcbdfe00640 1 -- start start 2024-02-26T06:20:26.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.929+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd81033c0 0x7fcbd8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:26.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.929+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81045b0 0x7fcbd8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:26.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.929+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd8104f70 0x7fcbd81a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:26.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdfe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd81a0f10 con 0x7fcbd81045b0 2024-02-26T06:20:26.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdfe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd81a1080 con 0x7fcbd8104f70 2024-02-26T06:20:26.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdfe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd81a1240 con 0x7fcbd81033c0 2024-02-26T06:20:26.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdd400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd81033c0 0x7fcbd8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:26.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd8104f70 0x7fcbd81a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:26.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdd400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd81033c0 0x7fcbd8199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60144/0 (socket says 172.21.15.49:60144) 2024-02-26T06:20:26.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdd400640 1 -- 172.21.15.49:0/1098061939 learned_addr learned my addr 172.21.15.49:0/1098061939 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:26.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd8104f70 0x7fcbd81a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50584/0 (socket says 172.21.15.49:50584) 2024-02-26T06:20:26.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdca00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81045b0 0x7fcbd8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:26.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdde00640 1 -- 172.21.15.49:0/1098061939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd81033c0 msgr2=0x7fcbd8199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdde00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd81033c0 0x7fcbd8199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdde00640 1 -- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81045b0 msgr2=0x7fcbd8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdde00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81045b0 0x7fcbd8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdde00640 1 -- 172.21.15.49:0/1098061939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc4009560 con 0x7fcbd8104f70 2024-02-26T06:20:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.930+0000 7fcbdd400640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd81033c0 0x7fcbd8199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.931+0000 7fcbdde00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd8104f70 0x7fcbd81a09d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009fd0 tx=0x7fcbd000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.931+0000 7fcbd6000640 1 -- 172.21.15.49:0/1098061939 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000ce20 con 0x7fcbd8104f70 2024-02-26T06:20:26.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.931+0000 7fcbd6000640 1 -- 172.21.15.49:0/1098061939 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbd0005020 con 0x7fcbd8104f70 2024-02-26T06:20:26.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.931+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd81a1520 con 0x7fcbd8104f70 2024-02-26T06:20:26.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.931+0000 7fcbd6000640 1 -- 172.21.15.49:0/1098061939 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0010690 con 0x7fcbd8104f70 2024-02-26T06:20:26.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.931+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd81a1a00 con 0x7fcbd8104f70 2024-02-26T06:20:26.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.932+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd810a6e0 con 0x7fcbd8104f70 2024-02-26T06:20:26.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.934+0000 7fcbd6000640 1 -- 172.21.15.49:0/1098061939 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcbd0005500 con 0x7fcbd8104f70 2024-02-26T06:20:26.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.935+0000 7fcbd6000640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fcbc00a2140 0x7fcbc00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:26.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.935+0000 7fcbd6000640 1 -- 172.21.15.49:0/1098061939 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcbd0014070 con 0x7fcbd8104f70 2024-02-26T06:20:26.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.936+0000 7fcbdd400640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fcbc00a2140 0x7fcbc00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.937+0000 7fcbdd400640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fcbc00a2140 0x7fcbc00a4600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005fd0 tx=0x7fcbcc00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:26.939+0000 7fcbd6000640 1 -- 172.21.15.49:0/1098061939 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcbd00972c0 con 0x7fcbd8104f70 2024-02-26T06:20:27.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.193+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbd810ee20 con 0x7fcbd8104f70 2024-02-26T06:20:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.194+0000 7fcbd6000640 1 -- 172.21.15.49:0/1098061939 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10031 (secure 0 0 0) 0x7fcbd0005190 con 0x7fcbd8104f70 2024-02-26T06:20:27.195 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:27.196 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24572,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-02-26T06:20:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 13 2024-02-26T06:20:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fcbc00a2140 msgr2=0x7fcbc00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fcbc00a2140 0x7fcbc00a4600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005fd0 tx=0x7fcbcc00b4a0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd8104f70 msgr2=0x7fcbd81a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd8104f70 0x7fcbd81a09d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009fd0 tx=0x7fcbd000ef10 comp rx=0 tx=0).stop 2024-02-26T06:20:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 shutdown_connections 2024-02-26T06:20:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fcbc00a2140 0x7fcbc00a4600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbd8104f70 0x7fcbd81a09d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81045b0 0x7fcbd8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 --2- 172.21.15.49:0/1098061939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd81033c0 0x7fcbd8199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.196+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 >> 172.21.15.49:0/1098061939 conn(0x7fcbd80fe970 msgr2=0x7fcbd81078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:27.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.197+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 shutdown_connections 2024-02-26T06:20:27.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.197+0000 7fcbdfe00640 1 -- 172.21.15.49:0/1098061939 wait complete. 2024-02-26T06:20:27.217 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:20:27.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 -- 172.21.15.49:0/2715499353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818073b80 msgr2=0x7f4818073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 --2- 172.21.15.49:0/2715499353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818073b80 0x7f4818073fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4804009900 tx=0x7f480402bcc0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 -- 172.21.15.49:0/2715499353 shutdown_connections 2024-02-26T06:20:27.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 --2- 172.21.15.49:0/2715499353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818073b80 0x7f4818073fe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 --2- 172.21.15.49:0/2715499353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4818108e70 0x7f481810b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 --2- 172.21.15.49:0/2715499353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4818075160 0x7f48180735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 -- 172.21.15.49:0/2715499353 >> 172.21.15.49:0/2715499353 conn(0x7f48180fc550 msgr2=0x7f48180fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:27.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 -- 172.21.15.49:0/2715499353 shutdown_connections 2024-02-26T06:20:27.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.426+0000 7f481d000640 1 -- 172.21.15.49:0/2715499353 wait complete. 2024-02-26T06:20:27.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.427+0000 7f481d000640 1 Processor -- start 2024-02-26T06:20:27.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f481d000640 1 -- start start 2024-02-26T06:20:27.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f481d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4818073b80 0x7f48181992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f481d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4818075160 0x7f48181997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f481d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818108e70 0x7f48181a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f481d000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48181a0da0 con 0x7f4818075160 2024-02-26T06:20:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f481d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48181a0f10 con 0x7f4818108e70 2024-02-26T06:20:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f481d000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48181a10d0 con 0x7f4818073b80 2024-02-26T06:20:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4816a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4818075160 0x7f48181997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4818073b80 0x7f48181992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4818073b80 0x7f48181992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60160/0 (socket says 172.21.15.49:60160) 2024-02-26T06:20:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817400640 1 -- 172.21.15.49:0/498883495 learned_addr learned my addr 172.21.15.49:0/498883495 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:27.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817e00640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818108e70 0x7f48181a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:27.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817400640 1 -- 172.21.15.49:0/498883495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818108e70 msgr2=0x7f48181a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817400640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818108e70 0x7f48181a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817400640 1 -- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4818075160 msgr2=0x7f48181997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817400640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4818075160 0x7f48181997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.428+0000 7f4817400640 1 -- 172.21.15.49:0/498883495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808009490 con 0x7f4818073b80 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.429+0000 7f4817400640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4818073b80 0x7f48181992b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f480800ea60 tx=0x7f480800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.429+0000 7f480fe00640 1 -- 172.21.15.49:0/498883495 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800cd20 con 0x7f4818073b80 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.429+0000 7f480fe00640 1 -- 172.21.15.49:0/498883495 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f480800ce80 con 0x7f4818073b80 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.429+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4810009840 con 0x7f4818073b80 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.429+0000 7f480fe00640 1 -- 172.21.15.49:0/498883495 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808010690 con 0x7f4818073b80 2024-02-26T06:20:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.429+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4804009560 con 0x7f4818073b80 2024-02-26T06:20:27.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.430+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4818107c30 con 0x7f4818073b80 2024-02-26T06:20:27.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.432+0000 7f480fe00640 1 -- 172.21.15.49:0/498883495 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4808003ee0 con 0x7f4818073b80 2024-02-26T06:20:27.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.433+0000 7f480fe00640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47fc0a20c0 0x7f47fc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:27.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.434+0000 7f480fe00640 1 -- 172.21.15.49:0/498883495 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4808014070 con 0x7f4818073b80 2024-02-26T06:20:27.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.434+0000 7f4816a00640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47fc0a20c0 0x7f47fc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:27.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.435+0000 7f4816a00640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47fc0a20c0 0x7f47fc0a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4804000c00 tx=0x7f480402c870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:27.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.437+0000 7f480fe00640 1 -- 172.21.15.49:0/498883495 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f48080970b0 con 0x7f4818073b80 2024-02-26T06:20:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.642+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f481802cda0 con 0x7f4818073b80 2024-02-26T06:20:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.644+0000 7f480fe00640 1 -- 172.21.15.49:0/498883495 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18443 (secure 0 0 0) 0x7f480809a770 con 0x7f4818073b80 2024-02-26T06:20:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:27.645 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:19:34.587856+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:20:27.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.646+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47fc0a20c0 msgr2=0x7f47fc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.646+0000 7f481d000640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47fc0a20c0 0x7f47fc0a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4804000c00 tx=0x7f480402c870 comp rx=0 tx=0).stop 2024-02-26T06:20:27.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.646+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4818073b80 msgr2=0x7f48181992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.646+0000 7f481d000640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4818073b80 0x7f48181992b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f480800ea60 tx=0x7f480800ee60 comp rx=0 tx=0).stop 2024-02-26T06:20:27.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.647+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 shutdown_connections 2024-02-26T06:20:27.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.647+0000 7f481d000640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f47fc0a20c0 0x7f47fc0a4580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.647+0000 7f481d000640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818108e70 0x7f48181a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.647+0000 7f481d000640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4818075160 0x7f48181997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.647+0000 7f481d000640 1 --2- 172.21.15.49:0/498883495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4818073b80 0x7f48181992b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.647+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 >> 172.21.15.49:0/498883495 conn(0x7f48180fc550 msgr2=0x7f4818104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:27.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.647+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 shutdown_connections 2024-02-26T06:20:27.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.647+0000 7f481d000640 1 -- 172.21.15.49:0/498883495 wait complete. 2024-02-26T06:20:27.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:20:27.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:20:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:27 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1098061939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 -- 172.21.15.49:0/315441142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb28104410 msgr2=0x7feb28106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 --2- 172.21.15.49:0/315441142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb28104410 0x7feb28106800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb10009900 tx=0x7feb1002bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 -- 172.21.15.49:0/315441142 shutdown_connections 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 --2- 172.21.15.49:0/315441142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb28104410 0x7feb28106800 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 --2- 172.21.15.49:0/315441142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb28106d40 0x7feb28109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 --2- 172.21.15.49:0/315441142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb28101ae0 0x7feb28103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 -- 172.21.15.49:0/315441142 >> 172.21.15.49:0/315441142 conn(0x7feb280fb4f0 msgr2=0x7feb280fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 -- 172.21.15.49:0/315441142 shutdown_connections 2024-02-26T06:20:27.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.896+0000 7feb2da00640 1 -- 172.21.15.49:0/315441142 wait complete. 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.897+0000 7feb2da00640 1 Processor -- start 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.897+0000 7feb2da00640 1 -- start start 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb2da00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb28101ae0 0x7feb281992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb2da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb28104410 0x7feb28199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb2da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb28106d40 0x7feb281a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb2da00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb281a0de0 con 0x7feb28101ae0 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb2da00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb281a0f50 con 0x7feb28104410 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb2da00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb281a1110 con 0x7feb28106d40 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb27e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb28101ae0 0x7feb281992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb27e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb28101ae0 0x7feb281992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37402/0 (socket says 172.21.15.49:37402) 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb27e00640 1 -- 172.21.15.49:0/4153300662 learned_addr learned my addr 172.21.15.49:0/4153300662 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:27.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb2d000640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb28106d40 0x7feb281a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:27.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.898+0000 7feb1fe00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb28104410 0x7feb28199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:27.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb2d000640 1 -- 172.21.15.49:0/4153300662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb28104410 msgr2=0x7feb28199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb2d000640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb28104410 0x7feb28199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb2d000640 1 -- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb28101ae0 msgr2=0x7feb281992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:27.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb2d000640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb28101ae0 0x7feb281992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:27.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb2d000640 1 -- 172.21.15.49:0/4153300662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10009560 con 0x7feb28106d40 2024-02-26T06:20:27.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb2d000640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb28106d40 0x7feb281a08a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb20009fd0 tx=0x7feb2000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:27.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb25600640 1 -- 172.21.15.49:0/4153300662 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2000ce20 con 0x7feb28106d40 2024-02-26T06:20:27.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb25600640 1 -- 172.21.15.49:0/4153300662 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb20005020 con 0x7feb28106d40 2024-02-26T06:20:27.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb25600640 1 -- 172.21.15.49:0/4153300662 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20010690 con 0x7feb28106d40 2024-02-26T06:20:27.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb281a13f0 con 0x7feb28106d40 2024-02-26T06:20:27.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.899+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb281a1880 con 0x7feb28106d40 2024-02-26T06:20:27.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.901+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb28108890 con 0x7feb28106d40 2024-02-26T06:20:27.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.902+0000 7feb25600640 1 -- 172.21.15.49:0/4153300662 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7feb20005500 con 0x7feb28106d40 2024-02-26T06:20:27.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.904+0000 7feb25600640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7feb0c0a2140 0x7feb0c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:27.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.904+0000 7feb25600640 1 -- 172.21.15.49:0/4153300662 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb20014070 con 0x7feb28106d40 2024-02-26T06:20:27.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.904+0000 7feb27e00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7feb0c0a2140 0x7feb0c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:27.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.905+0000 7feb27e00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7feb0c0a2140 0x7feb0c0a4600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb18005fd0 tx=0x7feb1800b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:27.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:27.907+0000 7feb25600640 1 -- 172.21.15.49:0/4153300662 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb20097310 con 0x7feb28106d40 2024-02-26T06:20:27.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:27 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1098061939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:27.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:28.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:27 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1098061939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:28.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.129+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb2810ca90 con 0x7feb28106d40 2024-02-26T06:20:28.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.130+0000 7feb25600640 1 -- 172.21.15.49:0/4153300662 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb2810ca90 con 0x7feb28106d40 2024-02-26T06:20:28.130 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7feb0c0a2140 msgr2=0x7feb0c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7feb0c0a2140 0x7feb0c0a4600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb18005fd0 tx=0x7feb1800b480 comp rx=0 tx=0).stop 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb28106d40 msgr2=0x7feb281a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb28106d40 0x7feb281a08a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb20009fd0 tx=0x7feb2000ef10 comp rx=0 tx=0).stop 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 shutdown_connections 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7feb0c0a2140 0x7feb0c0a4600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb28104410 0x7feb28199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb28106d40 0x7feb281a08a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 --2- 172.21.15.49:0/4153300662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb28101ae0 0x7feb281992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 >> 172.21.15.49:0/4153300662 conn(0x7feb280fb4f0 msgr2=0x7feb281011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 shutdown_connections 2024-02-26T06:20:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.133+0000 7feb2da00640 1 -- 172.21.15.49:0/4153300662 wait complete. 2024-02-26T06:20:28.160 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T06:20:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 -- 172.21.15.49:0/1926147667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f329c104340 msgr2=0x7f329c106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 --2- 172.21.15.49:0/1926147667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f329c104340 0x7f329c106730 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3288009900 tx=0x7f328802bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 -- 172.21.15.49:0/1926147667 shutdown_connections 2024-02-26T06:20:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 --2- 172.21.15.49:0/1926147667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f329c106c70 0x7f329c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 --2- 172.21.15.49:0/1926147667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f329c104340 0x7f329c106730 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 --2- 172.21.15.49:0/1926147667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f329c101a10 0x7f329c103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 -- 172.21.15.49:0/1926147667 >> 172.21.15.49:0/1926147667 conn(0x7f329c0fb3e0 msgr2=0x7f329c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 -- 172.21.15.49:0/1926147667 shutdown_connections 2024-02-26T06:20:28.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.380+0000 7f329b200640 1 -- 172.21.15.49:0/1926147667 wait complete. 2024-02-26T06:20:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.381+0000 7f329b200640 1 Processor -- start 2024-02-26T06:20:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.382+0000 7f329b200640 1 -- start start 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.382+0000 7f329b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f329c101a10 0x7f329c197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.382+0000 7f329b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f329c104340 0x7f329c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.382+0000 7f329b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f329c106c70 0x7f329c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.382+0000 7f329b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c19eb40 con 0x7f329c104340 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.382+0000 7f329b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c19ecb0 con 0x7f329c101a10 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.382+0000 7f329b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c19ee70 con 0x7f329c106c70 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.382+0000 7f3299e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f329c101a10 0x7f329c197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f329c101a10 0x7f329c197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50632/0 (socket says 172.21.15.49:50632) 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299e00640 1 -- 172.21.15.49:0/492867500 learned_addr learned my addr 172.21.15.49:0/492867500 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299400640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f329c104340 0x7f329c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:28.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f329a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f329c106c70 0x7f329c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299e00640 1 -- 172.21.15.49:0/492867500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f329c106c70 msgr2=0x7f329c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299e00640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f329c106c70 0x7f329c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299e00640 1 -- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f329c104340 msgr2=0x7f329c197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299e00640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f329c104340 0x7f329c197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299e00640 1 -- 172.21.15.49:0/492867500 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294009840 con 0x7f329c101a10 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.383+0000 7f3299e00640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f329c101a10 0x7f329c197050 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f328c00e9f0 tx=0x7f328c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.384+0000 7f3292a00640 1 -- 172.21.15.49:0/492867500 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c009720 con 0x7f329c101a10 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.384+0000 7f3292a00640 1 -- 172.21.15.49:0/492867500 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f328c005500 con 0x7f329c101a10 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.384+0000 7f3292a00640 1 -- 172.21.15.49:0/492867500 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c010430 con 0x7f329c101a10 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.384+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3288009560 con 0x7f329c101a10 2024-02-26T06:20:28.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.384+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c19f4a0 con 0x7f329c101a10 2024-02-26T06:20:28.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.385+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c0fced0 con 0x7f329c101a10 2024-02-26T06:20:28.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.387+0000 7f3292a00640 1 -- 172.21.15.49:0/492867500 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f328c005020 con 0x7f329c101a10 2024-02-26T06:20:28.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.388+0000 7f3292a00640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32800a2140 0x7f32800a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:28.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.389+0000 7f3292a00640 1 -- 172.21.15.49:0/492867500 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f328c014070 con 0x7f329c101a10 2024-02-26T06:20:28.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.389+0000 7f3299400640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32800a2140 0x7f32800a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:28.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.389+0000 7f3299400640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32800a2140 0x7f32800a4600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f328802c240 tx=0x7f328802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.391+0000 7f3292a00640 1 -- 172.21.15.49:0/492867500 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f328c096fa0 con 0x7f329c101a10 2024-02-26T06:20:28.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.610+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f329c10c9f0 con 0x7f329c101a10 2024-02-26T06:20:28.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.611+0000 7f3292a00640 1 -- 172.21.15.49:0/492867500 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f328c09a660 con 0x7f329c101a10 2024-02-26T06:20:28.611 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:28.612 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-02-26T06:20:28.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:20:28.612 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2024-02-26T06:20:28.612 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2024-02-26T06:20:28.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:20:28.612 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-02-26T06:20:28.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.614+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32800a2140 msgr2=0x7f32800a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.614+0000 7f329b200640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32800a2140 0x7f32800a4600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f328802c240 tx=0x7f328802c7e0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.614+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f329c101a10 msgr2=0x7f329c197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.614+0000 7f329b200640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f329c101a10 0x7f329c197050 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f328c00e9f0 tx=0x7f328c00edf0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.615+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 shutdown_connections 2024-02-26T06:20:28.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.615+0000 7f329b200640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32800a2140 0x7f32800a4600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.615+0000 7f329b200640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f329c106c70 0x7f329c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.615+0000 7f329b200640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f329c104340 0x7f329c197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.615+0000 7f329b200640 1 --2- 172.21.15.49:0/492867500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f329c101a10 0x7f329c197050 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.615+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 >> 172.21.15.49:0/492867500 conn(0x7f329c0fb3e0 msgr2=0x7f329c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:28.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.615+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 shutdown_connections 2024-02-26T06:20:28.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.615+0000 7f329b200640 1 -- 172.21.15.49:0/492867500 wait complete. 2024-02-26T06:20:28.633 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-26T06:20:28.633 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 -- 172.21.15.49:0/423078572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2be0102c10 msgr2=0x7f2be01094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 --2- 172.21.15.49:0/423078572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2be0102c10 0x7f2be01094a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009ca0 tx=0x7f2bd802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 -- 172.21.15.49:0/423078572 shutdown_connections 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 --2- 172.21.15.49:0/423078572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2be0102250 0x7f2be01026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 --2- 172.21.15.49:0/423078572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2be0102c10 0x7f2be01094a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 --2- 172.21.15.49:0/423078572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be0101060 0x7f2be0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 -- 172.21.15.49:0/423078572 >> 172.21.15.49:0/423078572 conn(0x7f2be00fc5f0 msgr2=0x7f2be00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 -- 172.21.15.49:0/423078572 shutdown_connections 2024-02-26T06:20:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.841+0000 7f2be6200640 1 -- 172.21.15.49:0/423078572 wait complete. 2024-02-26T06:20:28.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.842+0000 7f2be6200640 1 Processor -- start 2024-02-26T06:20:28.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.842+0000 7f2be6200640 1 -- start start 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be6200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be0101060 0x7f2be01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be6200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2be0102250 0x7f2be01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be6200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2be0102c10 0x7f2be01a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be6200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a0d90 con 0x7f2be0102c10 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be6200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a0f00 con 0x7f2be0102250 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be6200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a10c0 con 0x7f2be0101060 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be4e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be0101060 0x7f2be01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2be0102250 0x7f2be01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2be0102c10 0x7f2be01a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be4e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be0101060 0x7f2be01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60198/0 (socket says 172.21.15.49:60198) 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2be0102250 0x7f2be01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50640/0 (socket says 172.21.15.49:50640) 2024-02-26T06:20:28.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be4e00640 1 -- 172.21.15.49:0/426627524 learned_addr learned my addr 172.21.15.49:0/426627524 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be4e00640 1 -- 172.21.15.49:0/426627524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2be0102250 msgr2=0x7f2be01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be4e00640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2be0102250 0x7f2be01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be4e00640 1 -- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2be0102c10 msgr2=0x7f2be01a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.843+0000 7f2be4e00640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2be0102c10 0x7f2be01a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.844+0000 7f2be4e00640 1 -- 172.21.15.49:0/426627524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8009900 con 0x7f2be0101060 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.844+0000 7f2bdfe00640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2be0102250 0x7f2be01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.844+0000 7f2be4e00640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be0101060 0x7f2be01992a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400e9c0 tx=0x7f2bd400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.844+0000 7f2bdd600640 1 -- 172.21.15.49:0/426627524 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4009700 con 0x7f2be0101060 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.844+0000 7f2bdd600640 1 -- 172.21.15.49:0/426627524 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bd400cd80 con 0x7f2be0101060 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.844+0000 7f2bdd600640 1 -- 172.21.15.49:0/426627524 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4010430 con 0x7f2be0101060 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.844+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be01a13a0 con 0x7f2be0101060 2024-02-26T06:20:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.845+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be01a1900 con 0x7f2be0101060 2024-02-26T06:20:28.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.846+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be010ca40 con 0x7f2be0101060 2024-02-26T06:20:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.849+0000 7f2bdd600640 1 -- 172.21.15.49:0/426627524 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2bd4003680 con 0x7f2be0101060 2024-02-26T06:20:28.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.851+0000 7f2bdd600640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2bc80a2140 0x7f2bc80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:28.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.851+0000 7f2bdd600640 1 -- 172.21.15.49:0/426627524 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2bd4014070 con 0x7f2be0101060 2024-02-26T06:20:28.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.851+0000 7f2bdfe00640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2bc80a2140 0x7f2bc80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.851+0000 7f2bdfe00640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2bc80a2140 0x7f2bc80a4600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2be019a7b0 tx=0x7f2bcc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:28.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:28.855+0000 7f2bdd600640 1 -- 172.21.15.49:0/426627524 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2bd4096f30 con 0x7f2be0101060 2024-02-26T06:20:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:28 smithi049 ceph-mon[31696]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:28 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/498883495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:28 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4153300662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:20:29.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.060+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2be010cc00 con 0x7f2be0101060 2024-02-26T06:20:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:28 smithi080 ceph-mon[36206]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:28 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/498883495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:28 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4153300662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:20:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:28 smithi115 ceph-mon[36426]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:28 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/498883495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:29.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:28 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4153300662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:20:29.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.848+0000 7f2bdd600640 1 -- 172.21.15.49:0/426627524 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2bd409a5f0 con 0x7f2be0101060 2024-02-26T06:20:29.852 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2024-02-26T06:20:29.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2bc80a2140 msgr2=0x7f2bc80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:29.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2bc80a2140 0x7f2bc80a4600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2be019a7b0 tx=0x7f2bcc00b410 comp rx=0 tx=0).stop 2024-02-26T06:20:29.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be0101060 msgr2=0x7f2be01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:29.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be0101060 0x7f2be01992a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400e9c0 tx=0x7f2bd400ee80 comp rx=0 tx=0).stop 2024-02-26T06:20:29.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 shutdown_connections 2024-02-26T06:20:29.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2bc80a2140 0x7f2bc80a4600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:29.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2be0102250 0x7f2be01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:29.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2be0102c10 0x7f2be01a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:29.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 --2- 172.21.15.49:0/426627524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be0101060 0x7f2be01992a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:29.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.856+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 >> 172.21.15.49:0/426627524 conn(0x7f2be00fc5f0 msgr2=0x7f2be00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:29.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.857+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 shutdown_connections 2024-02-26T06:20:29.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:29.857+0000 7f2be6200640 1 -- 172.21.15.49:0/426627524 wait complete. 2024-02-26T06:20:29.876 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.066+0000 7fca84600640 1 -- 172.21.15.49:0/2870640207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0751c0 msgr2=0x7fca7c073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.066+0000 7fca84600640 1 --2- 172.21.15.49:0/2870640207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0751c0 0x7fca7c073610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fca70009860 tx=0x7fca7002bb70 comp rx=0 tx=0).stop 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.067+0000 7fca84600640 1 -- 172.21.15.49:0/2870640207 shutdown_connections 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.067+0000 7fca84600640 1 --2- 172.21.15.49:0/2870640207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7c109050 0x7fca7c10b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.067+0000 7fca84600640 1 --2- 172.21.15.49:0/2870640207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca7c073be0 0x7fca7c074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.067+0000 7fca84600640 1 --2- 172.21.15.49:0/2870640207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0751c0 0x7fca7c073610 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.067+0000 7fca84600640 1 -- 172.21.15.49:0/2870640207 >> 172.21.15.49:0/2870640207 conn(0x7fca7c0fc700 msgr2=0x7fca7c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.067+0000 7fca84600640 1 -- 172.21.15.49:0/2870640207 shutdown_connections 2024-02-26T06:20:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.067+0000 7fca84600640 1 -- 172.21.15.49:0/2870640207 wait complete. 2024-02-26T06:20:30.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.068+0000 7fca84600640 1 Processor -- start 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.068+0000 7fca84600640 1 -- start start 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.068+0000 7fca84600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c073be0 0x7fca7c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.068+0000 7fca84600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7c0751c0 0x7fca7c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca84600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca7c109050 0x7fca7c1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca84600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c1a0f60 con 0x7fca7c109050 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca84600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c1a10d0 con 0x7fca7c0751c0 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca84600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c1a1290 con 0x7fca7c073be0 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c073be0 0x7fca7c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7c0751c0 0x7fca7c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c073be0 0x7fca7c199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:52288/0 (socket says 172.21.15.49:52288) 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81c00640 1 -- 172.21.15.49:0/3111678063 learned_addr learned my addr 172.21.15.49:0/3111678063 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca82600640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca7c109050 0x7fca7c1a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81200640 1 -- 172.21.15.49:0/3111678063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c073be0 msgr2=0x7fca7c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81200640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c073be0 0x7fca7c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81200640 1 -- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca7c109050 msgr2=0x7fca7c1a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81200640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca7c109050 0x7fca7c1a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81200640 1 -- 172.21.15.49:0/3111678063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca70009490 con 0x7fca7c0751c0 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.069+0000 7fca81200640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7c0751c0 0x7fca7c1999b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca6800e9f0 tx=0x7fca6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.070+0000 7fca7a000640 1 -- 172.21.15.49:0/3111678063 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6800ccc0 con 0x7fca7c0751c0 2024-02-26T06:20:30.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.070+0000 7fca7a000640 1 -- 172.21.15.49:0/3111678063 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca6800ce20 con 0x7fca7c0751c0 2024-02-26T06:20:30.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.070+0000 7fca7a000640 1 -- 172.21.15.49:0/3111678063 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68018920 con 0x7fca7c0751c0 2024-02-26T06:20:30.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.070+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c1a1570 con 0x7fca7c0751c0 2024-02-26T06:20:30.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.070+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c1a1a30 con 0x7fca7c0751c0 2024-02-26T06:20:30.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.071+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c107d80 con 0x7fca7c0751c0 2024-02-26T06:20:30.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.073+0000 7fca7a000640 1 -- 172.21.15.49:0/3111678063 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fca68018a80 con 0x7fca7c0751c0 2024-02-26T06:20:30.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.075+0000 7fca7a000640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca640a20c0 0x7fca640a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:30.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.075+0000 7fca7a000640 1 -- 172.21.15.49:0/3111678063 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fca68014070 con 0x7fca7c0751c0 2024-02-26T06:20:30.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.075+0000 7fca81c00640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca640a20c0 0x7fca640a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:30.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.076+0000 7fca81c00640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca640a20c0 0x7fca640a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca7002c180 tx=0x7fca700058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:30.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.078+0000 7fca7a000640 1 -- 172.21.15.49:0/3111678063 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca68052d20 con 0x7fca7c0751c0 2024-02-26T06:20:30.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:29 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/492867500' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T06:20:30.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:30.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:30.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:29 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:29 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/492867500' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T06:20:30.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:30.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:30.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:29 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:29 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/492867500' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T06:20:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:30.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:30.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:29 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:30.274+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fca7c02ce40 con 0x7fca7c0751c0 2024-02-26T06:20:31.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.049+0000 7fca7a000640 1 -- 172.21.15.49:0/3111678063 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7fca6809aac0 con 0x7fca7c0751c0 2024-02-26T06:20:31.052 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2024-02-26T06:20:31.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.056+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca640a20c0 msgr2=0x7fca640a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:31.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.056+0000 7fca84600640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca640a20c0 0x7fca640a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca7002c180 tx=0x7fca700058b0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.056+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7c0751c0 msgr2=0x7fca7c1999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:31.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.056+0000 7fca84600640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7c0751c0 0x7fca7c1999b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca6800e9f0 tx=0x7fca6800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.057+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 shutdown_connections 2024-02-26T06:20:31.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.057+0000 7fca84600640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca640a20c0 0x7fca640a4580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.057+0000 7fca84600640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7c0751c0 0x7fca7c1999b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.057+0000 7fca84600640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca7c109050 0x7fca7c1a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.057+0000 7fca84600640 1 --2- 172.21.15.49:0/3111678063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c073be0 0x7fca7c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.057+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 >> 172.21.15.49:0/3111678063 conn(0x7fca7c0fc700 msgr2=0x7fca7c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.059+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 shutdown_connections 2024-02-26T06:20:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.059+0000 7fca84600640 1 -- 172.21.15.49:0/3111678063 wait complete. 2024-02-26T06:20:31.093 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-26T06:20:31.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 -- 172.21.15.49:0/344658899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d8102d40 msgr2=0x7fc8d81095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:31.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 --2- 172.21.15.49:0/344658899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d8102d40 0x7fc8d81095d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009be0 tx=0x7fc8d002bef0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 -- 172.21.15.49:0/344658899 shutdown_connections 2024-02-26T06:20:31.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 --2- 172.21.15.49:0/344658899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d8102d40 0x7fc8d81095d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 --2- 172.21.15.49:0/344658899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8d8102380 0x7fc8d8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 --2- 172.21.15.49:0/344658899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8d8101190 0x7fc8d81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 -- 172.21.15.49:0/344658899 >> 172.21.15.49:0/344658899 conn(0x7fc8d80fc720 msgr2=0x7fc8d80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:31.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 -- 172.21.15.49:0/344658899 shutdown_connections 2024-02-26T06:20:31.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.277+0000 7fc8dd000640 1 -- 172.21.15.49:0/344658899 wait complete. 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.278+0000 7fc8dd000640 1 Processor -- start 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.278+0000 7fc8dd000640 1 -- start start 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.278+0000 7fc8dd000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d8101190 0x7fc8d8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.278+0000 7fc8dd000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8d8102380 0x7fc8d8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.278+0000 7fc8dd000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8d8102d40 0x7fc8d819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.278+0000 7fc8dd000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d819ed30 con 0x7fc8d8102380 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.278+0000 7fc8dd000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d819eea0 con 0x7fc8d8101190 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.278+0000 7fc8dd000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d819f060 con 0x7fc8d8102d40 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8d8102d40 0x7fc8d819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d8101190 0x7fc8d8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d5400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8d8102380 0x7fc8d8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8d8102d40 0x7fc8d819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:52298/0 (socket says 172.21.15.49:52298) 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 -- 172.21.15.49:0/3491448748 learned_addr learned my addr 172.21.15.49:0/3491448748 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 -- 172.21.15.49:0/3491448748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d8101190 msgr2=0x7fc8d8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d8101190 0x7fc8d8197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 -- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8d8102380 msgr2=0x7fc8d8197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8d8102380 0x7fc8d8197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 -- 172.21.15.49:0/3491448748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c8009560 con 0x7fc8d8102d40 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.279+0000 7fc8d6800640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8d8102d40 0x7fc8d819e7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0033420 tx=0x7fc8d0033cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.280+0000 7fc8cea00640 1 -- 172.21.15.49:0/3491448748 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0003b80 con 0x7fc8d8102d40 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.280+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d0009840 con 0x7fc8d8102d40 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.280+0000 7fc8cea00640 1 -- 172.21.15.49:0/3491448748 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8d0003ce0 con 0x7fc8d8102d40 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.280+0000 7fc8cea00640 1 -- 172.21.15.49:0/3491448748 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00348c0 con 0x7fc8d8102d40 2024-02-26T06:20:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.280+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d819f600 con 0x7fc8d8102d40 2024-02-26T06:20:31.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.281+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d8108450 con 0x7fc8d8102d40 2024-02-26T06:20:31.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.283+0000 7fc8cea00640 1 -- 172.21.15.49:0/3491448748 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc8d0034a20 con 0x7fc8d8102d40 2024-02-26T06:20:31.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.284+0000 7fc8cea00640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc8bc0a20c0 0x7fc8bc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:31.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.284+0000 7fc8cea00640 1 -- 172.21.15.49:0/3491448748 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fc8d00f0260 con 0x7fc8d8102d40 2024-02-26T06:20:31.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.284+0000 7fc8d5e00640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc8bc0a20c0 0x7fc8bc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:31.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.285+0000 7fc8d5e00640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc8bc0a20c0 0x7fc8bc0a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009530 tx=0x7fc8c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:31.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.289+0000 7fc8cea00640 1 -- 172.21.15.49:0/3491448748 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc8d00b93f0 con 0x7fc8d8102d40 2024-02-26T06:20:31.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:31 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/426627524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:20:31.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:31 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:20:31.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:31 smithi080 ceph-mon[36206]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:31.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:31 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:31.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:31 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T06:20:31.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:31 smithi080 ceph-mon[36206]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T06:20:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:31 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/426627524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:20:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:31 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:20:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:31 smithi115 ceph-mon[36426]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:31.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:31 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:31.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:31 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T06:20:31.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:31 smithi115 ceph-mon[36426]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T06:20:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:31 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/426627524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:20:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:31 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:20:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:31 smithi049 ceph-mon[31696]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:31 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:31 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T06:20:31.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:31 smithi049 ceph-mon[31696]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T06:20:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:31.546+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc8d802ceb0 con 0x7fc8d8102d40 2024-02-26T06:20:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3111678063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:20:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:20:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3491448748' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:32 smithi080 ceph-mon[36206]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T06:20:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3111678063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:20:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:20:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T06:20:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T06:20:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3491448748' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:20:32.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:20:32.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T06:20:32.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:32 smithi115 ceph-mon[36426]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3111678063' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3491448748' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T06:20:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:32.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:32 smithi049 ceph-mon[31696]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T06:20:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.552+0000 7fc8cea00640 1 -- 172.21.15.49:0/3491448748 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc8d00bcab0 con 0x7fc8d8102d40 2024-02-26T06:20:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 2 and data pool 3 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc8bc0a20c0 msgr2=0x7fc8bc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc8bc0a20c0 0x7fc8bc0a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009530 tx=0x7fc8c8009260 comp rx=0 tx=0).stop 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8d8102d40 msgr2=0x7fc8d819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8d8102d40 0x7fc8d819e7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0033420 tx=0x7fc8d0033cb0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 shutdown_connections 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc8bc0a20c0 0x7fc8bc0a4580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8d8102d40 0x7fc8d819e7f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8d8102380 0x7fc8d8197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 --2- 172.21.15.49:0/3491448748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d8101190 0x7fc8d8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.556+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 >> 172.21.15.49:0/3491448748 conn(0x7fc8d80fc720 msgr2=0x7fc8d8105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.557+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 shutdown_connections 2024-02-26T06:20:32.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.557+0000 7fc8dd000640 1 -- 172.21.15.49:0/3491448748 wait complete. 2024-02-26T06:20:32.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:20:32.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.783+0000 7f5570200640 1 -- 172.21.15.49:0/659782619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5568104f70 msgr2=0x7f556810b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.783+0000 7f5570200640 1 --2- 172.21.15.49:0/659782619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5568104f70 0x7f556810b800 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5560009ca0 tx=0x7f556002c0f0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.783+0000 7f5570200640 1 -- 172.21.15.49:0/659782619 shutdown_connections 2024-02-26T06:20:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.783+0000 7f5570200640 1 --2- 172.21.15.49:0/659782619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55681045b0 0x7f5568104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.783+0000 7f5570200640 1 --2- 172.21.15.49:0/659782619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5568104f70 0x7f556810b800 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.783+0000 7f5570200640 1 --2- 172.21.15.49:0/659782619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55681033c0 0x7f55681037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.784+0000 7f5570200640 1 -- 172.21.15.49:0/659782619 >> 172.21.15.49:0/659782619 conn(0x7f55680fe970 msgr2=0x7f5568100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.784+0000 7f5570200640 1 -- 172.21.15.49:0/659782619 shutdown_connections 2024-02-26T06:20:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.784+0000 7f5570200640 1 -- 172.21.15.49:0/659782619 wait complete. 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.785+0000 7f5570200640 1 Processor -- start 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.785+0000 7f5570200640 1 -- start start 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.785+0000 7f5570200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55681033c0 0x7f55681993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.785+0000 7f5570200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55681045b0 0x7f5568199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.785+0000 7f5570200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5568104f70 0x7f55681a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.785+0000 7f5570200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55681a0ec0 con 0x7f55681045b0 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.785+0000 7f5570200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55681a1030 con 0x7f5568104f70 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.785+0000 7f5570200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55681a11f0 con 0x7f55681033c0 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556ce00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55681045b0 0x7f5568199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556e200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5568104f70 0x7f55681a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556d800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55681033c0 0x7f55681993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556ce00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55681045b0 0x7f5568199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44922/0 (socket says 172.21.15.49:44922) 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556ce00640 1 -- 172.21.15.49:0/3404424436 learned_addr learned my addr 172.21.15.49:0/3404424436 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556e200640 1 -- 172.21.15.49:0/3404424436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55681033c0 msgr2=0x7f55681993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556e200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55681033c0 0x7f55681993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556e200640 1 -- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55681045b0 msgr2=0x7f5568199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556e200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55681045b0 0x7f5568199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556e200640 1 -- 172.21.15.49:0/3404424436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560009900 con 0x7f5568104f70 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.786+0000 7f556e200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5568104f70 0x7f55681a0980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f556002c630 tx=0x7f5560034a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.787+0000 7f5566000640 1 -- 172.21.15.49:0/3404424436 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600329b0 con 0x7f5568104f70 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.787+0000 7f5566000640 1 -- 172.21.15.49:0/3404424436 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5560003e50 con 0x7f5568104f70 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.787+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55681a1470 con 0x7f5568104f70 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.787+0000 7f5566000640 1 -- 172.21.15.49:0/3404424436 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560040450 con 0x7f5568104f70 2024-02-26T06:20:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.787+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55681a1940 con 0x7f5568104f70 2024-02-26T06:20:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.788+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556810a6e0 con 0x7f5568104f70 2024-02-26T06:20:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.790+0000 7f5566000640 1 -- 172.21.15.49:0/3404424436 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f556000a040 con 0x7f5568104f70 2024-02-26T06:20:32.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.791+0000 7f5566000640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f55500a2140 0x7f55500a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:32.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.791+0000 7f5566000640 1 -- 172.21.15.49:0/3404424436 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55600f04d0 con 0x7f5568104f70 2024-02-26T06:20:32.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.791+0000 7f556d800640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f55500a2140 0x7f55500a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:32.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.792+0000 7f556d800640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f55500a2140 0x7f55500a4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5568104410 tx=0x7f555c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:32.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:32.795+0000 7f5566000640 1 -- 172.21.15.49:0/3404424436 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55600b9550 con 0x7f5568104f70 2024-02-26T06:20:33.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.001+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f556802ce40 con 0x7f5568104f70 2024-02-26T06:20:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.003+0000 7f5566000640 1 -- 172.21.15.49:0/3404424436 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22206 (secure 0 0 0) 0x7f55600bcc10 con 0x7f5568104f70 2024-02-26T06:20:33.004 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:33.004 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:20:32.561576+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f55500a2140 msgr2=0x7f55500a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f55500a2140 0x7f55500a4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5568104410 tx=0x7f555c005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5568104f70 msgr2=0x7f55681a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5568104f70 0x7f55681a0980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f556002c630 tx=0x7f5560034a90 comp rx=0 tx=0).stop 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 shutdown_connections 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f55500a2140 0x7f55500a4600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5568104f70 0x7f55681a0980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55681045b0 0x7f5568199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 --2- 172.21.15.49:0/3404424436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55681033c0 0x7f55681993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 >> 172.21.15.49:0/3404424436 conn(0x7f55680fe970 msgr2=0x7f55681078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:33.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.006+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 shutdown_connections 2024-02-26T06:20:33.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.007+0000 7f5570200640 1 -- 172.21.15.49:0/3404424436 wait complete. 2024-02-26T06:20:33.026 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:20:33.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.239+0000 7f6fe3a00640 1 -- 172.21.15.49:0/1352088974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc101cc0 msgr2=0x7f6fdc1040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.239+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/1352088974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc101cc0 0x7f6fdc1040b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc02bc40 comp rx=0 tx=0).stop 2024-02-26T06:20:33.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.239+0000 7f6fe3a00640 1 -- 172.21.15.49:0/1352088974 shutdown_connections 2024-02-26T06:20:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.239+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/1352088974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc106f20 0x7f6fdc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.239+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/1352088974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fdc1045f0 0x7f6fdc1069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.239+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/1352088974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc101cc0 0x7f6fdc1040b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.239+0000 7f6fe3a00640 1 -- 172.21.15.49:0/1352088974 >> 172.21.15.49:0/1352088974 conn(0x7f6fdc0fb610 msgr2=0x7f6fdc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.239+0000 7f6fe3a00640 1 -- 172.21.15.49:0/1352088974 shutdown_connections 2024-02-26T06:20:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.240+0000 7f6fe3a00640 1 -- 172.21.15.49:0/1352088974 wait complete. 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.240+0000 7f6fe3a00640 1 Processor -- start 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe3a00640 1 -- start start 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe3a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101cc0 0x7f6fdc199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe3a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc1045f0 0x7f6fdc1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe3a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fdc106f20 0x7f6fdc1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe3a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc1a0f70 con 0x7f6fdc1045f0 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe3a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc1a10e0 con 0x7f6fdc101cc0 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe3a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc1a12a0 con 0x7f6fdc106f20 2024-02-26T06:20:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc1045f0 0x7f6fdc1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc1045f0 0x7f6fdc1999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44936/0 (socket says 172.21.15.49:44936) 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fdbe00640 1 -- 172.21.15.49:0/696667876 learned_addr learned my addr 172.21.15.49:0/696667876 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe1a00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fdc106f20 0x7f6fdc1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.241+0000 7f6fe1000640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101cc0 0x7f6fdc199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fdbe00640 1 -- 172.21.15.49:0/696667876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fdc106f20 msgr2=0x7f6fdc1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fdbe00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fdc106f20 0x7f6fdc1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fdbe00640 1 -- 172.21.15.49:0/696667876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101cc0 msgr2=0x7f6fdc199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fdbe00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101cc0 0x7f6fdc199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fdbe00640 1 -- 172.21.15.49:0/696667876 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc009560 con 0x7f6fdc1045f0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fdbe00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc1045f0 0x7f6fdc1999c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400e9f0 tx=0x7f6fc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fd9600640 1 -- 172.21.15.49:0/696667876 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40096c0 con 0x7f6fdc1045f0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc1a1580 con 0x7f6fdc1045f0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fd9600640 1 -- 172.21.15.49:0/696667876 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fc4005500 con 0x7f6fdc1045f0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.242+0000 7f6fd9600640 1 -- 172.21.15.49:0/696667876 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4010430 con 0x7f6fdc1045f0 2024-02-26T06:20:33.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.243+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc1a1a40 con 0x7f6fdc1045f0 2024-02-26T06:20:33.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.244+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc108a10 con 0x7f6fdc1045f0 2024-02-26T06:20:33.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.246+0000 7f6fd9600640 1 -- 172.21.15.49:0/696667876 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6fc4005000 con 0x7f6fdc1045f0 2024-02-26T06:20:33.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.248+0000 7f6fd9600640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6fc00a20c0 0x7f6fc00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:33.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.248+0000 7f6fe1000640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6fc00a20c0 0x7f6fc00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:33.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.248+0000 7f6fd9600640 1 -- 172.21.15.49:0/696667876 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6fc4014070 con 0x7f6fdc1045f0 2024-02-26T06:20:33.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.249+0000 7f6fe1000640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6fc00a20c0 0x7f6fc00a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009230 tx=0x7f6fcc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:33.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.251+0000 7f6fd9600640 1 -- 172.21.15.49:0/696667876 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6fc4097190 con 0x7f6fdc1045f0 2024-02-26T06:20:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: pgmap v316: 97 pgs: 4 creating+peering, 1 active+clean, 92 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T06:20:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:33 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3404424436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: pgmap v316: 97 pgs: 4 creating+peering, 1 active+clean, 92 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T06:20:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T06:20:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T06:20:33.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T06:20:33.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T06:20:33.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:33.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:33.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:33 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3404424436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:33.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: pgmap v316: 97 pgs: 4 creating+peering, 1 active+clean, 92 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:20:33.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:33 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3404424436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:33.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.456+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fdc10cbd0 con 0x7f6fdc1045f0 2024-02-26T06:20:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.461+0000 7f6fd9600640 1 -- 172.21.15.49:0/696667876 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22206 (secure 0 0 0) 0x7f6fc409a850 con 0x7f6fdc1045f0 2024-02-26T06:20:33.462 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:33.462 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:20:32.561576+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:20:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6fc00a20c0 msgr2=0x7f6fc00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6fc00a20c0 0x7f6fc00a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009230 tx=0x7f6fcc005fd0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc1045f0 msgr2=0x7f6fdc1999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc1045f0 0x7f6fdc1999c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400e9f0 tx=0x7f6fc400eeb0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 shutdown_connections 2024-02-26T06:20:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f6fc00a20c0 0x7f6fc00a4580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fdc106f20 0x7f6fdc1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fdc1045f0 0x7f6fdc1999c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.463+0000 7f6fe3a00640 1 --2- 172.21.15.49:0/696667876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101cc0 0x7f6fdc199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.464+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 >> 172.21.15.49:0/696667876 conn(0x7f6fdc0fb610 msgr2=0x7f6fdc101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.464+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 shutdown_connections 2024-02-26T06:20:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.464+0000 7f6fe3a00640 1 -- 172.21.15.49:0/696667876 wait complete. 2024-02-26T06:20:33.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-26T06:20:33.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 -- 172.21.15.49:0/1714455730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd8100bd0 msgr2=0x7effd8101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 --2- 172.21.15.49:0/1714455730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd8100bd0 0x7effd8101050 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7effd0009860 tx=0x7effd002bb70 comp rx=0 tx=0).stop 2024-02-26T06:20:33.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 -- 172.21.15.49:0/1714455730 shutdown_connections 2024-02-26T06:20:33.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 --2- 172.21.15.49:0/1714455730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd8100bd0 0x7effd8101050 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 --2- 172.21.15.49:0/1714455730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd8100230 0x7effd8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 --2- 172.21.15.49:0/1714455730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80ff8d0 0x7effd80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 -- 172.21.15.49:0/1714455730 >> 172.21.15.49:0/1714455730 conn(0x7effd80fb4f0 msgr2=0x7effd80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:33.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 -- 172.21.15.49:0/1714455730 shutdown_connections 2024-02-26T06:20:33.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.722+0000 7effd7e00640 1 -- 172.21.15.49:0/1714455730 wait complete. 2024-02-26T06:20:33.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.724+0000 7effd7e00640 1 Processor -- start 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.724+0000 7effd7e00640 1 -- start start 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.724+0000 7effd7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd80ff8d0 0x7effd8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.724+0000 7effd7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd8100230 0x7effd81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.724+0000 7effd7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd8100bd0 0x7effd81a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.724+0000 7effd7e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81a0d60 con 0x7effd8100230 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.724+0000 7effd7e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81a0ed0 con 0x7effd8100bd0 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.724+0000 7effd7e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81a1090 con 0x7effd80ff8d0 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd8100230 0x7effd81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd8100230 0x7effd81997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44962/0 (socket says 172.21.15.49:44962) 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 -- 172.21.15.49:0/977798699 learned_addr learned my addr 172.21.15.49:0/977798699 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6a00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd80ff8d0 0x7effd8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd7400640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd8100bd0 0x7effd81a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 -- 172.21.15.49:0/977798699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd80ff8d0 msgr2=0x7effd8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd80ff8d0 0x7effd8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 -- 172.21.15.49:0/977798699 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd8100bd0 msgr2=0x7effd81a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:33.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd8100bd0 0x7effd81a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:33.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 -- 172.21.15.49:0/977798699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8009840 con 0x7effd8100230 2024-02-26T06:20:33.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.725+0000 7effd6000640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd8100230 0x7effd81997b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effc8009d50 tx=0x7effc800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:33.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.726+0000 7effcf400640 1 -- 172.21.15.49:0/977798699 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8009b40 con 0x7effd8100230 2024-02-26T06:20:33.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.726+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd0009490 con 0x7effd8100230 2024-02-26T06:20:33.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.726+0000 7effcf400640 1 -- 172.21.15.49:0/977798699 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effc8003b70 con 0x7effd8100230 2024-02-26T06:20:33.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.726+0000 7effcf400640 1 -- 172.21.15.49:0/977798699 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc80107c0 con 0x7effd8100230 2024-02-26T06:20:33.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.726+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd81a16f0 con 0x7effd8100230 2024-02-26T06:20:33.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.727+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd810a620 con 0x7effd8100230 2024-02-26T06:20:33.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.729+0000 7effcf400640 1 -- 172.21.15.49:0/977798699 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7effc8003f90 con 0x7effd8100230 2024-02-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.730+0000 7effcf400640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7effbc0a20c0 0x7effbc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.731+0000 7effcf400640 1 -- 172.21.15.49:0/977798699 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7effc8014070 con 0x7effd8100230 2024-02-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.731+0000 7effd6a00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7effbc0a20c0 0x7effbc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:33.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.732+0000 7effd6a00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7effbc0a20c0 0x7effbc0a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7effc0005fd0 tx=0x7effc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.735+0000 7effcf400640 1 -- 172.21.15.49:0/977798699 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7effc8097450 con 0x7effd8100230 2024-02-26T06:20:33.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:33.983+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7effd802cda0 con 0x7effd8100230 2024-02-26T06:20:34.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:34 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/696667876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:34.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:34 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] up:active 2024-02-26T06:20:34.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:34 smithi049 ceph-mon[31696]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:34.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:34 smithi049 ceph-mon[31696]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:34.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:34 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/977798699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:20:34.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:34 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/696667876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:34.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:34 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] up:active 2024-02-26T06:20:34.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:34 smithi080 ceph-mon[36206]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:34.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:34 smithi080 ceph-mon[36206]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:34.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:34 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/977798699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:20:34.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:34 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/696667876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:34.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:34 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] up:active 2024-02-26T06:20:34.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:34 smithi115 ceph-mon[36426]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:34.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:34 smithi115 ceph-mon[36426]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:34.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:34 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/977798699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:20:34.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.580+0000 7effcf400640 1 -- 172.21.15.49:0/977798699 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7effc809ab10 con 0x7effd8100230 2024-02-26T06:20:34.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.584+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7effbc0a20c0 msgr2=0x7effbc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.584+0000 7effd7e00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7effbc0a20c0 0x7effbc0a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7effc0005fd0 tx=0x7effc0009260 comp rx=0 tx=0).stop 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.584+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd8100230 msgr2=0x7effd81997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.584+0000 7effd7e00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd8100230 0x7effd81997b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effc8009d50 tx=0x7effc800c6a0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.585+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 shutdown_connections 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.585+0000 7effd7e00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7effbc0a20c0 0x7effbc0a4580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.585+0000 7effd7e00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd8100bd0 0x7effd81a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.585+0000 7effd7e00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd8100230 0x7effd81997b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.585+0000 7effd7e00640 1 --2- 172.21.15.49:0/977798699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd80ff8d0 0x7effd8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.585+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 >> 172.21.15.49:0/977798699 conn(0x7effd80fb4f0 msgr2=0x7effd80fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.585+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 shutdown_connections 2024-02-26T06:20:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.585+0000 7effd7e00640 1 -- 172.21.15.49:0/977798699 wait complete. 2024-02-26T06:20:34.609 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-02-26T06:20:34.610 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-02-26T06:20:34.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.852+0000 7f9c6dc00640 1 -- 172.21.15.49:0/3328508976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104e30 msgr2=0x7f9c6810b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:34.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.852+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/3328508976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104e30 0x7f9c6810b6c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009ca0 tx=0x7f9c6002c0f0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.853+0000 7f9c6dc00640 1 -- 172.21.15.49:0/3328508976 shutdown_connections 2024-02-26T06:20:34.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.853+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/3328508976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68104470 0x7f9c681048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.853+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/3328508976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104e30 0x7f9c6810b6c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.853+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/3328508976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68103280 0x7f9c681036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.853+0000 7f9c6dc00640 1 -- 172.21.15.49:0/3328508976 >> 172.21.15.49:0/3328508976 conn(0x7f9c680fe7f0 msgr2=0x7f9c68100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:34.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.853+0000 7f9c6dc00640 1 -- 172.21.15.49:0/3328508976 shutdown_connections 2024-02-26T06:20:34.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.853+0000 7f9c6dc00640 1 -- 172.21.15.49:0/3328508976 wait complete. 2024-02-26T06:20:34.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.854+0000 7f9c6dc00640 1 Processor -- start 2024-02-26T06:20:34.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.855+0000 7f9c6dc00640 1 -- start start 2024-02-26T06:20:34.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.855+0000 7f9c6dc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68103280 0x7f9c681992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:34.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.855+0000 7f9c6dc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104470 0x7f9c68199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:34.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.855+0000 7f9c6dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68104e30 0x7f9c681a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:34.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.855+0000 7f9c6dc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c681a0db0 con 0x7f9c68103280 2024-02-26T06:20:34.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.855+0000 7f9c6dc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c681a0f20 con 0x7f9c68104e30 2024-02-26T06:20:34.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.855+0000 7f9c6dc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c681a10e0 con 0x7f9c68104470 2024-02-26T06:20:34.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68103280 0x7f9c681992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:34.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104470 0x7f9c68199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104470 0x7f9c68199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:52360/0 (socket says 172.21.15.49:52360) 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 -- 172.21.15.49:0/1564075652 learned_addr learned my addr 172.21.15.49:0/1564075652 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c6d200640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68104e30 0x7f9c681a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 -- 172.21.15.49:0/1564075652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68104e30 msgr2=0x7f9c681a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68104e30 0x7f9c681a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 -- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68103280 msgr2=0x7f9c681992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68103280 0x7f9c681992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 -- 172.21.15.49:0/1564075652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c60009900 con 0x7f9c68104470 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.856+0000 7f9c67400640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104470 0x7f9c68199800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400e9f0 tx=0x7f9c5400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.857+0000 7f9c64c00640 1 -- 172.21.15.49:0/1564075652 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54009720 con 0x7f9c68104470 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.857+0000 7f9c64c00640 1 -- 172.21.15.49:0/1564075652 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c5400cd80 con 0x7f9c68104470 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.857+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c681a13c0 con 0x7f9c68104470 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.857+0000 7f9c64c00640 1 -- 172.21.15.49:0/1564075652 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54010690 con 0x7f9c68104470 2024-02-26T06:20:34.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.857+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c681a1880 con 0x7f9c68104470 2024-02-26T06:20:34.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.858+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6810a5a0 con 0x7f9c68104470 2024-02-26T06:20:34.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.861+0000 7f9c64c00640 1 -- 172.21.15.49:0/1564075652 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9c540046c0 con 0x7f9c68104470 2024-02-26T06:20:34.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.862+0000 7f9c64c00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c4c0a20c0 0x7f9c4c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:34.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.862+0000 7f9c64c00640 1 -- 172.21.15.49:0/1564075652 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c54014070 con 0x7f9c68104470 2024-02-26T06:20:34.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.862+0000 7f9c67e00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c4c0a20c0 0x7f9c4c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:34.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.863+0000 7f9c67e00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c4c0a20c0 0x7f9c4c0a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9c681042d0 tx=0x7f9c58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:34.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:34.866+0000 7f9c64c00640 1 -- 172.21.15.49:0/1564075652 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c54096fd0 con 0x7f9c68104470 2024-02-26T06:20:35.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:35.119+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f9c6810eee0 con 0x7f9c68104470 2024-02-26T06:20:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:35 smithi080 ceph-mon[36206]: pgmap v319: 97 pgs: 4 creating+peering, 48 active+clean, 45 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-02-26T06:20:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:35 smithi080 ceph-mon[36206]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-26T06:20:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:35 smithi080 ceph-mon[36206]: Cluster is now healthy 2024-02-26T06:20:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:35 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/977798699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T06:20:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:35 smithi080 ceph-mon[36206]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:35 smithi115 ceph-mon[36426]: pgmap v319: 97 pgs: 4 creating+peering, 48 active+clean, 45 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-02-26T06:20:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:35 smithi115 ceph-mon[36426]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-26T06:20:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:35 smithi115 ceph-mon[36426]: Cluster is now healthy 2024-02-26T06:20:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:35 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/977798699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T06:20:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:35 smithi115 ceph-mon[36426]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:35.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:35 smithi049 ceph-mon[31696]: pgmap v319: 97 pgs: 4 creating+peering, 48 active+clean, 45 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-02-26T06:20:35.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:35 smithi049 ceph-mon[31696]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-26T06:20:35.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:35 smithi049 ceph-mon[31696]: Cluster is now healthy 2024-02-26T06:20:35.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:35 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/977798699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T06:20:35.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:35 smithi049 ceph-mon[31696]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:36.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.063+0000 7f9c64c00640 1 -- 172.21.15.49:0/1564075652 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f9c5409a690 con 0x7f9c68104470 2024-02-26T06:20:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c4c0a20c0 msgr2=0x7f9c4c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c4c0a20c0 0x7f9c4c0a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9c681042d0 tx=0x7f9c58005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104470 msgr2=0x7f9c68199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104470 0x7f9c68199800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400e9f0 tx=0x7f9c5400edf0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 shutdown_connections 2024-02-26T06:20:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c4c0a20c0 0x7f9c4c0a4580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68104e30 0x7f9c681a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c68104470 0x7f9c68199800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/1564075652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68103280 0x7f9c681992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 >> 172.21.15.49:0/1564075652 conn(0x7f9c680fe7f0 msgr2=0x7f9c68107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 shutdown_connections 2024-02-26T06:20:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.067+0000 7f9c6dc00640 1 -- 172.21.15.49:0/1564075652 wait complete. 2024-02-26T06:20:36.092 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-26T06:20:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.313+0000 7f51afc00640 1 -- 172.21.15.49:0/637119079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 msgr2=0x7f51a81037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.313+0000 7f51afc00640 1 --2- 172.21.15.49:0/637119079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 0x7f51a81037c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5198009860 tx=0x7f519802bb70 comp rx=0 tx=0).stop 2024-02-26T06:20:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.314+0000 7f51afc00640 1 -- 172.21.15.49:0/637119079 shutdown_connections 2024-02-26T06:20:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.314+0000 7f51afc00640 1 --2- 172.21.15.49:0/637119079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51a8104f50 0x7f51a810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.314+0000 7f51afc00640 1 --2- 172.21.15.49:0/637119079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a8104590 0x7f51a8104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.314+0000 7f51afc00640 1 --2- 172.21.15.49:0/637119079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 0x7f51a81037c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.314+0000 7f51afc00640 1 -- 172.21.15.49:0/637119079 >> 172.21.15.49:0/637119079 conn(0x7f51a80fe930 msgr2=0x7f51a8100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.314+0000 7f51afc00640 1 -- 172.21.15.49:0/637119079 shutdown_connections 2024-02-26T06:20:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.314+0000 7f51afc00640 1 -- 172.21.15.49:0/637119079 wait complete. 2024-02-26T06:20:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.315+0000 7f51afc00640 1 Processor -- start 2024-02-26T06:20:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.315+0000 7f51afc00640 1 -- start start 2024-02-26T06:20:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.315+0000 7f51afc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 0x7f51a8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.315+0000 7f51afc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a8104590 0x7f51a8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.315+0000 7f51afc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51a8104f50 0x7f51a81a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.315+0000 7f51afc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a81a0f40 con 0x7f51a81033a0 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.315+0000 7f51afc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a81a10b0 con 0x7f51a8104f50 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.315+0000 7f51afc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a81a1270 con 0x7f51a8104590 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51ad200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 0x7f51a8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51ad200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 0x7f51a8199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45000/0 (socket says 172.21.15.49:45000) 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51ad200640 1 -- 172.21.15.49:0/876913373 learned_addr learned my addr 172.21.15.49:0/876913373 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51a7e00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a8104590 0x7f51a8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51adc00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51a8104f50 0x7f51a81a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51a7e00640 1 -- 172.21.15.49:0/876913373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51a8104f50 msgr2=0x7f51a81a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51a7e00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51a8104f50 0x7f51a81a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51a7e00640 1 -- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 msgr2=0x7f51a8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51a7e00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 0x7f51a8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51a7e00640 1 -- 172.21.15.49:0/876913373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009560 con 0x7f51a8104590 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.316+0000 7f51a7e00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a8104590 0x7f51a8199990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5194005fd0 tx=0x7f519400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.317+0000 7f51a5600640 1 -- 172.21.15.49:0/876913373 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519400cd00 con 0x7f51a8104590 2024-02-26T06:20:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.317+0000 7f51a5600640 1 -- 172.21.15.49:0/876913373 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f519400ce60 con 0x7f51a8104590 2024-02-26T06:20:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.317+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a0009840 con 0x7f51a8104590 2024-02-26T06:20:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.317+0000 7f51a5600640 1 -- 172.21.15.49:0/876913373 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194010430 con 0x7f51a8104590 2024-02-26T06:20:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.317+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5198009490 con 0x7f51a8104590 2024-02-26T06:20:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.318+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a810a790 con 0x7f51a8104590 2024-02-26T06:20:36.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.320+0000 7f51a5600640 1 -- 172.21.15.49:0/876913373 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5194003760 con 0x7f51a8104590 2024-02-26T06:20:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.321+0000 7f51a5600640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f518c0a2140 0x7f518c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.321+0000 7f51a5600640 1 -- 172.21.15.49:0/876913373 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5194014070 con 0x7f51a8104590 2024-02-26T06:20:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.322+0000 7f51ad200640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f518c0a2140 0x7f518c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.322+0000 7f51ad200640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f518c0a2140 0x7f518c0a4600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f51980057c0 tx=0x7f5198034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:36.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.323+0000 7f51a5600640 1 -- 172.21.15.49:0/876913373 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5194096fa0 con 0x7f51a8104590 2024-02-26T06:20:36.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:36 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1564075652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T06:20:36.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:36 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T06:20:36.331 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:20:35 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: 2024-02-26T06:20:35.832+0000 7f2b856e8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T06:20:36.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:36 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1564075652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T06:20:36.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:36 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T06:20:36.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:36 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1564075652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T06:20:36.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:36 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T06:20:36.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:36.566+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f51a810f000 con 0x7f51a8104590 2024-02-26T06:20:37.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.075+0000 7f51a5600640 1 -- 172.21.15.49:0/876913373 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f519409a660 con 0x7f51a8104590 2024-02-26T06:20:37.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f518c0a2140 msgr2=0x7f518c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:37.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f518c0a2140 0x7f518c0a4600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f51980057c0 tx=0x7f5198034c00 comp rx=0 tx=0).stop 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a8104590 msgr2=0x7f51a8199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a8104590 0x7f51a8199990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5194005fd0 tx=0x7f519400ee50 comp rx=0 tx=0).stop 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 shutdown_connections 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f518c0a2140 0x7f518c0a4600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51a8104f50 0x7f51a81a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a8104590 0x7f51a8199990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 --2- 172.21.15.49:0/876913373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51a81033a0 0x7f51a8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.080+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 >> 172.21.15.49:0/876913373 conn(0x7f51a80fe930 msgr2=0x7f51a8107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.081+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 shutdown_connections 2024-02-26T06:20:37.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.081+0000 7f51afc00640 1 -- 172.21.15.49:0/876913373 wait complete. 2024-02-26T06:20:37.109 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-26T06:20:37.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 -- 172.21.15.49:0/1829095985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 msgr2=0x7f58981036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:37.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 --2- 172.21.15.49:0/1829095985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 0x7f58981036a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5888009ca0 tx=0x7f588802c100 comp rx=0 tx=0).stop 2024-02-26T06:20:37.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 -- 172.21.15.49:0/1829095985 shutdown_connections 2024-02-26T06:20:37.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 --2- 172.21.15.49:0/1829095985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898104e30 0x7f589810b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 --2- 172.21.15.49:0/1829095985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5898104470 0x7f58981048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 --2- 172.21.15.49:0/1829095985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 0x7f58981036a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 -- 172.21.15.49:0/1829095985 >> 172.21.15.49:0/1829095985 conn(0x7f58980fe7f0 msgr2=0x7f5898100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:37.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 -- 172.21.15.49:0/1829095985 shutdown_connections 2024-02-26T06:20:37.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.324+0000 7f5897e00640 1 -- 172.21.15.49:0/1829095985 wait complete. 2024-02-26T06:20:37.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.326+0000 7f5897e00640 1 Processor -- start 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.326+0000 7f5897e00640 1 -- start start 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.326+0000 7f5897e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 0x7f58981992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.326+0000 7f5897e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898104470 0x7f5898199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.326+0000 7f5897e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5898104e30 0x7f58981a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.326+0000 7f5897e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981a0dd0 con 0x7f5898104e30 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.326+0000 7f5897e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981a0f40 con 0x7f5898103280 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.326+0000 7f5897e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981a1100 con 0x7f5898104470 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5898104e30 0x7f58981a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5896000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898104470 0x7f5898199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5898104e30 0x7f58981a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45014/0 (socket says 172.21.15.49:45014) 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 -- 172.21.15.49:0/1058907340 learned_addr learned my addr 172.21.15.49:0/1058907340 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5896a00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 0x7f58981992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 -- 172.21.15.49:0/1058907340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898104470 msgr2=0x7f5898199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898104470 0x7f5898199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 -- 172.21.15.49:0/1058907340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 msgr2=0x7f58981992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:37.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 0x7f58981992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:37.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 -- 172.21.15.49:0/1058907340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888009900 con 0x7f5898104e30 2024-02-26T06:20:37.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5896a00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 0x7f58981992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.327+0000 7f5897400640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5898104e30 0x7f58981a0890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f589000e970 tx=0x7f589000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.328+0000 7f588f400640 1 -- 172.21.15.49:0/1058907340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890009720 con 0x7f5898104e30 2024-02-26T06:20:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.328+0000 7f588f400640 1 -- 172.21.15.49:0/1058907340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5890005500 con 0x7f5898104e30 2024-02-26T06:20:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.328+0000 7f588f400640 1 -- 172.21.15.49:0/1058907340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890010430 con 0x7f5898104e30 2024-02-26T06:20:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.328+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58981a13e0 con 0x7f5898104e30 2024-02-26T06:20:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.328+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58981a1940 con 0x7f5898104e30 2024-02-26T06:20:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.329+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589810a5a0 con 0x7f5898104e30 2024-02-26T06:20:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: pgmap v320: 97 pgs: 4 creating+peering, 85 active+clean, 8 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-02-26T06:20:37.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T06:20:37.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T06:20:37.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T06:20:37.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:37.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-26T06:20:37.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-26T06:20:37.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/876913373' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:20:37.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:37 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:20:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.331+0000 7f588f400640 1 -- 172.21.15.49:0/1058907340 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5890004ff0 con 0x7f5898104e30 2024-02-26T06:20:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.334+0000 7f588f400640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f587c0a2140 0x7f587c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.334+0000 7f588f400640 1 -- 172.21.15.49:0/1058907340 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5890014070 con 0x7f5898104e30 2024-02-26T06:20:37.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.334+0000 7f5896a00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f587c0a2140 0x7f587c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:37.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.335+0000 7f5896a00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f587c0a2140 0x7f587c0a4600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f588802c710 tx=0x7f58880326d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:37.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.338+0000 7f588f400640 1 -- 172.21.15.49:0/1058907340 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58900d4080 con 0x7f5898104e30 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: pgmap v320: 97 pgs: 4 creating+peering, 85 active+clean, 8 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/876913373' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:20:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:37 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:20:37.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: pgmap v320: 97 pgs: 4 creating+peering, 85 active+clean, 8 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-02-26T06:20:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T06:20:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T06:20:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T06:20:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:20:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-26T06:20:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-26T06:20:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/876913373' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:20:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:37 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:20:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:37.584+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f589802cda0 con 0x7f5898104e30 2024-02-26T06:20:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.092+0000 7f588f400640 1 -- 172.21.15.49:0/1058907340 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5890096f00 con 0x7f5898104e30 2024-02-26T06:20:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f587c0a2140 msgr2=0x7f587c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f587c0a2140 0x7f587c0a4600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f588802c710 tx=0x7f58880326d0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5898104e30 msgr2=0x7f58981a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5898104e30 0x7f58981a0890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f589000e970 tx=0x7f589000ed70 comp rx=0 tx=0).stop 2024-02-26T06:20:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 shutdown_connections 2024-02-26T06:20:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f587c0a2140 0x7f587c0a4600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5898104e30 0x7f58981a0890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898104470 0x7f5898199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 --2- 172.21.15.49:0/1058907340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5898103280 0x7f58981992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.097+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 >> 172.21.15.49:0/1058907340 conn(0x7f58980fe7f0 msgr2=0x7f5898107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.098+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 shutdown_connections 2024-02-26T06:20:38.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.098+0000 7f5897e00640 1 -- 172.21.15.49:0/1058907340 wait complete. 2024-02-26T06:20:38.122 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-02-26T06:20:38.123 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-26T06:20:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T06:20:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] up:active 2024-02-26T06:20:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T06:20:38.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T06:20:38.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: Cluster is now healthy 2024-02-26T06:20:38.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-26T06:20:38.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-26T06:20:38.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-26T06:20:38.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:38 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1058907340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:20:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T06:20:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] up:active 2024-02-26T06:20:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T06:20:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T06:20:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: Cluster is now healthy 2024-02-26T06:20:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-26T06:20:38.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-26T06:20:38.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-26T06:20:38.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:38 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1058907340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:20:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.356+0000 7fca44e00640 1 -- 172.21.15.49:0/1426768048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 msgr2=0x7fca40107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.356+0000 7fca44e00640 1 --2- 172.21.15.49:0/1426768048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 0x7fca40107ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca30009900 tx=0x7fca3002bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.356+0000 7fca44e00640 1 -- 172.21.15.49:0/1426768048 shutdown_connections 2024-02-26T06:20:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.356+0000 7fca44e00640 1 --2- 172.21.15.49:0/1426768048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca40109250 0x7fca4010fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.356+0000 7fca44e00640 1 --2- 172.21.15.49:0/1426768048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca40108890 0x7fca40108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.356+0000 7fca44e00640 1 --2- 172.21.15.49:0/1426768048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 0x7fca40107ac0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.356+0000 7fca44e00640 1 -- 172.21.15.49:0/1426768048 >> 172.21.15.49:0/1426768048 conn(0x7fca40076650 msgr2=0x7fca40078a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.356+0000 7fca44e00640 1 -- 172.21.15.49:0/1426768048 shutdown_connections 2024-02-26T06:20:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.357+0000 7fca44e00640 1 -- 172.21.15.49:0/1426768048 wait complete. 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.357+0000 7fca44e00640 1 Processor -- start 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca44e00640 1 -- start start 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca44e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 0x7fca4019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca44e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca40108890 0x7fca4019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca44e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca40109250 0x7fca401a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca44e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca401a5250 con 0x7fca401076a0 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca44e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca401a53c0 con 0x7fca40108890 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca44e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca401a5580 con 0x7fca40109250 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca3f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 0x7fca4019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca3f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 0x7fca4019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45032/0 (socket says 172.21.15.49:45032) 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca3f400640 1 -- 172.21.15.49:0/2352370752 learned_addr learned my addr 172.21.15.49:0/2352370752 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca37e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca40108890 0x7fca4019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.358+0000 7fca3fe00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca40109250 0x7fca401a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3f400640 1 -- 172.21.15.49:0/2352370752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca40109250 msgr2=0x7fca401a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3f400640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca40109250 0x7fca401a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3f400640 1 -- 172.21.15.49:0/2352370752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca40108890 msgr2=0x7fca4019dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3f400640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca40108890 0x7fca4019dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3f400640 1 -- 172.21.15.49:0/2352370752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca30009560 con 0x7fca401076a0 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca37e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca40108890 0x7fca4019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3f400640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 0x7fca4019d760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca30000c00 tx=0x7fca30005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3cc00640 1 -- 172.21.15.49:0/2352370752 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300043d0 con 0x7fca401076a0 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3cc00640 1 -- 172.21.15.49:0/2352370752 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca300052d0 con 0x7fca401076a0 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca3cc00640 1 -- 172.21.15.49:0/2352370752 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30003da0 con 0x7fca401076a0 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.359+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca401a5800 con 0x7fca401076a0 2024-02-26T06:20:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.360+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca401a5d10 con 0x7fca401076a0 2024-02-26T06:20:38.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.361+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4010e9c0 con 0x7fca401076a0 2024-02-26T06:20:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.363+0000 7fca3cc00640 1 -- 172.21.15.49:0/2352370752 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fca30005440 con 0x7fca401076a0 2024-02-26T06:20:38.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.364+0000 7fca3cc00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca240a20c0 0x7fca240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:38.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.364+0000 7fca3cc00640 1 -- 172.21.15.49:0/2352370752 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fca300f0370 con 0x7fca401076a0 2024-02-26T06:20:38.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.364+0000 7fca37e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca240a20c0 0x7fca240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:38.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.365+0000 7fca37e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca240a20c0 0x7fca240a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca4019ec70 tx=0x7fca28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:38.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.368+0000 7fca3cc00640 1 -- 172.21.15.49:0/2352370752 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca300b9510 con 0x7fca401076a0 2024-02-26T06:20:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T06:20:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] up:active 2024-02-26T06:20:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T06:20:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T06:20:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: Cluster is now healthy 2024-02-26T06:20:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-26T06:20:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-26T06:20:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-26T06:20:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1058907340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:20:38.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.596+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fca4002ce10 con 0x7fca240a20c0 2024-02-26T06:20:38.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.625+0000 7fca3cc00640 1 -- 172.21.15.49:0/2352370752 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fca4002ce10 con 0x7fca240a20c0 2024-02-26T06:20:38.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca240a20c0 msgr2=0x7fca240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca240a20c0 0x7fca240a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca4019ec70 tx=0x7fca28005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:38.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 msgr2=0x7fca4019d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 0x7fca4019d760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca30000c00 tx=0x7fca30005e80 comp rx=0 tx=0).stop 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 shutdown_connections 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca240a20c0 0x7fca240a4580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca40108890 0x7fca4019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca40109250 0x7fca401a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 --2- 172.21.15.49:0/2352370752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca401076a0 0x7fca4019d760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 >> 172.21.15.49:0/2352370752 conn(0x7fca40076650 msgr2=0x7fca4010bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 shutdown_connections 2024-02-26T06:20:38.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.628+0000 7fca44e00640 1 -- 172.21.15.49:0/2352370752 wait complete. 2024-02-26T06:20:38.650 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.875+0000 7f43cfc00640 1 -- 172.21.15.49:0/3522133580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81033a0 msgr2=0x7f43c81037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.875+0000 7f43cfc00640 1 --2- 172.21.15.49:0/3522133580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81033a0 0x7f43c81037c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009860 tx=0x7f43bc02bb70 comp rx=0 tx=0).stop 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.875+0000 7f43cfc00640 1 -- 172.21.15.49:0/3522133580 shutdown_connections 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.875+0000 7f43cfc00640 1 --2- 172.21.15.49:0/3522133580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43c8104f50 0x7f43c810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.875+0000 7f43cfc00640 1 --2- 172.21.15.49:0/3522133580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c8104590 0x7f43c8104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.875+0000 7f43cfc00640 1 --2- 172.21.15.49:0/3522133580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c81033a0 0x7f43c81037c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.875+0000 7f43cfc00640 1 -- 172.21.15.49:0/3522133580 >> 172.21.15.49:0/3522133580 conn(0x7f43c80fe930 msgr2=0x7f43c8100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.876+0000 7f43cfc00640 1 -- 172.21.15.49:0/3522133580 shutdown_connections 2024-02-26T06:20:38.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.876+0000 7f43cfc00640 1 -- 172.21.15.49:0/3522133580 wait complete. 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.877+0000 7f43cfc00640 1 Processor -- start 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.877+0000 7f43cfc00640 1 -- start start 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cfc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c81033a0 0x7f43c8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cfc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43c8104590 0x7f43c81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cfc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c8104f50 0x7f43c81a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cfc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81a0f80 con 0x7f43c8104590 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cfc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81a10f0 con 0x7f43c8104f50 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cfc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81a12b0 con 0x7f43c81033a0 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cd200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c81033a0 0x7f43c8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43c7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43c8104590 0x7f43c81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:38.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cd200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c81033a0 0x7f43c8199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:52460/0 (socket says 172.21.15.49:52460) 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cd200640 1 -- 172.21.15.49:0/1041536578 learned_addr learned my addr 172.21.15.49:0/1041536578 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cdc00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c8104f50 0x7f43c81a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cd200640 1 -- 172.21.15.49:0/1041536578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c8104f50 msgr2=0x7f43c81a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cd200640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c8104f50 0x7f43c81a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cd200640 1 -- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43c8104590 msgr2=0x7f43c81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cd200640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43c8104590 0x7f43c81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.878+0000 7f43cd200640 1 -- 172.21.15.49:0/1041536578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0009900 con 0x7f43c81033a0 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.879+0000 7f43cd200640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c81033a0 0x7f43c8199490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43bc02c0b0 tx=0x7f43bc034440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.879+0000 7f43c5600640 1 -- 172.21.15.49:0/1041536578 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc02c510 con 0x7f43c81033a0 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.879+0000 7f43c5600640 1 -- 172.21.15.49:0/1041536578 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43bc02cb50 con 0x7f43c81033a0 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.879+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc009490 con 0x7f43c81033a0 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.879+0000 7f43c5600640 1 -- 172.21.15.49:0/1041536578 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc003ea0 con 0x7f43c81033a0 2024-02-26T06:20:38.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.879+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c81a1710 con 0x7f43c81033a0 2024-02-26T06:20:38.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.880+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c810a760 con 0x7f43c81033a0 2024-02-26T06:20:38.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.882+0000 7f43c5600640 1 -- 172.21.15.49:0/1041536578 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f43bc03b040 con 0x7f43c81033a0 2024-02-26T06:20:38.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.884+0000 7f43c5600640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43b00a20c0 0x7f43b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:38.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.884+0000 7f43c5600640 1 -- 172.21.15.49:0/1041536578 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f43bc0ef850 con 0x7f43c81033a0 2024-02-26T06:20:38.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.884+0000 7f43c7e00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43b00a20c0 0x7f43b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:38.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.885+0000 7f43c7e00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43b00a20c0 0x7f43b00a4580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f43c819a9a0 tx=0x7f43b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:38.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:38.887+0000 7f43c5600640 1 -- 172.21.15.49:0/1041536578 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43bc0b8980 con 0x7f43c81033a0 2024-02-26T06:20:38.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:20:38.605+0000 7fc989200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:20:38.920 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:20:38.605+0000 7fc989200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:20:38.920 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:20:38.605+0000 7fc989200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:20:38.920 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:20:38.605+0000 7fc989200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:20:38.920 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:20:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:20:38.605+0000 7fc989200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:20:39.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.126+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f43c802ce40 con 0x7f43b00a20c0 2024-02-26T06:20:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.159+0000 7f43c5600640 1 -- 172.21.15.49:0/1041536578 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f43c802ce40 con 0x7f43b00a20c0 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43b00a20c0 msgr2=0x7f43b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43b00a20c0 0x7f43b00a4580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f43c819a9a0 tx=0x7f43b400b390 comp rx=0 tx=0).stop 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c81033a0 msgr2=0x7f43c8199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c81033a0 0x7f43c8199490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43bc02c0b0 tx=0x7f43bc034440 comp rx=0 tx=0).stop 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 shutdown_connections 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43b00a20c0 0x7f43b00a4580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43c8104f50 0x7f43c81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43c8104590 0x7f43c81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 --2- 172.21.15.49:0/1041536578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c81033a0 0x7f43c8199490 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 >> 172.21.15.49:0/1041536578 conn(0x7f43c80fe930 msgr2=0x7f43c8107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 shutdown_connections 2024-02-26T06:20:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.162+0000 7f43cfc00640 1 -- 172.21.15.49:0/1041536578 wait complete. 2024-02-26T06:20:39.183 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-26T06:20:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:39 smithi049 ceph-mon[31696]: pgmap v321: 97 pgs: 97 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2024-02-26T06:20:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:39 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1058907340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T06:20:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:39 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:active 2024-02-26T06:20:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:39 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-02-26T06:20:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:39 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:20:39.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.422+0000 7fa8ccc00640 1 -- 172.21.15.49:0/1734943797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8075b40 msgr2=0x7fa8c8075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.422+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/1734943797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8075b40 0x7fa8c8075f60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009900 tx=0x7fa8b802bc50 comp rx=0 tx=0).stop 2024-02-26T06:20:39.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.422+0000 7fa8ccc00640 1 -- 172.21.15.49:0/1734943797 shutdown_connections 2024-02-26T06:20:39.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.422+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/1734943797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8c8077720 0x7fa8c8077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.422+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/1734943797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c8076d80 0x7fa8c80771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.422+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/1734943797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8075b40 0x7fa8c8075f60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.422+0000 7fa8ccc00640 1 -- 172.21.15.49:0/1734943797 >> 172.21.15.49:0/1734943797 conn(0x7fa8c80fe7f0 msgr2=0x7fa8c8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:39.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.423+0000 7fa8ccc00640 1 -- 172.21.15.49:0/1734943797 shutdown_connections 2024-02-26T06:20:39.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.423+0000 7fa8ccc00640 1 -- 172.21.15.49:0/1734943797 wait complete. 2024-02-26T06:20:39.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.424+0000 7fa8ccc00640 1 Processor -- start 2024-02-26T06:20:39.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.424+0000 7fa8ccc00640 1 -- start start 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8ccc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8c8075b40 0x7fa8c819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8ccc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c8076d80 0x7fa8c819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8ccc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8077720 0x7fa8c81a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8ccc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81a51c0 con 0x7fa8c8076d80 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8ccc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81a5330 con 0x7fa8c8075b40 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8ccc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81a54f0 con 0x7fa8c8077720 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8bfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c8076d80 0x7fa8c819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8bfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c8076d80 0x7fa8c819dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45084/0 (socket says 172.21.15.49:45084) 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8bfe00640 1 -- 172.21.15.49:0/912559645 learned_addr learned my addr 172.21.15.49:0/912559645 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8c7400640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8c8075b40 0x7fa8c819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8c7e00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8077720 0x7fa8c81a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8bfe00640 1 -- 172.21.15.49:0/912559645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8077720 msgr2=0x7fa8c81a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8bfe00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8077720 0x7fa8c81a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8bfe00640 1 -- 172.21.15.49:0/912559645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8c8075b40 msgr2=0x7fa8c819d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8bfe00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8c8075b40 0x7fa8c819d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.425+0000 7fa8bfe00640 1 -- 172.21.15.49:0/912559645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b8009560 con 0x7fa8c8076d80 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.426+0000 7fa8bfe00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c8076d80 0x7fa8c819dc10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000e9f0 tx=0x7fa8b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.426+0000 7fa8c4c00640 1 -- 172.21.15.49:0/912559645 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0009700 con 0x7fa8c8076d80 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.426+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c81a57d0 con 0x7fa8c8076d80 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.426+0000 7fa8c4c00640 1 -- 172.21.15.49:0/912559645 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b000cd80 con 0x7fa8c8076d80 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.426+0000 7fa8c4c00640 1 -- 172.21.15.49:0/912559645 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0018920 con 0x7fa8c8076d80 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.426+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c81a5d30 con 0x7fa8c8076d80 2024-02-26T06:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.428+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c810fd80 con 0x7fa8c8076d80 2024-02-26T06:20:39.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.430+0000 7fa8c4c00640 1 -- 172.21.15.49:0/912559645 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa8b0018a80 con 0x7fa8c8076d80 2024-02-26T06:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.431+0000 7fa8c4c00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8ac0a20c0 0x7fa8ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.431+0000 7fa8c4c00640 1 -- 172.21.15.49:0/912559645 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa8b0014070 con 0x7fa8c8076d80 2024-02-26T06:20:39.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.431+0000 7fa8c7400640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8ac0a20c0 0x7fa8ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.432+0000 7fa8c7400640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8ac0a20c0 0x7fa8ac0a4580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009230 tx=0x7fa8b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:39.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.435+0000 7fa8c4c00640 1 -- 172.21.15.49:0/912559645 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8b0052d20 con 0x7fa8c8076d80 2024-02-26T06:20:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:39 smithi080 ceph-mon[36206]: pgmap v321: 97 pgs: 97 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2024-02-26T06:20:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:39 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1058907340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T06:20:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:39 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:active 2024-02-26T06:20:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:39 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-02-26T06:20:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:39 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:20:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:39 smithi115 ceph-mon[36426]: pgmap v321: 97 pgs: 97 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2024-02-26T06:20:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:39 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1058907340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T06:20:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:39 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:active 2024-02-26T06:20:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:39 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-02-26T06:20:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:39 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:20:39.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.678+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8c80615c0 con 0x7fa8ac0a20c0 2024-02-26T06:20:39.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.685+0000 7fa8c4c00640 1 -- 172.21.15.49:0/912559645 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fa8c80615c0 con 0x7fa8ac0a20c0 2024-02-26T06:20:39.684 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/qa/sv_0/ab6b6149-17d2-4415-ad42-e2f8f717c0c3 2024-02-26T06:20:39.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8ac0a20c0 msgr2=0x7fa8ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8ac0a20c0 0x7fa8ac0a4580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009230 tx=0x7fa8b8005fd0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c8076d80 msgr2=0x7fa8c819dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c8076d80 0x7fa8c819dc10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000e9f0 tx=0x7fa8b000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 shutdown_connections 2024-02-26T06:20:39.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8ac0a20c0 0x7fa8ac0a4580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8077720 0x7fa8c81a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c8076d80 0x7fa8c819dc10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 --2- 172.21.15.49:0/912559645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8c8075b40 0x7fa8c819d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 >> 172.21.15.49:0/912559645 conn(0x7fa8c80fe7f0 msgr2=0x7fa8c80fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:39.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.687+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 shutdown_connections 2024-02-26T06:20:39.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.688+0000 7fa8ccc00640 1 -- 172.21.15.49:0/912559645 wait complete. 2024-02-26T06:20:39.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-26T06:20:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.951+0000 7f0083e00640 1 -- 172.21.15.49:0/167255343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 msgr2=0x7f0084104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.951+0000 7f0083e00640 1 --2- 172.21.15.49:0/167255343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 0x7f0084104910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0070009900 tx=0x7f007002bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.951+0000 7f0083e00640 1 -- 172.21.15.49:0/167255343 shutdown_connections 2024-02-26T06:20:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.951+0000 7f0083e00640 1 --2- 172.21.15.49:0/167255343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084104e50 0x7f008410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.951+0000 7f0083e00640 1 --2- 172.21.15.49:0/167255343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 0x7f0084104910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.951+0000 7f0083e00640 1 --2- 172.21.15.49:0/167255343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00841032a0 0x7f00841036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.951+0000 7f0083e00640 1 -- 172.21.15.49:0/167255343 >> 172.21.15.49:0/167255343 conn(0x7f00840fe830 msgr2=0x7f0084100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.952+0000 7f0083e00640 1 -- 172.21.15.49:0/167255343 shutdown_connections 2024-02-26T06:20:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.952+0000 7f0083e00640 1 -- 172.21.15.49:0/167255343 wait complete. 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.952+0000 7f0083e00640 1 Processor -- start 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0083e00640 1 -- start start 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0083e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00841032a0 0x7f0084199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0083e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 0x7f0084199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0083e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084104e50 0x7f00841a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0083e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841a0e00 con 0x7f0084104490 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0083e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841a0f70 con 0x7f00841032a0 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0083e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841a1130 con 0x7f0084104e50 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0082000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 0x7f0084199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0082a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00841032a0 0x7f0084199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0082000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 0x7f0084199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57840/0 (socket says 172.21.15.49:57840) 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0082000640 1 -- 172.21.15.49:0/3346486443 learned_addr learned my addr 172.21.15.49:0/3346486443 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.953+0000 7f0083400640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084104e50 0x7f00841a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f0082a00640 1 -- 172.21.15.49:0/3346486443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084104e50 msgr2=0x7f00841a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f0082a00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084104e50 0x7f00841a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f0082a00640 1 -- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 msgr2=0x7f0084199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f0082a00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 0x7f0084199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f0082a00640 1 -- 172.21.15.49:0/3346486443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074009490 con 0x7f00841032a0 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f0082000640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 0x7f0084199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f0082a00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00841032a0 0x7f0084199310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f007400e9c0 tx=0x7f007400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f007b400640 1 -- 172.21.15.49:0/3346486443 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400cd80 con 0x7f00841032a0 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.954+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0070009560 con 0x7f00841032a0 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.955+0000 7f007b400640 1 -- 172.21.15.49:0/3346486443 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0074010910 con 0x7f00841032a0 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.955+0000 7f007b400640 1 -- 172.21.15.49:0/3346486443 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074018850 con 0x7f00841032a0 2024-02-26T06:20:39.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.955+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00841a17e0 con 0x7f00841032a0 2024-02-26T06:20:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.956+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00840682e0 con 0x7f00841032a0 2024-02-26T06:20:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.958+0000 7f007b400640 1 -- 172.21.15.49:0/3346486443 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0074010a80 con 0x7f00841032a0 2024-02-26T06:20:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.959+0000 7f007b400640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f00680a20c0 0x7f00680a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.959+0000 7f007b400640 1 -- 172.21.15.49:0/3346486443 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0074014070 con 0x7f00841032a0 2024-02-26T06:20:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.960+0000 7f0082000640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f00680a20c0 0x7f00680a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.960+0000 7f0082000640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f00680a20c0 0x7f00680a4580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f007002cdd0 tx=0x7f007002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:39.961+0000 7f007b400640 1 -- 172.21.15.49:0/3346486443 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0074097600 con 0x7f00841032a0 2024-02-26T06:20:40.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.128+0000 7f007b400640 1 -- 172.21.15.49:0/3346486443 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f00740979d0 con 0x7f00841032a0 2024-02-26T06:20:40.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.205+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f00840615c0 con 0x7f00680a20c0 2024-02-26T06:20:40.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.238+0000 7f007b400640 1 -- 172.21.15.49:0/3346486443 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f00840615c0 con 0x7f00680a20c0 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f00680a20c0 msgr2=0x7f00680a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f00680a20c0 0x7f00680a4580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f007002cdd0 tx=0x7f007002c7e0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00841032a0 msgr2=0x7f0084199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00841032a0 0x7f0084199310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f007400e9c0 tx=0x7f007400ee80 comp rx=0 tx=0).stop 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 shutdown_connections 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f00680a20c0 0x7f00680a4580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084104e50 0x7f00841a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084104490 0x7f0084199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 --2- 172.21.15.49:0/3346486443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00841032a0 0x7f0084199310 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 >> 172.21.15.49:0/3346486443 conn(0x7f00840fe830 msgr2=0x7f0084107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 shutdown_connections 2024-02-26T06:20:40.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.241+0000 7f0083e00640 1 -- 172.21.15.49:0/3346486443 wait complete. 2024-02-26T06:20:40.263 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-26T06:20:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:40 smithi049 ceph-mon[31696]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:40.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:40 smithi049 ceph-mon[31696]: from='client.24697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 -- 172.21.15.49:0/1283505015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7460104e30 msgr2=0x7f746010b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 --2- 172.21.15.49:0/1283505015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7460104e30 0x7f746010b6c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f745c009ca0 tx=0x7f745c02c100 comp rx=0 tx=0).stop 2024-02-26T06:20:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 -- 172.21.15.49:0/1283505015 shutdown_connections 2024-02-26T06:20:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 --2- 172.21.15.49:0/1283505015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7460104e30 0x7f746010b6c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 --2- 172.21.15.49:0/1283505015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7460104470 0x7f74601048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 --2- 172.21.15.49:0/1283505015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7460103280 0x7f74601036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 -- 172.21.15.49:0/1283505015 >> 172.21.15.49:0/1283505015 conn(0x7f74600fe7f0 msgr2=0x7f7460100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 -- 172.21.15.49:0/1283505015 shutdown_connections 2024-02-26T06:20:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.489+0000 7f7467200640 1 -- 172.21.15.49:0/1283505015 wait complete. 2024-02-26T06:20:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.490+0000 7f7467200640 1 Processor -- start 2024-02-26T06:20:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.490+0000 7f7467200640 1 -- start start 2024-02-26T06:20:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7467200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7460103280 0x7f7460199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7467200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7460104470 0x7f74601997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7467200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7460104e30 0x7f74601a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7467200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74601a0d80 con 0x7f7460103280 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7467200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74601a0ef0 con 0x7f7460104e30 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7467200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74601a10b0 con 0x7f7460104470 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7460103280 0x7f7460199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7460103280 0x7f7460199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57858/0 (socket says 172.21.15.49:57858) 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465e00640 1 -- 172.21.15.49:0/3250103942 learned_addr learned my addr 172.21.15.49:0/3250103942 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465400640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7460104470 0x7f74601997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7466800640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7460104e30 0x7f74601a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465e00640 1 -- 172.21.15.49:0/3250103942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7460104470 msgr2=0x7f74601997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465e00640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7460104470 0x7f74601997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465e00640 1 -- 172.21.15.49:0/3250103942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7460104e30 msgr2=0x7f74601a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465e00640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7460104e30 0x7f74601a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.491+0000 7f7465e00640 1 -- 172.21.15.49:0/3250103942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c009900 con 0x7f7460103280 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.492+0000 7f7465e00640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7460103280 0x7f7460199290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f745000e9c0 tx=0x7f745000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.492+0000 7f745aa00640 1 -- 172.21.15.49:0/3250103942 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450009700 con 0x7f7460103280 2024-02-26T06:20:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.492+0000 7f745aa00640 1 -- 172.21.15.49:0/3250103942 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f745000cd80 con 0x7f7460103280 2024-02-26T06:20:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.492+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74601a1390 con 0x7f7460103280 2024-02-26T06:20:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.492+0000 7f745aa00640 1 -- 172.21.15.49:0/3250103942 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450010430 con 0x7f7460103280 2024-02-26T06:20:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.492+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74601a18f0 con 0x7f7460103280 2024-02-26T06:20:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.493+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746010a540 con 0x7f7460103280 2024-02-26T06:20:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.495+0000 7f745aa00640 1 -- 172.21.15.49:0/3250103942 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f7450003680 con 0x7f7460103280 2024-02-26T06:20:40.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.497+0000 7f745aa00640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f74480a21d0 0x7f74480a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:40.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.497+0000 7f745aa00640 1 -- 172.21.15.49:0/3250103942 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7450014070 con 0x7f7460103280 2024-02-26T06:20:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.497+0000 7f7465400640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f74480a21d0 0x7f74480a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.498+0000 7f7465400640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f74480a21d0 0x7f74480a4690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f746019a7a0 tx=0x7f7454005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:40.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.501+0000 7f745aa00640 1 -- 172.21.15.49:0/3250103942 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7450096fa0 con 0x7f7460103280 2024-02-26T06:20:40.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:40 smithi080 ceph-mon[36206]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:40.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:40 smithi080 ceph-mon[36206]: from='client.24697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:40.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:40 smithi115 ceph-mon[36426]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:40.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:40 smithi115 ceph-mon[36426]: from='client.24697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.734+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f746002ce10 con 0x7f74480a21d0 2024-02-26T06:20:40.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.743+0000 7f745aa00640 1 -- 172.21.15.49:0/3250103942 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f746002ce10 con 0x7f74480a21d0 2024-02-26T06:20:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 2024-02-26T06:20:40.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.745+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f74480a21d0 msgr2=0x7f74480a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:40.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f74480a21d0 0x7f74480a4690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f746019a7a0 tx=0x7f7454005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7460103280 msgr2=0x7f7460199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7460103280 0x7f7460199290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f745000e9c0 tx=0x7f745000ee80 comp rx=0 tx=0).stop 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 shutdown_connections 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f74480a21d0 0x7f74480a4690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7460104e30 0x7f74601a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7460104470 0x7f74601997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 --2- 172.21.15.49:0/3250103942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7460103280 0x7f7460199290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 >> 172.21.15.49:0/3250103942 conn(0x7f74600fe7f0 msgr2=0x7f7460107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 shutdown_connections 2024-02-26T06:20:40.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.746+0000 7f7467200640 1 -- 172.21.15.49:0/3250103942 wait complete. 2024-02-26T06:20:40.766 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:20:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 -- 172.21.15.49:0/2637087443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae41033c0 msgr2=0x7f4ae41037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 --2- 172.21.15.49:0/2637087443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae41033c0 0x7f4ae41037e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009900 tx=0x7f4ad402bc30 comp rx=0 tx=0).stop 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 -- 172.21.15.49:0/2637087443 shutdown_connections 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 --2- 172.21.15.49:0/2637087443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae4104f70 0x7f4ae410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 --2- 172.21.15.49:0/2637087443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ae41045b0 0x7f4ae4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 --2- 172.21.15.49:0/2637087443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae41033c0 0x7f4ae41037e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 -- 172.21.15.49:0/2637087443 >> 172.21.15.49:0/2637087443 conn(0x7f4ae40fe970 msgr2=0x7f4ae4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 -- 172.21.15.49:0/2637087443 shutdown_connections 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.997+0000 7f4aeac00640 1 -- 172.21.15.49:0/2637087443 wait complete. 2024-02-26T06:20:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.998+0000 7f4aeac00640 1 Processor -- start 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.998+0000 7f4aeac00640 1 -- start start 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.998+0000 7f4aeac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ae41033c0 0x7f4ae4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4aeac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae41045b0 0x7f4ae41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4aeac00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae4104f70 0x7f4ae41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4aeac00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41a0f70 con 0x7f4ae4104f70 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4aeac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41a10e0 con 0x7f4ae41045b0 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4aeac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41a12a0 con 0x7f4ae41033c0 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae4104f70 0x7f4ae41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae41045b0 0x7f4ae41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ae41033c0 0x7f4ae4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae4104f70 0x7f4ae41a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57870/0 (socket says 172.21.15.49:57870) 2024-02-26T06:20:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae41045b0 0x7f4ae41999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58036/0 (socket says 172.21.15.49:58036) 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae8c00640 1 -- 172.21.15.49:0/3249745520 learned_addr learned my addr 172.21.15.49:0/3249745520 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae8c00640 1 -- 172.21.15.49:0/3249745520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ae41033c0 msgr2=0x7f4ae4199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae8c00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ae41033c0 0x7f4ae4199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae8c00640 1 -- 172.21.15.49:0/3249745520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae41045b0 msgr2=0x7f4ae41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae8c00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae41045b0 0x7f4ae41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae8c00640 1 -- 172.21.15.49:0/3249745520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad4009560 con 0x7f4ae4104f70 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:40.999+0000 7f4ae3400640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae41045b0 0x7f4ae41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.000+0000 7f4ae8c00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae4104f70 0x7f4ae41a0a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00eb10 tx=0x7f4adc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.000+0000 7f4ae0c00640 1 -- 172.21.15.49:0/3249745520 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc004b20 con 0x7f4ae4104f70 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.000+0000 7f4ae0c00640 1 -- 172.21.15.49:0/3249745520 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4adc004c80 con 0x7f4ae4104f70 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.000+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae41a1580 con 0x7f4ae4104f70 2024-02-26T06:20:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.000+0000 7f4ae0c00640 1 -- 172.21.15.49:0/3249745520 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc010650 con 0x7f4ae4104f70 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.000+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae41a1a90 con 0x7f4ae4104f70 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.001+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae410a700 con 0x7f4ae4104f70 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.003+0000 7f4ae0c00640 1 -- 172.21.15.49:0/3249745520 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4adc005020 con 0x7f4ae4104f70 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.005+0000 7f4ae0c00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4ac80a21d0 0x7f4ac80a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.005+0000 7f4ae0c00640 1 -- 172.21.15.49:0/3249745520 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4adc014070 con 0x7f4ae4104f70 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.005+0000 7f4ae3e00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4ac80a21d0 0x7f4ac80a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.006+0000 7f4ae3e00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4ac80a21d0 0x7f4ac80a4690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4005ab0 tx=0x7f4ad4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.010+0000 7f4ae0c00640 1 -- 172.21.15.49:0/3249745520 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4adc097120 con 0x7f4ae4104f70 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.254+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ae410eda0 con 0x7f4ae4104f70 2024-02-26T06:20:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.256+0000 7f4ae0c00640 1 -- 172.21.15.49:0/3249745520 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11331 (secure 0 0 0) 0x7f4adc09a7e0 con 0x7f4ae4104f70 2024-02-26T06:20:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:41.550 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:20:38.098815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:20:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-02-26T06:20:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4ac80a21d0 msgr2=0x7f4ac80a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4ac80a21d0 0x7f4ac80a4690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4005ab0 tx=0x7f4ad4005fd0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae4104f70 msgr2=0x7f4ae41a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae4104f70 0x7f4ae41a0a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00eb10 tx=0x7f4adc00c2d0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 shutdown_connections 2024-02-26T06:20:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4ac80a21d0 0x7f4ac80a4690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae41045b0 0x7f4ae41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ae4104f70 0x7f4ae41a0a30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 --2- 172.21.15.49:0/3249745520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ae41033c0 0x7f4ae4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 >> 172.21.15.49:0/3249745520 conn(0x7f4ae40fe970 msgr2=0x7f4ae41078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:41.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 shutdown_connections 2024-02-26T06:20:41.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.259+0000 7f4aeac00640 1 -- 172.21.15.49:0/3249745520 wait complete. 2024-02-26T06:20:41.552 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:20:41.553 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:41 smithi049 ceph-mon[31696]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 3.5 KiB/s wr, 11 op/s 2024-02-26T06:20:41.553 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:41 smithi049 ceph-mon[31696]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:41.553 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:41 smithi049 ceph-mon[31696]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-02-26T06:20:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:41 smithi080 ceph-mon[36206]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 3.5 KiB/s wr, 11 op/s 2024-02-26T06:20:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:41 smithi080 ceph-mon[36206]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:41 smithi080 ceph-mon[36206]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-02-26T06:20:41.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:41 smithi115 ceph-mon[36426]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 3.5 KiB/s wr, 11 op/s 2024-02-26T06:20:41.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:41 smithi115 ceph-mon[36426]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:41.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:41 smithi115 ceph-mon[36426]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-02-26T06:20:41.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 -- 172.21.15.49:0/3428765485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 msgr2=0x7f4364101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:41.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 --2- 172.21.15.49:0/3428765485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 0x7f4364101400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4354009ca0 tx=0x7f435402c100 comp rx=0 tx=0).stop 2024-02-26T06:20:41.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 -- 172.21.15.49:0/3428765485 shutdown_connections 2024-02-26T06:20:41.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 --2- 172.21.15.49:0/3428765485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4364102b90 0x7f4364109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 --2- 172.21.15.49:0/3428765485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43641021d0 0x7f4364102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 --2- 172.21.15.49:0/3428765485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 0x7f4364101400 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 -- 172.21.15.49:0/3428765485 >> 172.21.15.49:0/3428765485 conn(0x7f43640fc550 msgr2=0x7f43640fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:41.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 -- 172.21.15.49:0/3428765485 shutdown_connections 2024-02-26T06:20:41.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.796+0000 7f436a000640 1 -- 172.21.15.49:0/3428765485 wait complete. 2024-02-26T06:20:41.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.797+0000 7f436a000640 1 Processor -- start 2024-02-26T06:20:41.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.797+0000 7f436a000640 1 -- start start 2024-02-26T06:20:41.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f436a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 0x7f43641992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:41.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f436a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43641021d0 0x7f4364199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:41.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f436a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4364102b90 0x7f43641a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:41.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f436a000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43641a0dc0 con 0x7f43641021d0 2024-02-26T06:20:41.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f436a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43641a0f30 con 0x7f4364100fe0 2024-02-26T06:20:41.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f436a000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43641a10f0 con 0x7f4364102b90 2024-02-26T06:20:41.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f4363e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43641021d0 0x7f4364199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f4368c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 0x7f43641992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f4368c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 0x7f43641992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58044/0 (socket says 172.21.15.49:58044) 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f4368c00640 1 -- 172.21.15.49:0/1966039842 learned_addr learned my addr 172.21.15.49:0/1966039842 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.798+0000 7f4369600640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4364102b90 0x7f43641a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4368c00640 1 -- 172.21.15.49:0/1966039842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4364102b90 msgr2=0x7f43641a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4368c00640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4364102b90 0x7f43641a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4368c00640 1 -- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43641021d0 msgr2=0x7f4364199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4368c00640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43641021d0 0x7f4364199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4368c00640 1 -- 172.21.15.49:0/1966039842 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4354009900 con 0x7f4364100fe0 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4363e00640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43641021d0 0x7f4364199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4368c00640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 0x7f43641992d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4354037040 tx=0x7f4354034bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4361600640 1 -- 172.21.15.49:0/1966039842 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354034850 con 0x7f4364100fe0 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43641a1370 con 0x7f4364100fe0 2024-02-26T06:20:41.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.799+0000 7f4361600640 1 -- 172.21.15.49:0/1966039842 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4354034e10 con 0x7f4364100fe0 2024-02-26T06:20:41.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.800+0000 7f4361600640 1 -- 172.21.15.49:0/1966039842 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354040450 con 0x7f4364100fe0 2024-02-26T06:20:41.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.800+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43641a1870 con 0x7f4364100fe0 2024-02-26T06:20:41.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.801+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436410c9b0 con 0x7f4364100fe0 2024-02-26T06:20:41.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.803+0000 7f4361600640 1 -- 172.21.15.49:0/1966039842 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f435400a040 con 0x7f4364100fe0 2024-02-26T06:20:41.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.805+0000 7f4361600640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43480a2150 0x7f43480a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:41.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.805+0000 7f4361600640 1 -- 172.21.15.49:0/1966039842 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f43540f05d0 con 0x7f4364100fe0 2024-02-26T06:20:41.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.805+0000 7f4363e00640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43480a2150 0x7f43480a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:41.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.806+0000 7f4363e00640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43480a2150 0x7f43480a4610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f436419a7e0 tx=0x7f4350005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:41.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:41.809+0000 7f4361600640 1 -- 172.21.15.49:0/1966039842 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43540b9700 con 0x7f4364100fe0 2024-02-26T06:20:42.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.012+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f436410cb40 con 0x7f4364100fe0 2024-02-26T06:20:42.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.014+0000 7f4361600640 1 -- 172.21.15.49:0/1966039842 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22206 (secure 0 0 0) 0x7f43540bcdc0 con 0x7f4364100fe0 2024-02-26T06:20:42.014 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:42.014 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:20:32.561576+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:20:42.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43480a2150 msgr2=0x7f43480a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43480a2150 0x7f43480a4610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f436419a7e0 tx=0x7f4350005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:42.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 msgr2=0x7f43641992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 0x7f43641992d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4354037040 tx=0x7f4354034bf0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 shutdown_connections 2024-02-26T06:20:42.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f43480a2150 0x7f43480a4610 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4364102b90 0x7f43641a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43641021d0 0x7f4364199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 --2- 172.21.15.49:0/1966039842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4364100fe0 0x7f43641992d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 >> 172.21.15.49:0/1966039842 conn(0x7f43640fc550 msgr2=0x7f43640733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:42.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 shutdown_connections 2024-02-26T06:20:42.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.017+0000 7f436a000640 1 -- 172.21.15.49:0/1966039842 wait complete. 2024-02-26T06:20:42.037 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T06:20:42.037 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:20:42.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.273+0000 7f4b11000640 1 -- 172.21.15.49:0/2659387888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 msgr2=0x7f4b0c10b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.273+0000 7f4b11000640 1 --2- 172.21.15.49:0/2659387888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 0x7f4b0c10b610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009ca0 tx=0x7f4b0402c100 comp rx=0 tx=0).stop 2024-02-26T06:20:42.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.273+0000 7f4b11000640 1 -- 172.21.15.49:0/2659387888 shutdown_connections 2024-02-26T06:20:42.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.273+0000 7f4b11000640 1 --2- 172.21.15.49:0/2659387888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 0x7f4b0c10b610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.273+0000 7f4b11000640 1 --2- 172.21.15.49:0/2659387888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0c1043c0 0x7f4b0c104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.273+0000 7f4b11000640 1 --2- 172.21.15.49:0/2659387888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b0c1031d0 0x7f4b0c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.273+0000 7f4b11000640 1 -- 172.21.15.49:0/2659387888 >> 172.21.15.49:0/2659387888 conn(0x7f4b0c0fe760 msgr2=0x7f4b0c100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:42.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.274+0000 7f4b11000640 1 -- 172.21.15.49:0/2659387888 shutdown_connections 2024-02-26T06:20:42.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.274+0000 7f4b11000640 1 -- 172.21.15.49:0/2659387888 wait complete. 2024-02-26T06:20:42.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.275+0000 7f4b11000640 1 Processor -- start 2024-02-26T06:20:42.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.275+0000 7f4b11000640 1 -- start start 2024-02-26T06:20:42.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.275+0000 7f4b11000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0c1031d0 0x7f4b0c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:42.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.275+0000 7f4b11000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b0c1043c0 0x7f4b0c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:42.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.275+0000 7f4b11000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 0x7f4b0c1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:42.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.275+0000 7f4b11000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c1a0cf0 con 0x7f4b0c1031d0 2024-02-26T06:20:42.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.275+0000 7f4b11000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c1a0e60 con 0x7f4b0c104d80 2024-02-26T06:20:42.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.275+0000 7f4b11000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c1a1020 con 0x7f4b0c1043c0 2024-02-26T06:20:42.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0c1031d0 0x7f4b0c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:42.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0c1031d0 0x7f4b0c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57908/0 (socket says 172.21.15.49:57908) 2024-02-26T06:20:42.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0b400640 1 -- 172.21.15.49:0/3526491125 learned_addr learned my addr 172.21.15.49:0/3526491125 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:42.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0be00640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 0x7f4b0c1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:42.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0aa00640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b0c1043c0 0x7f4b0c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:42.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0b400640 1 -- 172.21.15.49:0/3526491125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b0c1043c0 msgr2=0x7f4b0c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0b400640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b0c1043c0 0x7f4b0c199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0b400640 1 -- 172.21.15.49:0/3526491125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 msgr2=0x7f4b0c1a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0b400640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 0x7f4b0c1a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0b400640 1 -- 172.21.15.49:0/3526491125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b04009900 con 0x7f4b0c1031d0 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.276+0000 7f4b0be00640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 0x7f4b0c1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.277+0000 7f4b0b400640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0c1031d0 0x7f4b0c199200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00e9c0 tx=0x7f4afc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.277+0000 7f4b03e00640 1 -- 172.21.15.49:0/3526491125 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0096e0 con 0x7f4b0c1031d0 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.277+0000 7f4b03e00640 1 -- 172.21.15.49:0/3526491125 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4afc00cd80 con 0x7f4b0c1031d0 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.277+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c1a1300 con 0x7f4b0c1031d0 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.277+0000 7f4b03e00640 1 -- 172.21.15.49:0/3526491125 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc010430 con 0x7f4b0c1031d0 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.277+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c1a1810 con 0x7f4b0c1031d0 2024-02-26T06:20:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.278+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0c10a4f0 con 0x7f4b0c1031d0 2024-02-26T06:20:42.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.280+0000 7f4b03e00640 1 -- 172.21.15.49:0/3526491125 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4afc003680 con 0x7f4b0c1031d0 2024-02-26T06:20:42.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.282+0000 7f4b03e00640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4af00a21d0 0x7f4af00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:42.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.282+0000 7f4b03e00640 1 -- 172.21.15.49:0/3526491125 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4afc014070 con 0x7f4b0c1031d0 2024-02-26T06:20:42.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.282+0000 7f4b0aa00640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4af00a21d0 0x7f4af00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:42.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.283+0000 7f4b0aa00640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4af00a21d0 0x7f4af00a4690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c19a710 tx=0x7f4af8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:42.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.286+0000 7f4b03e00640 1 -- 172.21.15.49:0/3526491125 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4afc096fc0 con 0x7f4b0c1031d0 2024-02-26T06:20:42.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:42 smithi049 ceph-mon[31696]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:42.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:42 smithi049 ceph-mon[31696]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:42.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:42 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3249745520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:42.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:42 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1966039842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:42.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.485+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b0c02ceb0 con 0x7f4af00a21d0 2024-02-26T06:20:42.491 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:42 smithi115 ceph-mon[36426]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:42.492 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:42 smithi115 ceph-mon[36426]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:42.492 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:42 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3249745520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:42.492 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:42 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1966039842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:42.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.524+0000 7f4b03e00640 1 -- 172.21.15.49:0/3526491125 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334186 (secure 0 0 0) 0x7f4b0c02ceb0 con 0x7f4af00a21d0 2024-02-26T06:20:42.526 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:42.533 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-02-26T06:20:42.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.532+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4af00a21d0 msgr2=0x7f4af00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.532+0000 7f4b11000640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4af00a21d0 0x7f4af00a4690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c19a710 tx=0x7f4af8005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:42.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.532+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0c1031d0 msgr2=0x7f4b0c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.532+0000 7f4b11000640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0c1031d0 0x7f4b0c199200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00e9c0 tx=0x7f4afc00ee80 comp rx=0 tx=0).stop 2024-02-26T06:20:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.533+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 shutdown_connections 2024-02-26T06:20:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.533+0000 7f4b11000640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4af00a21d0 0x7f4af00a4690 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.533+0000 7f4b11000640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b0c104d80 0x7f4b0c1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.533+0000 7f4b11000640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b0c1043c0 0x7f4b0c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.533+0000 7f4b11000640 1 --2- 172.21.15.49:0/3526491125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0c1031d0 0x7f4b0c199200 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.533+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 >> 172.21.15.49:0/3526491125 conn(0x7f4b0c0fe760 msgr2=0x7f4b0c1076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.533+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 shutdown_connections 2024-02-26T06:20:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.533+0000 7f4b11000640 1 -- 172.21.15.49:0/3526491125 wait complete. 2024-02-26T06:20:42.558 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-02-26T06:20:41.421647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":152,"num_read_kb":134,"num_write":310,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":247,"ondisk_log_size":247,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360364,"kb_used_data":11212,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581460,"statfs":{"total":1151940427776,"available":1151571415040,"internally_reserved":0,"allocated":11481088,"data_stored":7134137,"data_compressed":110276,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":129,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.858444"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943903+0000","last_change":"2024-02-26T06:20:31.603060+0000","last_active":"2024-02-26T06:20:32.943903+0000","last_peered":"2024-02-26T06:20:32.943903+0000","last_clean":"2024-02-26T06:20:32.943903+0000","last_became_active":"2024-02-26T06:20:31.602762+0000","last_became_peered":"2024-02-26T06:20:31.602762+0000","last_unstale":"2024-02-26T06:20:32.943903+0000","last_undegraded":"2024-02-26T06:20:32.943903+0000","last_fullsized":"2024-02-26T06:20:32.943903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:19:42.465510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029996+0000","last_change":"2024-02-26T06:20:31.595138+0000","last_active":"2024-02-26T06:20:33.029996+0000","last_peered":"2024-02-26T06:20:33.029996+0000","last_clean":"2024-02-26T06:20:33.029996+0000","last_became_active":"2024-02-26T06:20:31.594826+0000","last_became_peered":"2024-02-26T06:20:31.594826+0000","last_unstale":"2024-02-26T06:20:33.029996+0000","last_undegraded":"2024-02-26T06:20:33.029996+0000","last_fullsized":"2024-02-26T06:20:33.029996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:49.900464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030816+0000","last_change":"2024-02-26T06:20:31.600415+0000","last_active":"2024-02-26T06:20:33.030816+0000","last_peered":"2024-02-26T06:20:33.030816+0000","last_clean":"2024-02-26T06:20:33.030816+0000","last_became_active":"2024-02-26T06:20:31.596273+0000","last_became_peered":"2024-02-26T06:20:31.596273+0000","last_unstale":"2024-02-26T06:20:33.030816+0000","last_undegraded":"2024-02-26T06:20:33.030816+0000","last_fullsized":"2024-02-26T06:20:33.030816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:15:20.626515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029941+0000","last_change":"2024-02-26T06:20:31.581149+0000","last_active":"2024-02-26T06:20:33.029941+0000","last_peered":"2024-02-26T06:20:33.029941+0000","last_clean":"2024-02-26T06:20:33.029941+0000","last_became_active":"2024-02-26T06:20:31.580831+0000","last_became_peered":"2024-02-26T06:20:31.580831+0000","last_unstale":"2024-02-26T06:20:33.029941+0000","last_undegraded":"2024-02-26T06:20:33.029941+0000","last_fullsized":"2024-02-26T06:20:33.029941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:47:12.466502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.908966+0000","last_change":"2024-02-26T06:20:31.608618+0000","last_active":"2024-02-26T06:20:32.908966+0000","last_peered":"2024-02-26T06:20:32.908966+0000","last_clean":"2024-02-26T06:20:32.908966+0000","last_became_active":"2024-02-26T06:20:31.608285+0000","last_became_peered":"2024-02-26T06:20:31.608285+0000","last_unstale":"2024-02-26T06:20:32.908966+0000","last_undegraded":"2024-02-26T06:20:32.908966+0000","last_fullsized":"2024-02-26T06:20:32.908966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:34:48.400836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.014089+0000","last_change":"2024-02-26T06:20:31.586929+0000","last_active":"2024-02-26T06:20:33.014089+0000","last_peered":"2024-02-26T06:20:33.014089+0000","last_clean":"2024-02-26T06:20:33.014089+0000","last_became_active":"2024-02-26T06:20:31.586630+0000","last_became_peered":"2024-02-26T06:20:31.586630+0000","last_unstale":"2024-02-26T06:20:33.014089+0000","last_undegraded":"2024-02-26T06:20:33.014089+0000","last_fullsized":"2024-02-26T06:20:33.014089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:09:27.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571978+0000","last_change":"2024-02-26T06:20:31.587285+0000","last_active":"2024-02-26T06:20:32.571978+0000","last_peered":"2024-02-26T06:20:32.571978+0000","last_clean":"2024-02-26T06:20:32.571978+0000","last_became_active":"2024-02-26T06:20:31.586985+0000","last_became_peered":"2024-02-26T06:20:31.586985+0000","last_unstale":"2024-02-26T06:20:32.571978+0000","last_undegraded":"2024-02-26T06:20:32.571978+0000","last_fullsized":"2024-02-26T06:20:32.571978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:55:56.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570246+0000","last_change":"2024-02-26T06:20:31.591136+0000","last_active":"2024-02-26T06:20:32.570246+0000","last_peered":"2024-02-26T06:20:32.570246+0000","last_clean":"2024-02-26T06:20:32.570246+0000","last_became_active":"2024-02-26T06:20:31.590180+0000","last_became_peered":"2024-02-26T06:20:31.590180+0000","last_unstale":"2024-02-26T06:20:32.570246+0000","last_undegraded":"2024-02-26T06:20:32.570246+0000","last_fullsized":"2024-02-26T06:20:32.570246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:02.649407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029893+0000","last_change":"2024-02-26T06:20:31.583890+0000","last_active":"2024-02-26T06:20:33.029893+0000","last_peered":"2024-02-26T06:20:33.029893+0000","last_clean":"2024-02-26T06:20:33.029893+0000","last_became_active":"2024-02-26T06:20:31.583459+0000","last_became_peered":"2024-02-26T06:20:31.583459+0000","last_unstale":"2024-02-26T06:20:33.029893+0000","last_undegraded":"2024-02-26T06:20:33.029893+0000","last_fullsized":"2024-02-26T06:20:33.029893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:14.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.014344+0000","last_change":"2024-02-26T06:20:31.579041+0000","last_active":"2024-02-26T06:20:33.014344+0000","last_peered":"2024-02-26T06:20:33.014344+0000","last_clean":"2024-02-26T06:20:33.014344+0000","last_became_active":"2024-02-26T06:20:31.578725+0000","last_became_peered":"2024-02-26T06:20:31.578725+0000","last_unstale":"2024-02-26T06:20:33.014344+0000","last_undegraded":"2024-02-26T06:20:33.014344+0000","last_fullsized":"2024-02-26T06:20:33.014344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:21.083206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572384+0000","last_change":"2024-02-26T06:20:31.581146+0000","last_active":"2024-02-26T06:20:32.572384+0000","last_peered":"2024-02-26T06:20:32.572384+0000","last_clean":"2024-02-26T06:20:32.572384+0000","last_became_active":"2024-02-26T06:20:31.580774+0000","last_became_peered":"2024-02-26T06:20:31.580774+0000","last_unstale":"2024-02-26T06:20:32.572384+0000","last_undegraded":"2024-02-26T06:20:32.572384+0000","last_fullsized":"2024-02-26T06:20:32.572384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:58:36.851504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029862+0000","last_change":"2024-02-26T06:20:31.582232+0000","last_active":"2024-02-26T06:20:33.029862+0000","last_peered":"2024-02-26T06:20:33.029862+0000","last_clean":"2024-02-26T06:20:33.029862+0000","last_became_active":"2024-02-26T06:20:31.581805+0000","last_became_peered":"2024-02-26T06:20:31.581805+0000","last_unstale":"2024-02-26T06:20:33.029862+0000","last_undegraded":"2024-02-26T06:20:33.029862+0000","last_fullsized":"2024-02-26T06:20:33.029862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:04.513814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.032654+0000","last_change":"2024-02-26T06:20:31.606373+0000","last_active":"2024-02-26T06:20:33.032654+0000","last_peered":"2024-02-26T06:20:33.032654+0000","last_clean":"2024-02-26T06:20:33.032654+0000","last_became_active":"2024-02-26T06:20:31.603226+0000","last_became_peered":"2024-02-26T06:20:31.603226+0000","last_unstale":"2024-02-26T06:20:33.032654+0000","last_undegraded":"2024-02-26T06:20:33.032654+0000","last_fullsized":"2024-02-26T06:20:33.032654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:29:49.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030346+0000","last_change":"2024-02-26T06:20:31.594587+0000","last_active":"2024-02-26T06:20:33.030346+0000","last_peered":"2024-02-26T06:20:33.030346+0000","last_clean":"2024-02-26T06:20:33.030346+0000","last_became_active":"2024-02-26T06:20:31.594032+0000","last_became_peered":"2024-02-26T06:20:31.594032+0000","last_unstale":"2024-02-26T06:20:33.030346+0000","last_undegraded":"2024-02-26T06:20:33.030346+0000","last_fullsized":"2024-02-26T06:20:33.030346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:49:18.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030313+0000","last_change":"2024-02-26T06:20:31.593115+0000","last_active":"2024-02-26T06:20:33.030313+0000","last_peered":"2024-02-26T06:20:33.030313+0000","last_clean":"2024-02-26T06:20:33.030313+0000","last_became_active":"2024-02-26T06:20:31.592782+0000","last_became_peered":"2024-02-26T06:20:31.592782+0000","last_unstale":"2024-02-26T06:20:33.030313+0000","last_undegraded":"2024-02-26T06:20:33.030313+0000","last_fullsized":"2024-02-26T06:20:33.030313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:10.709124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029179+0000","last_change":"2024-02-26T06:20:31.597938+0000","last_active":"2024-02-26T06:20:33.029179+0000","last_peered":"2024-02-26T06:20:33.029179+0000","last_clean":"2024-02-26T06:20:33.029179+0000","last_became_active":"2024-02-26T06:20:31.597600+0000","last_became_peered":"2024-02-26T06:20:31.597600+0000","last_unstale":"2024-02-26T06:20:33.029179+0000","last_undegraded":"2024-02-26T06:20:33.029179+0000","last_fullsized":"2024-02-26T06:20:33.029179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:13.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943986+0000","last_change":"2024-02-26T06:20:31.603283+0000","last_active":"2024-02-26T06:20:32.943986+0000","last_peered":"2024-02-26T06:20:32.943986+0000","last_clean":"2024-02-26T06:20:32.943986+0000","last_became_active":"2024-02-26T06:20:31.602912+0000","last_became_peered":"2024-02-26T06:20:31.602912+0000","last_unstale":"2024-02-26T06:20:32.943986+0000","last_undegraded":"2024-02-26T06:20:32.943986+0000","last_fullsized":"2024-02-26T06:20:32.943986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:32.250914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031497+0000","last_change":"2024-02-26T06:20:31.588670+0000","last_active":"2024-02-26T06:20:33.031497+0000","last_peered":"2024-02-26T06:20:33.031497+0000","last_clean":"2024-02-26T06:20:33.031497+0000","last_became_active":"2024-02-26T06:20:31.587971+0000","last_became_peered":"2024-02-26T06:20:31.587971+0000","last_unstale":"2024-02-26T06:20:33.031497+0000","last_undegraded":"2024-02-26T06:20:33.031497+0000","last_fullsized":"2024-02-26T06:20:33.031497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:13.053456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029382+0000","last_change":"2024-02-26T06:20:31.604266+0000","last_active":"2024-02-26T06:20:33.029382+0000","last_peered":"2024-02-26T06:20:33.029382+0000","last_clean":"2024-02-26T06:20:33.029382+0000","last_became_active":"2024-02-26T06:20:31.604027+0000","last_became_peered":"2024-02-26T06:20:31.604027+0000","last_unstale":"2024-02-26T06:20:33.029382+0000","last_undegraded":"2024-02-26T06:20:33.029382+0000","last_fullsized":"2024-02-26T06:20:33.029382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:13.345138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.101093+0000","last_change":"2024-02-26T06:20:31.068090+0000","last_active":"2024-02-26T06:20:37.101093+0000","last_peered":"2024-02-26T06:20:37.101093+0000","last_clean":"2024-02-26T06:20:37.101093+0000","last_became_active":"2024-02-26T06:20:31.067731+0000","last_became_peered":"2024-02-26T06:20:31.067731+0000","last_unstale":"2024-02-26T06:20:37.101093+0000","last_undegraded":"2024-02-26T06:20:37.101093+0000","last_fullsized":"2024-02-26T06:20:37.101093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:24.734040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571397+0000","last_change":"2024-02-26T06:20:31.597200+0000","last_active":"2024-02-26T06:20:32.571397+0000","last_peered":"2024-02-26T06:20:32.571397+0000","last_clean":"2024-02-26T06:20:32.571397+0000","last_became_active":"2024-02-26T06:20:31.596851+0000","last_became_peered":"2024-02-26T06:20:31.596851+0000","last_unstale":"2024-02-26T06:20:32.571397+0000","last_undegraded":"2024-02-26T06:20:32.571397+0000","last_fullsized":"2024-02-26T06:20:32.571397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:21:32.720172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571326+0000","last_change":"2024-02-26T06:20:31.083236+0000","last_active":"2024-02-26T06:20:32.571326+0000","last_peered":"2024-02-26T06:20:32.571326+0000","last_clean":"2024-02-26T06:20:32.571326+0000","last_became_active":"2024-02-26T06:20:31.081640+0000","last_became_peered":"2024-02-26T06:20:31.081640+0000","last_unstale":"2024-02-26T06:20:32.571326+0000","last_undegraded":"2024-02-26T06:20:32.571326+0000","last_fullsized":"2024-02-26T06:20:32.571326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:44.607108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031046+0000","last_change":"2024-02-26T06:20:31.593807+0000","last_active":"2024-02-26T06:20:33.031046+0000","last_peered":"2024-02-26T06:20:33.031046+0000","last_clean":"2024-02-26T06:20:33.031046+0000","last_became_active":"2024-02-26T06:20:31.593435+0000","last_became_peered":"2024-02-26T06:20:31.593435+0000","last_unstale":"2024-02-26T06:20:33.031046+0000","last_undegraded":"2024-02-26T06:20:33.031046+0000","last_fullsized":"2024-02-26T06:20:33.031046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:43.193196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.091324+0000","last_change":"2024-02-26T06:20:31.061339+0000","last_active":"2024-02-26T06:20:37.091324+0000","last_peered":"2024-02-26T06:20:37.091324+0000","last_clean":"2024-02-26T06:20:37.091324+0000","last_became_active":"2024-02-26T06:20:31.060769+0000","last_became_peered":"2024-02-26T06:20:31.060769+0000","last_unstale":"2024-02-26T06:20:37.091324+0000","last_undegraded":"2024-02-26T06:20:37.091324+0000","last_fullsized":"2024-02-26T06:20:37.091324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:28:30.501375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029392+0000","last_change":"2024-02-26T06:20:31.606936+0000","last_active":"2024-02-26T06:20:33.029392+0000","last_peered":"2024-02-26T06:20:33.029392+0000","last_clean":"2024-02-26T06:20:33.029392+0000","last_became_active":"2024-02-26T06:20:31.606582+0000","last_became_peered":"2024-02-26T06:20:31.606582+0000","last_unstale":"2024-02-26T06:20:33.029392+0000","last_undegraded":"2024-02-26T06:20:33.029392+0000","last_fullsized":"2024-02-26T06:20:33.029392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:09.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.909176+0000","last_change":"2024-02-26T06:20:31.063472+0000","last_active":"2024-02-26T06:20:32.909176+0000","last_peered":"2024-02-26T06:20:32.909176+0000","last_clean":"2024-02-26T06:20:32.909176+0000","last_became_active":"2024-02-26T06:20:31.063132+0000","last_became_peered":"2024-02-26T06:20:31.063132+0000","last_unstale":"2024-02-26T06:20:32.909176+0000","last_undegraded":"2024-02-26T06:20:32.909176+0000","last_fullsized":"2024-02-26T06:20:32.909176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:58.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569954+0000","last_change":"2024-02-26T06:20:31.589238+0000","last_active":"2024-02-26T06:20:32.569954+0000","last_peered":"2024-02-26T06:20:32.569954+0000","last_clean":"2024-02-26T06:20:32.569954+0000","last_became_active":"2024-02-26T06:20:31.588929+0000","last_became_peered":"2024-02-26T06:20:31.588929+0000","last_unstale":"2024-02-26T06:20:32.569954+0000","last_undegraded":"2024-02-26T06:20:32.569954+0000","last_fullsized":"2024-02-26T06:20:32.569954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:19.282943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.082307+0000","last_change":"2024-02-26T06:20:31.061714+0000","last_active":"2024-02-26T06:20:36.082307+0000","last_peered":"2024-02-26T06:20:36.082307+0000","last_clean":"2024-02-26T06:20:36.082307+0000","last_became_active":"2024-02-26T06:20:31.061234+0000","last_became_peered":"2024-02-26T06:20:31.061234+0000","last_unstale":"2024-02-26T06:20:36.082307+0000","last_undegraded":"2024-02-26T06:20:36.082307+0000","last_fullsized":"2024-02-26T06:20:36.082307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:29:17.668327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.908980+0000","last_change":"2024-02-26T06:20:31.588861+0000","last_active":"2024-02-26T06:20:32.908980+0000","last_peered":"2024-02-26T06:20:32.908980+0000","last_clean":"2024-02-26T06:20:32.908980+0000","last_became_active":"2024-02-26T06:20:31.588530+0000","last_became_peered":"2024-02-26T06:20:31.588530+0000","last_unstale":"2024-02-26T06:20:32.908980+0000","last_undegraded":"2024-02-26T06:20:32.908980+0000","last_fullsized":"2024-02-26T06:20:32.908980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:12.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.095338+0000","last_change":"2024-02-26T06:20:31.077656+0000","last_active":"2024-02-26T06:20:37.095338+0000","last_peered":"2024-02-26T06:20:37.095338+0000","last_clean":"2024-02-26T06:20:37.095338+0000","last_became_active":"2024-02-26T06:20:31.077288+0000","last_became_peered":"2024-02-26T06:20:31.077288+0000","last_unstale":"2024-02-26T06:20:37.095338+0000","last_undegraded":"2024-02-26T06:20:37.095338+0000","last_fullsized":"2024-02-26T06:20:37.095338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:56:05.173751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569652+0000","last_change":"2024-02-26T06:20:31.608089+0000","last_active":"2024-02-26T06:20:32.569652+0000","last_peered":"2024-02-26T06:20:32.569652+0000","last_clean":"2024-02-26T06:20:32.569652+0000","last_became_active":"2024-02-26T06:20:31.607786+0000","last_became_peered":"2024-02-26T06:20:31.607786+0000","last_unstale":"2024-02-26T06:20:32.569652+0000","last_undegraded":"2024-02-26T06:20:32.569652+0000","last_fullsized":"2024-02-26T06:20:32.569652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:24.113286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.101458+0000","last_change":"2024-02-26T06:20:31.077063+0000","last_active":"2024-02-26T06:20:37.101458+0000","last_peered":"2024-02-26T06:20:37.101458+0000","last_clean":"2024-02-26T06:20:37.101458+0000","last_became_active":"2024-02-26T06:20:31.075147+0000","last_became_peered":"2024-02-26T06:20:31.075147+0000","last_unstale":"2024-02-26T06:20:37.101458+0000","last_undegraded":"2024-02-26T06:20:37.101458+0000","last_fullsized":"2024-02-26T06:20:37.101458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:07:13.434635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570264+0000","last_change":"2024-02-26T06:20:31.600135+0000","last_active":"2024-02-26T06:20:32.570264+0000","last_peered":"2024-02-26T06:20:32.570264+0000","last_clean":"2024-02-26T06:20:32.570264+0000","last_became_active":"2024-02-26T06:20:31.595622+0000","last_became_peered":"2024-02-26T06:20:31.595622+0000","last_unstale":"2024-02-26T06:20:32.570264+0000","last_undegraded":"2024-02-26T06:20:32.570264+0000","last_fullsized":"2024-02-26T06:20:32.570264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:55.835700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569382+0000","last_change":"2024-02-26T06:20:31.060299+0000","last_active":"2024-02-26T06:20:32.569382+0000","last_peered":"2024-02-26T06:20:32.569382+0000","last_clean":"2024-02-26T06:20:32.569382+0000","last_became_active":"2024-02-26T06:20:31.059938+0000","last_became_peered":"2024-02-26T06:20:31.059938+0000","last_unstale":"2024-02-26T06:20:32.569382+0000","last_undegraded":"2024-02-26T06:20:32.569382+0000","last_fullsized":"2024-02-26T06:20:32.569382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:24.144958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031185+0000","last_change":"2024-02-26T06:20:31.601945+0000","last_active":"2024-02-26T06:20:33.031185+0000","last_peered":"2024-02-26T06:20:33.031185+0000","last_clean":"2024-02-26T06:20:33.031185+0000","last_became_active":"2024-02-26T06:20:31.601560+0000","last_became_peered":"2024-02-26T06:20:31.601560+0000","last_unstale":"2024-02-26T06:20:33.031185+0000","last_undegraded":"2024-02-26T06:20:33.031185+0000","last_fullsized":"2024-02-26T06:20:33.031185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:13.932623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.085496+0000","last_change":"2024-02-26T06:20:31.068334+0000","last_active":"2024-02-26T06:20:36.085496+0000","last_peered":"2024-02-26T06:20:36.085496+0000","last_clean":"2024-02-26T06:20:36.085496+0000","last_became_active":"2024-02-26T06:20:31.068020+0000","last_became_peered":"2024-02-26T06:20:31.068020+0000","last_unstale":"2024-02-26T06:20:36.085496+0000","last_undegraded":"2024-02-26T06:20:36.085496+0000","last_fullsized":"2024-02-26T06:20:36.085496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:23:01.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029825+0000","last_change":"2024-02-26T06:20:31.587862+0000","last_active":"2024-02-26T06:20:33.029825+0000","last_peered":"2024-02-26T06:20:33.029825+0000","last_clean":"2024-02-26T06:20:33.029825+0000","last_became_active":"2024-02-26T06:20:31.587551+0000","last_became_peered":"2024-02-26T06:20:31.587551+0000","last_unstale":"2024-02-26T06:20:33.029825+0000","last_undegraded":"2024-02-26T06:20:33.029825+0000","last_fullsized":"2024-02-26T06:20:33.029825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:07:15.933125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.080146+0000","last_change":"2024-02-26T06:20:31.058378+0000","last_active":"2024-02-26T06:20:36.080146+0000","last_peered":"2024-02-26T06:20:36.080146+0000","last_clean":"2024-02-26T06:20:36.080146+0000","last_became_active":"2024-02-26T06:20:31.058040+0000","last_became_peered":"2024-02-26T06:20:31.058040+0000","last_unstale":"2024-02-26T06:20:36.080146+0000","last_undegraded":"2024-02-26T06:20:36.080146+0000","last_fullsized":"2024-02-26T06:20:36.080146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:33:40.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030756+0000","last_change":"2024-02-26T06:20:31.601837+0000","last_active":"2024-02-26T06:20:33.030756+0000","last_peered":"2024-02-26T06:20:33.030756+0000","last_clean":"2024-02-26T06:20:33.030756+0000","last_became_active":"2024-02-26T06:20:31.597310+0000","last_became_peered":"2024-02-26T06:20:31.597310+0000","last_unstale":"2024-02-26T06:20:33.030756+0000","last_undegraded":"2024-02-26T06:20:33.030756+0000","last_fullsized":"2024-02-26T06:20:33.030756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.383510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570020+0000","last_change":"2024-02-26T06:20:31.076158+0000","last_active":"2024-02-26T06:20:32.570020+0000","last_peered":"2024-02-26T06:20:32.570020+0000","last_clean":"2024-02-26T06:20:32.570020+0000","last_became_active":"2024-02-26T06:20:31.075795+0000","last_became_peered":"2024-02-26T06:20:31.075795+0000","last_unstale":"2024-02-26T06:20:32.570020+0000","last_undegraded":"2024-02-26T06:20:32.570020+0000","last_fullsized":"2024-02-26T06:20:32.570020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:38.491674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029630+0000","last_change":"2024-02-26T06:20:31.586122+0000","last_active":"2024-02-26T06:20:33.029630+0000","last_peered":"2024-02-26T06:20:33.029630+0000","last_clean":"2024-02-26T06:20:33.029630+0000","last_became_active":"2024-02-26T06:20:31.585070+0000","last_became_peered":"2024-02-26T06:20:31.585070+0000","last_unstale":"2024-02-26T06:20:33.029630+0000","last_undegraded":"2024-02-26T06:20:33.029630+0000","last_fullsized":"2024-02-26T06:20:33.029630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:11.034252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.079382+0000","last_change":"2024-02-26T06:20:31.060917+0000","last_active":"2024-02-26T06:20:36.079382+0000","last_peered":"2024-02-26T06:20:36.079382+0000","last_clean":"2024-02-26T06:20:36.079382+0000","last_became_active":"2024-02-26T06:20:31.060555+0000","last_became_peered":"2024-02-26T06:20:31.060555+0000","last_unstale":"2024-02-26T06:20:36.079382+0000","last_undegraded":"2024-02-26T06:20:36.079382+0000","last_fullsized":"2024-02-26T06:20:36.079382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.909013+0000","last_change":"2024-02-26T06:20:31.584757+0000","last_active":"2024-02-26T06:20:32.909013+0000","last_peered":"2024-02-26T06:20:32.909013+0000","last_clean":"2024-02-26T06:20:32.909013+0000","last_became_active":"2024-02-26T06:20:31.583890+0000","last_became_peered":"2024-02-26T06:20:31.583890+0000","last_unstale":"2024-02-26T06:20:32.909013+0000","last_undegraded":"2024-02-26T06:20:32.909013+0000","last_fullsized":"2024-02-26T06:20:32.909013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:10:58.280082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.098544+0000","last_change":"2024-02-26T06:20:31.076044+0000","last_active":"2024-02-26T06:20:37.098544+0000","last_peered":"2024-02-26T06:20:37.098544+0000","last_clean":"2024-02-26T06:20:37.098544+0000","last_became_active":"2024-02-26T06:20:31.075744+0000","last_became_peered":"2024-02-26T06:20:31.075744+0000","last_unstale":"2024-02-26T06:20:37.098544+0000","last_undegraded":"2024-02-26T06:20:37.098544+0000","last_fullsized":"2024-02-26T06:20:37.098544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:35.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.032611+0000","last_change":"2024-02-26T06:20:31.602169+0000","last_active":"2024-02-26T06:20:33.032611+0000","last_peered":"2024-02-26T06:20:33.032611+0000","last_clean":"2024-02-26T06:20:33.032611+0000","last_became_active":"2024-02-26T06:20:31.601280+0000","last_became_peered":"2024-02-26T06:20:31.601280+0000","last_unstale":"2024-02-26T06:20:33.032611+0000","last_undegraded":"2024-02-26T06:20:33.032611+0000","last_fullsized":"2024-02-26T06:20:33.032611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:07.652549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.081938+0000","last_change":"2024-02-26T06:20:31.080482+0000","last_active":"2024-02-26T06:20:36.081938+0000","last_peered":"2024-02-26T06:20:36.081938+0000","last_clean":"2024-02-26T06:20:36.081938+0000","last_became_active":"2024-02-26T06:20:31.080188+0000","last_became_peered":"2024-02-26T06:20:31.080188+0000","last_unstale":"2024-02-26T06:20:36.081938+0000","last_undegraded":"2024-02-26T06:20:36.081938+0000","last_fullsized":"2024-02-26T06:20:36.081938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:32.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572110+0000","last_change":"2024-02-26T06:20:31.599753+0000","last_active":"2024-02-26T06:20:32.572110+0000","last_peered":"2024-02-26T06:20:32.572110+0000","last_clean":"2024-02-26T06:20:32.572110+0000","last_became_active":"2024-02-26T06:20:31.599409+0000","last_became_peered":"2024-02-26T06:20:31.599409+0000","last_unstale":"2024-02-26T06:20:32.572110+0000","last_undegraded":"2024-02-26T06:20:32.572110+0000","last_fullsized":"2024-02-26T06:20:32.572110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:43.887431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:38.108492+0000","last_change":"2024-02-26T06:20:31.059656+0000","last_active":"2024-02-26T06:20:38.108492+0000","last_peered":"2024-02-26T06:20:38.108492+0000","last_clean":"2024-02-26T06:20:38.108492+0000","last_became_active":"2024-02-26T06:20:31.059289+0000","last_became_peered":"2024-02-26T06:20:31.059289+0000","last_unstale":"2024-02-26T06:20:38.108492+0000","last_undegraded":"2024-02-26T06:20:38.108492+0000","last_fullsized":"2024-02-26T06:20:38.108492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:08:57.650876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571892+0000","last_change":"2024-02-26T06:20:31.583044+0000","last_active":"2024-02-26T06:20:32.571892+0000","last_peered":"2024-02-26T06:20:32.571892+0000","last_clean":"2024-02-26T06:20:32.571892+0000","last_became_active":"2024-02-26T06:20:31.582757+0000","last_became_peered":"2024-02-26T06:20:31.582757+0000","last_unstale":"2024-02-26T06:20:32.571892+0000","last_undegraded":"2024-02-26T06:20:32.571892+0000","last_fullsized":"2024-02-26T06:20:32.571892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:59.100921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.097297+0000","last_change":"2024-02-26T06:20:31.075077+0000","last_active":"2024-02-26T06:20:37.097297+0000","last_peered":"2024-02-26T06:20:37.097297+0000","last_clean":"2024-02-26T06:20:37.097297+0000","last_became_active":"2024-02-26T06:20:31.074751+0000","last_became_peered":"2024-02-26T06:20:31.074751+0000","last_unstale":"2024-02-26T06:20:37.097297+0000","last_undegraded":"2024-02-26T06:20:37.097297+0000","last_fullsized":"2024-02-26T06:20:37.097297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:53.559320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569376+0000","last_change":"2024-02-26T06:20:31.586151+0000","last_active":"2024-02-26T06:20:32.569376+0000","last_peered":"2024-02-26T06:20:32.569376+0000","last_clean":"2024-02-26T06:20:32.569376+0000","last_became_active":"2024-02-26T06:20:31.585615+0000","last_became_peered":"2024-02-26T06:20:31.585615+0000","last_unstale":"2024-02-26T06:20:32.569376+0000","last_undegraded":"2024-02-26T06:20:32.569376+0000","last_fullsized":"2024-02-26T06:20:32.569376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:38.105298+0000","last_change":"2024-02-26T06:20:31.062340+0000","last_active":"2024-02-26T06:20:38.105298+0000","last_peered":"2024-02-26T06:20:38.105298+0000","last_clean":"2024-02-26T06:20:38.105298+0000","last_became_active":"2024-02-26T06:20:31.062023+0000","last_became_peered":"2024-02-26T06:20:31.062023+0000","last_unstale":"2024-02-26T06:20:38.105298+0000","last_undegraded":"2024-02-26T06:20:38.105298+0000","last_fullsized":"2024-02-26T06:20:38.105298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:45.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943872+0000","last_change":"2024-02-26T06:20:31.585190+0000","last_active":"2024-02-26T06:20:32.943872+0000","last_peered":"2024-02-26T06:20:32.943872+0000","last_clean":"2024-02-26T06:20:32.943872+0000","last_became_active":"2024-02-26T06:20:31.584883+0000","last_became_peered":"2024-02-26T06:20:31.584883+0000","last_unstale":"2024-02-26T06:20:32.943872+0000","last_undegraded":"2024-02-26T06:20:32.943872+0000","last_fullsized":"2024-02-26T06:20:32.943872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:47:51.843824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031726+0000","last_change":"2024-02-26T06:20:31.075573+0000","last_active":"2024-02-26T06:20:33.031726+0000","last_peered":"2024-02-26T06:20:33.031726+0000","last_clean":"2024-02-26T06:20:33.031726+0000","last_became_active":"2024-02-26T06:20:31.075153+0000","last_became_peered":"2024-02-26T06:20:31.075153+0000","last_unstale":"2024-02-26T06:20:33.031726+0000","last_undegraded":"2024-02-26T06:20:33.031726+0000","last_fullsized":"2024-02-26T06:20:33.031726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:43.886858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.014146+0000","last_change":"2024-02-26T06:20:31.604400+0000","last_active":"2024-02-26T06:20:33.014146+0000","last_peered":"2024-02-26T06:20:33.014146+0000","last_clean":"2024-02-26T06:20:33.014146+0000","last_became_active":"2024-02-26T06:20:31.604068+0000","last_became_peered":"2024-02-26T06:20:31.604068+0000","last_unstale":"2024-02-26T06:20:33.014146+0000","last_undegraded":"2024-02-26T06:20:33.014146+0000","last_fullsized":"2024-02-26T06:20:33.014146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:28.792297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.381406+0000","last_change":"2024-02-26T06:20:31.072896+0000","last_active":"2024-02-26T06:20:33.381406+0000","last_peered":"2024-02-26T06:20:33.381406+0000","last_clean":"2024-02-26T06:20:33.381406+0000","last_became_active":"2024-02-26T06:20:31.072571+0000","last_became_peered":"2024-02-26T06:20:31.072571+0000","last_unstale":"2024-02-26T06:20:33.381406+0000","last_undegraded":"2024-02-26T06:20:33.381406+0000","last_fullsized":"2024-02-26T06:20:33.381406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:39.054737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.014303+0000","last_change":"2024-02-26T06:20:31.597409+0000","last_active":"2024-02-26T06:20:33.014303+0000","last_peered":"2024-02-26T06:20:33.014303+0000","last_clean":"2024-02-26T06:20:33.014303+0000","last_became_active":"2024-02-26T06:20:31.597067+0000","last_became_peered":"2024-02-26T06:20:31.597067+0000","last_unstale":"2024-02-26T06:20:33.014303+0000","last_undegraded":"2024-02-26T06:20:33.014303+0000","last_fullsized":"2024-02-26T06:20:33.014303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:58.947435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569580+0000","last_change":"2024-02-26T06:20:31.588989+0000","last_active":"2024-02-26T06:20:32.569580+0000","last_peered":"2024-02-26T06:20:32.569580+0000","last_clean":"2024-02-26T06:20:32.569580+0000","last_became_active":"2024-02-26T06:20:31.588683+0000","last_became_peered":"2024-02-26T06:20:31.588683+0000","last_unstale":"2024-02-26T06:20:32.569580+0000","last_undegraded":"2024-02-26T06:20:32.569580+0000","last_fullsized":"2024-02-26T06:20:32.569580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:29.687622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.101806+0000","last_change":"2024-02-26T06:20:31.081111+0000","last_active":"2024-02-26T06:20:37.101806+0000","last_peered":"2024-02-26T06:20:37.101806+0000","last_clean":"2024-02-26T06:20:37.101806+0000","last_became_active":"2024-02-26T06:20:31.080788+0000","last_became_peered":"2024-02-26T06:20:31.080788+0000","last_unstale":"2024-02-26T06:20:37.101806+0000","last_undegraded":"2024-02-26T06:20:37.101806+0000","last_fullsized":"2024-02-26T06:20:37.101806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:44:34.400256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570722+0000","last_change":"2024-02-26T06:20:31.595575+0000","last_active":"2024-02-26T06:20:32.570722+0000","last_peered":"2024-02-26T06:20:32.570722+0000","last_clean":"2024-02-26T06:20:32.570722+0000","last_became_active":"2024-02-26T06:20:31.595221+0000","last_became_peered":"2024-02-26T06:20:31.595221+0000","last_unstale":"2024-02-26T06:20:32.570722+0000","last_undegraded":"2024-02-26T06:20:32.570722+0000","last_fullsized":"2024-02-26T06:20:32.570722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:06.060667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:38.105976+0000","last_change":"2024-02-26T06:20:31.061702+0000","last_active":"2024-02-26T06:20:38.105976+0000","last_peered":"2024-02-26T06:20:38.105976+0000","last_clean":"2024-02-26T06:20:38.105976+0000","last_became_active":"2024-02-26T06:20:31.061358+0000","last_became_peered":"2024-02-26T06:20:31.061358+0000","last_unstale":"2024-02-26T06:20:38.105976+0000","last_undegraded":"2024-02-26T06:20:38.105976+0000","last_fullsized":"2024-02-26T06:20:38.105976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:25:41.652032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571819+0000","last_change":"2024-02-26T06:19:10.834318+0000","last_active":"2024-02-26T06:20:32.571819+0000","last_peered":"2024-02-26T06:20:32.571819+0000","last_clean":"2024-02-26T06:20:32.571819+0000","last_became_active":"2024-02-26T06:19:10.833869+0000","last_became_peered":"2024-02-26T06:19:10.833869+0000","last_unstale":"2024-02-26T06:20:32.571819+0000","last_undegraded":"2024-02-26T06:20:32.571819+0000","last_fullsized":"2024-02-26T06:20:32.571819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_clean_scrub_stamp":"2024-02-26T06:15:16.238016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:57.078533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.032010+0000","last_change":"2024-02-26T06:20:31.600738+0000","last_active":"2024-02-26T06:20:33.032010+0000","last_peered":"2024-02-26T06:20:33.032010+0000","last_clean":"2024-02-26T06:20:33.032010+0000","last_became_active":"2024-02-26T06:20:31.599144+0000","last_became_peered":"2024-02-26T06:20:31.599144+0000","last_unstale":"2024-02-26T06:20:33.032010+0000","last_undegraded":"2024-02-26T06:20:33.032010+0000","last_fullsized":"2024-02-26T06:20:33.032010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:01:16.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.091051+0000","last_change":"2024-02-26T06:20:31.063563+0000","last_active":"2024-02-26T06:20:37.091051+0000","last_peered":"2024-02-26T06:20:37.091051+0000","last_clean":"2024-02-26T06:20:37.091051+0000","last_became_active":"2024-02-26T06:20:31.063212+0000","last_became_peered":"2024-02-26T06:20:31.063212+0000","last_unstale":"2024-02-26T06:20:37.091051+0000","last_undegraded":"2024-02-26T06:20:37.091051+0000","last_fullsized":"2024-02-26T06:20:37.091051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:16.480553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.085793+0000","last_change":"2024-02-26T06:20:31.075104+0000","last_active":"2024-02-26T06:20:36.085793+0000","last_peered":"2024-02-26T06:20:36.085793+0000","last_clean":"2024-02-26T06:20:36.085793+0000","last_became_active":"2024-02-26T06:20:31.074777+0000","last_became_peered":"2024-02-26T06:20:31.074777+0000","last_unstale":"2024-02-26T06:20:36.085793+0000","last_undegraded":"2024-02-26T06:20:36.085793+0000","last_fullsized":"2024-02-26T06:20:36.085793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:32:20.511703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.033060+0000","last_change":"2024-02-26T06:20:31.605038+0000","last_active":"2024-02-26T06:20:33.033060+0000","last_peered":"2024-02-26T06:20:33.033060+0000","last_clean":"2024-02-26T06:20:33.033060+0000","last_became_active":"2024-02-26T06:20:31.603329+0000","last_became_peered":"2024-02-26T06:20:31.603329+0000","last_unstale":"2024-02-26T06:20:33.033060+0000","last_undegraded":"2024-02-26T06:20:33.033060+0000","last_fullsized":"2024-02-26T06:20:33.033060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:45.066836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:39.560355+0000","last_change":"2024-02-26T06:20:31.084613+0000","last_active":"2024-02-26T06:20:39.560355+0000","last_peered":"2024-02-26T06:20:39.560355+0000","last_clean":"2024-02-26T06:20:39.560355+0000","last_became_active":"2024-02-26T06:20:31.084021+0000","last_became_peered":"2024-02-26T06:20:31.084021+0000","last_unstale":"2024-02-26T06:20:39.560355+0000","last_undegraded":"2024-02-26T06:20:39.560355+0000","last_fullsized":"2024-02-26T06:20:39.560355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:52:42.096437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571618+0000","last_change":"2024-02-26T06:20:31.590367+0000","last_active":"2024-02-26T06:20:32.571618+0000","last_peered":"2024-02-26T06:20:32.571618+0000","last_clean":"2024-02-26T06:20:32.571618+0000","last_became_active":"2024-02-26T06:20:31.582914+0000","last_became_peered":"2024-02-26T06:20:31.582914+0000","last_unstale":"2024-02-26T06:20:32.571618+0000","last_undegraded":"2024-02-26T06:20:32.571618+0000","last_fullsized":"2024-02-26T06:20:32.571618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:02:47.496527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.083187+0000","last_change":"2024-02-26T06:20:31.070190+0000","last_active":"2024-02-26T06:20:36.083187+0000","last_peered":"2024-02-26T06:20:36.083187+0000","last_clean":"2024-02-26T06:20:36.083187+0000","last_became_active":"2024-02-26T06:20:31.069854+0000","last_became_peered":"2024-02-26T06:20:31.069854+0000","last_unstale":"2024-02-26T06:20:36.083187+0000","last_undegraded":"2024-02-26T06:20:36.083187+0000","last_fullsized":"2024-02-26T06:20:36.083187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:57.728821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030075+0000","last_change":"2024-02-26T06:20:31.595388+0000","last_active":"2024-02-26T06:20:33.030075+0000","last_peered":"2024-02-26T06:20:33.030075+0000","last_clean":"2024-02-26T06:20:33.030075+0000","last_became_active":"2024-02-26T06:20:31.594970+0000","last_became_peered":"2024-02-26T06:20:31.594970+0000","last_unstale":"2024-02-26T06:20:33.030075+0000","last_undegraded":"2024-02-26T06:20:33.030075+0000","last_fullsized":"2024-02-26T06:20:33.030075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:12:52.182988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.085526+0000","last_change":"2024-02-26T06:20:31.066256+0000","last_active":"2024-02-26T06:20:36.085526+0000","last_peered":"2024-02-26T06:20:36.085526+0000","last_clean":"2024-02-26T06:20:36.085526+0000","last_became_active":"2024-02-26T06:20:31.065956+0000","last_became_peered":"2024-02-26T06:20:31.065956+0000","last_unstale":"2024-02-26T06:20:36.085526+0000","last_undegraded":"2024-02-26T06:20:36.085526+0000","last_fullsized":"2024-02-26T06:20:36.085526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:35.405114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.908867+0000","last_change":"2024-02-26T06:20:31.585102+0000","last_active":"2024-02-26T06:20:32.908867+0000","last_peered":"2024-02-26T06:20:32.908867+0000","last_clean":"2024-02-26T06:20:32.908867+0000","last_became_active":"2024-02-26T06:20:31.584351+0000","last_became_peered":"2024-02-26T06:20:31.584351+0000","last_unstale":"2024-02-26T06:20:32.908867+0000","last_undegraded":"2024-02-26T06:20:32.908867+0000","last_fullsized":"2024-02-26T06:20:32.908867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:09.708752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.086845+0000","last_change":"2024-02-26T06:20:31.079664+0000","last_active":"2024-02-26T06:20:36.086845+0000","last_peered":"2024-02-26T06:20:36.086845+0000","last_clean":"2024-02-26T06:20:36.086845+0000","last_became_active":"2024-02-26T06:20:31.079234+0000","last_became_peered":"2024-02-26T06:20:31.079234+0000","last_unstale":"2024-02-26T06:20:36.086845+0000","last_undegraded":"2024-02-26T06:20:36.086845+0000","last_fullsized":"2024-02-26T06:20:36.086845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:37.407215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943574+0000","last_change":"2024-02-26T06:20:31.597416+0000","last_active":"2024-02-26T06:20:32.943574+0000","last_peered":"2024-02-26T06:20:32.943574+0000","last_clean":"2024-02-26T06:20:32.943574+0000","last_became_active":"2024-02-26T06:20:31.597094+0000","last_became_peered":"2024-02-26T06:20:31.597094+0000","last_unstale":"2024-02-26T06:20:32.943574+0000","last_undegraded":"2024-02-26T06:20:32.943574+0000","last_fullsized":"2024-02-26T06:20:32.943574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:24.774298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:34.456816+0000","last_change":"2024-02-26T06:20:31.074372+0000","last_active":"2024-02-26T06:20:34.456816+0000","last_peered":"2024-02-26T06:20:34.456816+0000","last_clean":"2024-02-26T06:20:34.456816+0000","last_became_active":"2024-02-26T06:20:31.074041+0000","last_became_peered":"2024-02-26T06:20:31.074041+0000","last_unstale":"2024-02-26T06:20:34.456816+0000","last_undegraded":"2024-02-26T06:20:34.456816+0000","last_fullsized":"2024-02-26T06:20:34.456816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:01.367668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570829+0000","last_change":"2024-02-26T06:20:31.599588+0000","last_active":"2024-02-26T06:20:32.570829+0000","last_peered":"2024-02-26T06:20:32.570829+0000","last_clean":"2024-02-26T06:20:32.570829+0000","last_became_active":"2024-02-26T06:20:31.599240+0000","last_became_peered":"2024-02-26T06:20:31.599240+0000","last_unstale":"2024-02-26T06:20:32.570829+0000","last_undegraded":"2024-02-26T06:20:32.570829+0000","last_fullsized":"2024-02-26T06:20:32.570829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:07:14.799565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.080378+0000","last_change":"2024-02-26T06:20:31.062593+0000","last_active":"2024-02-26T06:20:36.080378+0000","last_peered":"2024-02-26T06:20:36.080378+0000","last_clean":"2024-02-26T06:20:36.080378+0000","last_became_active":"2024-02-26T06:20:31.061814+0000","last_became_peered":"2024-02-26T06:20:31.061814+0000","last_unstale":"2024-02-26T06:20:36.080378+0000","last_undegraded":"2024-02-26T06:20:36.080378+0000","last_fullsized":"2024-02-26T06:20:36.080378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:04.506156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572269+0000","last_change":"2024-02-26T06:20:31.603955+0000","last_active":"2024-02-26T06:20:32.572269+0000","last_peered":"2024-02-26T06:20:32.572269+0000","last_clean":"2024-02-26T06:20:32.572269+0000","last_became_active":"2024-02-26T06:20:31.603635+0000","last_became_peered":"2024-02-26T06:20:31.603635+0000","last_unstale":"2024-02-26T06:20:32.572269+0000","last_undegraded":"2024-02-26T06:20:32.572269+0000","last_fullsized":"2024-02-26T06:20:32.572269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:56:59.060204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.099730+0000","last_change":"2024-02-26T06:20:31.074917+0000","last_active":"2024-02-26T06:20:37.099730+0000","last_peered":"2024-02-26T06:20:37.099730+0000","last_clean":"2024-02-26T06:20:37.099730+0000","last_became_active":"2024-02-26T06:20:31.074590+0000","last_became_peered":"2024-02-26T06:20:31.074590+0000","last_unstale":"2024-02-26T06:20:37.099730+0000","last_undegraded":"2024-02-26T06:20:37.099730+0000","last_fullsized":"2024-02-26T06:20:37.099730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:43.160268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572409+0000","last_change":"2024-02-26T06:20:31.582236+0000","last_active":"2024-02-26T06:20:32.572409+0000","last_peered":"2024-02-26T06:20:32.572409+0000","last_clean":"2024-02-26T06:20:32.572409+0000","last_became_active":"2024-02-26T06:20:31.581851+0000","last_became_peered":"2024-02-26T06:20:31.581851+0000","last_unstale":"2024-02-26T06:20:32.572409+0000","last_undegraded":"2024-02-26T06:20:32.572409+0000","last_fullsized":"2024-02-26T06:20:32.572409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:53.214338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569834+0000","last_change":"2024-02-26T06:20:31.068179+0000","last_active":"2024-02-26T06:20:32.569834+0000","last_peered":"2024-02-26T06:20:32.569834+0000","last_clean":"2024-02-26T06:20:32.569834+0000","last_became_active":"2024-02-26T06:20:31.067888+0000","last_became_peered":"2024-02-26T06:20:31.067888+0000","last_unstale":"2024-02-26T06:20:32.569834+0000","last_undegraded":"2024-02-26T06:20:32.569834+0000","last_fullsized":"2024-02-26T06:20:32.569834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:47:39.223749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572284+0000","last_change":"2024-02-26T06:20:31.598820+0000","last_active":"2024-02-26T06:20:32.572284+0000","last_peered":"2024-02-26T06:20:32.572284+0000","last_clean":"2024-02-26T06:20:32.572284+0000","last_became_active":"2024-02-26T06:20:31.598469+0000","last_became_peered":"2024-02-26T06:20:31.598469+0000","last_unstale":"2024-02-26T06:20:32.572284+0000","last_undegraded":"2024-02-26T06:20:32.572284+0000","last_fullsized":"2024-02-26T06:20:32.572284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:41:15.713748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.033276+0000","last_change":"2024-02-26T06:20:31.080988+0000","last_active":"2024-02-26T06:20:33.033276+0000","last_peered":"2024-02-26T06:20:33.033276+0000","last_clean":"2024-02-26T06:20:33.033276+0000","last_became_active":"2024-02-26T06:20:31.080654+0000","last_became_peered":"2024-02-26T06:20:31.080654+0000","last_unstale":"2024-02-26T06:20:33.033276+0000","last_undegraded":"2024-02-26T06:20:33.033276+0000","last_fullsized":"2024-02-26T06:20:33.033276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:50.315162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572476+0000","last_change":"2024-02-26T06:20:31.586057+0000","last_active":"2024-02-26T06:20:32.572476+0000","last_peered":"2024-02-26T06:20:32.572476+0000","last_clean":"2024-02-26T06:20:32.572476+0000","last_became_active":"2024-02-26T06:20:31.585756+0000","last_became_peered":"2024-02-26T06:20:31.585756+0000","last_unstale":"2024-02-26T06:20:32.572476+0000","last_undegraded":"2024-02-26T06:20:32.572476+0000","last_fullsized":"2024-02-26T06:20:32.572476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:59:07.032287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943736+0000","last_change":"2024-02-26T06:20:31.589067+0000","last_active":"2024-02-26T06:20:32.943736+0000","last_peered":"2024-02-26T06:20:32.943736+0000","last_clean":"2024-02-26T06:20:32.943736+0000","last_became_active":"2024-02-26T06:20:31.588769+0000","last_became_peered":"2024-02-26T06:20:31.588769+0000","last_unstale":"2024-02-26T06:20:32.943736+0000","last_undegraded":"2024-02-26T06:20:32.943736+0000","last_fullsized":"2024-02-26T06:20:32.943736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:43.211127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029166+0000","last_change":"2024-02-26T06:20:31.583838+0000","last_active":"2024-02-26T06:20:33.029166+0000","last_peered":"2024-02-26T06:20:33.029166+0000","last_clean":"2024-02-26T06:20:33.029166+0000","last_became_active":"2024-02-26T06:20:31.583524+0000","last_became_peered":"2024-02-26T06:20:31.583524+0000","last_unstale":"2024-02-26T06:20:33.029166+0000","last_undegraded":"2024-02-26T06:20:33.029166+0000","last_fullsized":"2024-02-26T06:20:33.029166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:21.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029228+0000","last_change":"2024-02-26T06:20:31.597901+0000","last_active":"2024-02-26T06:20:33.029228+0000","last_peered":"2024-02-26T06:20:33.029228+0000","last_clean":"2024-02-26T06:20:33.029228+0000","last_became_active":"2024-02-26T06:20:31.597633+0000","last_became_peered":"2024-02-26T06:20:31.597633+0000","last_unstale":"2024-02-26T06:20:33.029228+0000","last_undegraded":"2024-02-26T06:20:33.029228+0000","last_fullsized":"2024-02-26T06:20:33.029228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:14.084579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031667+0000","last_change":"2024-02-26T06:20:31.608754+0000","last_active":"2024-02-26T06:20:33.031667+0000","last_peered":"2024-02-26T06:20:33.031667+0000","last_clean":"2024-02-26T06:20:33.031667+0000","last_became_active":"2024-02-26T06:20:31.608392+0000","last_became_peered":"2024-02-26T06:20:31.608392+0000","last_unstale":"2024-02-26T06:20:33.031667+0000","last_undegraded":"2024-02-26T06:20:33.031667+0000","last_fullsized":"2024-02-26T06:20:33.031667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:57:23.283252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.033338+0000","last_change":"2024-02-26T06:20:31.605866+0000","last_active":"2024-02-26T06:20:33.033338+0000","last_peered":"2024-02-26T06:20:33.033338+0000","last_clean":"2024-02-26T06:20:33.033338+0000","last_became_active":"2024-02-26T06:20:31.605404+0000","last_became_peered":"2024-02-26T06:20:31.605404+0000","last_unstale":"2024-02-26T06:20:33.033338+0000","last_undegraded":"2024-02-26T06:20:33.033338+0000","last_fullsized":"2024-02-26T06:20:33.033338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:03:25.055462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943823+0000","last_change":"2024-02-26T06:20:31.585214+0000","last_active":"2024-02-26T06:20:32.943823+0000","last_peered":"2024-02-26T06:20:32.943823+0000","last_clean":"2024-02-26T06:20:32.943823+0000","last_became_active":"2024-02-26T06:20:31.584895+0000","last_became_peered":"2024-02-26T06:20:31.584895+0000","last_unstale":"2024-02-26T06:20:32.943823+0000","last_undegraded":"2024-02-26T06:20:32.943823+0000","last_fullsized":"2024-02-26T06:20:32.943823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:38.507499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029549+0000","last_change":"2024-02-26T06:20:31.582189+0000","last_active":"2024-02-26T06:20:33.029549+0000","last_peered":"2024-02-26T06:20:33.029549+0000","last_clean":"2024-02-26T06:20:33.029549+0000","last_became_active":"2024-02-26T06:20:31.580729+0000","last_became_peered":"2024-02-26T06:20:31.580729+0000","last_unstale":"2024-02-26T06:20:33.029549+0000","last_undegraded":"2024-02-26T06:20:33.029549+0000","last_fullsized":"2024-02-26T06:20:33.029549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:21.337261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571411+0000","last_change":"2024-02-26T06:20:31.588894+0000","last_active":"2024-02-26T06:20:32.571411+0000","last_peered":"2024-02-26T06:20:32.571411+0000","last_clean":"2024-02-26T06:20:32.571411+0000","last_became_active":"2024-02-26T06:20:31.586362+0000","last_became_peered":"2024-02-26T06:20:31.586362+0000","last_unstale":"2024-02-26T06:20:32.571411+0000","last_undegraded":"2024-02-26T06:20:32.571411+0000","last_fullsized":"2024-02-26T06:20:32.571411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:28:30.009901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569429+0000","last_change":"2024-02-26T06:20:31.608264+0000","last_active":"2024-02-26T06:20:32.569429+0000","last_peered":"2024-02-26T06:20:32.569429+0000","last_clean":"2024-02-26T06:20:32.569429+0000","last_became_active":"2024-02-26T06:20:31.607855+0000","last_became_peered":"2024-02-26T06:20:31.607855+0000","last_unstale":"2024-02-26T06:20:32.569429+0000","last_undegraded":"2024-02-26T06:20:32.569429+0000","last_fullsized":"2024-02-26T06:20:32.569429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:20.169489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570362+0000","last_change":"2024-02-26T06:20:31.591000+0000","last_active":"2024-02-26T06:20:32.570362+0000","last_peered":"2024-02-26T06:20:32.570362+0000","last_clean":"2024-02-26T06:20:32.570362+0000","last_became_active":"2024-02-26T06:20:31.588201+0000","last_became_peered":"2024-02-26T06:20:31.588201+0000","last_unstale":"2024-02-26T06:20:32.570362+0000","last_undegraded":"2024-02-26T06:20:32.570362+0000","last_fullsized":"2024-02-26T06:20:32.570362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:47:59.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029772+0000","last_change":"2024-02-26T06:20:31.590398+0000","last_active":"2024-02-26T06:20:33.029772+0000","last_peered":"2024-02-26T06:20:33.029772+0000","last_clean":"2024-02-26T06:20:33.029772+0000","last_became_active":"2024-02-26T06:20:31.590042+0000","last_became_peered":"2024-02-26T06:20:31.590042+0000","last_unstale":"2024-02-26T06:20:33.029772+0000","last_undegraded":"2024-02-26T06:20:33.029772+0000","last_fullsized":"2024-02-26T06:20:33.029772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:40:01.233978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.032924+0000","last_change":"2024-02-26T06:20:31.611698+0000","last_active":"2024-02-26T06:20:33.032924+0000","last_peered":"2024-02-26T06:20:33.032924+0000","last_clean":"2024-02-26T06:20:33.032924+0000","last_became_active":"2024-02-26T06:20:31.610979+0000","last_became_peered":"2024-02-26T06:20:31.610979+0000","last_unstale":"2024-02-26T06:20:33.032924+0000","last_undegraded":"2024-02-26T06:20:33.032924+0000","last_fullsized":"2024-02-26T06:20:33.032924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:44.604373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943823+0000","last_change":"2024-02-26T06:20:31.582016+0000","last_active":"2024-02-26T06:20:32.943823+0000","last_peered":"2024-02-26T06:20:32.943823+0000","last_clean":"2024-02-26T06:20:32.943823+0000","last_became_active":"2024-02-26T06:20:31.581689+0000","last_became_peered":"2024-02-26T06:20:31.581689+0000","last_unstale":"2024-02-26T06:20:32.943823+0000","last_undegraded":"2024-02-26T06:20:32.943823+0000","last_fullsized":"2024-02-26T06:20:32.943823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:04:11.030182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284221,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":129,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":77474,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1034508,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.624},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.56000000000000005}]},{"osd":2,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.877}]},{"osd":4,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.85699999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.2629999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70999999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.83599999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.222},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.94799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.93799999999999994}]},{"osd":1,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.407,"5min":2.407,"15min":2.407},"last":1.252}]},{"osd":2,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":1.4610000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.968,"5min":1.968,"15min":1.968},"min":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.851,"5min":1.851,"15min":1.851},"min":{"1min":1.151,"5min":1.151,"15min":1.151},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.302}]},{"osd":4,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.857,"5min":1.857,"15min":1.857},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":3.4990000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":2.0579999999999998},{"interface":"front","average":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":3.98,"5min":3.98,"15min":3.98},"last":1.7669999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":3.6800000000000002,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":1.883},{"interface":"front","average":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"min":{"1min":1.429,"5min":1.429,"15min":1.429},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.5089999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":1.635,"5min":1.635,"15min":1.635},"max":{"1min":4.2779999999999996,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.3779999999999999},{"interface":"front","average":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"min":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":2.181}]},{"osd":7,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"min":{"1min":1.357,"5min":1.357,"15min":1.357},"max":{"1min":4.2000000000000002,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":2.1280000000000001},{"interface":"front","average":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"min":{"1min":1.736,"5min":1.736,"15min":1.736},"max":{"1min":3.4489999999999998,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":1.619}]},{"osd":8,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"min":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":4.077,"5min":4.077,"15min":4.077},"last":2.2959999999999998},{"interface":"front","average":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"min":{"1min":1.52,"5min":1.52,"15min":1.52},"max":{"1min":4.1239999999999997,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":1.9990000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:20:11 2024","interfaces":[{"interface":"back","average":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":4.3170000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":2.242},{"interface":"front","average":{"1min":2.105,"5min":2.105,"15min":2.105},"min":{"1min":1.147,"5min":1.147,"15min":1.147},"max":{"1min":3.1619999999999999,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":2.375}]}]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.75},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.78700000000000003},"min":{"1min":0.54400000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0800000000000001,"5min":1.198,"15min":1.198},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64500000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65700000000000003,"5min":0.755,"15min":0.755},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59699999999999998,"5min":0.624,"15min":0.624},"last":0.52400000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77600000000000002,"15min":0.76800000000000002},"min":{"1min":0.54800000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.879,"15min":0.86899999999999999},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.879}]},{"osd":4,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.504},"min":{"1min":0.31900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.67900000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63}]},{"osd":6,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.61099999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.1240000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.92600000000000005}]},{"osd":7,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.84599999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":1.083}]},{"osd":10,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.96499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":1033732,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51700000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.42699999999999999,"15min":0.434},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56699999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.499}]},{"osd":2,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.64900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.85299999999999998,"15min":0.85699999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.236,"5min":1.591,"15min":1.591},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.129,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0189999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.495,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.60699999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.5,"15min":0.51600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60399999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.61599999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.61299999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.621,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80400000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69999999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2330000000000001,"5min":1.353,"15min":1.353},"last":0.873}]},{"osd":8,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83599999999999997,"5min":0.88,"15min":0.88},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.50800000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84599999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.78100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":2.0270000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.282,"15min":1.2390000000000001},"min":{"1min":0.91200000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.109,"5min":2.508,"15min":2.508},"last":1.575}]},{"osd":1,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.075,"15min":1.073},"min":{"1min":0.85199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.292},{"interface":"front","average":{"1min":1.175,"5min":1.095,"15min":1.0940000000000001},"min":{"1min":0.76700000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5149999999999999,"5min":1.944,"15min":1.944},"last":1.361}]},{"osd":2,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2569999999999999,"15min":1.26},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.9910000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.423},{"interface":"front","average":{"1min":1.456,"5min":1.302,"15min":1.27},"min":{"1min":0.65700000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8720000000000001,"5min":2.359,"15min":2.359},"last":1.6830000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.8720000000000001,"5min":1.641,"15min":1.585},"min":{"1min":1.335,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4710000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.9830000000000001},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.6990000000000001,"15min":1.679},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":2.677,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":2.1539999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.4419999999999999,"15min":1.427},"min":{"1min":0.74199999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1560000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":2.198},{"interface":"front","average":{"1min":1.863,"5min":1.6279999999999999,"15min":1.5900000000000001},"min":{"1min":1.4239999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.833}]},{"osd":5,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.6559999999999999,"15min":1.627},"min":{"1min":0.81699999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.29,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.7629999999999999},{"interface":"front","average":{"1min":1.677,"5min":1.722,"15min":1.716},"min":{"1min":1.036,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.4199999999999999,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":2.0779999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.9590000000000001,"5min":1.631,"15min":1.5760000000000001},"min":{"1min":1.3240000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.4159999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":2.282},{"interface":"front","average":{"1min":2.105,"5min":1.79,"15min":1.738},"min":{"1min":1.5389999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":2.347}]},{"osd":8,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":2.0150000000000001,"5min":1.7709999999999999,"15min":1.7310000000000001},"min":{"1min":1.385,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":2.129},{"interface":"front","average":{"1min":2.0049999999999999,"5min":1.8400000000000001,"15min":1.8120000000000001},"min":{"1min":0.999,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":2.5750000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"min":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"max":{"1min":3.258,"5min":3.258,"15min":3.258},"last":2.4849999999999999},{"interface":"front","average":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"min":{"1min":1.786,"5min":1.786,"15min":1.786},"max":{"1min":3.3380000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":2.246}]},{"osd":10,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":2.101,"15min":2.101},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":2.4510000000000001},{"interface":"front","average":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"min":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.4020000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":485799,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70099999999999996,"15min":0.67100000000000004},"min":{"1min":0.59699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95999999999999996,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.28,"5min":0.82199999999999995,"15min":0.746},"min":{"1min":0.49199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":8.9190000000000005,"5min":8.9190000000000005,"15min":8.9190000000000005},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":0.72699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40100000000000002,"5min":0.316,"15min":0.316},"max":{"1min":9.2629999999999999,"5min":9.2629999999999999,"15min":9.2629999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63400000000000001,"5min":0.745,"15min":0.745},"last":0.80900000000000005}]},{"osd":3,"last update":"Mon Feb 26 06:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.52500000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.232,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.63700000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.87}]},{"osd":4,"last update":"Mon Feb 26 06:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40600000000000003,"15min":0.39400000000000002},"min":{"1min":0.317,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66200000000000003,"5min":0.748,"15min":0.748},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.39900000000000002},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.64700000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.48399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78800000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.89200000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.80300000000000005,"15min":0.78400000000000003},"min":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.135,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.02},{"interface":"front","average":{"1min":1.3340000000000001,"5min":0.93400000000000005,"15min":0.875},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":8.8729999999999993,"5min":8.8729999999999993,"15min":8.8729999999999993},"last":0.90300000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.497,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.83399999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.71999999999999997,"15min":0.64900000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.9380000000000006,"5min":8.9380000000000006,"15min":8.9380000000000006},"last":0.95299999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.85799999999999998},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.089}]},{"osd":10,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.93799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:19:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.88900000000000001,"15min":0.80400000000000005},"min":{"1min":0.71799999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.90600000000000003,"15min":0.83199999999999996},"min":{"1min":0.60299999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4470000000000001,"5min":1.649,"15min":1.649},"last":1.2070000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.0269999999999999,"15min":0.90800000000000003},"min":{"1min":0.73199999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":6.6070000000000002,"5min":6.6070000000000002,"15min":6.6070000000000002},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.1220000000000001,"15min":0.96599999999999997},"min":{"1min":0.80100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.871,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.609}]},{"osd":3,"last update":"Mon Feb 26 06:18:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.194,"15min":1.1200000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1230000000000002,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.246,"15min":1.159},"min":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.8819999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:20:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.167,"15min":1.0469999999999999},"min":{"1min":0.88400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9319999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.946},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2170000000000001,"15min":1.147},"min":{"1min":1.0029999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0840000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.47,"15min":1.4419999999999999},"min":{"1min":1.238,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.2080000000000002,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.4370000000000001,"15min":1.373},"min":{"1min":1.0029999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":6.5510000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":2.032}]},{"osd":6,"last update":"Mon Feb 26 06:20:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9359999999999999,"5min":1.8109999999999999,"15min":1.802},"min":{"1min":1.1240000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":2.1349999999999998},{"interface":"front","average":{"1min":1.8999999999999999,"5min":1.825,"15min":1.841},"min":{"1min":1.548,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.387,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":2.3100000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.73,"15min":1.7190000000000001},"min":{"1min":1.2150000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.45},{"interface":"front","average":{"1min":1.73,"5min":1.599,"15min":1.577},"min":{"1min":1.123,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":1.8640000000000001,"5min":1.742,"15min":1.722},"min":{"1min":1.353,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.468,"5min":2.548,"15min":2.548},"last":2.0680000000000001},{"interface":"front","average":{"1min":2.0510000000000002,"5min":1.7949999999999999,"15min":1.7529999999999999},"min":{"1min":1.5980000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":2.105}]},{"osd":9,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"min":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.6930000000000001},{"interface":"front","average":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"min":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"max":{"1min":2.5630000000000002,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":2.2679999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"min":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":2.1680000000000001},{"interface":"front","average":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"min":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":1.8180000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:20:34 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.883,"15min":1.883},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":2.1890000000000001},{"interface":"front","average":{"1min":2.016,"5min":2.016,"15min":2.016},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":2.2229999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":451134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0609999999999999,"5min":1.091,"15min":1.091},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.54500000000000004,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46500000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.51500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.0700000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.56200000000000006,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.085,"5min":1.367,"15min":1.367},"last":0.68400000000000005}]},{"osd":4,"last update":"Mon Feb 26 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76400000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.43099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.41999999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45800000000000002,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.497},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52600000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.57099999999999995,"5min":0.499,"15min":0.499},"max":{"1min":1.071,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.82599999999999996,"15min":0.81399999999999995},"min":{"1min":0.65500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.073,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47399999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71899999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.96599999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":441766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.115,"15min":1.054},"min":{"1min":0.70699999999999996,"5min":0.433,"15min":0.433},"max":{"1min":1.9490000000000001,"5min":2.109,"15min":2.109},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.861,"5min":1.25,"15min":1.0880000000000001},"min":{"1min":0.88600000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.7919999999999998,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":1.647}]},{"osd":1,"last update":"Mon Feb 26 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.099,"15min":1.0109999999999999},"min":{"1min":0.80500000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.772,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.0860000000000001,"15min":1.0580000000000001},"min":{"1min":0.86699999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.593,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.367}]},{"osd":2,"last update":"Mon Feb 26 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.2450000000000001,"15min":1.165},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0409999999999999,"15min":1.026},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.294}]},{"osd":4,"last update":"Mon Feb 26 06:20:18 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.2949999999999999,"15min":1.2110000000000001},"min":{"1min":0.92300000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.9119999999999999,"5min":2.613,"15min":2.613},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.472,"15min":1.3640000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.0649999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.8140000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.415,"15min":1.363},"min":{"1min":1.2689999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.0150000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.3260000000000001,"15min":1.325},"min":{"1min":0.872,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.911}]},{"osd":6,"last update":"Mon Feb 26 06:20:18 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.5229999999999999,"15min":1.4670000000000001},"min":{"1min":0.90800000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4039999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":2.383},{"interface":"front","average":{"1min":2.0830000000000002,"5min":1.758,"15min":1.6419999999999999},"min":{"1min":0.97199999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.681,"5min":4.681,"15min":4.681},"last":2.173}]},{"osd":7,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.774,"15min":1.738},"min":{"1min":1.516,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.363,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.98},{"interface":"front","average":{"1min":2.1619999999999999,"5min":1.756,"15min":1.6879999999999999},"min":{"1min":1.577,"5min":0.998,"15min":0.998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":2.2349999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.806},"min":{"1min":0.97399999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.587,"15min":1.5649999999999999},"min":{"1min":0.84999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.23,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.143}]},{"osd":9,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"min":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":2.2879999999999998},{"interface":"front","average":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"min":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"max":{"1min":4.266,"5min":4.266,"15min":4.266},"last":2.4300000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.476,"5min":2.476,"15min":2.476},"last":2.1080000000000001},{"interface":"front","average":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"min":{"1min":1.127,"5min":1.127,"15min":1.127},"max":{"1min":4.0609999999999999,"5min":4.0609999999999999,"15min":4.0609999999999999},"last":2.327}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.056}]}]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.50600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.079,"5min":1.093,"15min":1.093},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.36299999999999999,"15min":0.34999999999999998},"min":{"1min":0.26100000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.32000000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36699999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":0.65500000000000003,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":12.468,"5min":12.468,"15min":12.468},"last":0.62}]},{"osd":3,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73099999999999998,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.075,"5min":1.167,"15min":1.167},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.99099999999999999,"5min":1.069,"15min":1.069},"last":0.88400000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54900000000000004,"15min":0.52700000000000002},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82999999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71799999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.61499999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0820000000000001,"5min":1.327,"15min":1.327},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.623,"5min":0.502,"15min":0.502},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.47399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.92600000000000005}]},{"osd":9,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.089}]},{"osd":10,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68799999999999994,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.71599999999999997,"15min":0.68200000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0660000000000001,"5min":1.131,"15min":1.131},"last":0.92200000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66300000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.29999999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.315,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80000000000000004,"15min":0.78200000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.74299999999999999,"15min":0.71099999999999997},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0840000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.98399999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.77000000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.61799999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83099999999999996,"15min":0.80700000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.96199999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.86599999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83899999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.54300000000000004,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67600000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48099999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.71299999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.93999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":44227,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:20:42.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:20:42.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:42 smithi080 ceph-mon[36206]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:42.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:42 smithi080 ceph-mon[36206]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:42.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:42 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3249745520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:20:42.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:42 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1966039842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 -- 172.21.15.49:0/1264834908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098104f70 msgr2=0x7fe09810b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 --2- 172.21.15.49:0/1264834908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098104f70 0x7fe09810b800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe090009ca0 tx=0x7fe09002c0f0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 -- 172.21.15.49:0/1264834908 shutdown_connections 2024-02-26T06:20:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 --2- 172.21.15.49:0/1264834908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe098104f70 0x7fe09810b800 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 --2- 172.21.15.49:0/1264834908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0981045b0 0x7fe098104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 --2- 172.21.15.49:0/1264834908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0981033c0 0x7fe0981037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 -- 172.21.15.49:0/1264834908 >> 172.21.15.49:0/1264834908 conn(0x7fe0980fe970 msgr2=0x7fe098100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 -- 172.21.15.49:0/1264834908 shutdown_connections 2024-02-26T06:20:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.837+0000 7fe09fe00640 1 -- 172.21.15.49:0/1264834908 wait complete. 2024-02-26T06:20:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.838+0000 7fe09fe00640 1 Processor -- start 2024-02-26T06:20:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09fe00640 1 -- start start 2024-02-26T06:20:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0981033c0 0x7fe0981993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0981045b0 0x7fe098199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe098104f70 0x7fe0981a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09fe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981a0ee0 con 0x7fe098104f70 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981a1050 con 0x7fe0981045b0 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09fe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981a1210 con 0x7fe0981033c0 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0981033c0 0x7fe0981993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe098104f70 0x7fe0981a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0981033c0 0x7fe0981993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34938/0 (socket says 172.21.15.49:34938) 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09d400640 1 -- 172.21.15.49:0/3541701495 learned_addr learned my addr 172.21.15.49:0/3541701495 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.839+0000 7fe09ca00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0981045b0 0x7fe098199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe09d400640 1 -- 172.21.15.49:0/3541701495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0981045b0 msgr2=0x7fe098199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe09d400640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0981045b0 0x7fe098199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe09d400640 1 -- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe098104f70 msgr2=0x7fe0981a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe09d400640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe098104f70 0x7fe0981a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe09d400640 1 -- 172.21.15.49:0/3541701495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090009900 con 0x7fe0981033c0 2024-02-26T06:20:42.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe09d400640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0981033c0 0x7fe0981993f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00e9c0 tx=0x7fe08c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:42.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe096000640 1 -- 172.21.15.49:0/3541701495 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0096e0 con 0x7fe0981033c0 2024-02-26T06:20:42.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe096000640 1 -- 172.21.15.49:0/3541701495 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe08c00cd80 con 0x7fe0981033c0 2024-02-26T06:20:42.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe096000640 1 -- 172.21.15.49:0/3541701495 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c010430 con 0x7fe0981033c0 2024-02-26T06:20:42.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.840+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0981a14f0 con 0x7fe0981033c0 2024-02-26T06:20:42.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.841+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0981a19b0 con 0x7fe0981033c0 2024-02-26T06:20:42.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.842+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09810a6e0 con 0x7fe0981033c0 2024-02-26T06:20:42.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.844+0000 7fe096000640 1 -- 172.21.15.49:0/3541701495 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe08c003680 con 0x7fe0981033c0 2024-02-26T06:20:42.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.845+0000 7fe096000640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe0800a2150 0x7fe0800a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.845+0000 7fe096000640 1 -- 172.21.15.49:0/3541701495 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe08c014070 con 0x7fe0981033c0 2024-02-26T06:20:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.845+0000 7fe09ca00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe0800a2150 0x7fe0800a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.846+0000 7fe09ca00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe0800a2150 0x7fe0800a4610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe09819a900 tx=0x7fe084005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:42.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:42.849+0000 7fe096000640 1 -- 172.21.15.49:0/3541701495 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe08c0970f0 con 0x7fe0981033c0 2024-02-26T06:20:43.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.048+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe09802ceb0 con 0x7fe0800a2150 2024-02-26T06:20:43.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.079+0000 7fe096000640 1 -- 172.21.15.49:0/3541701495 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334186 (secure 0 0 0) 0x7fe09802ceb0 con 0x7fe0800a2150 2024-02-26T06:20:43.082 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:43.087 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.088+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe0800a2150 msgr2=0x7fe0800a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.088+0000 7fe09fe00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe0800a2150 0x7fe0800a4610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe09819a900 tx=0x7fe084005f90 comp rx=0 tx=0).stop 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.088+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0981033c0 msgr2=0x7fe0981993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.088+0000 7fe09fe00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0981033c0 0x7fe0981993f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00e9c0 tx=0x7fe08c00ee80 comp rx=0 tx=0).stop 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.088+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 shutdown_connections 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.088+0000 7fe09fe00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe0800a2150 0x7fe0800a4610 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.088+0000 7fe09fe00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0981045b0 0x7fe098199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.088+0000 7fe09fe00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe098104f70 0x7fe0981a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.089+0000 7fe09fe00640 1 --2- 172.21.15.49:0/3541701495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0981033c0 0x7fe0981993f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.089+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 >> 172.21.15.49:0/3541701495 conn(0x7fe0980fe970 msgr2=0x7fe0981078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.089+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 shutdown_connections 2024-02-26T06:20:43.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:43.089+0000 7fe09fe00640 1 -- 172.21.15.49:0/3541701495 wait complete. 2024-02-26T06:20:43.111 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-02-26T06:20:41.421647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":152,"num_read_kb":134,"num_write":310,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":247,"ondisk_log_size":247,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360364,"kb_used_data":11212,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581460,"statfs":{"total":1151940427776,"available":1151571415040,"internally_reserved":0,"allocated":11481088,"data_stored":7134137,"data_compressed":110276,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":129,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.858444"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943903+0000","last_change":"2024-02-26T06:20:31.603060+0000","last_active":"2024-02-26T06:20:32.943903+0000","last_peered":"2024-02-26T06:20:32.943903+0000","last_clean":"2024-02-26T06:20:32.943903+0000","last_became_active":"2024-02-26T06:20:31.602762+0000","last_became_peered":"2024-02-26T06:20:31.602762+0000","last_unstale":"2024-02-26T06:20:32.943903+0000","last_undegraded":"2024-02-26T06:20:32.943903+0000","last_fullsized":"2024-02-26T06:20:32.943903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:19:42.465510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029996+0000","last_change":"2024-02-26T06:20:31.595138+0000","last_active":"2024-02-26T06:20:33.029996+0000","last_peered":"2024-02-26T06:20:33.029996+0000","last_clean":"2024-02-26T06:20:33.029996+0000","last_became_active":"2024-02-26T06:20:31.594826+0000","last_became_peered":"2024-02-26T06:20:31.594826+0000","last_unstale":"2024-02-26T06:20:33.029996+0000","last_undegraded":"2024-02-26T06:20:33.029996+0000","last_fullsized":"2024-02-26T06:20:33.029996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:49.900464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030816+0000","last_change":"2024-02-26T06:20:31.600415+0000","last_active":"2024-02-26T06:20:33.030816+0000","last_peered":"2024-02-26T06:20:33.030816+0000","last_clean":"2024-02-26T06:20:33.030816+0000","last_became_active":"2024-02-26T06:20:31.596273+0000","last_became_peered":"2024-02-26T06:20:31.596273+0000","last_unstale":"2024-02-26T06:20:33.030816+0000","last_undegraded":"2024-02-26T06:20:33.030816+0000","last_fullsized":"2024-02-26T06:20:33.030816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:15:20.626515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029941+0000","last_change":"2024-02-26T06:20:31.581149+0000","last_active":"2024-02-26T06:20:33.029941+0000","last_peered":"2024-02-26T06:20:33.029941+0000","last_clean":"2024-02-26T06:20:33.029941+0000","last_became_active":"2024-02-26T06:20:31.580831+0000","last_became_peered":"2024-02-26T06:20:31.580831+0000","last_unstale":"2024-02-26T06:20:33.029941+0000","last_undegraded":"2024-02-26T06:20:33.029941+0000","last_fullsized":"2024-02-26T06:20:33.029941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:47:12.466502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.908966+0000","last_change":"2024-02-26T06:20:31.608618+0000","last_active":"2024-02-26T06:20:32.908966+0000","last_peered":"2024-02-26T06:20:32.908966+0000","last_clean":"2024-02-26T06:20:32.908966+0000","last_became_active":"2024-02-26T06:20:31.608285+0000","last_became_peered":"2024-02-26T06:20:31.608285+0000","last_unstale":"2024-02-26T06:20:32.908966+0000","last_undegraded":"2024-02-26T06:20:32.908966+0000","last_fullsized":"2024-02-26T06:20:32.908966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:34:48.400836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.014089+0000","last_change":"2024-02-26T06:20:31.586929+0000","last_active":"2024-02-26T06:20:33.014089+0000","last_peered":"2024-02-26T06:20:33.014089+0000","last_clean":"2024-02-26T06:20:33.014089+0000","last_became_active":"2024-02-26T06:20:31.586630+0000","last_became_peered":"2024-02-26T06:20:31.586630+0000","last_unstale":"2024-02-26T06:20:33.014089+0000","last_undegraded":"2024-02-26T06:20:33.014089+0000","last_fullsized":"2024-02-26T06:20:33.014089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:09:27.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571978+0000","last_change":"2024-02-26T06:20:31.587285+0000","last_active":"2024-02-26T06:20:32.571978+0000","last_peered":"2024-02-26T06:20:32.571978+0000","last_clean":"2024-02-26T06:20:32.571978+0000","last_became_active":"2024-02-26T06:20:31.586985+0000","last_became_peered":"2024-02-26T06:20:31.586985+0000","last_unstale":"2024-02-26T06:20:32.571978+0000","last_undegraded":"2024-02-26T06:20:32.571978+0000","last_fullsized":"2024-02-26T06:20:32.571978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:55:56.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570246+0000","last_change":"2024-02-26T06:20:31.591136+0000","last_active":"2024-02-26T06:20:32.570246+0000","last_peered":"2024-02-26T06:20:32.570246+0000","last_clean":"2024-02-26T06:20:32.570246+0000","last_became_active":"2024-02-26T06:20:31.590180+0000","last_became_peered":"2024-02-26T06:20:31.590180+0000","last_unstale":"2024-02-26T06:20:32.570246+0000","last_undegraded":"2024-02-26T06:20:32.570246+0000","last_fullsized":"2024-02-26T06:20:32.570246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:02.649407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029893+0000","last_change":"2024-02-26T06:20:31.583890+0000","last_active":"2024-02-26T06:20:33.029893+0000","last_peered":"2024-02-26T06:20:33.029893+0000","last_clean":"2024-02-26T06:20:33.029893+0000","last_became_active":"2024-02-26T06:20:31.583459+0000","last_became_peered":"2024-02-26T06:20:31.583459+0000","last_unstale":"2024-02-26T06:20:33.029893+0000","last_undegraded":"2024-02-26T06:20:33.029893+0000","last_fullsized":"2024-02-26T06:20:33.029893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:14.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.014344+0000","last_change":"2024-02-26T06:20:31.579041+0000","last_active":"2024-02-26T06:20:33.014344+0000","last_peered":"2024-02-26T06:20:33.014344+0000","last_clean":"2024-02-26T06:20:33.014344+0000","last_became_active":"2024-02-26T06:20:31.578725+0000","last_became_peered":"2024-02-26T06:20:31.578725+0000","last_unstale":"2024-02-26T06:20:33.014344+0000","last_undegraded":"2024-02-26T06:20:33.014344+0000","last_fullsized":"2024-02-26T06:20:33.014344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:21.083206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572384+0000","last_change":"2024-02-26T06:20:31.581146+0000","last_active":"2024-02-26T06:20:32.572384+0000","last_peered":"2024-02-26T06:20:32.572384+0000","last_clean":"2024-02-26T06:20:32.572384+0000","last_became_active":"2024-02-26T06:20:31.580774+0000","last_became_peered":"2024-02-26T06:20:31.580774+0000","last_unstale":"2024-02-26T06:20:32.572384+0000","last_undegraded":"2024-02-26T06:20:32.572384+0000","last_fullsized":"2024-02-26T06:20:32.572384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:58:36.851504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029862+0000","last_change":"2024-02-26T06:20:31.582232+0000","last_active":"2024-02-26T06:20:33.029862+0000","last_peered":"2024-02-26T06:20:33.029862+0000","last_clean":"2024-02-26T06:20:33.029862+0000","last_became_active":"2024-02-26T06:20:31.581805+0000","last_became_peered":"2024-02-26T06:20:31.581805+0000","last_unstale":"2024-02-26T06:20:33.029862+0000","last_undegraded":"2024-02-26T06:20:33.029862+0000","last_fullsized":"2024-02-26T06:20:33.029862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:04.513814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.032654+0000","last_change":"2024-02-26T06:20:31.606373+0000","last_active":"2024-02-26T06:20:33.032654+0000","last_peered":"2024-02-26T06:20:33.032654+0000","last_clean":"2024-02-26T06:20:33.032654+0000","last_became_active":"2024-02-26T06:20:31.603226+0000","last_became_peered":"2024-02-26T06:20:31.603226+0000","last_unstale":"2024-02-26T06:20:33.032654+0000","last_undegraded":"2024-02-26T06:20:33.032654+0000","last_fullsized":"2024-02-26T06:20:33.032654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:29:49.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030346+0000","last_change":"2024-02-26T06:20:31.594587+0000","last_active":"2024-02-26T06:20:33.030346+0000","last_peered":"2024-02-26T06:20:33.030346+0000","last_clean":"2024-02-26T06:20:33.030346+0000","last_became_active":"2024-02-26T06:20:31.594032+0000","last_became_peered":"2024-02-26T06:20:31.594032+0000","last_unstale":"2024-02-26T06:20:33.030346+0000","last_undegraded":"2024-02-26T06:20:33.030346+0000","last_fullsized":"2024-02-26T06:20:33.030346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:49:18.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030313+0000","last_change":"2024-02-26T06:20:31.593115+0000","last_active":"2024-02-26T06:20:33.030313+0000","last_peered":"2024-02-26T06:20:33.030313+0000","last_clean":"2024-02-26T06:20:33.030313+0000","last_became_active":"2024-02-26T06:20:31.592782+0000","last_became_peered":"2024-02-26T06:20:31.592782+0000","last_unstale":"2024-02-26T06:20:33.030313+0000","last_undegraded":"2024-02-26T06:20:33.030313+0000","last_fullsized":"2024-02-26T06:20:33.030313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:10.709124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029179+0000","last_change":"2024-02-26T06:20:31.597938+0000","last_active":"2024-02-26T06:20:33.029179+0000","last_peered":"2024-02-26T06:20:33.029179+0000","last_clean":"2024-02-26T06:20:33.029179+0000","last_became_active":"2024-02-26T06:20:31.597600+0000","last_became_peered":"2024-02-26T06:20:31.597600+0000","last_unstale":"2024-02-26T06:20:33.029179+0000","last_undegraded":"2024-02-26T06:20:33.029179+0000","last_fullsized":"2024-02-26T06:20:33.029179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:13.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943986+0000","last_change":"2024-02-26T06:20:31.603283+0000","last_active":"2024-02-26T06:20:32.943986+0000","last_peered":"2024-02-26T06:20:32.943986+0000","last_clean":"2024-02-26T06:20:32.943986+0000","last_became_active":"2024-02-26T06:20:31.602912+0000","last_became_peered":"2024-02-26T06:20:31.602912+0000","last_unstale":"2024-02-26T06:20:32.943986+0000","last_undegraded":"2024-02-26T06:20:32.943986+0000","last_fullsized":"2024-02-26T06:20:32.943986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:32.250914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031497+0000","last_change":"2024-02-26T06:20:31.588670+0000","last_active":"2024-02-26T06:20:33.031497+0000","last_peered":"2024-02-26T06:20:33.031497+0000","last_clean":"2024-02-26T06:20:33.031497+0000","last_became_active":"2024-02-26T06:20:31.587971+0000","last_became_peered":"2024-02-26T06:20:31.587971+0000","last_unstale":"2024-02-26T06:20:33.031497+0000","last_undegraded":"2024-02-26T06:20:33.031497+0000","last_fullsized":"2024-02-26T06:20:33.031497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:13.053456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029382+0000","last_change":"2024-02-26T06:20:31.604266+0000","last_active":"2024-02-26T06:20:33.029382+0000","last_peered":"2024-02-26T06:20:33.029382+0000","last_clean":"2024-02-26T06:20:33.029382+0000","last_became_active":"2024-02-26T06:20:31.604027+0000","last_became_peered":"2024-02-26T06:20:31.604027+0000","last_unstale":"2024-02-26T06:20:33.029382+0000","last_undegraded":"2024-02-26T06:20:33.029382+0000","last_fullsized":"2024-02-26T06:20:33.029382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:13.345138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.101093+0000","last_change":"2024-02-26T06:20:31.068090+0000","last_active":"2024-02-26T06:20:37.101093+0000","last_peered":"2024-02-26T06:20:37.101093+0000","last_clean":"2024-02-26T06:20:37.101093+0000","last_became_active":"2024-02-26T06:20:31.067731+0000","last_became_peered":"2024-02-26T06:20:31.067731+0000","last_unstale":"2024-02-26T06:20:37.101093+0000","last_undegraded":"2024-02-26T06:20:37.101093+0000","last_fullsized":"2024-02-26T06:20:37.101093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:24.734040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571397+0000","last_change":"2024-02-26T06:20:31.597200+0000","last_active":"2024-02-26T06:20:32.571397+0000","last_peered":"2024-02-26T06:20:32.571397+0000","last_clean":"2024-02-26T06:20:32.571397+0000","last_became_active":"2024-02-26T06:20:31.596851+0000","last_became_peered":"2024-02-26T06:20:31.596851+0000","last_unstale":"2024-02-26T06:20:32.571397+0000","last_undegraded":"2024-02-26T06:20:32.571397+0000","last_fullsized":"2024-02-26T06:20:32.571397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:21:32.720172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571326+0000","last_change":"2024-02-26T06:20:31.083236+0000","last_active":"2024-02-26T06:20:32.571326+0000","last_peered":"2024-02-26T06:20:32.571326+0000","last_clean":"2024-02-26T06:20:32.571326+0000","last_became_active":"2024-02-26T06:20:31.081640+0000","last_became_peered":"2024-02-26T06:20:31.081640+0000","last_unstale":"2024-02-26T06:20:32.571326+0000","last_undegraded":"2024-02-26T06:20:32.571326+0000","last_fullsized":"2024-02-26T06:20:32.571326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:44.607108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031046+0000","last_change":"2024-02-26T06:20:31.593807+0000","last_active":"2024-02-26T06:20:33.031046+0000","last_peered":"2024-02-26T06:20:33.031046+0000","last_clean":"2024-02-26T06:20:33.031046+0000","last_became_active":"2024-02-26T06:20:31.593435+0000","last_became_peered":"2024-02-26T06:20:31.593435+0000","last_unstale":"2024-02-26T06:20:33.031046+0000","last_undegraded":"2024-02-26T06:20:33.031046+0000","last_fullsized":"2024-02-26T06:20:33.031046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:43.193196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.091324+0000","last_change":"2024-02-26T06:20:31.061339+0000","last_active":"2024-02-26T06:20:37.091324+0000","last_peered":"2024-02-26T06:20:37.091324+0000","last_clean":"2024-02-26T06:20:37.091324+0000","last_became_active":"2024-02-26T06:20:31.060769+0000","last_became_peered":"2024-02-26T06:20:31.060769+0000","last_unstale":"2024-02-26T06:20:37.091324+0000","last_undegraded":"2024-02-26T06:20:37.091324+0000","last_fullsized":"2024-02-26T06:20:37.091324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:28:30.501375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029392+0000","last_change":"2024-02-26T06:20:31.606936+0000","last_active":"2024-02-26T06:20:33.029392+0000","last_peered":"2024-02-26T06:20:33.029392+0000","last_clean":"2024-02-26T06:20:33.029392+0000","last_became_active":"2024-02-26T06:20:31.606582+0000","last_became_peered":"2024-02-26T06:20:31.606582+0000","last_unstale":"2024-02-26T06:20:33.029392+0000","last_undegraded":"2024-02-26T06:20:33.029392+0000","last_fullsized":"2024-02-26T06:20:33.029392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:09.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.909176+0000","last_change":"2024-02-26T06:20:31.063472+0000","last_active":"2024-02-26T06:20:32.909176+0000","last_peered":"2024-02-26T06:20:32.909176+0000","last_clean":"2024-02-26T06:20:32.909176+0000","last_became_active":"2024-02-26T06:20:31.063132+0000","last_became_peered":"2024-02-26T06:20:31.063132+0000","last_unstale":"2024-02-26T06:20:32.909176+0000","last_undegraded":"2024-02-26T06:20:32.909176+0000","last_fullsized":"2024-02-26T06:20:32.909176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:58.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569954+0000","last_change":"2024-02-26T06:20:31.589238+0000","last_active":"2024-02-26T06:20:32.569954+0000","last_peered":"2024-02-26T06:20:32.569954+0000","last_clean":"2024-02-26T06:20:32.569954+0000","last_became_active":"2024-02-26T06:20:31.588929+0000","last_became_peered":"2024-02-26T06:20:31.588929+0000","last_unstale":"2024-02-26T06:20:32.569954+0000","last_undegraded":"2024-02-26T06:20:32.569954+0000","last_fullsized":"2024-02-26T06:20:32.569954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:19.282943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.082307+0000","last_change":"2024-02-26T06:20:31.061714+0000","last_active":"2024-02-26T06:20:36.082307+0000","last_peered":"2024-02-26T06:20:36.082307+0000","last_clean":"2024-02-26T06:20:36.082307+0000","last_became_active":"2024-02-26T06:20:31.061234+0000","last_became_peered":"2024-02-26T06:20:31.061234+0000","last_unstale":"2024-02-26T06:20:36.082307+0000","last_undegraded":"2024-02-26T06:20:36.082307+0000","last_fullsized":"2024-02-26T06:20:36.082307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:29:17.668327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.908980+0000","last_change":"2024-02-26T06:20:31.588861+0000","last_active":"2024-02-26T06:20:32.908980+0000","last_peered":"2024-02-26T06:20:32.908980+0000","last_clean":"2024-02-26T06:20:32.908980+0000","last_became_active":"2024-02-26T06:20:31.588530+0000","last_became_peered":"2024-02-26T06:20:31.588530+0000","last_unstale":"2024-02-26T06:20:32.908980+0000","last_undegraded":"2024-02-26T06:20:32.908980+0000","last_fullsized":"2024-02-26T06:20:32.908980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:12.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.095338+0000","last_change":"2024-02-26T06:20:31.077656+0000","last_active":"2024-02-26T06:20:37.095338+0000","last_peered":"2024-02-26T06:20:37.095338+0000","last_clean":"2024-02-26T06:20:37.095338+0000","last_became_active":"2024-02-26T06:20:31.077288+0000","last_became_peered":"2024-02-26T06:20:31.077288+0000","last_unstale":"2024-02-26T06:20:37.095338+0000","last_undegraded":"2024-02-26T06:20:37.095338+0000","last_fullsized":"2024-02-26T06:20:37.095338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:56:05.173751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569652+0000","last_change":"2024-02-26T06:20:31.608089+0000","last_active":"2024-02-26T06:20:32.569652+0000","last_peered":"2024-02-26T06:20:32.569652+0000","last_clean":"2024-02-26T06:20:32.569652+0000","last_became_active":"2024-02-26T06:20:31.607786+0000","last_became_peered":"2024-02-26T06:20:31.607786+0000","last_unstale":"2024-02-26T06:20:32.569652+0000","last_undegraded":"2024-02-26T06:20:32.569652+0000","last_fullsized":"2024-02-26T06:20:32.569652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:24.113286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.101458+0000","last_change":"2024-02-26T06:20:31.077063+0000","last_active":"2024-02-26T06:20:37.101458+0000","last_peered":"2024-02-26T06:20:37.101458+0000","last_clean":"2024-02-26T06:20:37.101458+0000","last_became_active":"2024-02-26T06:20:31.075147+0000","last_became_peered":"2024-02-26T06:20:31.075147+0000","last_unstale":"2024-02-26T06:20:37.101458+0000","last_undegraded":"2024-02-26T06:20:37.101458+0000","last_fullsized":"2024-02-26T06:20:37.101458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:07:13.434635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570264+0000","last_change":"2024-02-26T06:20:31.600135+0000","last_active":"2024-02-26T06:20:32.570264+0000","last_peered":"2024-02-26T06:20:32.570264+0000","last_clean":"2024-02-26T06:20:32.570264+0000","last_became_active":"2024-02-26T06:20:31.595622+0000","last_became_peered":"2024-02-26T06:20:31.595622+0000","last_unstale":"2024-02-26T06:20:32.570264+0000","last_undegraded":"2024-02-26T06:20:32.570264+0000","last_fullsized":"2024-02-26T06:20:32.570264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:55.835700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569382+0000","last_change":"2024-02-26T06:20:31.060299+0000","last_active":"2024-02-26T06:20:32.569382+0000","last_peered":"2024-02-26T06:20:32.569382+0000","last_clean":"2024-02-26T06:20:32.569382+0000","last_became_active":"2024-02-26T06:20:31.059938+0000","last_became_peered":"2024-02-26T06:20:31.059938+0000","last_unstale":"2024-02-26T06:20:32.569382+0000","last_undegraded":"2024-02-26T06:20:32.569382+0000","last_fullsized":"2024-02-26T06:20:32.569382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:24.144958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031185+0000","last_change":"2024-02-26T06:20:31.601945+0000","last_active":"2024-02-26T06:20:33.031185+0000","last_peered":"2024-02-26T06:20:33.031185+0000","last_clean":"2024-02-26T06:20:33.031185+0000","last_became_active":"2024-02-26T06:20:31.601560+0000","last_became_peered":"2024-02-26T06:20:31.601560+0000","last_unstale":"2024-02-26T06:20:33.031185+0000","last_undegraded":"2024-02-26T06:20:33.031185+0000","last_fullsized":"2024-02-26T06:20:33.031185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:13.932623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.085496+0000","last_change":"2024-02-26T06:20:31.068334+0000","last_active":"2024-02-26T06:20:36.085496+0000","last_peered":"2024-02-26T06:20:36.085496+0000","last_clean":"2024-02-26T06:20:36.085496+0000","last_became_active":"2024-02-26T06:20:31.068020+0000","last_became_peered":"2024-02-26T06:20:31.068020+0000","last_unstale":"2024-02-26T06:20:36.085496+0000","last_undegraded":"2024-02-26T06:20:36.085496+0000","last_fullsized":"2024-02-26T06:20:36.085496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:23:01.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029825+0000","last_change":"2024-02-26T06:20:31.587862+0000","last_active":"2024-02-26T06:20:33.029825+0000","last_peered":"2024-02-26T06:20:33.029825+0000","last_clean":"2024-02-26T06:20:33.029825+0000","last_became_active":"2024-02-26T06:20:31.587551+0000","last_became_peered":"2024-02-26T06:20:31.587551+0000","last_unstale":"2024-02-26T06:20:33.029825+0000","last_undegraded":"2024-02-26T06:20:33.029825+0000","last_fullsized":"2024-02-26T06:20:33.029825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:07:15.933125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.080146+0000","last_change":"2024-02-26T06:20:31.058378+0000","last_active":"2024-02-26T06:20:36.080146+0000","last_peered":"2024-02-26T06:20:36.080146+0000","last_clean":"2024-02-26T06:20:36.080146+0000","last_became_active":"2024-02-26T06:20:31.058040+0000","last_became_peered":"2024-02-26T06:20:31.058040+0000","last_unstale":"2024-02-26T06:20:36.080146+0000","last_undegraded":"2024-02-26T06:20:36.080146+0000","last_fullsized":"2024-02-26T06:20:36.080146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:33:40.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030756+0000","last_change":"2024-02-26T06:20:31.601837+0000","last_active":"2024-02-26T06:20:33.030756+0000","last_peered":"2024-02-26T06:20:33.030756+0000","last_clean":"2024-02-26T06:20:33.030756+0000","last_became_active":"2024-02-26T06:20:31.597310+0000","last_became_peered":"2024-02-26T06:20:31.597310+0000","last_unstale":"2024-02-26T06:20:33.030756+0000","last_undegraded":"2024-02-26T06:20:33.030756+0000","last_fullsized":"2024-02-26T06:20:33.030756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.383510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570020+0000","last_change":"2024-02-26T06:20:31.076158+0000","last_active":"2024-02-26T06:20:32.570020+0000","last_peered":"2024-02-26T06:20:32.570020+0000","last_clean":"2024-02-26T06:20:32.570020+0000","last_became_active":"2024-02-26T06:20:31.075795+0000","last_became_peered":"2024-02-26T06:20:31.075795+0000","last_unstale":"2024-02-26T06:20:32.570020+0000","last_undegraded":"2024-02-26T06:20:32.570020+0000","last_fullsized":"2024-02-26T06:20:32.570020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:38.491674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029630+0000","last_change":"2024-02-26T06:20:31.586122+0000","last_active":"2024-02-26T06:20:33.029630+0000","last_peered":"2024-02-26T06:20:33.029630+0000","last_clean":"2024-02-26T06:20:33.029630+0000","last_became_active":"2024-02-26T06:20:31.585070+0000","last_became_peered":"2024-02-26T06:20:31.585070+0000","last_unstale":"2024-02-26T06:20:33.029630+0000","last_undegraded":"2024-02-26T06:20:33.029630+0000","last_fullsized":"2024-02-26T06:20:33.029630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:11.034252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.079382+0000","last_change":"2024-02-26T06:20:31.060917+0000","last_active":"2024-02-26T06:20:36.079382+0000","last_peered":"2024-02-26T06:20:36.079382+0000","last_clean":"2024-02-26T06:20:36.079382+0000","last_became_active":"2024-02-26T06:20:31.060555+0000","last_became_peered":"2024-02-26T06:20:31.060555+0000","last_unstale":"2024-02-26T06:20:36.079382+0000","last_undegraded":"2024-02-26T06:20:36.079382+0000","last_fullsized":"2024-02-26T06:20:36.079382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:42.348988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.909013+0000","last_change":"2024-02-26T06:20:31.584757+0000","last_active":"2024-02-26T06:20:32.909013+0000","last_peered":"2024-02-26T06:20:32.909013+0000","last_clean":"2024-02-26T06:20:32.909013+0000","last_became_active":"2024-02-26T06:20:31.583890+0000","last_became_peered":"2024-02-26T06:20:31.583890+0000","last_unstale":"2024-02-26T06:20:32.909013+0000","last_undegraded":"2024-02-26T06:20:32.909013+0000","last_fullsized":"2024-02-26T06:20:32.909013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:10:58.280082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.098544+0000","last_change":"2024-02-26T06:20:31.076044+0000","last_active":"2024-02-26T06:20:37.098544+0000","last_peered":"2024-02-26T06:20:37.098544+0000","last_clean":"2024-02-26T06:20:37.098544+0000","last_became_active":"2024-02-26T06:20:31.075744+0000","last_became_peered":"2024-02-26T06:20:31.075744+0000","last_unstale":"2024-02-26T06:20:37.098544+0000","last_undegraded":"2024-02-26T06:20:37.098544+0000","last_fullsized":"2024-02-26T06:20:37.098544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:35.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.032611+0000","last_change":"2024-02-26T06:20:31.602169+0000","last_active":"2024-02-26T06:20:33.032611+0000","last_peered":"2024-02-26T06:20:33.032611+0000","last_clean":"2024-02-26T06:20:33.032611+0000","last_became_active":"2024-02-26T06:20:31.601280+0000","last_became_peered":"2024-02-26T06:20:31.601280+0000","last_unstale":"2024-02-26T06:20:33.032611+0000","last_undegraded":"2024-02-26T06:20:33.032611+0000","last_fullsized":"2024-02-26T06:20:33.032611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:07.652549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.081938+0000","last_change":"2024-02-26T06:20:31.080482+0000","last_active":"2024-02-26T06:20:36.081938+0000","last_peered":"2024-02-26T06:20:36.081938+0000","last_clean":"2024-02-26T06:20:36.081938+0000","last_became_active":"2024-02-26T06:20:31.080188+0000","last_became_peered":"2024-02-26T06:20:31.080188+0000","last_unstale":"2024-02-26T06:20:36.081938+0000","last_undegraded":"2024-02-26T06:20:36.081938+0000","last_fullsized":"2024-02-26T06:20:36.081938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:32.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572110+0000","last_change":"2024-02-26T06:20:31.599753+0000","last_active":"2024-02-26T06:20:32.572110+0000","last_peered":"2024-02-26T06:20:32.572110+0000","last_clean":"2024-02-26T06:20:32.572110+0000","last_became_active":"2024-02-26T06:20:31.599409+0000","last_became_peered":"2024-02-26T06:20:31.599409+0000","last_unstale":"2024-02-26T06:20:32.572110+0000","last_undegraded":"2024-02-26T06:20:32.572110+0000","last_fullsized":"2024-02-26T06:20:32.572110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:43.887431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:38.108492+0000","last_change":"2024-02-26T06:20:31.059656+0000","last_active":"2024-02-26T06:20:38.108492+0000","last_peered":"2024-02-26T06:20:38.108492+0000","last_clean":"2024-02-26T06:20:38.108492+0000","last_became_active":"2024-02-26T06:20:31.059289+0000","last_became_peered":"2024-02-26T06:20:31.059289+0000","last_unstale":"2024-02-26T06:20:38.108492+0000","last_undegraded":"2024-02-26T06:20:38.108492+0000","last_fullsized":"2024-02-26T06:20:38.108492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:08:57.650876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571892+0000","last_change":"2024-02-26T06:20:31.583044+0000","last_active":"2024-02-26T06:20:32.571892+0000","last_peered":"2024-02-26T06:20:32.571892+0000","last_clean":"2024-02-26T06:20:32.571892+0000","last_became_active":"2024-02-26T06:20:31.582757+0000","last_became_peered":"2024-02-26T06:20:31.582757+0000","last_unstale":"2024-02-26T06:20:32.571892+0000","last_undegraded":"2024-02-26T06:20:32.571892+0000","last_fullsized":"2024-02-26T06:20:32.571892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:59.100921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.097297+0000","last_change":"2024-02-26T06:20:31.075077+0000","last_active":"2024-02-26T06:20:37.097297+0000","last_peered":"2024-02-26T06:20:37.097297+0000","last_clean":"2024-02-26T06:20:37.097297+0000","last_became_active":"2024-02-26T06:20:31.074751+0000","last_became_peered":"2024-02-26T06:20:31.074751+0000","last_unstale":"2024-02-26T06:20:37.097297+0000","last_undegraded":"2024-02-26T06:20:37.097297+0000","last_fullsized":"2024-02-26T06:20:37.097297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:53.559320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569376+0000","last_change":"2024-02-26T06:20:31.586151+0000","last_active":"2024-02-26T06:20:32.569376+0000","last_peered":"2024-02-26T06:20:32.569376+0000","last_clean":"2024-02-26T06:20:32.569376+0000","last_became_active":"2024-02-26T06:20:31.585615+0000","last_became_peered":"2024-02-26T06:20:31.585615+0000","last_unstale":"2024-02-26T06:20:32.569376+0000","last_undegraded":"2024-02-26T06:20:32.569376+0000","last_fullsized":"2024-02-26T06:20:32.569376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:38.105298+0000","last_change":"2024-02-26T06:20:31.062340+0000","last_active":"2024-02-26T06:20:38.105298+0000","last_peered":"2024-02-26T06:20:38.105298+0000","last_clean":"2024-02-26T06:20:38.105298+0000","last_became_active":"2024-02-26T06:20:31.062023+0000","last_became_peered":"2024-02-26T06:20:31.062023+0000","last_unstale":"2024-02-26T06:20:38.105298+0000","last_undegraded":"2024-02-26T06:20:38.105298+0000","last_fullsized":"2024-02-26T06:20:38.105298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:45.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943872+0000","last_change":"2024-02-26T06:20:31.585190+0000","last_active":"2024-02-26T06:20:32.943872+0000","last_peered":"2024-02-26T06:20:32.943872+0000","last_clean":"2024-02-26T06:20:32.943872+0000","last_became_active":"2024-02-26T06:20:31.584883+0000","last_became_peered":"2024-02-26T06:20:31.584883+0000","last_unstale":"2024-02-26T06:20:32.943872+0000","last_undegraded":"2024-02-26T06:20:32.943872+0000","last_fullsized":"2024-02-26T06:20:32.943872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:47:51.843824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031726+0000","last_change":"2024-02-26T06:20:31.075573+0000","last_active":"2024-02-26T06:20:33.031726+0000","last_peered":"2024-02-26T06:20:33.031726+0000","last_clean":"2024-02-26T06:20:33.031726+0000","last_became_active":"2024-02-26T06:20:31.075153+0000","last_became_peered":"2024-02-26T06:20:31.075153+0000","last_unstale":"2024-02-26T06:20:33.031726+0000","last_undegraded":"2024-02-26T06:20:33.031726+0000","last_fullsized":"2024-02-26T06:20:33.031726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:43.886858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.014146+0000","last_change":"2024-02-26T06:20:31.604400+0000","last_active":"2024-02-26T06:20:33.014146+0000","last_peered":"2024-02-26T06:20:33.014146+0000","last_clean":"2024-02-26T06:20:33.014146+0000","last_became_active":"2024-02-26T06:20:31.604068+0000","last_became_peered":"2024-02-26T06:20:31.604068+0000","last_unstale":"2024-02-26T06:20:33.014146+0000","last_undegraded":"2024-02-26T06:20:33.014146+0000","last_fullsized":"2024-02-26T06:20:33.014146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:28.792297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.381406+0000","last_change":"2024-02-26T06:20:31.072896+0000","last_active":"2024-02-26T06:20:33.381406+0000","last_peered":"2024-02-26T06:20:33.381406+0000","last_clean":"2024-02-26T06:20:33.381406+0000","last_became_active":"2024-02-26T06:20:31.072571+0000","last_became_peered":"2024-02-26T06:20:31.072571+0000","last_unstale":"2024-02-26T06:20:33.381406+0000","last_undegraded":"2024-02-26T06:20:33.381406+0000","last_fullsized":"2024-02-26T06:20:33.381406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:39.054737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.014303+0000","last_change":"2024-02-26T06:20:31.597409+0000","last_active":"2024-02-26T06:20:33.014303+0000","last_peered":"2024-02-26T06:20:33.014303+0000","last_clean":"2024-02-26T06:20:33.014303+0000","last_became_active":"2024-02-26T06:20:31.597067+0000","last_became_peered":"2024-02-26T06:20:31.597067+0000","last_unstale":"2024-02-26T06:20:33.014303+0000","last_undegraded":"2024-02-26T06:20:33.014303+0000","last_fullsized":"2024-02-26T06:20:33.014303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:58.947435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569580+0000","last_change":"2024-02-26T06:20:31.588989+0000","last_active":"2024-02-26T06:20:32.569580+0000","last_peered":"2024-02-26T06:20:32.569580+0000","last_clean":"2024-02-26T06:20:32.569580+0000","last_became_active":"2024-02-26T06:20:31.588683+0000","last_became_peered":"2024-02-26T06:20:31.588683+0000","last_unstale":"2024-02-26T06:20:32.569580+0000","last_undegraded":"2024-02-26T06:20:32.569580+0000","last_fullsized":"2024-02-26T06:20:32.569580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:29.687622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.101806+0000","last_change":"2024-02-26T06:20:31.081111+0000","last_active":"2024-02-26T06:20:37.101806+0000","last_peered":"2024-02-26T06:20:37.101806+0000","last_clean":"2024-02-26T06:20:37.101806+0000","last_became_active":"2024-02-26T06:20:31.080788+0000","last_became_peered":"2024-02-26T06:20:31.080788+0000","last_unstale":"2024-02-26T06:20:37.101806+0000","last_undegraded":"2024-02-26T06:20:37.101806+0000","last_fullsized":"2024-02-26T06:20:37.101806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:44:34.400256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570722+0000","last_change":"2024-02-26T06:20:31.595575+0000","last_active":"2024-02-26T06:20:32.570722+0000","last_peered":"2024-02-26T06:20:32.570722+0000","last_clean":"2024-02-26T06:20:32.570722+0000","last_became_active":"2024-02-26T06:20:31.595221+0000","last_became_peered":"2024-02-26T06:20:31.595221+0000","last_unstale":"2024-02-26T06:20:32.570722+0000","last_undegraded":"2024-02-26T06:20:32.570722+0000","last_fullsized":"2024-02-26T06:20:32.570722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:06.060667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:38.105976+0000","last_change":"2024-02-26T06:20:31.061702+0000","last_active":"2024-02-26T06:20:38.105976+0000","last_peered":"2024-02-26T06:20:38.105976+0000","last_clean":"2024-02-26T06:20:38.105976+0000","last_became_active":"2024-02-26T06:20:31.061358+0000","last_became_peered":"2024-02-26T06:20:31.061358+0000","last_unstale":"2024-02-26T06:20:38.105976+0000","last_undegraded":"2024-02-26T06:20:38.105976+0000","last_fullsized":"2024-02-26T06:20:38.105976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:25:41.652032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571819+0000","last_change":"2024-02-26T06:19:10.834318+0000","last_active":"2024-02-26T06:20:32.571819+0000","last_peered":"2024-02-26T06:20:32.571819+0000","last_clean":"2024-02-26T06:20:32.571819+0000","last_became_active":"2024-02-26T06:19:10.833869+0000","last_became_peered":"2024-02-26T06:19:10.833869+0000","last_unstale":"2024-02-26T06:20:32.571819+0000","last_undegraded":"2024-02-26T06:20:32.571819+0000","last_fullsized":"2024-02-26T06:20:32.571819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_clean_scrub_stamp":"2024-02-26T06:15:16.238016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:57.078533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.032010+0000","last_change":"2024-02-26T06:20:31.600738+0000","last_active":"2024-02-26T06:20:33.032010+0000","last_peered":"2024-02-26T06:20:33.032010+0000","last_clean":"2024-02-26T06:20:33.032010+0000","last_became_active":"2024-02-26T06:20:31.599144+0000","last_became_peered":"2024-02-26T06:20:31.599144+0000","last_unstale":"2024-02-26T06:20:33.032010+0000","last_undegraded":"2024-02-26T06:20:33.032010+0000","last_fullsized":"2024-02-26T06:20:33.032010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:01:16.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.091051+0000","last_change":"2024-02-26T06:20:31.063563+0000","last_active":"2024-02-26T06:20:37.091051+0000","last_peered":"2024-02-26T06:20:37.091051+0000","last_clean":"2024-02-26T06:20:37.091051+0000","last_became_active":"2024-02-26T06:20:31.063212+0000","last_became_peered":"2024-02-26T06:20:31.063212+0000","last_unstale":"2024-02-26T06:20:37.091051+0000","last_undegraded":"2024-02-26T06:20:37.091051+0000","last_fullsized":"2024-02-26T06:20:37.091051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:16.480553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.085793+0000","last_change":"2024-02-26T06:20:31.075104+0000","last_active":"2024-02-26T06:20:36.085793+0000","last_peered":"2024-02-26T06:20:36.085793+0000","last_clean":"2024-02-26T06:20:36.085793+0000","last_became_active":"2024-02-26T06:20:31.074777+0000","last_became_peered":"2024-02-26T06:20:31.074777+0000","last_unstale":"2024-02-26T06:20:36.085793+0000","last_undegraded":"2024-02-26T06:20:36.085793+0000","last_fullsized":"2024-02-26T06:20:36.085793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:32:20.511703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.033060+0000","last_change":"2024-02-26T06:20:31.605038+0000","last_active":"2024-02-26T06:20:33.033060+0000","last_peered":"2024-02-26T06:20:33.033060+0000","last_clean":"2024-02-26T06:20:33.033060+0000","last_became_active":"2024-02-26T06:20:31.603329+0000","last_became_peered":"2024-02-26T06:20:31.603329+0000","last_unstale":"2024-02-26T06:20:33.033060+0000","last_undegraded":"2024-02-26T06:20:33.033060+0000","last_fullsized":"2024-02-26T06:20:33.033060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:45.066836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:39.560355+0000","last_change":"2024-02-26T06:20:31.084613+0000","last_active":"2024-02-26T06:20:39.560355+0000","last_peered":"2024-02-26T06:20:39.560355+0000","last_clean":"2024-02-26T06:20:39.560355+0000","last_became_active":"2024-02-26T06:20:31.084021+0000","last_became_peered":"2024-02-26T06:20:31.084021+0000","last_unstale":"2024-02-26T06:20:39.560355+0000","last_undegraded":"2024-02-26T06:20:39.560355+0000","last_fullsized":"2024-02-26T06:20:39.560355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:52:42.096437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571618+0000","last_change":"2024-02-26T06:20:31.590367+0000","last_active":"2024-02-26T06:20:32.571618+0000","last_peered":"2024-02-26T06:20:32.571618+0000","last_clean":"2024-02-26T06:20:32.571618+0000","last_became_active":"2024-02-26T06:20:31.582914+0000","last_became_peered":"2024-02-26T06:20:31.582914+0000","last_unstale":"2024-02-26T06:20:32.571618+0000","last_undegraded":"2024-02-26T06:20:32.571618+0000","last_fullsized":"2024-02-26T06:20:32.571618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:02:47.496527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.083187+0000","last_change":"2024-02-26T06:20:31.070190+0000","last_active":"2024-02-26T06:20:36.083187+0000","last_peered":"2024-02-26T06:20:36.083187+0000","last_clean":"2024-02-26T06:20:36.083187+0000","last_became_active":"2024-02-26T06:20:31.069854+0000","last_became_peered":"2024-02-26T06:20:31.069854+0000","last_unstale":"2024-02-26T06:20:36.083187+0000","last_undegraded":"2024-02-26T06:20:36.083187+0000","last_fullsized":"2024-02-26T06:20:36.083187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:57.728821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.030075+0000","last_change":"2024-02-26T06:20:31.595388+0000","last_active":"2024-02-26T06:20:33.030075+0000","last_peered":"2024-02-26T06:20:33.030075+0000","last_clean":"2024-02-26T06:20:33.030075+0000","last_became_active":"2024-02-26T06:20:31.594970+0000","last_became_peered":"2024-02-26T06:20:31.594970+0000","last_unstale":"2024-02-26T06:20:33.030075+0000","last_undegraded":"2024-02-26T06:20:33.030075+0000","last_fullsized":"2024-02-26T06:20:33.030075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:12:52.182988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.085526+0000","last_change":"2024-02-26T06:20:31.066256+0000","last_active":"2024-02-26T06:20:36.085526+0000","last_peered":"2024-02-26T06:20:36.085526+0000","last_clean":"2024-02-26T06:20:36.085526+0000","last_became_active":"2024-02-26T06:20:31.065956+0000","last_became_peered":"2024-02-26T06:20:31.065956+0000","last_unstale":"2024-02-26T06:20:36.085526+0000","last_undegraded":"2024-02-26T06:20:36.085526+0000","last_fullsized":"2024-02-26T06:20:36.085526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:35.405114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.908867+0000","last_change":"2024-02-26T06:20:31.585102+0000","last_active":"2024-02-26T06:20:32.908867+0000","last_peered":"2024-02-26T06:20:32.908867+0000","last_clean":"2024-02-26T06:20:32.908867+0000","last_became_active":"2024-02-26T06:20:31.584351+0000","last_became_peered":"2024-02-26T06:20:31.584351+0000","last_unstale":"2024-02-26T06:20:32.908867+0000","last_undegraded":"2024-02-26T06:20:32.908867+0000","last_fullsized":"2024-02-26T06:20:32.908867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:09.708752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.086845+0000","last_change":"2024-02-26T06:20:31.079664+0000","last_active":"2024-02-26T06:20:36.086845+0000","last_peered":"2024-02-26T06:20:36.086845+0000","last_clean":"2024-02-26T06:20:36.086845+0000","last_became_active":"2024-02-26T06:20:31.079234+0000","last_became_peered":"2024-02-26T06:20:31.079234+0000","last_unstale":"2024-02-26T06:20:36.086845+0000","last_undegraded":"2024-02-26T06:20:36.086845+0000","last_fullsized":"2024-02-26T06:20:36.086845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:37.407215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943574+0000","last_change":"2024-02-26T06:20:31.597416+0000","last_active":"2024-02-26T06:20:32.943574+0000","last_peered":"2024-02-26T06:20:32.943574+0000","last_clean":"2024-02-26T06:20:32.943574+0000","last_became_active":"2024-02-26T06:20:31.597094+0000","last_became_peered":"2024-02-26T06:20:31.597094+0000","last_unstale":"2024-02-26T06:20:32.943574+0000","last_undegraded":"2024-02-26T06:20:32.943574+0000","last_fullsized":"2024-02-26T06:20:32.943574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:24.774298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:34.456816+0000","last_change":"2024-02-26T06:20:31.074372+0000","last_active":"2024-02-26T06:20:34.456816+0000","last_peered":"2024-02-26T06:20:34.456816+0000","last_clean":"2024-02-26T06:20:34.456816+0000","last_became_active":"2024-02-26T06:20:31.074041+0000","last_became_peered":"2024-02-26T06:20:31.074041+0000","last_unstale":"2024-02-26T06:20:34.456816+0000","last_undegraded":"2024-02-26T06:20:34.456816+0000","last_fullsized":"2024-02-26T06:20:34.456816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:01.367668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570829+0000","last_change":"2024-02-26T06:20:31.599588+0000","last_active":"2024-02-26T06:20:32.570829+0000","last_peered":"2024-02-26T06:20:32.570829+0000","last_clean":"2024-02-26T06:20:32.570829+0000","last_became_active":"2024-02-26T06:20:31.599240+0000","last_became_peered":"2024-02-26T06:20:31.599240+0000","last_unstale":"2024-02-26T06:20:32.570829+0000","last_undegraded":"2024-02-26T06:20:32.570829+0000","last_fullsized":"2024-02-26T06:20:32.570829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:07:14.799565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:36.080378+0000","last_change":"2024-02-26T06:20:31.062593+0000","last_active":"2024-02-26T06:20:36.080378+0000","last_peered":"2024-02-26T06:20:36.080378+0000","last_clean":"2024-02-26T06:20:36.080378+0000","last_became_active":"2024-02-26T06:20:31.061814+0000","last_became_peered":"2024-02-26T06:20:31.061814+0000","last_unstale":"2024-02-26T06:20:36.080378+0000","last_undegraded":"2024-02-26T06:20:36.080378+0000","last_fullsized":"2024-02-26T06:20:36.080378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:04.506156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572269+0000","last_change":"2024-02-26T06:20:31.603955+0000","last_active":"2024-02-26T06:20:32.572269+0000","last_peered":"2024-02-26T06:20:32.572269+0000","last_clean":"2024-02-26T06:20:32.572269+0000","last_became_active":"2024-02-26T06:20:31.603635+0000","last_became_peered":"2024-02-26T06:20:31.603635+0000","last_unstale":"2024-02-26T06:20:32.572269+0000","last_undegraded":"2024-02-26T06:20:32.572269+0000","last_fullsized":"2024-02-26T06:20:32.572269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:56:59.060204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:37.099730+0000","last_change":"2024-02-26T06:20:31.074917+0000","last_active":"2024-02-26T06:20:37.099730+0000","last_peered":"2024-02-26T06:20:37.099730+0000","last_clean":"2024-02-26T06:20:37.099730+0000","last_became_active":"2024-02-26T06:20:31.074590+0000","last_became_peered":"2024-02-26T06:20:31.074590+0000","last_unstale":"2024-02-26T06:20:37.099730+0000","last_undegraded":"2024-02-26T06:20:37.099730+0000","last_fullsized":"2024-02-26T06:20:37.099730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:43.160268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572409+0000","last_change":"2024-02-26T06:20:31.582236+0000","last_active":"2024-02-26T06:20:32.572409+0000","last_peered":"2024-02-26T06:20:32.572409+0000","last_clean":"2024-02-26T06:20:32.572409+0000","last_became_active":"2024-02-26T06:20:31.581851+0000","last_became_peered":"2024-02-26T06:20:31.581851+0000","last_unstale":"2024-02-26T06:20:32.572409+0000","last_undegraded":"2024-02-26T06:20:32.572409+0000","last_fullsized":"2024-02-26T06:20:32.572409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:53.214338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569834+0000","last_change":"2024-02-26T06:20:31.068179+0000","last_active":"2024-02-26T06:20:32.569834+0000","last_peered":"2024-02-26T06:20:32.569834+0000","last_clean":"2024-02-26T06:20:32.569834+0000","last_became_active":"2024-02-26T06:20:31.067888+0000","last_became_peered":"2024-02-26T06:20:31.067888+0000","last_unstale":"2024-02-26T06:20:32.569834+0000","last_undegraded":"2024-02-26T06:20:32.569834+0000","last_fullsized":"2024-02-26T06:20:32.569834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:47:39.223749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572284+0000","last_change":"2024-02-26T06:20:31.598820+0000","last_active":"2024-02-26T06:20:32.572284+0000","last_peered":"2024-02-26T06:20:32.572284+0000","last_clean":"2024-02-26T06:20:32.572284+0000","last_became_active":"2024-02-26T06:20:31.598469+0000","last_became_peered":"2024-02-26T06:20:31.598469+0000","last_unstale":"2024-02-26T06:20:32.572284+0000","last_undegraded":"2024-02-26T06:20:32.572284+0000","last_fullsized":"2024-02-26T06:20:32.572284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:41:15.713748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.033276+0000","last_change":"2024-02-26T06:20:31.080988+0000","last_active":"2024-02-26T06:20:33.033276+0000","last_peered":"2024-02-26T06:20:33.033276+0000","last_clean":"2024-02-26T06:20:33.033276+0000","last_became_active":"2024-02-26T06:20:31.080654+0000","last_became_peered":"2024-02-26T06:20:31.080654+0000","last_unstale":"2024-02-26T06:20:33.033276+0000","last_undegraded":"2024-02-26T06:20:33.033276+0000","last_fullsized":"2024-02-26T06:20:33.033276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:50.315162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.572476+0000","last_change":"2024-02-26T06:20:31.586057+0000","last_active":"2024-02-26T06:20:32.572476+0000","last_peered":"2024-02-26T06:20:32.572476+0000","last_clean":"2024-02-26T06:20:32.572476+0000","last_became_active":"2024-02-26T06:20:31.585756+0000","last_became_peered":"2024-02-26T06:20:31.585756+0000","last_unstale":"2024-02-26T06:20:32.572476+0000","last_undegraded":"2024-02-26T06:20:32.572476+0000","last_fullsized":"2024-02-26T06:20:32.572476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:59:07.032287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943736+0000","last_change":"2024-02-26T06:20:31.589067+0000","last_active":"2024-02-26T06:20:32.943736+0000","last_peered":"2024-02-26T06:20:32.943736+0000","last_clean":"2024-02-26T06:20:32.943736+0000","last_became_active":"2024-02-26T06:20:31.588769+0000","last_became_peered":"2024-02-26T06:20:31.588769+0000","last_unstale":"2024-02-26T06:20:32.943736+0000","last_undegraded":"2024-02-26T06:20:32.943736+0000","last_fullsized":"2024-02-26T06:20:32.943736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:43.211127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029166+0000","last_change":"2024-02-26T06:20:31.583838+0000","last_active":"2024-02-26T06:20:33.029166+0000","last_peered":"2024-02-26T06:20:33.029166+0000","last_clean":"2024-02-26T06:20:33.029166+0000","last_became_active":"2024-02-26T06:20:31.583524+0000","last_became_peered":"2024-02-26T06:20:31.583524+0000","last_unstale":"2024-02-26T06:20:33.029166+0000","last_undegraded":"2024-02-26T06:20:33.029166+0000","last_fullsized":"2024-02-26T06:20:33.029166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:21.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029228+0000","last_change":"2024-02-26T06:20:31.597901+0000","last_active":"2024-02-26T06:20:33.029228+0000","last_peered":"2024-02-26T06:20:33.029228+0000","last_clean":"2024-02-26T06:20:33.029228+0000","last_became_active":"2024-02-26T06:20:31.597633+0000","last_became_peered":"2024-02-26T06:20:31.597633+0000","last_unstale":"2024-02-26T06:20:33.029228+0000","last_undegraded":"2024-02-26T06:20:33.029228+0000","last_fullsized":"2024-02-26T06:20:33.029228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:14.084579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.031667+0000","last_change":"2024-02-26T06:20:31.608754+0000","last_active":"2024-02-26T06:20:33.031667+0000","last_peered":"2024-02-26T06:20:33.031667+0000","last_clean":"2024-02-26T06:20:33.031667+0000","last_became_active":"2024-02-26T06:20:31.608392+0000","last_became_peered":"2024-02-26T06:20:31.608392+0000","last_unstale":"2024-02-26T06:20:33.031667+0000","last_undegraded":"2024-02-26T06:20:33.031667+0000","last_fullsized":"2024-02-26T06:20:33.031667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:57:23.283252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.033338+0000","last_change":"2024-02-26T06:20:31.605866+0000","last_active":"2024-02-26T06:20:33.033338+0000","last_peered":"2024-02-26T06:20:33.033338+0000","last_clean":"2024-02-26T06:20:33.033338+0000","last_became_active":"2024-02-26T06:20:31.605404+0000","last_became_peered":"2024-02-26T06:20:31.605404+0000","last_unstale":"2024-02-26T06:20:33.033338+0000","last_undegraded":"2024-02-26T06:20:33.033338+0000","last_fullsized":"2024-02-26T06:20:33.033338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:03:25.055462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943823+0000","last_change":"2024-02-26T06:20:31.585214+0000","last_active":"2024-02-26T06:20:32.943823+0000","last_peered":"2024-02-26T06:20:32.943823+0000","last_clean":"2024-02-26T06:20:32.943823+0000","last_became_active":"2024-02-26T06:20:31.584895+0000","last_became_peered":"2024-02-26T06:20:31.584895+0000","last_unstale":"2024-02-26T06:20:32.943823+0000","last_undegraded":"2024-02-26T06:20:32.943823+0000","last_fullsized":"2024-02-26T06:20:32.943823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:38.507499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029549+0000","last_change":"2024-02-26T06:20:31.582189+0000","last_active":"2024-02-26T06:20:33.029549+0000","last_peered":"2024-02-26T06:20:33.029549+0000","last_clean":"2024-02-26T06:20:33.029549+0000","last_became_active":"2024-02-26T06:20:31.580729+0000","last_became_peered":"2024-02-26T06:20:31.580729+0000","last_unstale":"2024-02-26T06:20:33.029549+0000","last_undegraded":"2024-02-26T06:20:33.029549+0000","last_fullsized":"2024-02-26T06:20:33.029549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:21.337261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.571411+0000","last_change":"2024-02-26T06:20:31.588894+0000","last_active":"2024-02-26T06:20:32.571411+0000","last_peered":"2024-02-26T06:20:32.571411+0000","last_clean":"2024-02-26T06:20:32.571411+0000","last_became_active":"2024-02-26T06:20:31.586362+0000","last_became_peered":"2024-02-26T06:20:31.586362+0000","last_unstale":"2024-02-26T06:20:32.571411+0000","last_undegraded":"2024-02-26T06:20:32.571411+0000","last_fullsized":"2024-02-26T06:20:32.571411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:28:30.009901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.569429+0000","last_change":"2024-02-26T06:20:31.608264+0000","last_active":"2024-02-26T06:20:32.569429+0000","last_peered":"2024-02-26T06:20:32.569429+0000","last_clean":"2024-02-26T06:20:32.569429+0000","last_became_active":"2024-02-26T06:20:31.607855+0000","last_became_peered":"2024-02-26T06:20:31.607855+0000","last_unstale":"2024-02-26T06:20:32.569429+0000","last_undegraded":"2024-02-26T06:20:32.569429+0000","last_fullsized":"2024-02-26T06:20:32.569429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:20.169489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.570362+0000","last_change":"2024-02-26T06:20:31.591000+0000","last_active":"2024-02-26T06:20:32.570362+0000","last_peered":"2024-02-26T06:20:32.570362+0000","last_clean":"2024-02-26T06:20:32.570362+0000","last_became_active":"2024-02-26T06:20:31.588201+0000","last_became_peered":"2024-02-26T06:20:31.588201+0000","last_unstale":"2024-02-26T06:20:32.570362+0000","last_undegraded":"2024-02-26T06:20:32.570362+0000","last_fullsized":"2024-02-26T06:20:32.570362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:47:59.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.029772+0000","last_change":"2024-02-26T06:20:31.590398+0000","last_active":"2024-02-26T06:20:33.029772+0000","last_peered":"2024-02-26T06:20:33.029772+0000","last_clean":"2024-02-26T06:20:33.029772+0000","last_became_active":"2024-02-26T06:20:31.590042+0000","last_became_peered":"2024-02-26T06:20:31.590042+0000","last_unstale":"2024-02-26T06:20:33.029772+0000","last_undegraded":"2024-02-26T06:20:33.029772+0000","last_fullsized":"2024-02-26T06:20:33.029772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:40:01.233978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:33.032924+0000","last_change":"2024-02-26T06:20:31.611698+0000","last_active":"2024-02-26T06:20:33.032924+0000","last_peered":"2024-02-26T06:20:33.032924+0000","last_clean":"2024-02-26T06:20:33.032924+0000","last_became_active":"2024-02-26T06:20:31.610979+0000","last_became_peered":"2024-02-26T06:20:31.610979+0000","last_unstale":"2024-02-26T06:20:33.032924+0000","last_undegraded":"2024-02-26T06:20:33.032924+0000","last_fullsized":"2024-02-26T06:20:33.032924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:44.604373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T06:20:32.943823+0000","last_change":"2024-02-26T06:20:31.582016+0000","last_active":"2024-02-26T06:20:32.943823+0000","last_peered":"2024-02-26T06:20:32.943823+0000","last_clean":"2024-02-26T06:20:32.943823+0000","last_became_active":"2024-02-26T06:20:31.581689+0000","last_became_peered":"2024-02-26T06:20:31.581689+0000","last_unstale":"2024-02-26T06:20:32.943823+0000","last_undegraded":"2024-02-26T06:20:32.943823+0000","last_fullsized":"2024-02-26T06:20:32.943823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:04:11.030182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284221,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":129,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":77474,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1034508,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.624},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.56000000000000005}]},{"osd":2,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.877}]},{"osd":4,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.85699999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.2629999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70999999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.83599999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.222},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.94799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.93799999999999994}]},{"osd":1,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.407,"5min":2.407,"15min":2.407},"last":1.252}]},{"osd":2,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":1.4610000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.968,"5min":1.968,"15min":1.968},"min":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.851,"5min":1.851,"15min":1.851},"min":{"1min":1.151,"5min":1.151,"15min":1.151},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.302}]},{"osd":4,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":1.857,"5min":1.857,"15min":1.857},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":3.4990000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":2.0579999999999998},{"interface":"front","average":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":3.98,"5min":3.98,"15min":3.98},"last":1.7669999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":3.6800000000000002,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":1.883},{"interface":"front","average":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"min":{"1min":1.429,"5min":1.429,"15min":1.429},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.5089999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":1.635,"5min":1.635,"15min":1.635},"max":{"1min":4.2779999999999996,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.3779999999999999},{"interface":"front","average":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"min":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":2.181}]},{"osd":7,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"min":{"1min":1.357,"5min":1.357,"15min":1.357},"max":{"1min":4.2000000000000002,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":2.1280000000000001},{"interface":"front","average":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"min":{"1min":1.736,"5min":1.736,"15min":1.736},"max":{"1min":3.4489999999999998,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":1.619}]},{"osd":8,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"min":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":4.077,"5min":4.077,"15min":4.077},"last":2.2959999999999998},{"interface":"front","average":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"min":{"1min":1.52,"5min":1.52,"15min":1.52},"max":{"1min":4.1239999999999997,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":1.9990000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:20:11 2024","interfaces":[{"interface":"back","average":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":4.3170000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":2.242},{"interface":"front","average":{"1min":2.105,"5min":2.105,"15min":2.105},"min":{"1min":1.147,"5min":1.147,"15min":1.147},"max":{"1min":3.1619999999999999,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":2.375}]}]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.75},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.78700000000000003},"min":{"1min":0.54400000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0800000000000001,"5min":1.198,"15min":1.198},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64500000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65700000000000003,"5min":0.755,"15min":0.755},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59699999999999998,"5min":0.624,"15min":0.624},"last":0.52400000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77600000000000002,"15min":0.76800000000000002},"min":{"1min":0.54800000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.879,"15min":0.86899999999999999},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.879}]},{"osd":4,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.504},"min":{"1min":0.31900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.67900000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63}]},{"osd":6,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.61099999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.1240000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.92600000000000005}]},{"osd":7,"last update":"Mon Feb 26 06:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.84599999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":1.083}]},{"osd":10,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.96499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":1033732,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51700000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.42699999999999999,"15min":0.434},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56699999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.499}]},{"osd":2,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.64900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.85299999999999998,"15min":0.85699999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.236,"5min":1.591,"15min":1.591},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.129,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0189999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.495,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.60699999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.5,"15min":0.51600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60399999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.61599999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.61299999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.621,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80400000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69999999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2330000000000001,"5min":1.353,"15min":1.353},"last":0.873}]},{"osd":8,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83599999999999997,"5min":0.88,"15min":0.88},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.50800000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84599999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.78100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":2.0270000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.282,"15min":1.2390000000000001},"min":{"1min":0.91200000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.109,"5min":2.508,"15min":2.508},"last":1.575}]},{"osd":1,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.075,"15min":1.073},"min":{"1min":0.85199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.292},{"interface":"front","average":{"1min":1.175,"5min":1.095,"15min":1.0940000000000001},"min":{"1min":0.76700000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5149999999999999,"5min":1.944,"15min":1.944},"last":1.361}]},{"osd":2,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2569999999999999,"15min":1.26},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.9910000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.423},{"interface":"front","average":{"1min":1.456,"5min":1.302,"15min":1.27},"min":{"1min":0.65700000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8720000000000001,"5min":2.359,"15min":2.359},"last":1.6830000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.8720000000000001,"5min":1.641,"15min":1.585},"min":{"1min":1.335,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4710000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.9830000000000001},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.6990000000000001,"15min":1.679},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":2.677,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":2.1539999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.4419999999999999,"15min":1.427},"min":{"1min":0.74199999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1560000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":2.198},{"interface":"front","average":{"1min":1.863,"5min":1.6279999999999999,"15min":1.5900000000000001},"min":{"1min":1.4239999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.833}]},{"osd":5,"last update":"Mon Feb 26 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.6559999999999999,"15min":1.627},"min":{"1min":0.81699999999999995,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.29,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.7629999999999999},{"interface":"front","average":{"1min":1.677,"5min":1.722,"15min":1.716},"min":{"1min":1.036,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.4199999999999999,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":2.0779999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.9590000000000001,"5min":1.631,"15min":1.5760000000000001},"min":{"1min":1.3240000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.4159999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":2.282},{"interface":"front","average":{"1min":2.105,"5min":1.79,"15min":1.738},"min":{"1min":1.5389999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":2.347}]},{"osd":8,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":2.0150000000000001,"5min":1.7709999999999999,"15min":1.7310000000000001},"min":{"1min":1.385,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":2.129},{"interface":"front","average":{"1min":2.0049999999999999,"5min":1.8400000000000001,"15min":1.8120000000000001},"min":{"1min":0.999,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":2.5750000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"min":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"max":{"1min":3.258,"5min":3.258,"15min":3.258},"last":2.4849999999999999},{"interface":"front","average":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"min":{"1min":1.786,"5min":1.786,"15min":1.786},"max":{"1min":3.3380000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":2.246}]},{"osd":10,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":2.101,"15min":2.101},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":2.4510000000000001},{"interface":"front","average":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"min":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.4020000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":485799,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70099999999999996,"15min":0.67100000000000004},"min":{"1min":0.59699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95999999999999996,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.28,"5min":0.82199999999999995,"15min":0.746},"min":{"1min":0.49199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":8.9190000000000005,"5min":8.9190000000000005,"15min":8.9190000000000005},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":0.72699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40100000000000002,"5min":0.316,"15min":0.316},"max":{"1min":9.2629999999999999,"5min":9.2629999999999999,"15min":9.2629999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63400000000000001,"5min":0.745,"15min":0.745},"last":0.80900000000000005}]},{"osd":3,"last update":"Mon Feb 26 06:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.52500000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.232,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.63700000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.87}]},{"osd":4,"last update":"Mon Feb 26 06:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40600000000000003,"15min":0.39400000000000002},"min":{"1min":0.317,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66200000000000003,"5min":0.748,"15min":0.748},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.39900000000000002},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.64700000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.48399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78800000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.89200000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.80300000000000005,"15min":0.78400000000000003},"min":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.135,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.02},{"interface":"front","average":{"1min":1.3340000000000001,"5min":0.93400000000000005,"15min":0.875},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":8.8729999999999993,"5min":8.8729999999999993,"15min":8.8729999999999993},"last":0.90300000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.497,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.83399999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.71999999999999997,"15min":0.64900000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.9380000000000006,"5min":8.9380000000000006,"15min":8.9380000000000006},"last":0.95299999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.85799999999999998},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.089}]},{"osd":10,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.93799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:19:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.88900000000000001,"15min":0.80400000000000005},"min":{"1min":0.71799999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.90600000000000003,"15min":0.83199999999999996},"min":{"1min":0.60299999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4470000000000001,"5min":1.649,"15min":1.649},"last":1.2070000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.0269999999999999,"15min":0.90800000000000003},"min":{"1min":0.73199999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":6.6070000000000002,"5min":6.6070000000000002,"15min":6.6070000000000002},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.1220000000000001,"15min":0.96599999999999997},"min":{"1min":0.80100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.871,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.609}]},{"osd":3,"last update":"Mon Feb 26 06:18:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.194,"15min":1.1200000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1230000000000002,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.246,"15min":1.159},"min":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.8819999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:20:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.167,"15min":1.0469999999999999},"min":{"1min":0.88400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9319999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.946},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2170000000000001,"15min":1.147},"min":{"1min":1.0029999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0840000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.47,"15min":1.4419999999999999},"min":{"1min":1.238,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.2080000000000002,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.4370000000000001,"15min":1.373},"min":{"1min":1.0029999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":6.5510000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":2.032}]},{"osd":6,"last update":"Mon Feb 26 06:20:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9359999999999999,"5min":1.8109999999999999,"15min":1.802},"min":{"1min":1.1240000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":2.1349999999999998},{"interface":"front","average":{"1min":1.8999999999999999,"5min":1.825,"15min":1.841},"min":{"1min":1.548,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.387,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":2.3100000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.73,"15min":1.7190000000000001},"min":{"1min":1.2150000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.45},{"interface":"front","average":{"1min":1.73,"5min":1.599,"15min":1.577},"min":{"1min":1.123,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":1.8640000000000001,"5min":1.742,"15min":1.722},"min":{"1min":1.353,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.468,"5min":2.548,"15min":2.548},"last":2.0680000000000001},{"interface":"front","average":{"1min":2.0510000000000002,"5min":1.7949999999999999,"15min":1.7529999999999999},"min":{"1min":1.5980000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":2.105}]},{"osd":9,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"min":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.6930000000000001},{"interface":"front","average":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"min":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"max":{"1min":2.5630000000000002,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":2.2679999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"min":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":2.1680000000000001},{"interface":"front","average":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"min":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":1.8180000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:20:34 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.883,"15min":1.883},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":2.1890000000000001},{"interface":"front","average":{"1min":2.016,"5min":2.016,"15min":2.016},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":2.2229999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":451134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0609999999999999,"5min":1.091,"15min":1.091},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.54500000000000004,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46500000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.51500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.0700000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.56200000000000006,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.085,"5min":1.367,"15min":1.367},"last":0.68400000000000005}]},{"osd":4,"last update":"Mon Feb 26 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76400000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.43099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.41999999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45800000000000002,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.497},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52600000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.57099999999999995,"5min":0.499,"15min":0.499},"max":{"1min":1.071,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.82599999999999996,"15min":0.81399999999999995},"min":{"1min":0.65500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.073,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47399999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71899999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.96599999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":441766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.115,"15min":1.054},"min":{"1min":0.70699999999999996,"5min":0.433,"15min":0.433},"max":{"1min":1.9490000000000001,"5min":2.109,"15min":2.109},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.861,"5min":1.25,"15min":1.0880000000000001},"min":{"1min":0.88600000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.7919999999999998,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":1.647}]},{"osd":1,"last update":"Mon Feb 26 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.099,"15min":1.0109999999999999},"min":{"1min":0.80500000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.772,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.0860000000000001,"15min":1.0580000000000001},"min":{"1min":0.86699999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.593,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.367}]},{"osd":2,"last update":"Mon Feb 26 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.2450000000000001,"15min":1.165},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0409999999999999,"15min":1.026},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.294}]},{"osd":4,"last update":"Mon Feb 26 06:20:18 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.2949999999999999,"15min":1.2110000000000001},"min":{"1min":0.92300000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.9119999999999999,"5min":2.613,"15min":2.613},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.472,"15min":1.3640000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.0649999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.8140000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.415,"15min":1.363},"min":{"1min":1.2689999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.0150000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.3260000000000001,"15min":1.325},"min":{"1min":0.872,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.911}]},{"osd":6,"last update":"Mon Feb 26 06:20:18 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.5229999999999999,"15min":1.4670000000000001},"min":{"1min":0.90800000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4039999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":2.383},{"interface":"front","average":{"1min":2.0830000000000002,"5min":1.758,"15min":1.6419999999999999},"min":{"1min":0.97199999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.681,"5min":4.681,"15min":4.681},"last":2.173}]},{"osd":7,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.774,"15min":1.738},"min":{"1min":1.516,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.363,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.98},{"interface":"front","average":{"1min":2.1619999999999999,"5min":1.756,"15min":1.6879999999999999},"min":{"1min":1.577,"5min":0.998,"15min":0.998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":2.2349999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.806},"min":{"1min":0.97399999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.587,"15min":1.5649999999999999},"min":{"1min":0.84999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.23,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.143}]},{"osd":9,"last update":"Mon Feb 26 06:19:41 2024","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"min":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":2.2879999999999998},{"interface":"front","average":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"min":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"max":{"1min":4.266,"5min":4.266,"15min":4.266},"last":2.4300000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.476,"5min":2.476,"15min":2.476},"last":2.1080000000000001},{"interface":"front","average":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"min":{"1min":1.127,"5min":1.127,"15min":1.127},"max":{"1min":4.0609999999999999,"5min":4.0609999999999999,"15min":4.0609999999999999},"last":2.327}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.056}]}]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.50600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.079,"5min":1.093,"15min":1.093},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.36299999999999999,"15min":0.34999999999999998},"min":{"1min":0.26100000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.32000000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36699999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":0.65500000000000003,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":12.468,"5min":12.468,"15min":12.468},"last":0.62}]},{"osd":3,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73099999999999998,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.075,"5min":1.167,"15min":1.167},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.99099999999999999,"5min":1.069,"15min":1.069},"last":0.88400000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54900000000000004,"15min":0.52700000000000002},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82999999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71799999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.61499999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0820000000000001,"5min":1.327,"15min":1.327},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.623,"5min":0.502,"15min":0.502},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.47399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.92600000000000005}]},{"osd":9,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.089}]},{"osd":10,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68799999999999994,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.71599999999999997,"15min":0.68200000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0660000000000001,"5min":1.131,"15min":1.131},"last":0.92200000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.66300000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.29999999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.315,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80000000000000004,"15min":0.78200000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.74299999999999999,"15min":0.71099999999999997},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0840000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.98399999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.77000000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.61799999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83099999999999996,"15min":0.80700000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.96199999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.86599999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83899999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.54300000000000004,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67600000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48099999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.71299999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.93999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":44227,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:20:43.117 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T06:20:43.118 INFO:tasks.cephadm:Setting up client nodes... 2024-02-26T06:20:43.118 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-26T06:20:43.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:43 smithi049 ceph-mon[31696]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.1 KiB/s wr, 14 op/s 2024-02-26T06:20:43.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:43 smithi080 ceph-mon[36206]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.1 KiB/s wr, 14 op/s 2024-02-26T06:20:43.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:43 smithi115 ceph-mon[36426]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.1 KiB/s wr, 14 op/s 2024-02-26T06:20:43.632 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:20:44.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:44 smithi049 ceph-mon[31696]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:20:44.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:44 smithi049 ceph-mon[31696]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:20:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:44 smithi080 ceph-mon[36206]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:20:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:44 smithi080 ceph-mon[36206]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:20:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:44 smithi115 ceph-mon[36426]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:20:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:44 smithi115 ceph-mon[36426]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:20:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:45 smithi049 ceph-mon[31696]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-02-26T06:20:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:45 smithi080 ceph-mon[36206]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-02-26T06:20:45.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:45 smithi115 ceph-mon[36426]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-02-26T06:20:46.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 -- 172.21.15.49:0/4174736636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c810e2a0 msgr2=0x7fa0c8110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:46.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 --2- 172.21.15.49:0/4174736636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c810e2a0 0x7fa0c8110690 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000b0a0 tx=0x7fa0c000b3b0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 -- 172.21.15.49:0/4174736636 shutdown_connections 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 --2- 172.21.15.49:0/4174736636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c810e2a0 0x7fa0c8110690 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 --2- 172.21.15.49:0/4174736636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c8075ca0 0x7fa0c8074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 --2- 172.21.15.49:0/4174736636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c80746d0 0x7fa0c8074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 -- 172.21.15.49:0/4174736636 >> 172.21.15.49:0/4174736636 conn(0x7fa0c80fd280 msgr2=0x7fa0c80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 -- 172.21.15.49:0/4174736636 shutdown_connections 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.069+0000 7fa0cda00700 1 -- 172.21.15.49:0/4174736636 wait complete. 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.071+0000 7fa0cda00700 1 Processor -- start 2024-02-26T06:20:46.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.071+0000 7fa0cda00700 1 -- start start 2024-02-26T06:20:46.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.071+0000 7fa0cda00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c80746d0 0x7fa0c819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:46.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.071+0000 7fa0cda00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c8075ca0 0x7fa0c819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:46.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.071+0000 7fa0cda00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c810e2a0 0x7fa0c81a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.071+0000 7fa0cda00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81a5e50 con 0x7fa0c810e2a0 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.071+0000 7fa0cda00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81a5f90 con 0x7fa0c8075ca0 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.071+0000 7fa0cda00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81a6120 con 0x7fa0c80746d0 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c80746d0 0x7fa0c819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c8075ca0 0x7fa0c819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c7400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c810e2a0 0x7fa0c81a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c8075ca0 0x7fa0c819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58092/0 (socket says 172.21.15.49:58092) 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6000700 1 -- 172.21.15.49:0/1888382392 learned_addr learned my addr 172.21.15.49:0/1888382392 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c80746d0 0x7fa0c819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34960/0 (socket says 172.21.15.49:34960) 2024-02-26T06:20:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6000700 1 -- 172.21.15.49:0/1888382392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c80746d0 msgr2=0x7fa0c819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6000700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c80746d0 0x7fa0c819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6000700 1 -- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c810e2a0 msgr2=0x7fa0c81a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6000700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c810e2a0 0x7fa0c81a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6000700 1 -- 172.21.15.49:0/1888382392 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8009520 con 0x7fa0c8075ca0 2024-02-26T06:20:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.072+0000 7fa0c6a00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c80746d0 0x7fa0c819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.073+0000 7fa0c6000700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c8075ca0 0x7fa0c819e800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400e970 tx=0x7fa0b400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.073+0000 7fa0bf400700 1 -- 172.21.15.49:0/1888382392 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4009790 con 0x7fa0c8075ca0 2024-02-26T06:20:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.073+0000 7fa0bf400700 1 -- 172.21.15.49:0/1888382392 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0b400ccc0 con 0x7fa0c8075ca0 2024-02-26T06:20:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.073+0000 7fa0bf400700 1 -- 172.21.15.49:0/1888382392 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4003d60 con 0x7fa0c8075ca0 2024-02-26T06:20:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.073+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c0009d00 con 0x7fa0c8075ca0 2024-02-26T06:20:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.073+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c81a6720 con 0x7fa0c8075ca0 2024-02-26T06:20:46.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.075+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c8104500 con 0x7fa0c8075ca0 2024-02-26T06:20:46.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.077+0000 7fa0bf400700 1 -- 172.21.15.49:0/1888382392 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa0b400c7e0 con 0x7fa0c8075ca0 2024-02-26T06:20:46.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.078+0000 7fa0bf400700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa0ac0a2310 0x7fa0ac0a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:46.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.078+0000 7fa0bf400700 1 -- 172.21.15.49:0/1888382392 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0b4014070 con 0x7fa0c8075ca0 2024-02-26T06:20:46.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.078+0000 7fa0c6a00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa0ac0a2310 0x7fa0ac0a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:46.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.079+0000 7fa0c6a00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa0ac0a2310 0x7fa0ac0a47d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009fd0 tx=0x7fa0b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.081+0000 7fa0bf400700 1 -- 172.21.15.49:0/1888382392 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa0b4096da0 con 0x7fa0c8075ca0 2024-02-26T06:20:46.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.398+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fa0c8113c70 con 0x7fa0c8075ca0 2024-02-26T06:20:46.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:46 smithi049 ceph-mon[31696]: pgmap v325: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-02-26T06:20:46.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.406+0000 7fa0bf400700 1 -- 172.21.15.49:0/1888382392 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fa0b409a460 con 0x7fa0c8075ca0 2024-02-26T06:20:46.406 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-02-26T06:20:46.406 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQC+Ldxl9TroFxAAXtGJr3RKC+z+zVuZJ5oVLw== 2024-02-26T06:20:46.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa0ac0a2310 msgr2=0x7fa0ac0a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:46.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa0ac0a2310 0x7fa0ac0a47d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009fd0 tx=0x7fa0b8009260 comp rx=0 tx=0).stop 2024-02-26T06:20:46.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c8075ca0 msgr2=0x7fa0c819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:46.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c8075ca0 0x7fa0c819e800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400e970 tx=0x7fa0b400ec80 comp rx=0 tx=0).stop 2024-02-26T06:20:46.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 shutdown_connections 2024-02-26T06:20:46.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa0ac0a2310 0x7fa0ac0a47d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c810e2a0 0x7fa0c81a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c80746d0 0x7fa0c819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 --2- 172.21.15.49:0/1888382392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0c8075ca0 0x7fa0c819e800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:46.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.409+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 >> 172.21.15.49:0/1888382392 conn(0x7fa0c80fd280 msgr2=0x7fa0c8101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:46.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.410+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 shutdown_connections 2024-02-26T06:20:46.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:46.410+0000 7fa0cda00700 1 -- 172.21.15.49:0/1888382392 wait complete. 2024-02-26T06:20:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:46 smithi080 ceph-mon[36206]: pgmap v325: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-02-26T06:20:46.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:46 smithi115 ceph-mon[36426]: pgmap v325: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-02-26T06:20:47.220 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:20:47.220 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-26T06:20:47.220 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-26T06:20:47.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:47 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1888382392' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:20:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:47 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:20:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:47 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T06:20:47.396 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-26T06:20:47.396 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-26T06:20:47.396 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph mgr dump --format=json 2024-02-26T06:20:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:47 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1888382392' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:20:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:47 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:20:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:47 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T06:20:47.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:47 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1888382392' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:20:47.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:47 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:20:47.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:47 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T06:20:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:20:48.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:48 smithi049 ceph-mon[31696]: pgmap v326: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-02-26T06:20:48.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:48 smithi080 ceph-mon[36206]: pgmap v326: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-02-26T06:20:48.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:48 smithi115 ceph-mon[36426]: pgmap v326: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 -- 172.21.15.49:0/2696399931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f377010e670 msgr2=0x7f3770110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 --2- 172.21.15.49:0/2696399931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f377010e670 0x7f3770110a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f376800b0a0 tx=0x7f376800b3b0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 -- 172.21.15.49:0/2696399931 shutdown_connections 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 --2- 172.21.15.49:0/2696399931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f377010e670 0x7f3770110a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 --2- 172.21.15.49:0/2696399931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3770073ea0 0x7f3770074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 --2- 172.21.15.49:0/2696399931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37700747c0 0x7f377010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 -- 172.21.15.49:0/2696399931 >> 172.21.15.49:0/2696399931 conn(0x7f3770078da0 msgr2=0x7f37700791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 -- 172.21.15.49:0/2696399931 shutdown_connections 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.270+0000 7f3775a00700 1 -- 172.21.15.49:0/2696399931 wait complete. 2024-02-26T06:20:50.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.271+0000 7f3775a00700 1 Processor -- start 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.271+0000 7f3775a00700 1 -- start start 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f3775a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770073ea0 0x7f377019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f3775a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37700747c0 0x7f377019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f3775a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377010e670 0x7f377019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f3775a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377019f430 con 0x7f3770073ea0 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f3775a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377019f5a0 con 0x7f377010e670 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f3775a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377019f760 con 0x7f37700747c0 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f376ea00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770073ea0 0x7f377019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f3766000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37700747c0 0x7f377019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f376ea00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770073ea0 0x7f377019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49312/0 (socket says 172.21.15.49:49312) 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f376ea00700 1 -- 172.21.15.49:0/731611410 learned_addr learned my addr 172.21.15.49:0/731611410 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.272+0000 7f376f400700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377010e670 0x7f377019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f376ea00700 1 -- 172.21.15.49:0/731611410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37700747c0 msgr2=0x7f377019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f376ea00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37700747c0 0x7f377019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f376ea00700 1 -- 172.21.15.49:0/731611410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377010e670 msgr2=0x7f377019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f376ea00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377010e670 0x7f377019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f376ea00700 1 -- 172.21.15.49:0/731611410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009520 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f3766000700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37700747c0 0x7f377019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f376ea00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770073ea0 0x7f377019a550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f376000eb30 tx=0x7f376000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f3767e00700 1 -- 172.21.15.49:0/731611410 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760009880 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f3767e00700 1 -- 172.21.15.49:0/731611410 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3760004500 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f3767e00700 1 -- 172.21.15.49:0/731611410 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760003e70 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3768009d00 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.273+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37701a70d0 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.275+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3770109880 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.277+0000 7f3767e00700 1 -- 172.21.15.49:0/731611410 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f376000cbd0 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.278+0000 7f3767e00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37540a21e0 0x7f37540a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.278+0000 7f3767e00700 1 -- 172.21.15.49:0/731611410 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3760014070 con 0x7f3770073ea0 2024-02-26T06:20:50.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.278+0000 7f3766000700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37540a21e0 0x7f37540a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:50.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.279+0000 7f3766000700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37540a21e0 0x7f37540a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3770110370 tx=0x7f3758009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:50.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.283+0000 7f3767e00700 1 -- 172.21.15.49:0/731611410 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37600970a0 con 0x7f3770073ea0 2024-02-26T06:20:50.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.573+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f377019b8b0 con 0x7f3770073ea0 2024-02-26T06:20:50.574 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:50 smithi049 ceph-mon[31696]: pgmap v327: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-02-26T06:20:50.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.588+0000 7f3767e00700 1 -- 172.21.15.49:0/731611410 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261485 (secure 0 0 0) 0x7f376009a760 con 0x7f3770073ea0 2024-02-26T06:20:50.593 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:50.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.594+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37540a21e0 msgr2=0x7f37540a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:50.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.594+0000 7f3775a00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37540a21e0 0x7f37540a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3770110370 tx=0x7f3758009260 comp rx=0 tx=0).stop 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.594+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770073ea0 msgr2=0x7f377019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.594+0000 7f3775a00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770073ea0 0x7f377019a550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f376000eb30 tx=0x7f376000ee40 comp rx=0 tx=0).stop 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.594+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 shutdown_connections 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.594+0000 7f3775a00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f37540a21e0 0x7f37540a46a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.594+0000 7f3775a00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377010e670 0x7f377019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.595+0000 7f3775a00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770073ea0 0x7f377019a550 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.595+0000 7f3775a00700 1 --2- 172.21.15.49:0/731611410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37700747c0 0x7f377019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.595+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 >> 172.21.15.49:0/731611410 conn(0x7f3770078da0 msgr2=0x7f37700feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.595+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 shutdown_connections 2024-02-26T06:20:50.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:50.595+0000 7f3775a00700 1 -- 172.21.15.49:0/731611410 wait complete. 2024-02-26T06:20:50.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:50 smithi080 ceph-mon[36206]: pgmap v327: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-02-26T06:20:50.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:50 smithi115 ceph-mon[36426]: pgmap v327: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-02-26T06:20:51.497 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1431851577},{"type":"v1","addr":"172.21.15.49:6801","nonce":1431851577}]},"active_addr":"172.21.15.49:6801/1431851577","active_change":"2024-02-26T06:12:27.275106+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.49:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3617020911}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":2527769498}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3210390195}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3477045895}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1647865560}]}]} 2024-02-26T06:20:51.503 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-26T06:20:51.503 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-26T06:20:51.503 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T06:20:51.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:51 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/731611410' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T06:20:51.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:51 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/731611410' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T06:20:51.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:51 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/731611410' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T06:20:52.043 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:20:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:52 smithi080 ceph-mon[36206]: pgmap v328: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-02-26T06:20:52.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:52 smithi115 ceph-mon[36426]: pgmap v328: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-02-26T06:20:52.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:52 smithi049 ceph-mon[31696]: pgmap v328: 97 pgs: 97 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-02-26T06:20:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 -- 172.21.15.49:0/3190666680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01010e670 msgr2=0x7fa010110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 --2- 172.21.15.49:0/3190666680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01010e670 0x7fa010110a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa00800b0a0 tx=0x7fa00800b3b0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 -- 172.21.15.49:0/3190666680 shutdown_connections 2024-02-26T06:20:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 --2- 172.21.15.49:0/3190666680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa01010e670 0x7fa010110a60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 --2- 172.21.15.49:0/3190666680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa010073ea0 0x7fa010074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 --2- 172.21.15.49:0/3190666680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0100747c0 0x7fa01010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 -- 172.21.15.49:0/3190666680 >> 172.21.15.49:0/3190666680 conn(0x7fa010078dd0 msgr2=0x7fa0100791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 -- 172.21.15.49:0/3190666680 shutdown_connections 2024-02-26T06:20:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.399+0000 7fa017a00700 1 -- 172.21.15.49:0/3190666680 wait complete. 2024-02-26T06:20:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.400+0000 7fa017a00700 1 Processor -- start 2024-02-26T06:20:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.400+0000 7fa017a00700 1 -- start start 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa017a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa010073ea0 0x7fa01019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa017a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0100747c0 0x7fa01019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa017a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01010e670 0x7fa01019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa017a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01019f410 con 0x7fa010073ea0 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa017a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01019f580 con 0x7fa01010e670 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa017a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01019f740 con 0x7fa0100747c0 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa015000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa010073ea0 0x7fa01019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa015000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa010073ea0 0x7fa01019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49330/0 (socket says 172.21.15.49:49330) 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa015a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01010e670 0x7fa01019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa015000700 1 -- 172.21.15.49:0/1484111161 learned_addr learned my addr 172.21.15.49:0/1484111161 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.401+0000 7fa007e00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0100747c0 0x7fa01019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa015a00700 1 -- 172.21.15.49:0/1484111161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0100747c0 msgr2=0x7fa01019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa015a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0100747c0 0x7fa01019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa015a00700 1 -- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa010073ea0 msgr2=0x7fa01019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:54.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa015a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa010073ea0 0x7fa01019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa015a00700 1 -- 172.21.15.49:0/1484111161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff4009520 con 0x7fa01010e670 2024-02-26T06:20:54.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa015000700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa010073ea0 0x7fa01019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:20:54.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa015a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01010e670 0x7fa01019ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa00800b070 tx=0x7fa00800bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:54.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa00e000700 1 -- 172.21.15.49:0/1484111161 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00800e040 con 0x7fa01010e670 2024-02-26T06:20:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc009520 con 0x7fa01010e670 2024-02-26T06:20:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.402+0000 7fa00e000700 1 -- 172.21.15.49:0/1484111161 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0080079d0 con 0x7fa01010e670 2024-02-26T06:20:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.403+0000 7fa00e000700 1 -- 172.21.15.49:0/1484111161 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008005040 con 0x7fa01010e670 2024-02-26T06:20:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.403+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa008009d00 con 0x7fa01010e670 2024-02-26T06:20:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.404+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa010068930 con 0x7fa01010e670 2024-02-26T06:20:54.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.406+0000 7fa00e000700 1 -- 172.21.15.49:0/1484111161 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa00801d600 con 0x7fa01010e670 2024-02-26T06:20:54.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.408+0000 7fa00e000700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9fec0a21e0 0x7f9fec0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:54.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.408+0000 7fa00e000700 1 -- 172.21.15.49:0/1484111161 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0080cf7b0 con 0x7fa01010e670 2024-02-26T06:20:54.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.408+0000 7fa015000700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9fec0a21e0 0x7f9fec0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:54.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.409+0000 7fa015000700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9fec0a21e0 0x7f9fec0a46a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4009fd0 tx=0x7f9ff4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.409+0000 7fa00e000700 1 -- 172.21.15.49:0/1484111161 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa008098760 con 0x7fa01010e670 2024-02-26T06:20:54.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.639+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa010113fd0 con 0x7fa01010e670 2024-02-26T06:20:54.639 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:54 smithi049 ceph-mon[31696]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-02-26T06:20:54.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.641+0000 7fa00e000700 1 -- 172.21.15.49:0/1484111161 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22206 (secure 0 0 0) 0x7fa00809be20 con 0x7fa01010e670 2024-02-26T06:20:54.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:54.641 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:20:32.561576+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:20:54.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9fec0a21e0 msgr2=0x7f9fec0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:54.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9fec0a21e0 0x7f9fec0a46a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4009fd0 tx=0x7f9ff4009260 comp rx=0 tx=0).stop 2024-02-26T06:20:54.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01010e670 msgr2=0x7fa01019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01010e670 0x7fa01019ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa00800b070 tx=0x7fa00800bcc0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 shutdown_connections 2024-02-26T06:20:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9fec0a21e0 0x7f9fec0a46a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01010e670 0x7fa01019ee00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa010073ea0 0x7fa01019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 --2- 172.21.15.49:0/1484111161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0100747c0 0x7fa01019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 >> 172.21.15.49:0/1484111161 conn(0x7fa010078dd0 msgr2=0x7fa0100fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 shutdown_connections 2024-02-26T06:20:54.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:54.644+0000 7fa017a00700 1 -- 172.21.15.49:0/1484111161 wait complete. 2024-02-26T06:20:54.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:54 smithi080 ceph-mon[36206]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-02-26T06:20:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:54 smithi115 ceph-mon[36426]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-02-26T06:20:55.496 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-26T06:20:55.497 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T06:20:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:55 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1484111161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:55.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:55 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1484111161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:55.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:55 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1484111161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:55.992 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:20:56.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:56 smithi080 ceph-mon[36206]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-26T06:20:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:56 smithi115 ceph-mon[36426]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-26T06:20:56.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:56 smithi049 ceph-mon[31696]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-26T06:20:58.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 -- 172.21.15.49:0/345717345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c06a2c0 msgr2=0x7f2d7c10eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:58.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 --2- 172.21.15.49:0/345717345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c06a2c0 0x7f2d7c10eb00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009f90 tx=0x7f2d7400b330 comp rx=0 tx=0).stop 2024-02-26T06:20:58.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 -- 172.21.15.49:0/345717345 shutdown_connections 2024-02-26T06:20:58.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 --2- 172.21.15.49:0/345717345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c06a2c0 0x7f2d7c10eb00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 --2- 172.21.15.49:0/345717345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d7c107cc0 0x7f2d7c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 --2- 172.21.15.49:0/345717345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7c069920 0x7f2d7c069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 -- 172.21.15.49:0/345717345 >> 172.21.15.49:0/345717345 conn(0x7f2d7c077aa0 msgr2=0x7f2d7c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:58.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 -- 172.21.15.49:0/345717345 shutdown_connections 2024-02-26T06:20:58.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.244+0000 7f2d84200700 1 -- 172.21.15.49:0/345717345 wait complete. 2024-02-26T06:20:58.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.245+0000 7f2d84200700 1 Processor -- start 2024-02-26T06:20:58.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d84200700 1 -- start start 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d84200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d7c069920 0x7f2d7c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d84200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7c06a2c0 0x7f2d7c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d84200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c107cc0 0x7f2d7c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d84200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c19f3e0 con 0x7f2d7c069920 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d84200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c19f550 con 0x7f2d7c107cc0 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d84200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c19f710 con 0x7f2d7c06a2c0 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d82200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c107cc0 0x7f2d7c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d82200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c107cc0 0x7f2d7c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36478/0 (socket says 172.21.15.49:36478) 2024-02-26T06:20:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d82200700 1 -- 172.21.15.49:0/778201894 learned_addr learned my addr 172.21.15.49:0/778201894 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.246+0000 7f2d81800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d7c069920 0x7f2d7c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.247+0000 7f2d80e00700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7c06a2c0 0x7f2d7c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.247+0000 7f2d82200700 1 -- 172.21.15.49:0/778201894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7c06a2c0 msgr2=0x7f2d7c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.247+0000 7f2d82200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7c06a2c0 0x7f2d7c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.247+0000 7f2d82200700 1 -- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d7c069920 msgr2=0x7f2d7c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.247+0000 7f2d82200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d7c069920 0x7f2d7c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.247+0000 7f2d82200700 1 -- 172.21.15.49:0/778201894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d74009c40 con 0x7f2d7c107cc0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.247+0000 7f2d82200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c107cc0 0x7f2d7c19edd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d74004fd0 tx=0x7f2d74009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.248+0000 7f2d7a000700 1 -- 172.21.15.49:0/778201894 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400e040 con 0x7f2d7c107cc0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.248+0000 7f2d7a000700 1 -- 172.21.15.49:0/778201894 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7400be60 con 0x7f2d7c107cc0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.248+0000 7f2d7a000700 1 -- 172.21.15.49:0/778201894 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74005040 con 0x7f2d7c107cc0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.248+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c19f990 con 0x7f2d7c107cc0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.248+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c1a7250 con 0x7f2d7c107cc0 2024-02-26T06:20:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.249+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c068930 con 0x7f2d7c107cc0 2024-02-26T06:20:58.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.251+0000 7f2d7a000700 1 -- 172.21.15.49:0/778201894 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2d7401d470 con 0x7f2d7c107cc0 2024-02-26T06:20:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.252+0000 7f2d7a000700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2d640a2260 0x7f2d640a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.252+0000 7f2d7a000700 1 -- 172.21.15.49:0/778201894 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2d740cf150 con 0x7f2d7c107cc0 2024-02-26T06:20:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.253+0000 7f2d81800700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2d640a2260 0x7f2d640a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:58.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.253+0000 7f2d81800700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2d640a2260 0x7f2d640a4720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d70005fd0 tx=0x7f2d70009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:58.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.256+0000 7f2d7a000700 1 -- 172.21.15.49:0/778201894 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d74098100 con 0x7f2d7c107cc0 2024-02-26T06:20:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.488+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d7c19fb80 con 0x7f2d7c107cc0 2024-02-26T06:20:58.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.490+0000 7f2d7a000700 1 -- 172.21.15.49:0/778201894 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22206 (secure 0 0 0) 0x7f2d7409b7c0 con 0x7f2d7c107cc0 2024-02-26T06:20:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:20:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:20:32.561576+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2d640a2260 msgr2=0x7f2d640a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2d640a2260 0x7f2d640a4720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d70005fd0 tx=0x7f2d70009310 comp rx=0 tx=0).stop 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c107cc0 msgr2=0x7f2d7c19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c107cc0 0x7f2d7c19edd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d74004fd0 tx=0x7f2d74009500 comp rx=0 tx=0).stop 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 shutdown_connections 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2d640a2260 0x7f2d640a4720 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d7c107cc0 0x7f2d7c19edd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d7c069920 0x7f2d7c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 --2- 172.21.15.49:0/778201894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7c06a2c0 0x7f2d7c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 >> 172.21.15.49:0/778201894 conn(0x7f2d7c077aa0 msgr2=0x7f2d7c10ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:58.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 shutdown_connections 2024-02-26T06:20:58.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:20:58.493+0000 7f2d84200700 1 -- 172.21.15.49:0/778201894 wait complete. 2024-02-26T06:20:58.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:58 smithi080 ceph-mon[36206]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-26T06:20:58.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:20:58 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/778201894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:58.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:58 smithi115 ceph-mon[36426]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-26T06:20:58.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:20:58 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/778201894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:58.907 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:58 smithi049 ceph-mon[31696]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-26T06:20:58.907 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:20:58 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/778201894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:20:59.339 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-26T06:20:59.339 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-26T06:20:59.340 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-26T06:20:59.340 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-26T06:20:59.340 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-26T06:20:59.341 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-26T06:20:59.341 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-26T06:20:59.342 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-26T06:20:59.342 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-26T06:20:59.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-26T06:20:59.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-26T06:20:59.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-26T06:21:00.330 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:00.448 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:00.481 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:00.498 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:00.500 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:00.509 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:00.784 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:00 smithi049 ceph-mon[31696]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-26T06:21:00.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:00 smithi080 ceph-mon[36206]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-26T06:21:00.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:00 smithi115 ceph-mon[36426]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-26T06:21:02.777 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:02 smithi049 ceph-mon[31696]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-26T06:21:02.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:02 smithi080 ceph-mon[36206]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-26T06:21:02.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:02 smithi115 ceph-mon[36426]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-26T06:21:03.642 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:04.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 -- 172.21.15.49:0/2691659780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ef4112e40 msgr2=0x7f0ef4115230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:04.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 --2- 172.21.15.49:0/2691659780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ef4112e40 0x7f0ef4115230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009f90 tx=0x7f0eec00b330 comp rx=0 tx=0).stop 2024-02-26T06:21:04.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 -- 172.21.15.49:0/2691659780 shutdown_connections 2024-02-26T06:21:04.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 --2- 172.21.15.49:0/2691659780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ef4112e40 0x7f0ef4115230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:04.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 --2- 172.21.15.49:0/2691659780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4073ea0 0x7f0ef4074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:04.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 --2- 172.21.15.49:0/2691659780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ef40747c0 0x7f0ef4112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:04.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 -- 172.21.15.49:0/2691659780 >> 172.21.15.49:0/2691659780 conn(0x7f0ef4078dd0 msgr2=0x7f0ef40791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:04.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 -- 172.21.15.49:0/2691659780 shutdown_connections 2024-02-26T06:21:04.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.482+0000 7f0ef8a00700 1 -- 172.21.15.49:0/2691659780 wait complete. 2024-02-26T06:21:04.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.484+0000 7f0ef8a00700 1 Processor -- start 2024-02-26T06:21:04.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.484+0000 7f0ef8a00700 1 -- start start 2024-02-26T06:21:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.484+0000 7f0ef8a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4073ea0 0x7f0ef41a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.484+0000 7f0ef8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ef40747c0 0x7f0ef41a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef8a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ef4112e40 0x7f0ef419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef8a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef419ef00 con 0x7f0ef4073ea0 2024-02-26T06:21:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef8a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef419f040 con 0x7f0ef4112e40 2024-02-26T06:21:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef8a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef419f200 con 0x7f0ef40747c0 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef1e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4073ea0 0x7f0ef41a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef1e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4073ea0 0x7f0ef41a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38264/0 (socket says 172.21.15.49:38264) 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef1e00700 1 -- 172.21.15.49:0/3806802009 learned_addr learned my addr 172.21.15.49:0/3806802009 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef1400700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ef40747c0 0x7f0ef41a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.485+0000 7f0ef2800700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ef4112e40 0x7f0ef419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.486+0000 7f0ef1e00700 1 -- 172.21.15.49:0/3806802009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ef40747c0 msgr2=0x7f0ef41a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.486+0000 7f0ef1e00700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ef40747c0 0x7f0ef41a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.486+0000 7f0ef1e00700 1 -- 172.21.15.49:0/3806802009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ef4112e40 msgr2=0x7f0ef419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.486+0000 7f0ef1e00700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ef4112e40 0x7f0ef419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.486+0000 7f0ef1e00700 1 -- 172.21.15.49:0/3806802009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee00095f0 con 0x7f0ef4073ea0 2024-02-26T06:21:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.486+0000 7f0ef2800700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ef4112e40 0x7f0ef419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:21:04.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.486+0000 7f0ef1e00700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4073ea0 0x7f0ef41a4640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e9a0 tx=0x7f0ee400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:04.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.486+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4009790 con 0x7f0ef4073ea0 2024-02-26T06:21:04.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.487+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ee400cbd0 con 0x7f0ef4073ea0 2024-02-26T06:21:04.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.487+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4018680 con 0x7f0ef4073ea0 2024-02-26T06:21:04.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.487+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec009c40 con 0x7f0ef4073ea0 2024-02-26T06:21:04.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.487+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef41ab510 con 0x7f0ef4073ea0 2024-02-26T06:21:04.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.488+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ef41186b0 con 0x7f0ef4073ea0 2024-02-26T06:21:04.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.490+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0ee400cd40 con 0x7f0ef4073ea0 2024-02-26T06:21:04.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.492+0000 7f0eeaa00700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0ed80a21e0 0x7f0ed80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:04.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.492+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ee4014070 con 0x7f0ef4073ea0 2024-02-26T06:21:04.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.492+0000 7f0ef1400700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0ed80a21e0 0x7f0ed80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:04.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.492+0000 7f0eeaa00700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] conn(0x7f0ed80ab9c0 0x7f0ed80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:04.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.492+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 --> [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ed80ae490 con 0x7f0ed80ab9c0 2024-02-26T06:21:04.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.492+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee40cea00 con 0x7f0ef4073ea0 2024-02-26T06:21:04.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.492+0000 7f0ef2800700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] conn(0x7f0ed80ab9c0 0x7f0ed80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:04.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.493+0000 7f0ef1400700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0ed80a21e0 0x7f0ed80a46a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4114c90 tx=0x7f0ee0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:04.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.493+0000 7f0ef2800700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] conn(0x7f0ed80ab9c0 0x7f0ed80adde0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:04.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.494+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 <== osd.2 v2:172.21.15.115:6800/3987885615 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0ed80ae490 con 0x7f0ed80ab9c0 2024-02-26T06:21:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.512+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 --> [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ef41124f0 con 0x7f0ed80ab9c0 2024-02-26T06:21:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.513+0000 7f0eeaa00700 1 -- 172.21.15.49:0/3806802009 <== osd.2 v2:172.21.15.115:6800/3987885615 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ef41124f0 con 0x7f0ed80ab9c0 2024-02-26T06:21:04.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 >> [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] conn(0x7f0ed80ab9c0 msgr2=0x7f0ed80adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:04.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] conn(0x7f0ed80ab9c0 0x7f0ed80adde0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:04.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0ed80a21e0 msgr2=0x7f0ed80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:04.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0ed80a21e0 0x7f0ed80a46a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4114c90 tx=0x7f0ee0009310 comp rx=0 tx=0).stop 2024-02-26T06:21:04.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4073ea0 msgr2=0x7f0ef41a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:04.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 --2- 172.21.15.49:0/3806802009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ef4073ea0 0x7f0ef41a4640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e9a0 tx=0x7f0ee400ecb0 comp rx=0 tx=0).stop 2024-02-26T06:21:04.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef2800700 1 -- 172.21.15.49:0/3806802009 reap_dead start 2024-02-26T06:21:04.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 shutdown_connections 2024-02-26T06:21:04.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 >> 172.21.15.49:0/3806802009 conn(0x7f0ef4078dd0 msgr2=0x7f0ef4111b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:04.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 shutdown_connections 2024-02-26T06:21:04.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.514+0000 7f0ef8a00700 1 -- 172.21.15.49:0/3806802009 wait complete. 2024-02-26T06:21:04.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:04 smithi049 ceph-mon[31696]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-26T06:21:04.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:04 smithi080 ceph-mon[36206]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-26T06:21:04.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:04 smithi115 ceph-mon[36426]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-26T06:21:06.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:06 smithi049 ceph-mon[31696]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-26T06:21:06.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:06 smithi080 ceph-mon[36206]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-26T06:21:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:06 smithi115 ceph-mon[36426]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 -- 172.21.15.49:0/2662546236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220101f20 msgr2=0x7f9220102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 --2- 172.21.15.49:0/2662546236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220101f20 0x7f9220102300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9214009990 tx=0x7f9214009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 -- 172.21.15.49:0/2662546236 shutdown_connections 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 --2- 172.21.15.49:0/2662546236 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9220106ff0 0x7f922010ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 --2- 172.21.15.49:0/2662546236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220101f20 0x7f9220102300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 --2- 172.21.15.49:0/2662546236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92201028d0 0x7f9220106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 -- 172.21.15.49:0/2662546236 >> 172.21.15.49:0/2662546236 conn(0x7f9220078dd0 msgr2=0x7f92200791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 -- 172.21.15.49:0/2662546236 shutdown_connections 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.384+0000 7f9228600700 1 -- 172.21.15.49:0/2662546236 wait complete. 2024-02-26T06:21:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.385+0000 7f9228600700 1 Processor -- start 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.386+0000 7f9228600700 1 -- start start 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.386+0000 7f9228600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9220101f20 0x7f9220198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.386+0000 7f9228600700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92201028d0 0x7f9220198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.386+0000 7f9228600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220106ff0 0x7f922019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.386+0000 7f9228600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922019d120 con 0x7f9220106ff0 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.386+0000 7f9228600700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922019d290 con 0x7f92201028d0 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.386+0000 7f9228600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922019d450 con 0x7f9220101f20 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9220101f20 0x7f9220198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9226600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220106ff0 0x7f922019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92201028d0 0x7f9220198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9220101f20 0x7f9220198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57508/0 (socket says 172.21.15.49:57508) 2024-02-26T06:21:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9226600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220106ff0 0x7f922019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38256/0 (socket says 172.21.15.49:38256) 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225c00700 1 -- 172.21.15.49:0/2673911371 learned_addr learned my addr 172.21.15.49:0/2673911371 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225c00700 1 -- 172.21.15.49:0/2673911371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92201028d0 msgr2=0x7f9220198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225c00700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92201028d0 0x7f9220198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225c00700 1 -- 172.21.15.49:0/2673911371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220106ff0 msgr2=0x7f922019cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225c00700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220106ff0 0x7f922019cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225c00700 1 -- 172.21.15.49:0/2673911371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140095f0 con 0x7f9220101f20 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9225200700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92201028d0 0x7f9220198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.387+0000 7f9226600700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9220106ff0 0x7f922019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.388+0000 7f9225c00700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9220101f20 0x7f9220198240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9214000c00 tx=0x7f9214005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.388+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921401d070 con 0x7f9220101f20 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.388+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f921400b610 con 0x7f9220101f20 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.388+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922019d6d0 con 0x7f9220101f20 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.388+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140173f0 con 0x7f9220101f20 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.388+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92201a4f90 con 0x7f9220101f20 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.389+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9220111fe0 con 0x7f9220101f20 2024-02-26T06:21:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.392+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f921400fb00 con 0x7f9220101f20 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.393+0000 7f921e000700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f92080a21e0 0x7f92080a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.393+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f92140cf9f0 con 0x7f9220101f20 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.393+0000 7f9225200700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f92080a21e0 0x7f92080a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.394+0000 7f921e000700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] conn(0x7f92080ab9c0 0x7f92080adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.394+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 --> [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92080ae490 con 0x7f92080ab9c0 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.394+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92140cfd60 con 0x7f9220101f20 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.394+0000 7f9226600700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] conn(0x7f92080ab9c0 0x7f92080adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.394+0000 7f9225200700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f92080a21e0 0x7f92080a46a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9220108570 tx=0x7f920c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.395+0000 7f9226600700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] conn(0x7f92080ab9c0 0x7f92080adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.396+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 <== osd.1 v2:172.21.15.80:6800/3019548533 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f92080ae490 con 0x7f92080ab9c0 2024-02-26T06:21:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.414+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 --> [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9220062300 con 0x7f92080ab9c0 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.415+0000 7f921e000700 1 -- 172.21.15.49:0/2673911371 <== osd.1 v2:172.21.15.80:6800/3019548533 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9220062300 con 0x7f92080ab9c0 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 >> [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] conn(0x7f92080ab9c0 msgr2=0x7f92080adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] conn(0x7f92080ab9c0 0x7f92080adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f92080a21e0 msgr2=0x7f92080a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f92080a21e0 0x7f92080a46a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9220108570 tx=0x7f920c009260 comp rx=0 tx=0).stop 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9220101f20 msgr2=0x7f9220198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 --2- 172.21.15.49:0/2673911371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9220101f20 0x7f9220198240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9214000c00 tx=0x7f9214005740 comp rx=0 tx=0).stop 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9226600700 1 -- 172.21.15.49:0/2673911371 reap_dead start 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 shutdown_connections 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 >> 172.21.15.49:0/2673911371 conn(0x7f9220078dd0 msgr2=0x7f92200ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.416+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 shutdown_connections 2024-02-26T06:21:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:04.417+0000 7f9228600700 1 -- 172.21.15.49:0/2673911371 wait complete. 2024-02-26T06:21:08.801 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:08 smithi049 ceph-mon[31696]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-26T06:21:08.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:08 smithi080 ceph-mon[36206]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-26T06:21:08.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:08 smithi115 ceph-mon[36426]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-26T06:21:09.819 INFO:teuthology.orchestra.run.smithi049.stdout:77309411400 2024-02-26T06:21:09.819 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-26T06:21:10.109 INFO:teuthology.orchestra.run.smithi049.stdout:55834574925 2024-02-26T06:21:10.109 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-26T06:21:10.818 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:10 smithi049 ceph-mon[31696]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:10.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:10 smithi080 ceph-mon[36206]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:10.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:10 smithi115 ceph-mon[36426]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 -- 172.21.15.49:0/3964091047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a41091a0 msgr2=0x7fa3a4110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 --2- 172.21.15.49:0/3964091047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a41091a0 0x7fa3a4110d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00b0a0 tx=0x7fa39c00b3b0 comp rx=0 tx=0).stop 2024-02-26T06:21:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 -- 172.21.15.49:0/3964091047 shutdown_connections 2024-02-26T06:21:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 --2- 172.21.15.49:0/3964091047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a41091a0 0x7fa3a4110d70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 --2- 172.21.15.49:0/3964091047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a41040c0 0x7fa3a41044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 --2- 172.21.15.49:0/3964091047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a4104a70 0x7fa3a4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 -- 172.21.15.49:0/3964091047 >> 172.21.15.49:0/3964091047 conn(0x7fa3a40ff950 msgr2=0x7fa3a4101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 -- 172.21.15.49:0/3964091047 shutdown_connections 2024-02-26T06:21:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.714+0000 7fa3aa800700 1 -- 172.21.15.49:0/3964091047 wait complete. 2024-02-26T06:21:11.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.715+0000 7fa3aa800700 1 Processor -- start 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa3aa800700 1 -- start start 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa3aa800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a41040c0 0x7fa3a419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa3aa800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a4104a70 0x7fa3a419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa3aa800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41091a0 0x7fa3a419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa3aa800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a419f3f0 con 0x7fa3a4104a70 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa3aa800700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a419f560 con 0x7fa3a41040c0 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa3aa800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a419f720 con 0x7fa3a41091a0 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa39be00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a4104a70 0x7fa3a419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa39be00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a4104a70 0x7fa3a419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47996/0 (socket says 172.21.15.49:47996) 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.716+0000 7fa39be00700 1 -- 172.21.15.49:0/1528187467 learned_addr learned my addr 172.21.15.49:0/1528187467 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.717+0000 7fa3a3400700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a41040c0 0x7fa3a419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.717+0000 7fa3a3e00700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41091a0 0x7fa3a419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.717+0000 7fa39be00700 1 -- 172.21.15.49:0/1528187467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41091a0 msgr2=0x7fa3a419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.717+0000 7fa39be00700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41091a0 0x7fa3a419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.717+0000 7fa39be00700 1 -- 172.21.15.49:0/1528187467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a41040c0 msgr2=0x7fa3a419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.717+0000 7fa39be00700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a41040c0 0x7fa3a419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.717+0000 7fa39be00700 1 -- 172.21.15.49:0/1528187467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c009d00 con 0x7fa3a4104a70 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.717+0000 7fa39be00700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a4104a70 0x7fa3a419aaa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00e970 tx=0x7fa38c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.718+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c009790 con 0x7fa3a4104a70 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.718+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a419fa00 con 0x7fa3a4104a70 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.718+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa38c00eec0 con 0x7fa3a4104a70 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.718+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c005690 con 0x7fa3a4104a70 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.718+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a41a7260 con 0x7fa3a4104a70 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.719+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3a41141e0 con 0x7fa3a4104a70 2024-02-26T06:21:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.721+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa38c00cbd0 con 0x7fa3a4104a70 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.723+0000 7fa3a0c00700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa3880a2260 0x7fa3880a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.723+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa38c014070 con 0x7fa3a4104a70 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.723+0000 7fa3a3400700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa3880a2260 0x7fa3880a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.723+0000 7fa3a0c00700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] conn(0x7fa3880aba40 0x7fa3880ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.723+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 --> [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3880ae510 con 0x7fa3880aba40 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.723+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa38c0ce4f0 con 0x7fa3a4104a70 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.724+0000 7fa3a3e00700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] conn(0x7fa3880aba40 0x7fa3880ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.724+0000 7fa3a3400700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa3880a2260 0x7fa3880a4720 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa394009c60 tx=0x7fa394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.724+0000 7fa3a3e00700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] conn(0x7fa3880aba40 0x7fa3880ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.725+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 <== osd.6 v2:172.21.15.49:6818/2967719452 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa3880ae510 con 0x7fa3880aba40 2024-02-26T06:21:11.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.744+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 --> [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa3a4062300 con 0x7fa3880aba40 2024-02-26T06:21:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.744+0000 7fa3a0c00700 1 -- 172.21.15.49:0/1528187467 <== osd.6 v2:172.21.15.49:6818/2967719452 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa3a4062300 con 0x7fa3880aba40 2024-02-26T06:21:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.744+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] conn(0x7fa3880aba40 msgr2=0x7fa3880ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.744+0000 7fa3aa800700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] conn(0x7fa3880aba40 0x7fa3880ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa3880a2260 msgr2=0x7fa3880a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3aa800700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa3880a2260 0x7fa3880a4720 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa394009c60 tx=0x7fa394009260 comp rx=0 tx=0).stop 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a4104a70 msgr2=0x7fa3a419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3aa800700 1 --2- 172.21.15.49:0/1528187467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a4104a70 0x7fa3a419aaa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00e970 tx=0x7fa38c00ec80 comp rx=0 tx=0).stop 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3a3e00700 1 -- 172.21.15.49:0/1528187467 reap_dead start 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 shutdown_connections 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 >> 172.21.15.49:0/1528187467 conn(0x7fa3a40ff950 msgr2=0x7fa3a4100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 shutdown_connections 2024-02-26T06:21:11.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:11.745+0000 7fa3aa800700 1 -- 172.21.15.49:0/1528187467 wait complete. 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 -- 172.21.15.49:0/1832729837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 msgr2=0x7f57801044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 --2- 172.21.15.49:0/1832729837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 0x7f57801044a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57700098f0 tx=0x7f5770009c00 comp rx=0 tx=0).stop 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 -- 172.21.15.49:0/1832729837 shutdown_connections 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 --2- 172.21.15.49:0/1832729837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57801091a0 0x7f5780110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 --2- 172.21.15.49:0/1832729837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 0x7f57801044a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 --2- 172.21.15.49:0/1832729837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5780104a70 0x7f5780108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 -- 172.21.15.49:0/1832729837 >> 172.21.15.49:0/1832729837 conn(0x7f57800ff950 msgr2=0x7f5780101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 -- 172.21.15.49:0/1832729837 shutdown_connections 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.397+0000 7f5785600700 1 -- 172.21.15.49:0/1832729837 wait complete. 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.399+0000 7f5785600700 1 Processor -- start 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.399+0000 7f5785600700 1 -- start start 2024-02-26T06:21:12.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.399+0000 7f5785600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 0x7f578019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.399+0000 7f5785600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5780104a70 0x7f578019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.399+0000 7f5785600700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57801091a0 0x7f578019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.399+0000 7f5785600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578019f2d0 con 0x7f5780104a70 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.399+0000 7f5785600700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578019f440 con 0x7f57801091a0 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.399+0000 7f5785600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578019f600 con 0x7f57801040c0 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 0x7f578019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 0x7f578019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48506/0 (socket says 172.21.15.49:48506) 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577ea00700 1 -- 172.21.15.49:0/3015842758 learned_addr learned my addr 172.21.15.49:0/3015842758 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577e000700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5780104a70 0x7f578019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577f400700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57801091a0 0x7f578019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577ea00700 1 -- 172.21.15.49:0/3015842758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57801091a0 msgr2=0x7f578019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577ea00700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57801091a0 0x7f578019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577ea00700 1 -- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5780104a70 msgr2=0x7f578019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577ea00700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5780104a70 0x7f578019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577ea00700 1 -- 172.21.15.49:0/3015842758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0095f0 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577f400700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57801091a0 0x7f578019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.400+0000 7f577e000700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5780104a70 0x7f578019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.401+0000 7f577ea00700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 0x7f578019a3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5770005e90 tx=0x7f577000fa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.402+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577001c070 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.402+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f577000b560 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.402+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770003c60 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.402+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5770009520 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.402+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578019fab0 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.403+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57801055a0 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.405+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f577000b6d0 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.406+0000 7f5777400700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f57640a2260 0x7f57640a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.407+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57700ceeb0 con 0x7f57801040c0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.407+0000 7f577e000700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f57640a2260 0x7f57640a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.407+0000 7f5777400700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] conn(0x7f57640ab910 0x7f57640add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:12.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.407+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 --> [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57640ae3e0 con 0x7f57640ab910 2024-02-26T06:21:12.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.407+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57700cf220 con 0x7f57801040c0 2024-02-26T06:21:12.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.407+0000 7f577f400700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] conn(0x7f57640ab910 0x7f57640add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:12.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.407+0000 7f577e000700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f57640a2260 0x7f57640a4720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5780100de0 tx=0x7f576c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:12.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.408+0000 7f577f400700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] conn(0x7f57640ab910 0x7f57640add30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:12.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.409+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 <== osd.8 v2:172.21.15.115:6816/2102490724 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f57640ae3e0 con 0x7f57640ab910 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.427+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 --> [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5780114250 con 0x7f57640ab910 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.428+0000 7f5777400700 1 -- 172.21.15.49:0/3015842758 <== osd.8 v2:172.21.15.115:6816/2102490724 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5780114250 con 0x7f57640ab910 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.428+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 >> [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] conn(0x7f57640ab910 msgr2=0x7f57640add30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.428+0000 7f5785600700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] conn(0x7f57640ab910 0x7f57640add30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f57640a2260 msgr2=0x7f57640a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f5785600700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f57640a2260 0x7f57640a4720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5780100de0 tx=0x7f576c009310 comp rx=0 tx=0).stop 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 msgr2=0x7f578019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f5785600700 1 --2- 172.21.15.49:0/3015842758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57801040c0 0x7f578019a3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5770005e90 tx=0x7f577000fa90 comp rx=0 tx=0).stop 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f577f400700 1 -- 172.21.15.49:0/3015842758 reap_dead start 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 shutdown_connections 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 >> 172.21.15.49:0/3015842758 conn(0x7f57800ff950 msgr2=0x7f57801016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 shutdown_connections 2024-02-26T06:21:12.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:12.429+0000 7f5785600700 1 -- 172.21.15.49:0/3015842758 wait complete. 2024-02-26T06:21:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:12 smithi080 ceph-mon[36206]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:12.847 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:12 smithi115 ceph-mon[36426]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:12.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:12 smithi049 ceph-mon[31696]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:13.479 INFO:teuthology.orchestra.run.smithi049.stdout:257698037800 2024-02-26T06:21:13.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-26T06:21:13.618 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:13.618 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:13.619 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:13.620 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:13.620 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:13.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.343+0000 7fc70ce00700 1 -- 172.21.15.49:0/1166777592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708101db0 msgr2=0x7fc708102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:13.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.343+0000 7fc70ce00700 1 --2- 172.21.15.49:0/1166777592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708101db0 0x7fc708102190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80098f0 tx=0x7fc6f8009c00 comp rx=0 tx=0).stop 2024-02-26T06:21:13.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.343+0000 7fc70ce00700 1 -- 172.21.15.49:0/1166777592 shutdown_connections 2024-02-26T06:21:13.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.343+0000 7fc70ce00700 1 --2- 172.21.15.49:0/1166777592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708106e90 0x7fc70810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:13.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.343+0000 7fc70ce00700 1 --2- 172.21.15.49:0/1166777592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708101db0 0x7fc708102190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:13.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.343+0000 7fc70ce00700 1 --2- 172.21.15.49:0/1166777592 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc708102760 0x7fc7081067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:13.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.343+0000 7fc70ce00700 1 -- 172.21.15.49:0/1166777592 >> 172.21.15.49:0/1166777592 conn(0x7fc7080fd640 msgr2=0x7fc7080ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:13.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.344+0000 7fc70ce00700 1 -- 172.21.15.49:0/1166777592 shutdown_connections 2024-02-26T06:21:13.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.344+0000 7fc70ce00700 1 -- 172.21.15.49:0/1166777592 wait complete. 2024-02-26T06:21:13.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.345+0000 7fc70ce00700 1 Processor -- start 2024-02-26T06:21:13.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.345+0000 7fc70ce00700 1 -- start start 2024-02-26T06:21:13.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.345+0000 7fc70ce00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708101db0 0x7fc708073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:13.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.345+0000 7fc70ce00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc708102760 0x7fc7080744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:13.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.345+0000 7fc70ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708106e90 0x7fc7080749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:13.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.345+0000 7fc70ce00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a6500 con 0x7fc708101db0 2024-02-26T06:21:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.345+0000 7fc70ce00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a6670 con 0x7fc708102760 2024-02-26T06:21:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.345+0000 7fc70ce00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a6830 con 0x7fc708106e90 2024-02-26T06:21:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708101db0 0x7fc708073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708101db0 0x7fc708073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48026/0 (socket says 172.21.15.49:48026) 2024-02-26T06:21:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc706a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc708102760 0x7fc7080744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707400700 1 -- 172.21.15.49:0/1363421167 learned_addr learned my addr 172.21.15.49:0/1363421167 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707e00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708106e90 0x7fc7080749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707400700 1 -- 172.21.15.49:0/1363421167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708106e90 msgr2=0x7fc7080749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707400700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708106e90 0x7fc7080749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707400700 1 -- 172.21.15.49:0/1363421167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc708102760 msgr2=0x7fc7080744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707400700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc708102760 0x7fc7080744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707400700 1 -- 172.21.15.49:0/1363421167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc700009d00 con 0x7fc708101db0 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc707e00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708106e90 0x7fc7080749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.346+0000 7fc706a00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc708102760 0x7fc7080744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.347+0000 7fc707400700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708101db0 0x7fc708073f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009230 tx=0x7fc6f800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.347+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f801c070 con 0x7fc708101db0 2024-02-26T06:21:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.347+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f800b560 con 0x7fc708101db0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.347+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f8009520 con 0x7fc708101db0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.347+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8004cc0 con 0x7fc708101db0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.347+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7081a6d30 con 0x7fc708101db0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.348+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc708111e40 con 0x7fc708101db0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.350+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc6f800b6d0 con 0x7fc708101db0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.352+0000 7fc6ffe00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc6ec0a21e0 0x7fc6ec0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.352+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc6f80cf950 con 0x7fc708101db0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.352+0000 7fc706a00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc6ec0a21e0 0x7fc6ec0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.352+0000 7fc6ffe00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] conn(0x7fc6ec0ab9c0 0x7fc6ec0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.352+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 --> [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6ec0ae490 con 0x7fc6ec0ab9c0 2024-02-26T06:21:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.352+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f80cfcc0 con 0x7fc708101db0 2024-02-26T06:21:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.353+0000 7fc707e00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] conn(0x7fc6ec0ab9c0 0x7fc6ec0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.353+0000 7fc706a00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc6ec0a21e0 0x7fc6ec0a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc7080ffc40 tx=0x7fc6f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.353+0000 7fc707e00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] conn(0x7fc6ec0ab9c0 0x7fc6ec0adde0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.354+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 <== osd.4 v2:172.21.15.80:6808/1597953633 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc6ec0ae490 con 0x7fc6ec0ab9c0 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.373+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 --> [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc708062380 con 0x7fc6ec0ab9c0 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.373+0000 7fc6ffe00700 1 -- 172.21.15.49:0/1363421167 <== osd.4 v2:172.21.15.80:6808/1597953633 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc708062380 con 0x7fc6ec0ab9c0 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 >> [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] conn(0x7fc6ec0ab9c0 msgr2=0x7fc6ec0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] conn(0x7fc6ec0ab9c0 0x7fc6ec0adde0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc6ec0a21e0 msgr2=0x7fc6ec0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc6ec0a21e0 0x7fc6ec0a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc7080ffc40 tx=0x7fc6f4009260 comp rx=0 tx=0).stop 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708101db0 msgr2=0x7fc708073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 --2- 172.21.15.49:0/1363421167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc708101db0 0x7fc708073f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009230 tx=0x7fc6f800fa70 comp rx=0 tx=0).stop 2024-02-26T06:21:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc707e00700 1 -- 172.21.15.49:0/1363421167 reap_dead start 2024-02-26T06:21:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 shutdown_connections 2024-02-26T06:21:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 >> 172.21.15.49:0/1363421167 conn(0x7fc7080fd640 msgr2=0x7fc708106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.374+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 shutdown_connections 2024-02-26T06:21:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.375+0000 7fc70ce00700 1 -- 172.21.15.49:0/1363421167 wait complete. 2024-02-26T06:21:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:14 smithi080 ceph-mon[36206]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T06:21:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:14 smithi115 ceph-mon[36426]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T06:21:14.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:14 smithi049 ceph-mon[31696]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T06:21:14.937 INFO:teuthology.orchestra.run.smithi049.stdout:146028888126 2024-02-26T06:21:14.938 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-26T06:21:15.022 INFO:teuthology.orchestra.run.smithi049.stdout:201863462962 2024-02-26T06:21:15.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-26T06:21:15.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.788+0000 7f4897200700 1 -- 172.21.15.49:0/2791400683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4890101f20 msgr2=0x7f4890102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.788+0000 7f4897200700 1 --2- 172.21.15.49:0/2791400683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4890101f20 0x7f4890102300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4880009990 tx=0x7f4880009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.788+0000 7f4897200700 1 -- 172.21.15.49:0/2791400683 shutdown_connections 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.788+0000 7f4897200700 1 --2- 172.21.15.49:0/2791400683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4890106ff0 0x7f489010ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.788+0000 7f4897200700 1 --2- 172.21.15.49:0/2791400683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4890101f20 0x7f4890102300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.788+0000 7f4897200700 1 --2- 172.21.15.49:0/2791400683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48901028d0 0x7f4890106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.788+0000 7f4897200700 1 -- 172.21.15.49:0/2791400683 >> 172.21.15.49:0/2791400683 conn(0x7f4890078dd0 msgr2=0x7f48900791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.789+0000 7f4897200700 1 -- 172.21.15.49:0/2791400683 shutdown_connections 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.789+0000 7f4897200700 1 -- 172.21.15.49:0/2791400683 wait complete. 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.790+0000 7f4897200700 1 Processor -- start 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.790+0000 7f4897200700 1 -- start start 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.790+0000 7f4897200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4890101f20 0x7f489019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.790+0000 7f4897200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48901028d0 0x7f489019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.790+0000 7f4897200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4890106ff0 0x7f489019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:15.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.790+0000 7f4897200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019f2e0 con 0x7f4890106ff0 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.790+0000 7f4897200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019f450 con 0x7f4890101f20 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.790+0000 7f4897200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019f610 con 0x7f48901028d0 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4890101f20 0x7f489019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4890101f20 0x7f489019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49624/0 (socket says 172.21.15.49:49624) 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 -- 172.21.15.49:0/401224771 learned_addr learned my addr 172.21.15.49:0/401224771 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f4895200700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4890106ff0 0x7f489019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488f400700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48901028d0 0x7f489019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 -- 172.21.15.49:0/401224771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48901028d0 msgr2=0x7f489019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48901028d0 0x7f489019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 -- 172.21.15.49:0/401224771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4890106ff0 msgr2=0x7f489019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:15.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4890106ff0 0x7f489019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 -- 172.21.15.49:0/401224771 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800095f0 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.791+0000 7f488fe00700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4890101f20 0x7f489019a400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4880005770 tx=0x7f48800057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.792+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488001d070 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.792+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f488000b610 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.792+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489019f890 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.792+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880021e10 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.792+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48901a7150 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.793+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4890111fa0 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.796+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4880021510 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.797+0000 7f488cc00700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f48740a21e0 0x7f48740a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.797+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48800cee30 con 0x7f4890101f20 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.798+0000 7f488f400700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f48740a21e0 0x7f48740a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.798+0000 7f488cc00700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] conn(0x7f48740ab9c0 0x7f48740adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.798+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 --> [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48740ae490 con 0x7f48740ab9c0 2024-02-26T06:21:15.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.798+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48800d1410 con 0x7f4890101f20 2024-02-26T06:21:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.798+0000 7f4895200700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] conn(0x7f48740ab9c0 0x7f48740adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.798+0000 7f488f400700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f48740a21e0 0x7f48740a46a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4890075f90 tx=0x7f487c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.799+0000 7f4895200700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] conn(0x7f48740ab9c0 0x7f48740adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.800+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 <== osd.7 v2:172.21.15.80:6816/4119828999 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f48740ae490 con 0x7f48740ab9c0 2024-02-26T06:21:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.820+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 --> [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4890105450 con 0x7f48740ab9c0 2024-02-26T06:21:15.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.821+0000 7f488cc00700 1 -- 172.21.15.49:0/401224771 <== osd.7 v2:172.21.15.80:6816/4119828999 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4890105450 con 0x7f48740ab9c0 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 >> [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] conn(0x7f48740ab9c0 msgr2=0x7f48740adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] conn(0x7f48740ab9c0 0x7f48740adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f48740a21e0 msgr2=0x7f48740a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f48740a21e0 0x7f48740a46a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4890075f90 tx=0x7f487c009260 comp rx=0 tx=0).stop 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4890101f20 msgr2=0x7f489019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 --2- 172.21.15.49:0/401224771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4890101f20 0x7f489019a400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4880005770 tx=0x7f48800057a0 comp rx=0 tx=0).stop 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4895200700 1 -- 172.21.15.49:0/401224771 reap_dead start 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 shutdown_connections 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 >> 172.21.15.49:0/401224771 conn(0x7f4890078dd0 msgr2=0x7f4890100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 shutdown_connections 2024-02-26T06:21:15.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:13.822+0000 7f4897200700 1 -- 172.21.15.49:0/401224771 wait complete. 2024-02-26T06:21:16.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:16 smithi080 ceph-mon[36206]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:16.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:16 smithi115 ceph-mon[36426]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:16.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:16 smithi049 ceph-mon[31696]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:21:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:223338299437 2024-02-26T06:21:17.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-26T06:21:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:18 smithi080 ceph-mon[36206]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-26T06:21:18.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:18 smithi115 ceph-mon[36426]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-26T06:21:18.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:18 smithi049 ceph-mon[31696]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-26T06:21:19.192 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:19.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.055+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3695036316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101f20 msgr2=0x7fc5f8102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.055+0000 7fc5fdc00700 1 --2- 172.21.15.49:0/3695036316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101f20 0x7fc5f8102300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b0a0 tx=0x7fc5e800b3b0 comp rx=0 tx=0).stop 2024-02-26T06:21:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.055+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3695036316 shutdown_connections 2024-02-26T06:21:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.055+0000 7fc5fdc00700 1 --2- 172.21.15.49:0/3695036316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5f8106ff0 0x7fc5f810ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.055+0000 7fc5fdc00700 1 --2- 172.21.15.49:0/3695036316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101f20 0x7fc5f8102300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.055+0000 7fc5fdc00700 1 --2- 172.21.15.49:0/3695036316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f81028d0 0x7fc5f8106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.055+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3695036316 >> 172.21.15.49:0/3695036316 conn(0x7fc5f8078dd0 msgr2=0x7fc5f80791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.055+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3695036316 shutdown_connections 2024-02-26T06:21:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.056+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3695036316 wait complete. 2024-02-26T06:21:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.057+0000 7fc5fdc00700 1 Processor -- start 2024-02-26T06:21:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.057+0000 7fc5fdc00700 1 -- start start 2024-02-26T06:21:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.057+0000 7fc5fdc00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5f8101f20 0x7fc5f819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.057+0000 7fc5fdc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f81028d0 0x7fc5f819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.057+0000 7fc5fdc00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8106ff0 0x7fc5f819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.057+0000 7fc5fdc00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f819f480 con 0x7fc5f8106ff0 2024-02-26T06:21:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.057+0000 7fc5fdc00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f819f5f0 con 0x7fc5f8101f20 2024-02-26T06:21:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.057+0000 7fc5fdc00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f819f7b0 con 0x7fc5f81028d0 2024-02-26T06:21:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8106ff0 0x7fc5f819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8106ff0 0x7fc5f819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48060/0 (socket says 172.21.15.49:48060) 2024-02-26T06:21:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f6a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5f8101f20 0x7fc5f819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 -- 172.21.15.49:0/3735437818 learned_addr learned my addr 172.21.15.49:0/3735437818 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f6000700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f81028d0 0x7fc5f819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 -- 172.21.15.49:0/3735437818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f81028d0 msgr2=0x7fc5f819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f81028d0 0x7fc5f819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 -- 172.21.15.49:0/3735437818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5f8101f20 msgr2=0x7fc5f819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5f8101f20 0x7fc5f819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 -- 172.21.15.49:0/3735437818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8009d00 con 0x7fc5f8106ff0 2024-02-26T06:21:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f6a00700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5f8101f20 0x7fc5f819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.058+0000 7fc5f7400700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8106ff0 0x7fc5f819ee70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000e970 tx=0x7fc5f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.059+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0009700 con 0x7fc5f8106ff0 2024-02-26T06:21:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.059+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5f0005020 con 0x7fc5f8106ff0 2024-02-26T06:21:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.059+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0010ca0 con 0x7fc5f8106ff0 2024-02-26T06:21:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.059+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f819fa90 con 0x7fc5f8106ff0 2024-02-26T06:21:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.059+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f81a72f0 con 0x7fc5f8106ff0 2024-02-26T06:21:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.060+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc5f8112060 con 0x7fc5f8106ff0 2024-02-26T06:21:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.062+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc5f0005500 con 0x7fc5f8106ff0 2024-02-26T06:21:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.064+0000 7fc5ef400700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc5dc0a21e0 0x7fc5dc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.064+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5f0014070 con 0x7fc5f8106ff0 2024-02-26T06:21:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.064+0000 7fc5f6a00700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc5dc0a21e0 0x7fc5dc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.064+0000 7fc5ef400700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] conn(0x7fc5dc0ab9c0 0x7fc5dc0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.064+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 --> [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5dc0ae490 con 0x7fc5dc0ab9c0 2024-02-26T06:21:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.064+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5f00ce630 con 0x7fc5f8106ff0 2024-02-26T06:21:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.065+0000 7fc5f6000700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] conn(0x7fc5dc0ab9c0 0x7fc5dc0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.065+0000 7fc5f6a00700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc5dc0a21e0 0x7fc5dc0a46a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b780 tx=0x7fc5e8006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.065+0000 7fc5f6000700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] conn(0x7fc5dc0ab9c0 0x7fc5dc0adde0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.066+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 <== osd.0 v2:172.21.15.49:6802/3449358654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc5dc0ae490 con 0x7fc5dc0ab9c0 2024-02-26T06:21:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.087+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 --> [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc5f810e200 con 0x7fc5dc0ab9c0 2024-02-26T06:21:19.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.087+0000 7fc5ef400700 1 -- 172.21.15.49:0/3735437818 <== osd.0 v2:172.21.15.49:6802/3449358654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc5f810e200 con 0x7fc5dc0ab9c0 2024-02-26T06:21:19.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] conn(0x7fc5dc0ab9c0 msgr2=0x7fc5dc0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:19.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5fdc00700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] conn(0x7fc5dc0ab9c0 0x7fc5dc0adde0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:19.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc5dc0a21e0 msgr2=0x7fc5dc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:19.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5fdc00700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc5dc0a21e0 0x7fc5dc0a46a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b780 tx=0x7fc5e8006210 comp rx=0 tx=0).stop 2024-02-26T06:21:19.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8106ff0 msgr2=0x7fc5f819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:19.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5fdc00700 1 --2- 172.21.15.49:0/3735437818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8106ff0 0x7fc5f819ee70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000e970 tx=0x7fc5f000ec80 comp rx=0 tx=0).stop 2024-02-26T06:21:19.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5f7400700 1 -- 172.21.15.49:0/3735437818 reap_dead start 2024-02-26T06:21:19.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 shutdown_connections 2024-02-26T06:21:19.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.088+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 >> 172.21.15.49:0/3735437818 conn(0x7fc5f8078dd0 msgr2=0x7fc5f80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:19.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.089+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 shutdown_connections 2024-02-26T06:21:19.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:19.089+0000 7fc5fdc00700 1 -- 172.21.15.49:0/3735437818 wait complete. 2024-02-26T06:21:20.289 INFO:teuthology.orchestra.run.smithi049.stdout:34359738453 2024-02-26T06:21:20.289 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-26T06:21:20.909 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:20 smithi049 ceph-mon[31696]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T06:21:21.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:20 smithi080 ceph-mon[36206]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T06:21:21.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:20 smithi115 ceph-mon[36426]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T06:21:22.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 -- 172.21.15.49:0/2932025854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18d4069920 msgr2=0x7f18d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:22.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 --2- 172.21.15.49:0/2932025854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18d4069920 0x7f18d4069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f18c00099c0 tx=0x7f18c0009cd0 comp rx=0 tx=0).stop 2024-02-26T06:21:22.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 -- 172.21.15.49:0/2932025854 shutdown_connections 2024-02-26T06:21:22.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 --2- 172.21.15.49:0/2932025854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18d406a2c0 0x7f18d410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:22.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 --2- 172.21.15.49:0/2932025854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18d4107cc0 0x7f18d41080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:22.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 --2- 172.21.15.49:0/2932025854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18d4069920 0x7f18d4069d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:22.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 -- 172.21.15.49:0/2932025854 >> 172.21.15.49:0/2932025854 conn(0x7f18d4077aa0 msgr2=0x7f18d4077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:22.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 -- 172.21.15.49:0/2932025854 shutdown_connections 2024-02-26T06:21:22.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.608+0000 7f18db800700 1 -- 172.21.15.49:0/2932025854 wait complete. 2024-02-26T06:21:22.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.609+0000 7f18db800700 1 Processor -- start 2024-02-26T06:21:22.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18db800700 1 -- start start 2024-02-26T06:21:22.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18db800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18d4069920 0x7f18d41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18db800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18d406a2c0 0x7f18d41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18db800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18d4107cc0 0x7f18d419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18db800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419d090 con 0x7f18d406a2c0 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18db800700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419d200 con 0x7f18d4069920 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18db800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419d3c0 con 0x7f18d4107cc0 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18d8e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18d4069920 0x7f18d41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18d3e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18d406a2c0 0x7f18d41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18d8e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18d4069920 0x7f18d41981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44036/0 (socket says 172.21.15.49:44036) 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.610+0000 7f18d8e00700 1 -- 172.21.15.49:0/1159177810 learned_addr learned my addr 172.21.15.49:0/1159177810 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.611+0000 7f18d9800700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18d4107cc0 0x7f18d419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.611+0000 7f18d8e00700 1 -- 172.21.15.49:0/1159177810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18d4107cc0 msgr2=0x7f18d419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.611+0000 7f18d8e00700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18d4107cc0 0x7f18d419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:22.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.611+0000 7f18d8e00700 1 -- 172.21.15.49:0/1159177810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18d406a2c0 msgr2=0x7f18d41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:22.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.611+0000 7f18d8e00700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18d406a2c0 0x7f18d41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:22.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.611+0000 7f18d8e00700 1 -- 172.21.15.49:0/1159177810 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8009520 con 0x7f18d4069920 2024-02-26T06:21:22.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.611+0000 7f18d8e00700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18d4069920 0x7f18d41981b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18c800eb30 tx=0x7f18c800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:22.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.612+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8009880 con 0x7f18d4069920 2024-02-26T06:21:22.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.612+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18c8004500 con 0x7f18d4069920 2024-02-26T06:21:22.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.612+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c00095f0 con 0x7f18d4069920 2024-02-26T06:21:22.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.612+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8010430 con 0x7f18d4069920 2024-02-26T06:21:22.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.612+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d41a4d30 con 0x7f18d4069920 2024-02-26T06:21:22.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.613+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18d4103370 con 0x7f18d4069920 2024-02-26T06:21:22.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.615+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f18c800cc90 con 0x7f18d4069920 2024-02-26T06:21:22.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.616+0000 7f18d1600700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f18bc0a2260 0x7f18bc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:22.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.616+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f18c8014070 con 0x7f18d4069920 2024-02-26T06:21:22.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.617+0000 7f18d3e00700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f18bc0a2260 0x7f18bc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:22.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.617+0000 7f18d1600700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] conn(0x7f18bc0aba40 0x7f18bc0ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:22.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.617+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 --> [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18bc0ae510 con 0x7f18bc0aba40 2024-02-26T06:21:22.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.617+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18c80ce490 con 0x7f18d4069920 2024-02-26T06:21:22.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.617+0000 7f18d3e00700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f18bc0a2260 0x7f18bc0a4720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18c000b580 tx=0x7f18c0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:22.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.617+0000 7f18d9800700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] conn(0x7f18bc0aba40 0x7f18bc0ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:22.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.618+0000 7f18d9800700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] conn(0x7f18bc0aba40 0x7f18bc0ade60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:22.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.619+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 <== osd.11 v2:172.21.15.115:6824/1497318098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f18bc0ae510 con 0x7f18bc0aba40 2024-02-26T06:21:22.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.638+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 --> [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18d4112010 con 0x7f18bc0aba40 2024-02-26T06:21:22.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.639+0000 7f18d1600700 1 -- 172.21.15.49:0/1159177810 <== osd.11 v2:172.21.15.115:6824/1497318098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f18d4112010 con 0x7f18bc0aba40 2024-02-26T06:21:22.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.639+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 >> [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] conn(0x7f18bc0aba40 msgr2=0x7f18bc0ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:22.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.639+0000 7f18db800700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] conn(0x7f18bc0aba40 0x7f18bc0ade60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f18bc0a2260 msgr2=0x7f18bc0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18db800700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f18bc0a2260 0x7f18bc0a4720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18c000b580 tx=0x7f18c0005fb0 comp rx=0 tx=0).stop 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18d4069920 msgr2=0x7f18d41981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18db800700 1 --2- 172.21.15.49:0/1159177810 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18d4069920 0x7f18d41981b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18c800eb30 tx=0x7f18c800ee40 comp rx=0 tx=0).stop 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18d9800700 1 -- 172.21.15.49:0/1159177810 reap_dead start 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 shutdown_connections 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 >> 172.21.15.49:0/1159177810 conn(0x7f18d4077aa0 msgr2=0x7f18d40ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 shutdown_connections 2024-02-26T06:21:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:22.640+0000 7f18db800700 1 -- 172.21.15.49:0/1159177810 wait complete. 2024-02-26T06:21:23.072 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:22 smithi049 ceph-mon[31696]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T06:21:23.214 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:23.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:22 smithi080 ceph-mon[36206]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T06:21:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:22 smithi115 ceph-mon[36426]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T06:21:24.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.730+0000 7f7294600700 1 -- 172.21.15.49:0/2093817180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c069920 msgr2=0x7f728c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:24.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.730+0000 7f7294600700 1 --2- 172.21.15.49:0/2093817180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c069920 0x7f728c069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f72780099c0 tx=0x7f7278009cd0 comp rx=0 tx=0).stop 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.730+0000 7f7294600700 1 -- 172.21.15.49:0/2093817180 shutdown_connections 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.730+0000 7f7294600700 1 --2- 172.21.15.49:0/2093817180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f728c06a2c0 0x7f728c10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.730+0000 7f7294600700 1 --2- 172.21.15.49:0/2093817180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f728c107cc0 0x7f728c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.730+0000 7f7294600700 1 --2- 172.21.15.49:0/2093817180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c069920 0x7f728c069d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.730+0000 7f7294600700 1 -- 172.21.15.49:0/2093817180 >> 172.21.15.49:0/2093817180 conn(0x7f728c077aa0 msgr2=0x7f728c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.731+0000 7f7294600700 1 -- 172.21.15.49:0/2093817180 shutdown_connections 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.731+0000 7f7294600700 1 -- 172.21.15.49:0/2093817180 wait complete. 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.732+0000 7f7294600700 1 Processor -- start 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.732+0000 7f7294600700 1 -- start start 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.732+0000 7f7294600700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f728c069920 0x7f728c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.732+0000 7f7294600700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c06a2c0 0x7f728c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.732+0000 7f7294600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f728c107cc0 0x7f728c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.732+0000 7f7294600700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c19f280 con 0x7f728c06a2c0 2024-02-26T06:21:24.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.732+0000 7f7294600700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c19f3f0 con 0x7f728c069920 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.732+0000 7f7294600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c19f5b0 con 0x7f728c107cc0 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c06a2c0 0x7f728c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c06a2c0 0x7f728c19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49742/0 (socket says 172.21.15.49:49742) 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f728c069920 0x7f728c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291200700 1 -- 172.21.15.49:0/3435409783 learned_addr learned my addr 172.21.15.49:0/3435409783 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7292600700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f728c107cc0 0x7f728c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291200700 1 -- 172.21.15.49:0/3435409783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f728c107cc0 msgr2=0x7f728c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291200700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f728c107cc0 0x7f728c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:24.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291200700 1 -- 172.21.15.49:0/3435409783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f728c069920 msgr2=0x7f728c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:24.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291200700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f728c069920 0x7f728c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:24.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291200700 1 -- 172.21.15.49:0/3435409783 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72780095f0 con 0x7f728c06a2c0 2024-02-26T06:21:24.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.733+0000 7f7291c00700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f728c069920 0x7f728c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:24.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.734+0000 7f7291200700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c06a2c0 0x7f728c19a8e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f727800c010 tx=0x7f727800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:24.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.734+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727801d070 con 0x7f728c06a2c0 2024-02-26T06:21:24.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.734+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7278003680 con 0x7f728c06a2c0 2024-02-26T06:21:24.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.734+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c19f830 con 0x7f728c06a2c0 2024-02-26T06:21:24.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.734+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780173f0 con 0x7f728c06a2c0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.734+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c1a70f0 con 0x7f728c06a2c0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.736+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f728c19fb00 con 0x7f728c06a2c0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.738+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f7278022470 con 0x7f728c06a2c0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.739+0000 7f728a000700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f72740a21e0 0x7f72740a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.739+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72780cf580 con 0x7f728c06a2c0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.740+0000 7f7291c00700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f72740a21e0 0x7f72740a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.740+0000 7f728a000700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] conn(0x7f72740ab9c0 0x7f72740adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.740+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 --> [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72740ae490 con 0x7f72740ab9c0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.740+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72780cf8f0 con 0x7f728c06a2c0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.740+0000 7f7292600700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] conn(0x7f72740ab9c0 0x7f72740adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.740+0000 7f7291c00700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f72740a21e0 0x7f72740a46a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7280007f50 tx=0x7f72800073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.741+0000 7f7292600700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] conn(0x7f72740ab9c0 0x7f72740adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.742+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 <== osd.3 v2:172.21.15.49:6810/2774029183 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f72740ae490 con 0x7f72740ab9c0 2024-02-26T06:21:24.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.760+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 --> [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f728c112000 con 0x7f72740ab9c0 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.761+0000 7f728a000700 1 -- 172.21.15.49:0/3435409783 <== osd.3 v2:172.21.15.49:6810/2774029183 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f728c112000 con 0x7f72740ab9c0 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.761+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] conn(0x7f72740ab9c0 msgr2=0x7f72740adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] conn(0x7f72740ab9c0 0x7f72740adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f72740a21e0 msgr2=0x7f72740a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f72740a21e0 0x7f72740a46a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7280007f50 tx=0x7f72800073d0 comp rx=0 tx=0).stop 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c06a2c0 msgr2=0x7f728c19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 --2- 172.21.15.49:0/3435409783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f728c06a2c0 0x7f728c19a8e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f727800c010 tx=0x7f727800f690 comp rx=0 tx=0).stop 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7292600700 1 -- 172.21.15.49:0/3435409783 reap_dead start 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 shutdown_connections 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 >> 172.21.15.49:0/3435409783 conn(0x7f728c077aa0 msgr2=0x7f728c10ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 shutdown_connections 2024-02-26T06:21:24.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:24.762+0000 7f7294600700 1 -- 172.21.15.49:0/3435409783 wait complete. 2024-02-26T06:21:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:24 smithi049 ceph-mon[31696]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:21:25.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:24 smithi080 ceph-mon[36206]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:21:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:24 smithi115 ceph-mon[36426]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:21:26.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:26 smithi049 ceph-mon[31696]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:21:27.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:26 smithi080 ceph-mon[36206]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:21:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:26 smithi115 ceph-mon[36426]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:21:28.123 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-26T06:21:28.123 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-02-26T06:21:28.123 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-26T06:21:28.123 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-02-26T06:21:28.123 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-02-26T06:21:28.123 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-02-26T06:21:28.123 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-26T06:21:28.124 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-02-26T06:21:28.124 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:27 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-02-26T06:21:28.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-26T06:21:28.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-02-26T06:21:28.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-26T06:21:28.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-02-26T06:21:28.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-02-26T06:21:28.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-02-26T06:21:28.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-26T06:21:28.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-02-26T06:21:28.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:27 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-02-26T06:21:28.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-26T06:21:28.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-02-26T06:21:28.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-26T06:21:28.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-02-26T06:21:28.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-02-26T06:21:28.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-02-26T06:21:28.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-26T06:21:28.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-02-26T06:21:28.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:27 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.668+0000 7fb704c00700 1 -- 172.21.15.49:0/2427239656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb70006a1b0 msgr2=0x7fb70006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.668+0000 7fb704c00700 1 --2- 172.21.15.49:0/2427239656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb70006a1b0 0x7fb70006a590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b0a0 tx=0x7fb6f000b3b0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.668+0000 7fb704c00700 1 -- 172.21.15.49:0/2427239656 shutdown_connections 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.668+0000 7fb704c00700 1 --2- 172.21.15.49:0/2427239656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70010e750 0x7fb700110b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.668+0000 7fb704c00700 1 --2- 172.21.15.49:0/2427239656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb70006a1b0 0x7fb70006a590 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.668+0000 7fb704c00700 1 --2- 172.21.15.49:0/2427239656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70006aad0 0x7fb70010e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.668+0000 7fb704c00700 1 -- 172.21.15.49:0/2427239656 >> 172.21.15.49:0/2427239656 conn(0x7fb700076bc0 msgr2=0x7fb700076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.669+0000 7fb704c00700 1 -- 172.21.15.49:0/2427239656 shutdown_connections 2024-02-26T06:21:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.669+0000 7fb704c00700 1 -- 172.21.15.49:0/2427239656 wait complete. 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.670+0000 7fb704c00700 1 Processor -- start 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.670+0000 7fb704c00700 1 -- start start 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb704c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006a1b0 0x7fb70019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb704c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70006aad0 0x7fb70019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb704c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb70010e750 0x7fb70019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb704c00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019f490 con 0x7fb70006a1b0 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb704c00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019f600 con 0x7fb70006aad0 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb704c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019f7c0 con 0x7fb70010e750 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb6fde00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006a1b0 0x7fb70019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb6fde00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006a1b0 0x7fb70019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49760/0 (socket says 172.21.15.49:49760) 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb6fde00700 1 -- 172.21.15.49:0/3828554645 learned_addr learned my addr 172.21.15.49:0/3828554645 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb6f5400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70006aad0 0x7fb70019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:28.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.671+0000 7fb6fe800700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb70010e750 0x7fb70019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f5400700 1 -- 172.21.15.49:0/3828554645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb70010e750 msgr2=0x7fb70019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f5400700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb70010e750 0x7fb70019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f5400700 1 -- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006a1b0 msgr2=0x7fb70019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f5400700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006a1b0 0x7fb70019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f5400700 1 -- 172.21.15.49:0/3828554645 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e8009520 con 0x7fb70006aad0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6fde00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006a1b0 0x7fb70019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f5400700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70006aad0 0x7fb70019ab40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9f0 tx=0x7fb6e800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f7400700 1 -- 172.21.15.49:0/3828554645 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800cb70 con 0x7fb70006aad0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f0009d00 con 0x7fb70006aad0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f7400700 1 -- 172.21.15.49:0/3828554645 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6e800ee90 con 0x7fb70006aad0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb6f7400700 1 -- 172.21.15.49:0/3828554645 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8003d80 con 0x7fb70006aad0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.672+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7001a7130 con 0x7fb70006aad0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.673+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb700109970 con 0x7fb70006aad0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.676+0000 7fb6f7400700 1 -- 172.21.15.49:0/3828554645 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb6e8003f20 con 0x7fb70006aad0 2024-02-26T06:21:28.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.677+0000 7fb6f7400700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb6e40a21e0 0x7fb6e40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.677+0000 7fb6f7400700 1 -- 172.21.15.49:0/3828554645 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb6e8014070 con 0x7fb70006aad0 2024-02-26T06:21:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.677+0000 7fb6fde00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb6e40a21e0 0x7fb6e40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.678+0000 7fb6fde00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb6e40a21e0 0x7fb6e40a46a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b780 tx=0x7fb6f0009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.679+0000 7fb6f7400700 1 -- 172.21.15.49:0/3828554645 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6e80970b0 con 0x7fb70006aad0 2024-02-26T06:21:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.930+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb700068930 con 0x7fb70006aad0 2024-02-26T06:21:28.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.931+0000 7fb6f7400700 1 -- 172.21.15.49:0/3828554645 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6e809a770 con 0x7fb70006aad0 2024-02-26T06:21:28.931 INFO:teuthology.orchestra.run.smithi049.stdout:55834574930 2024-02-26T06:21:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.933+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb6e40a21e0 msgr2=0x7fb6e40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.933+0000 7fb704c00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb6e40a21e0 0x7fb6e40a46a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b780 tx=0x7fb6f0009780 comp rx=0 tx=0).stop 2024-02-26T06:21:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.933+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70006aad0 msgr2=0x7fb70019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.933+0000 7fb704c00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70006aad0 0x7fb70019ab40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9f0 tx=0x7fb6e800ed00 comp rx=0 tx=0).stop 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.934+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 shutdown_connections 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.934+0000 7fb704c00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb6e40a21e0 0x7fb6e40a46a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.934+0000 7fb704c00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb70010e750 0x7fb70019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.934+0000 7fb704c00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006a1b0 0x7fb70019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.934+0000 7fb704c00700 1 --2- 172.21.15.49:0/3828554645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70006aad0 0x7fb70019ab40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.934+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 >> 172.21.15.49:0/3828554645 conn(0x7fb700076bc0 msgr2=0x7fb70010fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.934+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 shutdown_connections 2024-02-26T06:21:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.934+0000 7fb704c00700 1 -- 172.21.15.49:0/3828554645 wait complete. 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-02-26T06:21:29.058 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-02-26T06:21:29.059 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:28 smithi049 ceph-mon[31696]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T06:21:29.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:29.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-26T06:21:29.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-02-26T06:21:29.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-26T06:21:29.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-02-26T06:21:29.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-02-26T06:21:29.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-02-26T06:21:29.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-26T06:21:29.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-02-26T06:21:29.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-02-26T06:21:29.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:28 smithi080 ceph-mon[36206]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T06:21:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-26T06:21:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-02-26T06:21:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-26T06:21:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-02-26T06:21:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-02-26T06:21:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-02-26T06:21:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-26T06:21:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-02-26T06:21:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-02-26T06:21:29.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:28 smithi115 ceph-mon[36426]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T06:21:29.957 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:29 smithi049 ceph-mon[31696]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T06:21:29.957 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:29 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3828554645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T06:21:30.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.780+0000 7f0167200700 1 -- 172.21.15.49:0/1594903751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168102760 msgr2=0x7f01681067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.780+0000 7f0167200700 1 --2- 172.21.15.49:0/1594903751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168102760 0x7f01681067b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0154009990 tx=0x7f0154009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.780+0000 7f0167200700 1 -- 172.21.15.49:0/1594903751 shutdown_connections 2024-02-26T06:21:30.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.780+0000 7f0167200700 1 --2- 172.21.15.49:0/1594903751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0168106e90 0x7f016810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.780+0000 7f0167200700 1 --2- 172.21.15.49:0/1594903751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0168101db0 0x7f0168102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.780+0000 7f0167200700 1 --2- 172.21.15.49:0/1594903751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168102760 0x7f01681067b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.780+0000 7f0167200700 1 -- 172.21.15.49:0/1594903751 >> 172.21.15.49:0/1594903751 conn(0x7f01680fd640 msgr2=0x7f01680ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.781+0000 7f0167200700 1 -- 172.21.15.49:0/1594903751 shutdown_connections 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.781+0000 7f0167200700 1 -- 172.21.15.49:0/1594903751 wait complete. 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.782+0000 7f0167200700 1 Processor -- start 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.782+0000 7f0167200700 1 -- start start 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.782+0000 7f0167200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168101db0 0x7f016819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0167200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0168102760 0x7f016819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0167200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0168106e90 0x7f016819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0167200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016819f190 con 0x7f0168101db0 2024-02-26T06:21:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0167200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016819f300 con 0x7f0168102760 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0167200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016819f4c0 con 0x7f0168106e90 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168101db0 0x7f016819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168101db0 0x7f016819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49770/0 (socket says 172.21.15.49:49770) 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165e00700 1 -- 172.21.15.49:0/2862087844 learned_addr learned my addr 172.21.15.49:0/2862087844 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165400700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0168102760 0x7f016819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0166800700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0168106e90 0x7f016819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165400700 1 -- 172.21.15.49:0/2862087844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0168106e90 msgr2=0x7f016819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165400700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0168106e90 0x7f016819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165400700 1 -- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168101db0 msgr2=0x7f016819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165400700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168101db0 0x7f016819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165400700 1 -- 172.21.15.49:0/2862087844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0160009c40 con 0x7f0168102760 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.783+0000 7f0165e00700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0168101db0 0x7f016819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.784+0000 7f0165400700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0168102760 0x7f016819a7f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f01540098e0 tx=0x7f0154004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.784+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015401d070 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.784+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f015400b610 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.784+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01540095f0 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.784+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154022ea0 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.784+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01681a6e30 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.785+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0168103290 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.789+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0154022590 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.791+0000 7f015ea00700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f014c0a2260 0x7f014c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.791+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f01540cf770 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.791+0000 7f0165e00700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f014c0a2260 0x7f014c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.791+0000 7f015ea00700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] conn(0x7f014c0abe20 0x7f014c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.791+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 --> [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f014c0ae990 con 0x7f014c0abe20 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.791+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01540cfae0 con 0x7f0168102760 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.792+0000 7f0166800700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] conn(0x7f014c0abe20 0x7f014c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.792+0000 7f0165e00700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f014c0a2260 0x7f014c0a4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0158011550 tx=0x7f0158009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.792+0000 7f0166800700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] conn(0x7f014c0abe20 0x7f014c0ae2c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.793+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 <== osd.9 v2:172.21.15.49:6826/4117635859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f014c0ae990 con 0x7f014c0abe20 2024-02-26T06:21:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.812+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 --> [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0168111ed0 con 0x7f014c0abe20 2024-02-26T06:21:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.812+0000 7f015ea00700 1 -- 172.21.15.49:0/2862087844 <== osd.9 v2:172.21.15.49:6826/4117635859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0168111ed0 con 0x7f014c0abe20 2024-02-26T06:21:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] conn(0x7f014c0abe20 msgr2=0x7f014c0ae2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] conn(0x7f014c0abe20 0x7f014c0ae2c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f014c0a2260 msgr2=0x7f014c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f014c0a2260 0x7f014c0a4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0158011550 tx=0x7f0158009260 comp rx=0 tx=0).stop 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0168102760 msgr2=0x7f016819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 --2- 172.21.15.49:0/2862087844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0168102760 0x7f016819a7f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f01540098e0 tx=0x7f0154004730 comp rx=0 tx=0).stop 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0166800700 1 -- 172.21.15.49:0/2862087844 reap_dead start 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 shutdown_connections 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 >> 172.21.15.49:0/2862087844 conn(0x7f01680fd640 msgr2=0x7f016810d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 shutdown_connections 2024-02-26T06:21:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:28.813+0000 7f0167200700 1 -- 172.21.15.49:0/2862087844 wait complete. 2024-02-26T06:21:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:29 smithi080 ceph-mon[36206]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T06:21:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:29 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3828554645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T06:21:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:29 smithi115 ceph-mon[36426]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T06:21:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:29 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3828554645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 -- 172.21.15.49:0/3956480982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f68102680 msgr2=0x7f0f6810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 --2- 172.21.15.49:0/3956480982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f68102680 0x7f0f6810eac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009fc0 tx=0x7f0f6000b330 comp rx=0 tx=0).stop 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 -- 172.21.15.49:0/3956480982 shutdown_connections 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 --2- 172.21.15.49:0/3956480982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f68102680 0x7f0f6810eac0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 --2- 172.21.15.49:0/3956480982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f68107ca0 0x7f0f68108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 --2- 172.21.15.49:0/3956480982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f68101cc0 0x7f0f68102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 -- 172.21.15.49:0/3956480982 >> 172.21.15.49:0/3956480982 conn(0x7f0f68078da0 msgr2=0x7f0f680791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 -- 172.21.15.49:0/3956480982 shutdown_connections 2024-02-26T06:21:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.865+0000 7f0f6dc00700 1 -- 172.21.15.49:0/3956480982 wait complete. 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.866+0000 7f0f6dc00700 1 Processor -- start 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.866+0000 7f0f6dc00700 1 -- start start 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f6dc00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f68101cc0 0x7f0f68073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f6dc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f68102680 0x7f0f680744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f6dc00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f68107ca0 0x7f0f680749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f6dc00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f681a6650 con 0x7f0f68101cc0 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f6dc00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f681a67c0 con 0x7f0f68107ca0 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f6dc00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f681a6980 con 0x7f0f68102680 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f66a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f68101cc0 0x7f0f68073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f66a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f68101cc0 0x7f0f68073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40010/0 (socket says 172.21.15.49:40010) 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f66a00700 1 -- 172.21.15.49:0/1923749125 learned_addr learned my addr 172.21.15.49:0/1923749125 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f67400700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f68107ca0 0x7f0f680749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.867+0000 7f0f66000700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f68102680 0x7f0f680744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f66a00700 1 -- 172.21.15.49:0/1923749125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f68102680 msgr2=0x7f0f680744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f66a00700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f68102680 0x7f0f680744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f66a00700 1 -- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f68107ca0 msgr2=0x7f0f680749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f66a00700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f68107ca0 0x7f0f680749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f66a00700 1 -- 172.21.15.49:0/1923749125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54009520 con 0x7f0f68101cc0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f67400700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f68107ca0 0x7f0f680749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f66a00700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f68101cc0 0x7f0f68073f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800e9c0 tx=0x7f0f5800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58004a30 con 0x7f0f68101cc0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f5800cca0 con 0x7f0f68101cc0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f60009c40 con 0x7f0f68101cc0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.868+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58005580 con 0x7f0f68101cc0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.869+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f681a6fd0 con 0x7f0f68101cc0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.870+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f68112010 con 0x7f0f68101cc0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.872+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0f58004c20 con 0x7f0f68101cc0 2024-02-26T06:21:30.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.873+0000 7f0f5f400700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0f4c0a21e0 0x7f0f4c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.873+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0f58014070 con 0x7f0f68101cc0 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.873+0000 7f0f66000700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0f4c0a21e0 0x7f0f4c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.874+0000 7f0f5f400700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] conn(0x7f0f4c0abda0 0x7f0f4c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.874+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 --> [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f4c0ae910 con 0x7f0f4c0abda0 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.874+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f580ce4f0 con 0x7f0f68101cc0 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.874+0000 7f0f67400700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] conn(0x7f0f4c0abda0 0x7f0f4c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.874+0000 7f0f66000700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0f4c0a21e0 0x7f0f4c0a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f68079310 tx=0x7f0f54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.875+0000 7f0f67400700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] conn(0x7f0f4c0abda0 0x7f0f4c0ae240 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.876+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 <== osd.10 v2:172.21.15.80:6824/1162870008 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0f4c0ae910 con 0x7f0f4c0abda0 2024-02-26T06:21:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.896+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 --> [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f68062380 con 0x7f0f4c0abda0 2024-02-26T06:21:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.896+0000 7f0f5f400700 1 -- 172.21.15.49:0/1923749125 <== osd.10 v2:172.21.15.80:6824/1162870008 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0f68062380 con 0x7f0f4c0abda0 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] conn(0x7f0f4c0abda0 msgr2=0x7f0f4c0ae240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f6dc00700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] conn(0x7f0f4c0abda0 0x7f0f4c0ae240 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0f4c0a21e0 msgr2=0x7f0f4c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f6dc00700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0f4c0a21e0 0x7f0f4c0a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f68079310 tx=0x7f0f54009260 comp rx=0 tx=0).stop 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f68101cc0 msgr2=0x7f0f68073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f6dc00700 1 --2- 172.21.15.49:0/1923749125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f68101cc0 0x7f0f68073f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800e9c0 tx=0x7f0f5800ecd0 comp rx=0 tx=0).stop 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f67400700 1 -- 172.21.15.49:0/1923749125 reap_dead start 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 shutdown_connections 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.897+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 >> 172.21.15.49:0/1923749125 conn(0x7f0f68078da0 msgr2=0x7f0f68100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.898+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 shutdown_connections 2024-02-26T06:21:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:30.898+0000 7f0f6dc00700 1 -- 172.21.15.49:0/1923749125 wait complete. 2024-02-26T06:21:30.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:30 smithi049 ceph-mon[31696]: pgmap v349: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-02-26T06:21:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:30 smithi080 ceph-mon[36206]: pgmap v349: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-02-26T06:21:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:30 smithi115 ceph-mon[36426]: pgmap v349: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-02-26T06:21:32.207 INFO:teuthology.orchestra.run.smithi049.stdout:347892351000 2024-02-26T06:21:32.208 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-26T06:21:33.070 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574930 for osd.1 2024-02-26T06:21:33.070 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:32 smithi080 ceph-mon[36206]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-02-26T06:21:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:32 smithi115 ceph-mon[36426]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-02-26T06:21:33.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 -- 172.21.15.49:0/3918902621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e7c106ff0 msgr2=0x7f3e7c10eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 --2- 172.21.15.49:0/3918902621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e7c106ff0 0x7f3e7c10eb70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400b0a0 tx=0x7f3e7400b3b0 comp rx=0 tx=0).stop 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 -- 172.21.15.49:0/3918902621 shutdown_connections 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 --2- 172.21.15.49:0/3918902621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e7c106ff0 0x7f3e7c10eb70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 --2- 172.21.15.49:0/3918902621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c06a410 0x7f3e7c06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 --2- 172.21.15.49:0/3918902621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e7c06adc0 0x7f3e7c1068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 -- 172.21.15.49:0/3918902621 >> 172.21.15.49:0/3918902621 conn(0x7f3e7c076bc0 msgr2=0x7f3e7c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 -- 172.21.15.49:0/3918902621 shutdown_connections 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.185+0000 7f3e80a00700 1 -- 172.21.15.49:0/3918902621 wait complete. 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.186+0000 7f3e80a00700 1 Processor -- start 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.186+0000 7f3e80a00700 1 -- start start 2024-02-26T06:21:33.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e80a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e7c06a410 0x7f3e7c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:33.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e80a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e7c06adc0 0x7f3e7c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:33.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e80a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c106ff0 0x7f3e7c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:33.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e80a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19d190 con 0x7f3e7c106ff0 2024-02-26T06:21:33.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e80a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19d300 con 0x7f3e7c06a410 2024-02-26T06:21:33.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e80a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19d4c0 con 0x7f3e7c06adc0 2024-02-26T06:21:33.360 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:32 smithi049 ceph-mon[31696]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e7c06a410 0x7f3e7c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e7c06a410 0x7f3e7c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36104/0 (socket says 172.21.15.49:36104) 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e7c06adc0 0x7f3e7c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79e00700 1 -- 172.21.15.49:0/3321776266 learned_addr learned my addr 172.21.15.49:0/3321776266 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e7a800700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c106ff0 0x7f3e7c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79e00700 1 -- 172.21.15.49:0/3321776266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e7c06adc0 msgr2=0x7f3e7c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79e00700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e7c06adc0 0x7f3e7c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79e00700 1 -- 172.21.15.49:0/3321776266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c106ff0 msgr2=0x7f3e7c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79e00700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c106ff0 0x7f3e7c19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.187+0000 7f3e79e00700 1 -- 172.21.15.49:0/3321776266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74009d00 con 0x7f3e7c06a410 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.188+0000 7f3e79e00700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e7c06a410 0x7f3e7c198300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00e9a0 tx=0x7f3e6c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:33.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.188+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c009790 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.188+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e6c010430 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.188+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c018680 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.188+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c19d7a0 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.189+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c100d70 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.190+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e7c1033a0 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.192+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f3e6c010910 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.193+0000 7f3e72a00700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e600a2260 0x7f3e600a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.193+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3e6c014070 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.193+0000 7f3e72a00700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] conn(0x7f3e600abe20 0x7f3e600ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.193+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 --> [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e600ae990 con 0x7f3e600abe20 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.193+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e6c0d04a0 con 0x7f3e7c06a410 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.194+0000 7f3e79400700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e600a2260 0x7f3e600a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.194+0000 7f3e7a800700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] conn(0x7f3e600abe20 0x7f3e600ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:33.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.194+0000 7f3e7a800700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] conn(0x7f3e600abe20 0x7f3e600ae2c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:33.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.194+0000 7f3e79400700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e600a2260 0x7f3e600a4720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c100770 tx=0x7f3e68012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:33.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.195+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 <== osd.5 v2:172.21.15.115:6808/1277586468 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3e600ae990 con 0x7f3e600abe20 2024-02-26T06:21:33.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.214+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 --> [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3e7c112000 con 0x7f3e600abe20 2024-02-26T06:21:33.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.215+0000 7f3e72a00700 1 -- 172.21.15.49:0/3321776266 <== osd.5 v2:172.21.15.115:6808/1277586468 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3e7c112000 con 0x7f3e600abe20 2024-02-26T06:21:33.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 >> [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] conn(0x7f3e600abe20 msgr2=0x7f3e600ae2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] conn(0x7f3e600abe20 0x7f3e600ae2c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e600a2260 msgr2=0x7f3e600a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e600a2260 0x7f3e600a4720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c100770 tx=0x7f3e68012040 comp rx=0 tx=0).stop 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e7c06a410 msgr2=0x7f3e7c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 --2- 172.21.15.49:0/3321776266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e7c06a410 0x7f3e7c198300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00e9a0 tx=0x7f3e6c00ecb0 comp rx=0 tx=0).stop 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e7a800700 1 -- 172.21.15.49:0/3321776266 reap_dead start 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 shutdown_connections 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 >> 172.21.15.49:0/3321776266 conn(0x7f3e7c076bc0 msgr2=0x7f3e7c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 shutdown_connections 2024-02-26T06:21:33.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:32.216+0000 7f3e80a00700 1 -- 172.21.15.49:0/3321776266 wait complete. 2024-02-26T06:21:33.513 INFO:teuthology.orchestra.run.smithi049.stdout:292057776164 2024-02-26T06:21:33.513 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-26T06:21:33.628 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:33.629 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:34.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:33 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:34.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:33 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:34.253 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:33 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:34.668 INFO:teuthology.orchestra.run.smithi049.stdout:111669149766 2024-02-26T06:21:34.668 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-26T06:21:35.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:34 smithi049 ceph-mon[31696]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-02-26T06:21:35.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:34 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:34 smithi080 ceph-mon[36206]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-02-26T06:21:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:34 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:34 smithi115 ceph-mon[36426]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-02-26T06:21:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:34 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:21:35.341 INFO:teuthology.orchestra.run.smithi049.stdout:167503724605 2024-02-26T06:21:35.342 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-26T06:21:36.936 INFO:teuthology.orchestra.run.smithi049.stdout:313532612639 2024-02-26T06:21:36.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-26T06:21:37.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:36 smithi049 ceph-mon[31696]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-02-26T06:21:37.022 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:36 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1346687623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T06:21:37.023 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:36 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/166264535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T06:21:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:36 smithi080 ceph-mon[36206]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-02-26T06:21:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:36 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1346687623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T06:21:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:36 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/166264535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T06:21:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:36 smithi115 ceph-mon[36426]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-02-26T06:21:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:36 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1346687623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T06:21:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:36 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/166264535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T06:21:37.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 -- 172.21.15.49:0/1218558038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 msgr2=0x7f58e4106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:37.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1218558038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 0x7f58e4106910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009990 tx=0x7f58d0009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 -- 172.21.15.49:0/1218558038 shutdown_connections 2024-02-26T06:21:37.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1218558038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58e4106ff0 0x7f58e410ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1218558038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4101f20 0x7f58e4102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1218558038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 0x7f58e4106910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 -- 172.21.15.49:0/1218558038 >> 172.21.15.49:0/1218558038 conn(0x7f58e4078dd0 msgr2=0x7f58e40791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 -- 172.21.15.49:0/1218558038 shutdown_connections 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.166+0000 7f58e8a00700 1 -- 172.21.15.49:0/1218558038 wait complete. 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.167+0000 7f58e8a00700 1 Processor -- start 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.167+0000 7f58e8a00700 1 -- start start 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4101f20 0x7f58e419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e8a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 0x7f58e419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e8a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58e4106ff0 0x7f58e419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e8a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e419f350 con 0x7f58e4106ff0 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e8a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e419f4c0 con 0x7f58e41028d0 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e8a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e419f680 con 0x7f58e4101f20 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e2800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58e4106ff0 0x7f58e419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e1e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4101f20 0x7f58e419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:37.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e1400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 0x7f58e419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e2800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58e4106ff0 0x7f58e419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40032/0 (socket says 172.21.15.49:40032) 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.168+0000 7f58e2800700 1 -- 172.21.15.49:0/1346687623 learned_addr learned my addr 172.21.15.49:0/1346687623 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58e1400700 1 -- 172.21.15.49:0/1346687623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4101f20 msgr2=0x7f58e419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58e1400700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4101f20 0x7f58e419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58e1400700 1 -- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58e4106ff0 msgr2=0x7f58e419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58e1400700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58e4106ff0 0x7f58e419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58e1400700 1 -- 172.21.15.49:0/1346687623 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009c40 con 0x7f58e41028d0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58e1400700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 0x7f58e419a9b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f58d00098e0 tx=0x7f58d0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58daa00700 1 -- 172.21.15.49:0/1346687623 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d001d070 con 0x7f58e41028d0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58daa00700 1 -- 172.21.15.49:0/1346687623 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58d000b610 con 0x7f58e41028d0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d00095f0 con 0x7f58e41028d0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58daa00700 1 -- 172.21.15.49:0/1346687623 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0022e60 con 0x7f58e41028d0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.169+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e41a6ff0 con 0x7f58e41028d0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.171+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e41033f0 con 0x7f58e41028d0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.172+0000 7f58daa00700 1 -- 172.21.15.49:0/1346687623 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f58d0022470 con 0x7f58e41028d0 2024-02-26T06:21:37.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.174+0000 7f58daa00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f58c80a2260 0x7f58c80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:37.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.174+0000 7f58daa00700 1 -- 172.21.15.49:0/1346687623 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f58d00cf170 con 0x7f58e41028d0 2024-02-26T06:21:37.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.174+0000 7f58e1e00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f58c80a2260 0x7f58c80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:37.755 INFO:teuthology.orchestra.run.smithi049.stdout:77309411406 2024-02-26T06:21:37.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.174+0000 7f58e1e00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f58c80a2260 0x7f58c80a4720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009c60 tx=0x7f58d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:37.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.178+0000 7f58daa00700 1 -- 172.21.15.49:0/1346687623 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58d005dd20 con 0x7f58e41028d0 2024-02-26T06:21:37.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.410+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f58e419b830 con 0x7f58e41028d0 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.411+0000 7f58daa00700 1 -- 172.21.15.49:0/1346687623 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f58d0098120 con 0x7f58e41028d0 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f58c80a2260 msgr2=0x7f58c80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f58c80a2260 0x7f58c80a4720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009c60 tx=0x7f58d4009260 comp rx=0 tx=0).stop 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 msgr2=0x7f58e419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 0x7f58e419a9b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f58d00098e0 tx=0x7f58d0004730 comp rx=0 tx=0).stop 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 shutdown_connections 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f58c80a2260 0x7f58c80a4720 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58e4106ff0 0x7f58e419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4101f20 0x7f58e419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 --2- 172.21.15.49:0/1346687623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58e41028d0 0x7f58e419a9b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 >> 172.21.15.49:0/1346687623 conn(0x7f58e4078dd0 msgr2=0x7f58e41081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.414+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 shutdown_connections 2024-02-26T06:21:37.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.415+0000 7f58e8a00700 1 -- 172.21.15.49:0/1346687623 wait complete. 2024-02-26T06:21:38.081 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 -- 172.21.15.49:0/2454599915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 msgr2=0x7f5b8410ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 --2- 172.21.15.49:0/2454599915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 0x7f5b8410ea60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009f90 tx=0x7f5b7c00b330 comp rx=0 tx=0).stop 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 -- 172.21.15.49:0/2454599915 shutdown_connections 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 --2- 172.21.15.49:0/2454599915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 0x7f5b8410ea60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 --2- 172.21.15.49:0/2454599915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b84101db0 0x7f5b84102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 --2- 172.21.15.49:0/2454599915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b84102760 0x7f5b841067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 -- 172.21.15.49:0/2454599915 >> 172.21.15.49:0/2454599915 conn(0x7f5b840fd640 msgr2=0x7f5b840ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 -- 172.21.15.49:0/2454599915 shutdown_connections 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.509+0000 7f5b89200700 1 -- 172.21.15.49:0/2454599915 wait complete. 2024-02-26T06:21:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.510+0000 7f5b89200700 1 Processor -- start 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.510+0000 7f5b89200700 1 -- start start 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b89200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b84101db0 0x7f5b8419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b89200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b84102760 0x7f5b8419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b89200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 0x7f5b8419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b89200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8419f2c0 con 0x7f5b84101db0 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b89200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8419f430 con 0x7f5b84106e90 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b89200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8419f5f0 con 0x7f5b84102760 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b84101db0 0x7f5b8419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 0x7f5b8419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 0x7f5b8419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36152/0 (socket says 172.21.15.49:36152) 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83e00700 1 -- 172.21.15.49:0/166264535 learned_addr learned my addr 172.21.15.49:0/166264535 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b82a00700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b84102760 0x7f5b8419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83e00700 1 -- 172.21.15.49:0/166264535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b84102760 msgr2=0x7f5b8419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83e00700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b84102760 0x7f5b8419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83e00700 1 -- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b84101db0 msgr2=0x7f5b8419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83e00700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b84101db0 0x7f5b8419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b83e00700 1 -- 172.21.15.49:0/166264535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b74009520 con 0x7f5b84106e90 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.512+0000 7f5b83400700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b84101db0 0x7f5b8419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.511+0000 7f5b82a00700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b84102760 0x7f5b8419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.512+0000 7f5b83e00700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 0x7f5b8419ecb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c004fb0 tx=0x7f5b7c009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.512+0000 7f5b7be00700 1 -- 172.21.15.49:0/166264535 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00e040 con 0x7f5b84106e90 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.512+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b700095f0 con 0x7f5b84106e90 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.512+0000 7f5b7be00700 1 -- 172.21.15.49:0/166264535 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c01d470 con 0x7f5b84106e90 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.512+0000 7f5b7be00700 1 -- 172.21.15.49:0/166264535 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0079e0 con 0x7f5b84106e90 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.512+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c009c40 con 0x7f5b84106e90 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.513+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b84068930 con 0x7f5b84106e90 2024-02-26T06:21:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.515+0000 7f5b7be00700 1 -- 172.21.15.49:0/166264535 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f5b7c01d950 con 0x7f5b84106e90 2024-02-26T06:21:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.517+0000 7f5b7be00700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5b680a21e0 0x7f5b680a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.517+0000 7f5b7be00700 1 -- 172.21.15.49:0/166264535 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f5b7c0cf500 con 0x7f5b84106e90 2024-02-26T06:21:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.517+0000 7f5b83400700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5b680a21e0 0x7f5b680a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.518+0000 7f5b83400700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5b680a21e0 0x7f5b680a46a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009fd0 tx=0x7f5b74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.519+0000 7f5b7be00700 1 -- 172.21.15.49:0/166264535 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b7c098390 con 0x7f5b84106e90 2024-02-26T06:21:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.750+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f5b84111c30 con 0x7f5b84106e90 2024-02-26T06:21:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.751+0000 7f5b7be00700 1 -- 172.21.15.49:0/166264535 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b7c01d5e0 con 0x7f5b84106e90 2024-02-26T06:21:38.217 INFO:teuthology.orchestra.run.smithi049.stdout:257698037805 2024-02-26T06:21:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5b680a21e0 msgr2=0x7f5b680a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5b680a21e0 0x7f5b680a46a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009fd0 tx=0x7f5b74009260 comp rx=0 tx=0).stop 2024-02-26T06:21:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 msgr2=0x7f5b8419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 0x7f5b8419ecb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c004fb0 tx=0x7f5b7c009490 comp rx=0 tx=0).stop 2024-02-26T06:21:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 shutdown_connections 2024-02-26T06:21:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5b680a21e0 0x7f5b680a46a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b84106e90 0x7f5b8419ecb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b84101db0 0x7f5b8419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 --2- 172.21.15.49:0/166264535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b84102760 0x7f5b8419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.754+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 >> 172.21.15.49:0/166264535 conn(0x7f5b840fd640 msgr2=0x7f5b8410d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.755+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 shutdown_connections 2024-02-26T06:21:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:36.755+0000 7f5b89200700 1 -- 172.21.15.49:0/166264535 wait complete. 2024-02-26T06:21:39.016 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411406 for osd.2 2024-02-26T06:21:39.017 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:39.066 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:38 smithi049 ceph-mon[31696]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 6.2 KiB/s, 1 objects/s recovering 2024-02-26T06:21:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:38 smithi080 ceph-mon[36206]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 6.2 KiB/s, 1 objects/s recovering 2024-02-26T06:21:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:38 smithi115 ceph-mon[36426]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 6.2 KiB/s, 1 objects/s recovering 2024-02-26T06:21:41.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:40 smithi049 ceph-mon[31696]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.6 KiB/s, 1 objects/s recovering 2024-02-26T06:21:41.224 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:40 smithi080 ceph-mon[36206]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.6 KiB/s, 1 objects/s recovering 2024-02-26T06:21:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:40 smithi115 ceph-mon[36426]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.6 KiB/s, 1 objects/s recovering 2024-02-26T06:21:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 -- 172.21.15.49:0/3464989389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 msgr2=0x7f19ec100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3464989389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 0x7f19ec100ff0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f19d80098f0 tx=0x7f19d8009c00 comp rx=0 tx=0).stop 2024-02-26T06:21:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 -- 172.21.15.49:0/3464989389 shutdown_connections 2024-02-26T06:21:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3464989389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19ec101530 0x7f19ec10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3464989389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19ec106b80 0x7f19ec106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3464989389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 0x7f19ec100ff0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 -- 172.21.15.49:0/3464989389 >> 172.21.15.49:0/3464989389 conn(0x7f19ec077aa0 msgr2=0x7f19ec077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 -- 172.21.15.49:0/3464989389 shutdown_connections 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.043+0000 7f19f0e00700 1 -- 172.21.15.49:0/3464989389 wait complete. 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.045+0000 7f19f0e00700 1 Processor -- start 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.045+0000 7f19f0e00700 1 -- start start 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.045+0000 7f19f0e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 0x7f19ec199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.045+0000 7f19f0e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19ec101530 0x7f19ec1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19f0e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19ec106b80 0x7f19ec1a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19f0e00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec199e70 con 0x7f19ec100b70 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19f0e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1a4650 con 0x7f19ec106b80 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19f0e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1a4790 con 0x7f19ec101530 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 0x7f19ec199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 0x7f19ec199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48850/0 (socket says 172.21.15.49:48850) 2024-02-26T06:21:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9e00700 1 -- 172.21.15.49:0/3673279536 learned_addr learned my addr 172.21.15.49:0/3673279536 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9400700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19ec101530 0x7f19ec1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19ea800700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19ec106b80 0x7f19ec1a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9e00700 1 -- 172.21.15.49:0/3673279536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19ec101530 msgr2=0x7f19ec1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19ec101530 0x7f19ec1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9e00700 1 -- 172.21.15.49:0/3673279536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19ec106b80 msgr2=0x7f19ec1a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19ec106b80 0x7f19ec1a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.046+0000 7f19e9e00700 1 -- 172.21.15.49:0/3673279536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d8009520 con 0x7f19ec100b70 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.047+0000 7f19e9e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 0x7f19ec199090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e9d0 tx=0x7f19dc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.047+0000 7f19e2a00700 1 -- 172.21.15.49:0/3673279536 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc009790 con 0x7f19ec100b70 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.047+0000 7f19e2a00700 1 -- 172.21.15.49:0/3673279536 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19dc00ee70 con 0x7f19ec100b70 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.047+0000 7f19e2a00700 1 -- 172.21.15.49:0/3673279536 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc018680 con 0x7f19ec100b70 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.047+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec1a4a40 con 0x7f19ec100b70 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.047+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec1a4ec0 con 0x7f19ec100b70 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.048+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec109020 con 0x7f19ec100b70 2024-02-26T06:21:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.050+0000 7f19e2a00700 1 -- 172.21.15.49:0/3673279536 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f19dc00cbd0 con 0x7f19ec100b70 2024-02-26T06:21:43.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.051+0000 7f19e2a00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19d00a2260 0x7f19d00a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:43.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.052+0000 7f19e2a00700 1 -- 172.21.15.49:0/3673279536 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f19dc014070 con 0x7f19ec100b70 2024-02-26T06:21:43.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.052+0000 7f19e9400700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19d00a2260 0x7f19d00a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:43.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.053+0000 7f19e9400700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19d00a2260 0x7f19d00a4720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009810 tx=0x7f19d8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:43.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.056+0000 7f19e2a00700 1 -- 172.21.15.49:0/3673279536 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19dc052d20 con 0x7f19ec100b70 2024-02-26T06:21:43.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:42 smithi049 ceph-mon[31696]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.0 KiB/s, 1 objects/s recovering 2024-02-26T06:21:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.293+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f19ec110bd0 con 0x7f19ec100b70 2024-02-26T06:21:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.294+0000 7f19e2a00700 1 -- 172.21.15.49:0/3673279536 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19dc096d40 con 0x7f19ec100b70 2024-02-26T06:21:43.295 INFO:teuthology.orchestra.run.smithi049.stdout:223338299442 2024-02-26T06:21:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19d00a2260 msgr2=0x7f19d00a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19d00a2260 0x7f19d00a4720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009810 tx=0x7f19d8019040 comp rx=0 tx=0).stop 2024-02-26T06:21:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 msgr2=0x7f19ec199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 0x7f19ec199090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e9d0 tx=0x7f19dc00ece0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 shutdown_connections 2024-02-26T06:21:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19d00a2260 0x7f19d00a4720 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19ec106b80 0x7f19ec1a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec100b70 0x7f19ec199090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 --2- 172.21.15.49:0/3673279536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19ec101530 0x7f19ec1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.297+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 >> 172.21.15.49:0/3673279536 conn(0x7f19ec077aa0 msgr2=0x7f19ec0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.298+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 shutdown_connections 2024-02-26T06:21:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:43.298+0000 7f19f0e00700 1 -- 172.21.15.49:0/3673279536 wait complete. 2024-02-26T06:21:43.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:42 smithi080 ceph-mon[36206]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.0 KiB/s, 1 objects/s recovering 2024-02-26T06:21:43.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:42 smithi115 ceph-mon[36426]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.0 KiB/s, 1 objects/s recovering 2024-02-26T06:21:44.199 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037800 got 257698037805 for osd.8 2024-02-26T06:21:44.199 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:44.275 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:43 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3673279536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T06:21:44.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:43 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3673279536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T06:21:44.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:43 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3673279536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T06:21:45.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:44 smithi080 ceph-mon[36206]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-02-26T06:21:45.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:44 smithi115 ceph-mon[36426]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-02-26T06:21:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:44 smithi049 ceph-mon[31696]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-02-26T06:21:45.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 -- 172.21.15.49:0/2314289706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c104a70 msgr2=0x7f4e9c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 --2- 172.21.15.49:0/2314289706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c104a70 0x7f4e9c108ac0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009990 tx=0x7f4e88009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 -- 172.21.15.49:0/2314289706 shutdown_connections 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 --2- 172.21.15.49:0/2314289706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e9c1091a0 0x7f4e9c110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 --2- 172.21.15.49:0/2314289706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e9c1040c0 0x7f4e9c1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 --2- 172.21.15.49:0/2314289706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c104a70 0x7f4e9c108ac0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 -- 172.21.15.49:0/2314289706 >> 172.21.15.49:0/2314289706 conn(0x7f4e9c0ff950 msgr2=0x7f4e9c101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 -- 172.21.15.49:0/2314289706 shutdown_connections 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.818+0000 7f4ea1400700 1 -- 172.21.15.49:0/2314289706 wait complete. 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.819+0000 7f4ea1400700 1 Processor -- start 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.819+0000 7f4ea1400700 1 -- start start 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.819+0000 7f4ea1400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e9c1040c0 0x7f4e9c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:45.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4ea1400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e9c104a70 0x7f4e9c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4ea1400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c1091a0 0x7f4e9c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4ea1400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c19f280 con 0x7f4e9c104a70 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4ea1400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c19f3f0 con 0x7f4e9c1091a0 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4ea1400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c19f5b0 con 0x7f4e9c1040c0 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9a800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e9c1040c0 0x7f4e9c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e99e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e9c104a70 0x7f4e9c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9a800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e9c1040c0 0x7f4e9c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:50176/0 (socket says 172.21.15.49:50176) 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9a800700 1 -- 172.21.15.49:0/856256504 learned_addr learned my addr 172.21.15.49:0/856256504 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9b200700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c1091a0 0x7f4e9c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9a800700 1 -- 172.21.15.49:0/856256504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c1091a0 msgr2=0x7f4e9c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9a800700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c1091a0 0x7f4e9c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9a800700 1 -- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e9c104a70 msgr2=0x7f4e9c19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9a800700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e9c104a70 0x7f4e9c19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9a800700 1 -- 172.21.15.49:0/856256504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880095f0 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e99e00700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e9c104a70 0x7f4e9c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.820+0000 7f4e9b200700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c1091a0 0x7f4e9c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.821+0000 7f4e9a800700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e9c1040c0 0x7f4e9c19a3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00e9a0 tx=0x7f4e8c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.821+0000 7f4e93400700 1 -- 172.21.15.49:0/856256504 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c009790 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.821+0000 7f4e93400700 1 -- 172.21.15.49:0/856256504 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e8c010430 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.821+0000 7f4e93400700 1 -- 172.21.15.49:0/856256504 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c018680 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.821+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c19f890 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.821+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c1a70f0 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.822+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c10c4c0 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.824+0000 7f4e93400700 1 -- 172.21.15.49:0/856256504 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4e8c010910 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.826+0000 7f4e93400700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4e800a2260 0x7f4e800a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.826+0000 7f4e93400700 1 -- 172.21.15.49:0/856256504 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f4e8c014070 con 0x7f4e9c1040c0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.826+0000 7f4e99e00700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4e800a2260 0x7f4e800a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:45.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.827+0000 7f4e99e00700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4e800a2260 0x7f4e800a4720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4e88006010 tx=0x7f4e88005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:44.828+0000 7f4e93400700 1 -- 172.21.15.49:0/856256504 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e8c096de0 con 0x7f4e9c1040c0 2024-02-26T06:21:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.061+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4e9c19b730 con 0x7f4e9c1040c0 2024-02-26T06:21:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.062+0000 7f4e93400700 1 -- 172.21.15.49:0/856256504 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e8c0105a0 con 0x7f4e9c1040c0 2024-02-26T06:21:45.586 INFO:teuthology.orchestra.run.smithi049.stdout:34359738458 2024-02-26T06:21:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4e800a2260 msgr2=0x7f4e800a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4e800a2260 0x7f4e800a4720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4e88006010 tx=0x7f4e88005960 comp rx=0 tx=0).stop 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e9c1040c0 msgr2=0x7f4e9c19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e9c1040c0 0x7f4e9c19a3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00e9a0 tx=0x7f4e8c00ecb0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 shutdown_connections 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f4e800a2260 0x7f4e800a4720 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c1091a0 0x7f4e9c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e9c1040c0 0x7f4e9c19a3a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 --2- 172.21.15.49:0/856256504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e9c104a70 0x7f4e9c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 >> 172.21.15.49:0/856256504 conn(0x7f4e9c0ff950 msgr2=0x7f4e9c101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.065+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 shutdown_connections 2024-02-26T06:21:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:45.066+0000 7f4ea1400700 1 -- 172.21.15.49:0/856256504 wait complete. 2024-02-26T06:21:46.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:45 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/856256504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T06:21:46.334 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:45 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/856256504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T06:21:46.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:45 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/856256504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T06:21:46.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299437 got 223338299442 for osd.7 2024-02-26T06:21:46.682 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:46.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 -- 172.21.15.49:0/3235790684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 msgr2=0x7ff4d810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 --2- 172.21.15.49:0/3235790684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 0x7ff4d810d920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009990 tx=0x7ff4d0009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 -- 172.21.15.49:0/3235790684 shutdown_connections 2024-02-26T06:21:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 --2- 172.21.15.49:0/3235790684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 0x7ff4d810d920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 --2- 172.21.15.49:0/3235790684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d8106a50 0x7ff4d8106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 --2- 172.21.15.49:0/3235790684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4d8100a50 0x7ff4d8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 -- 172.21.15.49:0/3235790684 >> 172.21.15.49:0/3235790684 conn(0x7ff4d80fc540 msgr2=0x7ff4d80fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 -- 172.21.15.49:0/3235790684 shutdown_connections 2024-02-26T06:21:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.959+0000 7ff4dda00700 1 -- 172.21.15.49:0/3235790684 wait complete. 2024-02-26T06:21:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.961+0000 7ff4dda00700 1 Processor -- start 2024-02-26T06:21:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.961+0000 7ff4dda00700 1 -- start start 2024-02-26T06:21:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.961+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4d8100a50 0x7ff4d8102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.961+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 0x7ff4d8102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d8106a50 0x7ff4d81030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4dda00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d81a6430 con 0x7ff4d8101410 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4dda00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d81a65a0 con 0x7ff4d8100a50 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4dda00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d81a6760 con 0x7ff4d8106a50 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4cf400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 0x7ff4d8102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4cf400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 0x7ff4d8102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48890/0 (socket says 172.21.15.49:48890) 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4d7e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4d8100a50 0x7ff4d8102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4cf400700 1 -- 172.21.15.49:0/156102703 learned_addr learned my addr 172.21.15.49:0/156102703 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.962+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d8106a50 0x7ff4d81030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4dd000700 1 -- 172.21.15.49:0/156102703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4d8100a50 msgr2=0x7ff4d8102620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4dd000700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4d8100a50 0x7ff4d8102620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4dd000700 1 -- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 msgr2=0x7ff4d8102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4dd000700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 0x7ff4d8102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4dd000700 1 -- 172.21.15.49:0/156102703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d00095f0 con 0x7ff4d8106a50 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4d7e00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4d8100a50 0x7ff4d8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4cf400700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 0x7ff4d8102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4dd000700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d8106a50 0x7ff4d81030a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009990 tx=0x7ff4d0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4d5600700 1 -- 172.21.15.49:0/156102703 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d001d070 con 0x7ff4d8106a50 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4d5600700 1 -- 172.21.15.49:0/156102703 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4d000b610 con 0x7ff4d8106a50 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.963+0000 7ff4d5600700 1 -- 172.21.15.49:0/156102703 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0022e90 con 0x7ff4d8106a50 2024-02-26T06:21:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.964+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d81a69e0 con 0x7ff4d8106a50 2024-02-26T06:21:46.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.964+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d81a6da0 con 0x7ff4d8106a50 2024-02-26T06:21:46.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.965+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d8068930 con 0x7ff4d8106a50 2024-02-26T06:21:46.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.972+0000 7ff4d5600700 1 -- 172.21.15.49:0/156102703 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff4d0022470 con 0x7ff4d8106a50 2024-02-26T06:21:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.974+0000 7ff4d5600700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4bc0a21e0 0x7ff4bc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.974+0000 7ff4d5600700 1 -- 172.21.15.49:0/156102703 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff4d00cf390 con 0x7ff4d8106a50 2024-02-26T06:21:46.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.974+0000 7ff4d7e00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4bc0a21e0 0x7ff4bc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:46.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.975+0000 7ff4d7e00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4bc0a21e0 0x7ff4bc0a46a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009480 tx=0x7ff4c800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:46.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:46.975+0000 7ff4d5600700 1 -- 172.21.15.49:0/156102703 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4d0098220 con 0x7ff4d8106a50 2024-02-26T06:21:47.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.205+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff4d8110bb0 con 0x7ff4d8106a50 2024-02-26T06:21:47.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.206+0000 7ff4d5600700 1 -- 172.21.15.49:0/156102703 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff4d0027020 con 0x7ff4d8106a50 2024-02-26T06:21:47.208 INFO:teuthology.orchestra.run.smithi049.stdout:146028888133 2024-02-26T06:21:47.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4bc0a21e0 msgr2=0x7ff4bc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:47.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4bc0a21e0 0x7ff4bc0a46a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009480 tx=0x7ff4c800b040 comp rx=0 tx=0).stop 2024-02-26T06:21:47.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d8106a50 msgr2=0x7ff4d81030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:47.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d8106a50 0x7ff4d81030a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009990 tx=0x7ff4d0005740 comp rx=0 tx=0).stop 2024-02-26T06:21:47.210 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:46 smithi049 ceph-mon[31696]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-02-26T06:21:47.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 shutdown_connections 2024-02-26T06:21:47.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff4bc0a21e0 0x7ff4bc0a46a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:47.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d8106a50 0x7ff4d81030a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:47.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4d8100a50 0x7ff4d8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:47.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 --2- 172.21.15.49:0/156102703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8101410 0x7ff4d8102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:47.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.209+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 >> 172.21.15.49:0/156102703 conn(0x7ff4d80fc540 msgr2=0x7ff4d8104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:47.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.210+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 shutdown_connections 2024-02-26T06:21:47.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:47.210+0000 7ff4dda00700 1 -- 172.21.15.49:0/156102703 wait complete. 2024-02-26T06:21:47.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:46 smithi080 ceph-mon[36206]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-02-26T06:21:47.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:46 smithi115 ceph-mon[36426]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-02-26T06:21:47.384 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738458 for osd.0 2024-02-26T06:21:47.384 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:48.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:47 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/156102703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T06:21:48.256 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:48.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888126 got 146028888133 for osd.4 2024-02-26T06:21:48.317 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:47 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/156102703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T06:21:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:47 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/156102703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T06:21:48.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 -- 172.21.15.49:0/3633104128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347406adc0 msgr2=0x7f34741068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:48.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 --2- 172.21.15.49:0/3633104128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347406adc0 0x7f34741068c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3460009990 tx=0x7f3460009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 -- 172.21.15.49:0/3633104128 shutdown_connections 2024-02-26T06:21:48.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 --2- 172.21.15.49:0/3633104128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3474106ff0 0x7f347410eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 --2- 172.21.15.49:0/3633104128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f347406a410 0x7f347406a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 --2- 172.21.15.49:0/3633104128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347406adc0 0x7f34741068c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 -- 172.21.15.49:0/3633104128 >> 172.21.15.49:0/3633104128 conn(0x7f3474076bc0 msgr2=0x7f3474076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:48.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 -- 172.21.15.49:0/3633104128 shutdown_connections 2024-02-26T06:21:48.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.552+0000 7f347b000700 1 -- 172.21.15.49:0/3633104128 wait complete. 2024-02-26T06:21:48.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.553+0000 7f347b000700 1 Processor -- start 2024-02-26T06:21:48.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.554+0000 7f347b000700 1 -- start start 2024-02-26T06:21:48.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.554+0000 7f347b000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347406a410 0x7f34741981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:48.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.554+0000 7f347b000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f347406adc0 0x7f3474198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:48.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.554+0000 7f347b000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3474106ff0 0x7f347419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:48.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.554+0000 7f347b000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347419d0d0 con 0x7f347406adc0 2024-02-26T06:21:48.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.554+0000 7f347b000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347419d240 con 0x7f3474106ff0 2024-02-26T06:21:48.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.554+0000 7f347b000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347419d400 con 0x7f347406a410 2024-02-26T06:21:48.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.555+0000 7f3473400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f347406adc0 0x7f3474198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:48.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.555+0000 7f3473400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f347406adc0 0x7f3474198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48898/0 (socket says 172.21.15.49:48898) 2024-02-26T06:21:48.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.555+0000 7f3473400700 1 -- 172.21.15.49:0/1919607074 learned_addr learned my addr 172.21.15.49:0/1919607074 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.555+0000 7f3473e00700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347406a410 0x7f34741981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.555+0000 7f3479000700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3474106ff0 0x7f347419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.555+0000 7f3473400700 1 -- 172.21.15.49:0/1919607074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347406a410 msgr2=0x7f34741981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.555+0000 7f3473400700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347406a410 0x7f34741981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.555+0000 7f3473400700 1 -- 172.21.15.49:0/1919607074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3474106ff0 msgr2=0x7f347419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.556+0000 7f3473400700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3474106ff0 0x7f347419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.556+0000 7f3473400700 1 -- 172.21.15.49:0/1919607074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600095f0 con 0x7f347406adc0 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.556+0000 7f3479000700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3474106ff0 0x7f347419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:48.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.556+0000 7f3473400700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f347406adc0 0x7f3474198730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f34600098e0 tx=0x7f3460005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:48.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.556+0000 7f3470c00700 1 -- 172.21.15.49:0/1919607074 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346001d070 con 0x7f347406adc0 2024-02-26T06:21:48.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.556+0000 7f347b000700 1 -- 172.21.15.49:0/1919607074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347419d680 con 0x7f347406adc0 2024-02-26T06:21:48.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.556+0000 7f3470c00700 1 -- 172.21.15.49:0/1919607074 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f346000b610 con 0x7f347406adc0 2024-02-26T06:21:48.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.556+0000 7f3470c00700 1 -- 172.21.15.49:0/1919607074 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34600173f0 con 0x7f347406adc0 2024-02-26T06:21:48.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.557+0000 7f347b000700 1 -- 172.21.15.49:0/1919607074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34741a4f40 con 0x7f347406adc0 2024-02-26T06:21:48.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.558+0000 7f346a000700 1 -- 172.21.15.49:0/1919607074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34741033a0 con 0x7f347406adc0 2024-02-26T06:21:48.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.561+0000 7f3470c00700 1 -- 172.21.15.49:0/1919607074 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f3460022600 con 0x7f347406adc0 2024-02-26T06:21:48.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.563+0000 7f3470c00700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f34580a21e0 0x7f34580a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:48.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.563+0000 7f3470c00700 1 -- 172.21.15.49:0/1919607074 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f34600cfa00 con 0x7f347406adc0 2024-02-26T06:21:48.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.563+0000 7f3473e00700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f34580a21e0 0x7f34580a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:48.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.564+0000 7f3473e00700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f34580a21e0 0x7f34580a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3464009c60 tx=0x7f3464009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:48.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.564+0000 7f3470c00700 1 -- 172.21.15.49:0/1919607074 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3460098890 con 0x7f347406adc0 2024-02-26T06:21:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:48.586 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.803+0000 7f346a000700 1 -- 172.21.15.49:0/1919607074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3474068930 con 0x7f347406adc0 2024-02-26T06:21:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.804+0000 7f3470c00700 1 -- 172.21.15.49:0/1919607074 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f346009bf50 con 0x7f347406adc0 2024-02-26T06:21:48.805 INFO:teuthology.orchestra.run.smithi049.stdout:201863462969 2024-02-26T06:21:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.808+0000 7f346a000700 1 -- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f34580a21e0 msgr2=0x7f34580a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.808+0000 7f346a000700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f34580a21e0 0x7f34580a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3464009c60 tx=0x7f3464009260 comp rx=0 tx=0).stop 2024-02-26T06:21:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.808+0000 7f346a000700 1 -- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f347406adc0 msgr2=0x7f3474198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.808+0000 7f346a000700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f347406adc0 0x7f3474198730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f34600098e0 tx=0x7f3460005740 comp rx=0 tx=0).stop 2024-02-26T06:21:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.809+0000 7f346a000700 1 -- 172.21.15.49:0/1919607074 shutdown_connections 2024-02-26T06:21:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.809+0000 7f346a000700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f34580a21e0 0x7f34580a46a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.809+0000 7f346a000700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3474106ff0 0x7f347419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.809+0000 7f346a000700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347406a410 0x7f34741981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.809+0000 7f346a000700 1 --2- 172.21.15.49:0/1919607074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f347406adc0 0x7f3474198730 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.809+0000 7f346a000700 1 -- 172.21.15.49:0/1919607074 >> 172.21.15.49:0/1919607074 conn(0x7f3474076bc0 msgr2=0x7f34740ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.810+0000 7f346a000700 1 -- 172.21.15.49:0/1919607074 shutdown_connections 2024-02-26T06:21:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:48.810+0000 7f346a000700 1 -- 172.21.15.49:0/1919607074 wait complete. 2024-02-26T06:21:49.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:48 smithi049 ceph-mon[31696]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:49.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:48 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1919607074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T06:21:49.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:48 smithi080 ceph-mon[36206]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:49.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:48 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1919607074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T06:21:49.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:48 smithi115 ceph-mon[36426]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:49.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:48 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1919607074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T06:21:50.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 -- 172.21.15.49:0/3625216205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 msgr2=0x7fcd081067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:50.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 --2- 172.21.15.49:0/3625216205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 0x7fcd081067b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009990 tx=0x7fccf0009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 -- 172.21.15.49:0/3625216205 shutdown_connections 2024-02-26T06:21:50.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 --2- 172.21.15.49:0/3625216205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd08106e90 0x7fcd0810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 --2- 172.21.15.49:0/3625216205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd08101db0 0x7fcd08102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 --2- 172.21.15.49:0/3625216205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 0x7fcd081067b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 -- 172.21.15.49:0/3625216205 >> 172.21.15.49:0/3625216205 conn(0x7fcd080fd640 msgr2=0x7fcd080ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:50.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 -- 172.21.15.49:0/3625216205 shutdown_connections 2024-02-26T06:21:50.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.135+0000 7fcd07e00700 1 -- 172.21.15.49:0/3625216205 wait complete. 2024-02-26T06:21:50.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.136+0000 7fcd07e00700 1 Processor -- start 2024-02-26T06:21:50.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.136+0000 7fcd07e00700 1 -- start start 2024-02-26T06:21:50.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd08101db0 0x7fcd0819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 0x7fcd0819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd08106e90 0x7fcd0819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07e00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0819f1a0 con 0x7fcd08106e90 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0819f310 con 0x7fcd08102760 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0819f4d0 con 0x7fcd08101db0 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd08106e90 0x7fcd0819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd06a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd08101db0 0x7fcd0819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd08106e90 0x7fcd0819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57232/0 (socket says 172.21.15.49:57232) 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd06000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 0x7fcd0819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd07400700 1 -- 172.21.15.49:0/727706325 learned_addr learned my addr 172.21.15.49:0/727706325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.137+0000 7fcd06a00700 1 -- 172.21.15.49:0/727706325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 msgr2=0x7fcd0819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fcd06a00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 0x7fcd0819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fcd06a00700 1 -- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd08106e90 msgr2=0x7fcd0819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:50.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fcd06a00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd08106e90 0x7fcd0819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fcd06a00700 1 -- 172.21.15.49:0/727706325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00095f0 con 0x7fcd08101db0 2024-02-26T06:21:50.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fcd07400700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd08106e90 0x7fcd0819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:50.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fcd06000700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 0x7fcd0819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:50.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fcd06a00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd08101db0 0x7fcd0819a2c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e9f0 tx=0x7fccf800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:50.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fccff400700 1 -- 172.21.15.49:0/727706325 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf800c9c0 con 0x7fcd08101db0 2024-02-26T06:21:50.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0819f7b0 con 0x7fcd08101db0 2024-02-26T06:21:50.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fccff400700 1 -- 172.21.15.49:0/727706325 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccf8010430 con 0x7fcd08101db0 2024-02-26T06:21:50.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.138+0000 7fccff400700 1 -- 172.21.15.49:0/727706325 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8018660 con 0x7fcd08101db0 2024-02-26T06:21:50.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.139+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd081a7010 con 0x7fcd08101db0 2024-02-26T06:21:50.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.140+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd08068930 con 0x7fcd08101db0 2024-02-26T06:21:50.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.142+0000 7fccff400700 1 -- 172.21.15.49:0/727706325 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fccf8010910 con 0x7fcd08101db0 2024-02-26T06:21:50.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.143+0000 7fccff400700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fccec0a21e0 0x7fccec0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:50.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.143+0000 7fccff400700 1 -- 172.21.15.49:0/727706325 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fccf8014070 con 0x7fcd08101db0 2024-02-26T06:21:50.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.144+0000 7fcd06000700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fccec0a21e0 0x7fccec0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:50.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.144+0000 7fcd06000700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fccec0a21e0 0x7fccec0a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd0810ef10 tx=0x7fccf0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:50.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.145+0000 7fccff400700 1 -- 172.21.15.49:0/727706325 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fccf80972d0 con 0x7fcd08101db0 2024-02-26T06:21:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.383+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fcd0819b620 con 0x7fcd08101db0 2024-02-26T06:21:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.383+0000 7fccff400700 1 -- 172.21.15.49:0/727706325 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fccf80105a0 con 0x7fcd08101db0 2024-02-26T06:21:50.384 INFO:teuthology.orchestra.run.smithi049.stdout:292057776168 2024-02-26T06:21:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.386+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fccec0a21e0 msgr2=0x7fccec0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.386+0000 7fcd07e00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fccec0a21e0 0x7fccec0a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd0810ef10 tx=0x7fccf0005fd0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.386+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd08101db0 msgr2=0x7fcd0819a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.386+0000 7fcd07e00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd08101db0 0x7fcd0819a2c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e9f0 tx=0x7fccf800ed00 comp rx=0 tx=0).stop 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.387+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 shutdown_connections 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.387+0000 7fcd07e00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fccec0a21e0 0x7fccec0a46a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.387+0000 7fcd07e00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd08106e90 0x7fcd0819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.387+0000 7fcd07e00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd08101db0 0x7fcd0819a2c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.387+0000 7fcd07e00700 1 --2- 172.21.15.49:0/727706325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd08102760 0x7fcd0819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.387+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 >> 172.21.15.49:0/727706325 conn(0x7fcd080fd640 msgr2=0x7fcd080fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.387+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 shutdown_connections 2024-02-26T06:21:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:50.387+0000 7fcd07e00700 1 -- 172.21.15.49:0/727706325 wait complete. 2024-02-26T06:21:51.050 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:50 smithi049 ceph-mon[31696]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:21:51.050 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:50 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/727706325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T06:21:51.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462962 got 201863462969 for osd.6 2024-02-26T06:21:51.112 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:50 smithi080 ceph-mon[36206]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:21:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:50 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/727706325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T06:21:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:50 smithi115 ceph-mon[36426]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:21:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:50 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/727706325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T06:21:53.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:52 smithi080 ceph-mon[36206]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:21:53.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:52 smithi115 ceph-mon[36426]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:21:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:52 smithi049 ceph-mon[31696]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:21:53.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776164 got 292057776168 for osd.9 2024-02-26T06:21:53.793 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:54.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.811+0000 7f3c49200700 1 -- 172.21.15.49:0/771657410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 msgr2=0x7f3c44102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:54.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.811+0000 7f3c49200700 1 --2- 172.21.15.49:0/771657410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 0x7f3c44102170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0099c0 tx=0x7f3c2c009cd0 comp rx=0 tx=0).stop 2024-02-26T06:21:54.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.812+0000 7f3c49200700 1 -- 172.21.15.49:0/771657410 shutdown_connections 2024-02-26T06:21:54.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.812+0000 7f3c49200700 1 --2- 172.21.15.49:0/771657410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c441026b0 0x7f3c4410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:54.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.812+0000 7f3c49200700 1 --2- 172.21.15.49:0/771657410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c44107d40 0x7f3c44108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:54.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.812+0000 7f3c49200700 1 --2- 172.21.15.49:0/771657410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 0x7f3c44102170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:54.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.812+0000 7f3c49200700 1 -- 172.21.15.49:0/771657410 >> 172.21.15.49:0/771657410 conn(0x7f3c44078dc0 msgr2=0x7f3c440791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:54.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.812+0000 7f3c49200700 1 -- 172.21.15.49:0/771657410 shutdown_connections 2024-02-26T06:21:54.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.812+0000 7f3c49200700 1 -- 172.21.15.49:0/771657410 wait complete. 2024-02-26T06:21:54.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.813+0000 7f3c49200700 1 Processor -- start 2024-02-26T06:21:54.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.814+0000 7f3c49200700 1 -- start start 2024-02-26T06:21:54.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.814+0000 7f3c49200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 0x7f3c4419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:54.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.814+0000 7f3c49200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c441026b0 0x7f3c4419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:54.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.814+0000 7f3c49200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c44107d40 0x7f3c4419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:54.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.814+0000 7f3c49200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4419f2a0 con 0x7f3c44107d40 2024-02-26T06:21:54.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.814+0000 7f3c49200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4419f410 con 0x7f3c441026b0 2024-02-26T06:21:54.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.814+0000 7f3c49200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4419f5d0 con 0x7f3c44101cf0 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c42800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c44107d40 0x7f3c4419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c41e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 0x7f3c4419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c39400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c441026b0 0x7f3c4419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c41e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 0x7f3c4419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43498/0 (socket says 172.21.15.49:43498) 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c41e00700 1 -- 172.21.15.49:0/2853491117 learned_addr learned my addr 172.21.15.49:0/2853491117 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c39400700 1 -- 172.21.15.49:0/2853491117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 msgr2=0x7f3c4419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c39400700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 0x7f3c4419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c39400700 1 -- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c44107d40 msgr2=0x7f3c4419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c39400700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c44107d40 0x7f3c4419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c39400700 1 -- 172.21.15.49:0/2853491117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c0095f0 con 0x7f3c441026b0 2024-02-26T06:21:54.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.815+0000 7f3c39400700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c441026b0 0x7f3c4419a900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098e0 tx=0x7f3c2c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:54.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.816+0000 7f3c3b400700 1 -- 172.21.15.49:0/2853491117 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c01d070 con 0x7f3c441026b0 2024-02-26T06:21:54.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.816+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4419f850 con 0x7f3c441026b0 2024-02-26T06:21:54.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.816+0000 7f3c3b400700 1 -- 172.21.15.49:0/2853491117 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c003680 con 0x7f3c441026b0 2024-02-26T06:21:54.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.816+0000 7f3c3b400700 1 -- 172.21.15.49:0/2853491117 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c022e00 con 0x7f3c441026b0 2024-02-26T06:21:54.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.816+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c441a7110 con 0x7f3c441026b0 2024-02-26T06:21:54.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.817+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c441033b0 con 0x7f3c441026b0 2024-02-26T06:21:54.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.820+0000 7f3c3b400700 1 -- 172.21.15.49:0/2853491117 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f3c2c022500 con 0x7f3c441026b0 2024-02-26T06:21:54.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.821+0000 7f3c3b400700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c280a21e0 0x7f3c280a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:54.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.821+0000 7f3c3b400700 1 -- 172.21.15.49:0/2853491117 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3c2c0cf090 con 0x7f3c441026b0 2024-02-26T06:21:54.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.821+0000 7f3c41e00700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c280a21e0 0x7f3c280a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:54.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.822+0000 7f3c41e00700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c280a21e0 0x7f3c280a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009ca0 tx=0x7f3c34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:54.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:54.823+0000 7f3c3b400700 1 -- 172.21.15.49:0/2853491117 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c2c097f20 con 0x7f3c441026b0 2024-02-26T06:21:55.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.060+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f3c4419b720 con 0x7f3c441026b0 2024-02-26T06:21:55.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.061+0000 7f3c3b400700 1 -- 172.21.15.49:0/2853491117 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f3c2c09b5e0 con 0x7f3c441026b0 2024-02-26T06:21:55.062 INFO:teuthology.orchestra.run.smithi049.stdout:347892351006 2024-02-26T06:21:55.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c280a21e0 msgr2=0x7f3c280a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:55.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c280a21e0 0x7f3c280a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009ca0 tx=0x7f3c34009260 comp rx=0 tx=0).stop 2024-02-26T06:21:55.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c441026b0 msgr2=0x7f3c4419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:55.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c441026b0 0x7f3c4419a900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098e0 tx=0x7f3c2c00bfd0 comp rx=0 tx=0).stop 2024-02-26T06:21:55.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 shutdown_connections 2024-02-26T06:21:55.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c280a21e0 0x7f3c280a46a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:55.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c44107d40 0x7f3c4419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:55.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c44101cf0 0x7f3c4419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:55.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 --2- 172.21.15.49:0/2853491117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c441026b0 0x7f3c4419a900 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:55.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 >> 172.21.15.49:0/2853491117 conn(0x7f3c44078dc0 msgr2=0x7f3c4410d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:55.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.064+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 shutdown_connections 2024-02-26T06:21:55.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:55.065+0000 7f3c49200700 1 -- 172.21.15.49:0/2853491117 wait complete. 2024-02-26T06:21:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:54 smithi080 ceph-mon[36206]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:21:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:54 smithi115 ceph-mon[36426]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:21:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:54 smithi049 ceph-mon[31696]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:21:56.230 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351000 got 347892351006 for osd.11 2024-02-26T06:21:56.231 DEBUG:teuthology.parallel:result is None 2024-02-26T06:21:56.291 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:55 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2853491117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T06:21:56.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:55 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2853491117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T06:21:56.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:55 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2853491117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T06:21:57.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:57 smithi080 ceph-mon[36206]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:21:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:57 smithi115 ceph-mon[36426]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:21:57.409 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:57 smithi049 ceph-mon[31696]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 -- 172.21.15.49:0/1076122392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 msgr2=0x7fa850074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 --2- 172.21.15.49:0/1076122392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 0x7fa850074280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa840009990 tx=0x7fa840009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 -- 172.21.15.49:0/1076122392 shutdown_connections 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 --2- 172.21.15.49:0/1076122392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa85010e670 0x7fa850110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 --2- 172.21.15.49:0/1076122392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 0x7fa850074280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 --2- 172.21.15.49:0/1076122392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8500747c0 0x7fa85010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 -- 172.21.15.49:0/1076122392 >> 172.21.15.49:0/1076122392 conn(0x7fa850078da0 msgr2=0x7fa8500791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 -- 172.21.15.49:0/1076122392 shutdown_connections 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.379+0000 7fa856c00700 1 -- 172.21.15.49:0/1076122392 wait complete. 2024-02-26T06:21:58.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.380+0000 7fa856c00700 1 Processor -- start 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.380+0000 7fa856c00700 1 -- start start 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.380+0000 7fa856c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 0x7fa85019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.380+0000 7fa856c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8500747c0 0x7fa85019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.380+0000 7fa856c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa85010e670 0x7fa85019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa856c00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85019f220 con 0x7fa8500747c0 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa856c00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85019f390 con 0x7fa850073ea0 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa856c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85019f550 con 0x7fa85010e670 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84fe00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 0x7fa85019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84fe00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 0x7fa85019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60948/0 (socket says 172.21.15.49:60948) 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84fe00700 1 -- 172.21.15.49:0/322069596 learned_addr learned my addr 172.21.15.49:0/322069596 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84f400700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8500747c0 0x7fa85019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa854c00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa85010e670 0x7fa85019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84fe00700 1 -- 172.21.15.49:0/322069596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa85010e670 msgr2=0x7fa85019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84fe00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa85010e670 0x7fa85019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84fe00700 1 -- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8500747c0 msgr2=0x7fa85019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84fe00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8500747c0 0x7fa85019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.381+0000 7fa84fe00700 1 -- 172.21.15.49:0/322069596 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400095f0 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.382+0000 7fa84fe00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 0x7fa85019a390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa840000c00 tx=0x7fa840004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.382+0000 7fa84cc00700 1 -- 172.21.15.49:0/322069596 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84001d070 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.382+0000 7fa84cc00700 1 -- 172.21.15.49:0/322069596 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa84000b610 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.382+0000 7fa84cc00700 1 -- 172.21.15.49:0/322069596 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8400173f0 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.382+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85019f7d0 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.382+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8501a7090 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.384+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa850109880 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.385+0000 7fa84cc00700 1 -- 172.21.15.49:0/322069596 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa840022670 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.386+0000 7fa84cc00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8340a2150 0x7fa8340a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.386+0000 7fa84cc00700 1 -- 172.21.15.49:0/322069596 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fa8400cee30 con 0x7fa850073ea0 2024-02-26T06:21:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.387+0000 7fa84f400700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8340a2150 0x7fa8340a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:58.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.387+0000 7fa84f400700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8340a2150 0x7fa8340a4610 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa850110390 tx=0x7fa83c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:58.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.394+0000 7fa84cc00700 1 -- 172.21.15.49:0/322069596 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa840097e20 con 0x7fa850073ea0 2024-02-26T06:21:58.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.639+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa85019b6a0 con 0x7fa850073ea0 2024-02-26T06:21:58.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.640+0000 7fa84cc00700 1 -- 172.21.15.49:0/322069596 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa84009b4e0 con 0x7fa850073ea0 2024-02-26T06:21:58.640 INFO:teuthology.orchestra.run.smithi049.stdout:167503724611 2024-02-26T06:21:58.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8340a2150 msgr2=0x7fa8340a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:58.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8340a2150 0x7fa8340a4610 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa850110390 tx=0x7fa83c009260 comp rx=0 tx=0).stop 2024-02-26T06:21:58.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 msgr2=0x7fa85019a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:58.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 0x7fa85019a390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa840000c00 tx=0x7fa840004730 comp rx=0 tx=0).stop 2024-02-26T06:21:58.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 shutdown_connections 2024-02-26T06:21:58.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa8340a2150 0x7fa8340a4610 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa85010e670 0x7fa85019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa850073ea0 0x7fa85019a390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 --2- 172.21.15.49:0/322069596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8500747c0 0x7fa85019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:58.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 >> 172.21.15.49:0/322069596 conn(0x7fa850078da0 msgr2=0x7fa8500fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:58.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 shutdown_connections 2024-02-26T06:21:58.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.643+0000 7fa856c00700 1 -- 172.21.15.49:0/322069596 wait complete. 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.503+0000 7f1409800700 1 -- 172.21.15.49:0/3715503886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 msgr2=0x7f140406a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.503+0000 7f1409800700 1 --2- 172.21.15.49:0/3715503886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 0x7f140406a950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009990 tx=0x7f13f4009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.504+0000 7f1409800700 1 -- 172.21.15.49:0/3715503886 shutdown_connections 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.504+0000 7f1409800700 1 --2- 172.21.15.49:0/3715503886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1404106fe0 0x7f140410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.504+0000 7f1409800700 1 --2- 172.21.15.49:0/3715503886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 0x7f140406a950 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.504+0000 7f1409800700 1 --2- 172.21.15.49:0/3715503886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f140406af20 0x7f14041068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.504+0000 7f1409800700 1 -- 172.21.15.49:0/3715503886 >> 172.21.15.49:0/3715503886 conn(0x7f1404077ae0 msgr2=0x7f1404077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.504+0000 7f1409800700 1 -- 172.21.15.49:0/3715503886 shutdown_connections 2024-02-26T06:21:59.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.504+0000 7f1409800700 1 -- 172.21.15.49:0/3715503886 wait complete. 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.505+0000 7f1409800700 1 Processor -- start 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.505+0000 7f1409800700 1 -- start start 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.505+0000 7f1409800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 0x7f140419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.505+0000 7f1409800700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f140406af20 0x7f140419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.505+0000 7f1409800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1404106fe0 0x7f140419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.505+0000 7f1409800700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140419f2e0 con 0x7f1404106fe0 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.505+0000 7f1409800700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140419f450 con 0x7f140406af20 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.505+0000 7f1409800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140419f610 con 0x7f140406a570 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1403400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1404106fe0 0x7f140419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1403400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1404106fe0 0x7f140419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57294/0 (socket says 172.21.15.49:57294) 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1402a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 0x7f140419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1403400700 1 -- 172.21.15.49:0/1307153442 learned_addr learned my addr 172.21.15.49:0/1307153442 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1402000700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f140406af20 0x7f140419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:59.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1402a00700 1 -- 172.21.15.49:0/1307153442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f140406af20 msgr2=0x7f140419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1402a00700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f140406af20 0x7f140419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1402a00700 1 -- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1404106fe0 msgr2=0x7f140419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1402a00700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1404106fe0 0x7f140419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1402a00700 1 -- 172.21.15.49:0/1307153442 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40095f0 con 0x7f140406a570 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1403400700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1404106fe0 0x7f140419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.506+0000 7f1402a00700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 0x7f140419a400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13f40098e0 tx=0x7f13f4005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.507+0000 7f13fb400700 1 -- 172.21.15.49:0/1307153442 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f401d070 con 0x7f140406a570 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.507+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140419f890 con 0x7f140406a570 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.507+0000 7f13fb400700 1 -- 172.21.15.49:0/1307153442 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13f400b610 con 0x7f140406a570 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.507+0000 7f13fb400700 1 -- 172.21.15.49:0/1307153442 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4022e00 con 0x7f140406a570 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.507+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14041a7150 con 0x7f140406a570 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.508+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1404069bc0 con 0x7f140406a570 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.512+0000 7f13fb400700 1 -- 172.21.15.49:0/1307153442 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f13f4022500 con 0x7f140406a570 2024-02-26T06:21:59.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.513+0000 7f13fb400700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f13e80a2150 0x7f13e80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:59.173 INFO:teuthology.orchestra.run.smithi049.stdout:111669149773 2024-02-26T06:21:59.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.514+0000 7f13fb400700 1 -- 172.21.15.49:0/1307153442 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f13f40cf090 con 0x7f140406a570 2024-02-26T06:21:59.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.514+0000 7f1402000700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f13e80a2150 0x7f13e80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:59.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.514+0000 7f1402000700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f13e80a2150 0x7f13e80a4610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1404102570 tx=0x7f13f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:59.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.515+0000 7f13fb400700 1 -- 172.21.15.49:0/1307153442 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13f4098080 con 0x7f140406a570 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.776+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1404103390 con 0x7f140406a570 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.777+0000 7f13fb400700 1 -- 172.21.15.49:0/1307153442 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13f409b740 con 0x7f140406a570 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f13e80a2150 msgr2=0x7f13e80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f13e80a2150 0x7f13e80a4610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1404102570 tx=0x7f13f0009260 comp rx=0 tx=0).stop 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 msgr2=0x7f140419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 0x7f140419a400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13f40098e0 tx=0x7f13f4005720 comp rx=0 tx=0).stop 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 shutdown_connections 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f13e80a2150 0x7f13e80a4610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1404106fe0 0x7f140419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f140406a570 0x7f140419a400 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 --2- 172.21.15.49:0/1307153442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f140406af20 0x7f140419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:59.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 >> 172.21.15.49:0/1307153442 conn(0x7f1404077ae0 msgr2=0x7f14040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:59.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 shutdown_connections 2024-02-26T06:21:59.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:21:58.780+0000 7f1409800700 1 -- 172.21.15.49:0/1307153442 wait complete. 2024-02-26T06:21:59.282 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:59 smithi049 ceph-mon[31696]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:59.282 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:59 smithi049 ceph-mon[31696]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-02-26T06:21:59.283 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:59 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/322069596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T06:21:59.283 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:21:59 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1307153442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T06:21:59.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:59 smithi080 ceph-mon[36206]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:59.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:59 smithi080 ceph-mon[36206]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-02-26T06:21:59.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:59 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/322069596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T06:21:59.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:21:59 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1307153442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T06:21:59.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:59 smithi115 ceph-mon[36426]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:21:59.343 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:59 smithi115 ceph-mon[36426]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-02-26T06:21:59.343 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:59 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/322069596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T06:21:59.344 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:21:59 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1307153442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T06:22:00.141 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724605 got 167503724611 for osd.5 2024-02-26T06:22:00.141 DEBUG:teuthology.parallel:result is None 2024-02-26T06:22:00.696 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149766 got 111669149773 for osd.3 2024-02-26T06:22:00.697 DEBUG:teuthology.parallel:result is None 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 -- 172.21.15.49:0/207383027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0101eb0 msgr2=0x7ff5b0102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 --2- 172.21.15.49:0/207383027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0101eb0 0x7ff5b0102290 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009990 tx=0x7ff5a0009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 -- 172.21.15.49:0/207383027 shutdown_connections 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 --2- 172.21.15.49:0/207383027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b0106f90 0x7ff5b010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 --2- 172.21.15.49:0/207383027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0101eb0 0x7ff5b0102290 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 --2- 172.21.15.49:0/207383027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b0102860 0x7ff5b01068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 -- 172.21.15.49:0/207383027 >> 172.21.15.49:0/207383027 conn(0x7ff5b00fd720 msgr2=0x7ff5b00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 -- 172.21.15.49:0/207383027 shutdown_connections 2024-02-26T06:22:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.967+0000 7ff5b7000700 1 -- 172.21.15.49:0/207383027 wait complete. 2024-02-26T06:22:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.968+0000 7ff5b7000700 1 Processor -- start 2024-02-26T06:22:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.968+0000 7ff5b7000700 1 -- start start 2024-02-26T06:22:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.969+0000 7ff5b7000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b0101eb0 0x7ff5b0073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.969+0000 7ff5b7000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0102860 0x7ff5b00744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.969+0000 7ff5b7000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b0106f90 0x7ff5b00749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.969+0000 7ff5b7000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a6580 con 0x7ff5b0106f90 2024-02-26T06:22:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.969+0000 7ff5b7000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a66f0 con 0x7ff5b0102860 2024-02-26T06:22:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.969+0000 7ff5b7000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a68b0 con 0x7ff5b0101eb0 2024-02-26T06:22:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.969+0000 7ff5afe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b0101eb0 0x7ff5b0073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5af400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0102860 0x7ff5b00744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5afe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b0101eb0 0x7ff5b0073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48954/0 (socket says 172.21.15.49:48954) 2024-02-26T06:22:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5afe00700 1 -- 172.21.15.49:0/2959367879 learned_addr learned my addr 172.21.15.49:0/2959367879 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5b5000700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b0106f90 0x7ff5b00749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5af400700 1 -- 172.21.15.49:0/2959367879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b0101eb0 msgr2=0x7ff5b0073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5af400700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b0101eb0 0x7ff5b0073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5af400700 1 -- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b0106f90 msgr2=0x7ff5b00749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5af400700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b0106f90 0x7ff5b00749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5af400700 1 -- 172.21.15.49:0/2959367879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a00095f0 con 0x7ff5b0102860 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.970+0000 7ff5afe00700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b0101eb0 0x7ff5b0073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.971+0000 7ff5af400700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0102860 0x7ff5b00744b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00e9c0 tx=0x7ff59c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.971+0000 7ff5acc00700 1 -- 172.21.15.49:0/2959367879 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00c9c0 con 0x7ff5b0102860 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.971+0000 7ff5acc00700 1 -- 172.21.15.49:0/2959367879 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff59c010430 con 0x7ff5b0102860 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.971+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b01a6b90 con 0x7ff5b0102860 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.971+0000 7ff5acc00700 1 -- 172.21.15.49:0/2959367879 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c018640 con 0x7ff5b0102860 2024-02-26T06:22:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.971+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b01a70e0 con 0x7ff5b0102860 2024-02-26T06:22:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.972+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b010a1c0 con 0x7ff5b0102860 2024-02-26T06:22:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.974+0000 7ff5acc00700 1 -- 172.21.15.49:0/2959367879 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff59c0105a0 con 0x7ff5b0102860 2024-02-26T06:22:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.976+0000 7ff5acc00700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff5940a21d0 0x7ff5940a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.976+0000 7ff5acc00700 1 -- 172.21.15.49:0/2959367879 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff59c014070 con 0x7ff5b0102860 2024-02-26T06:22:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.976+0000 7ff5afe00700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff5940a21d0 0x7ff5940a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.977+0000 7ff5afe00700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff5940a21d0 0x7ff5940a4690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0006010 tx=0x7ff5a000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:00.978+0000 7ff5acc00700 1 -- 172.21.15.49:0/2959367879 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff59c097120 con 0x7ff5b0102860 2024-02-26T06:22:01.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.254+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7ff5b0111cf0 con 0x7ff5b0102860 2024-02-26T06:22:01.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:01 smithi049 ceph-mon[31696]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-26T06:22:01.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.255+0000 7ff5acc00700 1 -- 172.21.15.49:0/2959367879 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff59c026020 con 0x7ff5b0102860 2024-02-26T06:22:01.255 INFO:teuthology.orchestra.run.smithi049.stdout:313532612645 2024-02-26T06:22:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff5940a21d0 msgr2=0x7ff5940a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff5940a21d0 0x7ff5940a4690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0006010 tx=0x7ff5a000b610 comp rx=0 tx=0).stop 2024-02-26T06:22:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0102860 msgr2=0x7ff5b00744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0102860 0x7ff5b00744b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00e9c0 tx=0x7ff59c00ecd0 comp rx=0 tx=0).stop 2024-02-26T06:22:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 shutdown_connections 2024-02-26T06:22:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff5940a21d0 0x7ff5940a4690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5b0106f90 0x7ff5b00749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b0101eb0 0x7ff5b0073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 --2- 172.21.15.49:0/2959367879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5b0102860 0x7ff5b00744b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.258+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 >> 172.21.15.49:0/2959367879 conn(0x7ff5b00fd720 msgr2=0x7ff5b00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.259+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 shutdown_connections 2024-02-26T06:22:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:01.259+0000 7ff5b7000700 1 -- 172.21.15.49:0/2959367879 wait complete. 2024-02-26T06:22:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:01 smithi080 ceph-mon[36206]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-26T06:22:01.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:01 smithi115 ceph-mon[36426]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-26T06:22:02.038 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612639 got 313532612645 for osd.10 2024-02-26T06:22:02.038 DEBUG:teuthology.parallel:result is None 2024-02-26T06:22:02.038 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-26T06:22:02.039 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-26T06:22:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:02 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2959367879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T06:22:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:02 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:02 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2959367879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T06:22:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:02.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:02 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:02 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2959367879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T06:22:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:02 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:02.527 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:03.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:03 smithi080 ceph-mon[36206]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-26T06:22:03.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:03 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:03 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:03.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:03 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:03 smithi115 ceph-mon[36426]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-26T06:22:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:03 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:03 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:03 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:03.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:03 smithi049 ceph-mon[31696]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-26T06:22:03.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:03.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:22:04.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 -- 172.21.15.49:0/1474789173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09b4069920 msgr2=0x7f09b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:04.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 --2- 172.21.15.49:0/1474789173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09b4069920 0x7f09b4069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f099c009920 tx=0x7f099c009c30 comp rx=0 tx=0).stop 2024-02-26T06:22:04.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 -- 172.21.15.49:0/1474789173 shutdown_connections 2024-02-26T06:22:04.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 --2- 172.21.15.49:0/1474789173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09b406a2c0 0x7f09b410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:04.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 --2- 172.21.15.49:0/1474789173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09b4107cc0 0x7f09b41080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:04.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 --2- 172.21.15.49:0/1474789173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09b4069920 0x7f09b4069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:04.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 -- 172.21.15.49:0/1474789173 >> 172.21.15.49:0/1474789173 conn(0x7f09b4077aa0 msgr2=0x7f09b4077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:04.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 -- 172.21.15.49:0/1474789173 shutdown_connections 2024-02-26T06:22:04.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.957+0000 7f09ba400700 1 -- 172.21.15.49:0/1474789173 wait complete. 2024-02-26T06:22:04.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.958+0000 7f09ba400700 1 Processor -- start 2024-02-26T06:22:04.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.958+0000 7f09ba400700 1 -- start start 2024-02-26T06:22:04.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09ba400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09b4069920 0x7f09b4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:04.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09ba400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09b406a2c0 0x7f09b4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:04.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09ba400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09b4107cc0 0x7f09b419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:04.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09ba400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b419d110 con 0x7f09b4107cc0 2024-02-26T06:22:04.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09ba400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b419d280 con 0x7f09b406a2c0 2024-02-26T06:22:04.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09ba400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b419d440 con 0x7f09b4069920 2024-02-26T06:22:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09b3e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09b4107cc0 0x7f09b419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09b3e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09b4107cc0 0x7f09b419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39482/0 (socket says 172.21.15.49:39482) 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09b3e00700 1 -- 172.21.15.49:0/2029746455 learned_addr learned my addr 172.21.15.49:0/2029746455 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09aaa00700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09b406a2c0 0x7f09b4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.959+0000 7f09b3400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09b4069920 0x7f09b4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09aaa00700 1 -- 172.21.15.49:0/2029746455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09b4069920 msgr2=0x7f09b4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09aaa00700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09b4069920 0x7f09b4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09aaa00700 1 -- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09b4107cc0 msgr2=0x7f09b419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09aaa00700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09b4107cc0 0x7f09b419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09aaa00700 1 -- 172.21.15.49:0/2029746455 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c009520 con 0x7f09b406a2c0 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09aaa00700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09b406a2c0 0x7f09b4198770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f099c000c00 tx=0x7f099c00fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09b0c00700 1 -- 172.21.15.49:0/2029746455 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c01c070 con 0x7f09b406a2c0 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b419d6c0 con 0x7f09b406a2c0 2024-02-26T06:22:04.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09b0c00700 1 -- 172.21.15.49:0/2029746455 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f099c0056a0 con 0x7f09b406a2c0 2024-02-26T06:22:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.960+0000 7f09b0c00700 1 -- 172.21.15.49:0/2029746455 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0038e0 con 0x7f09b406a2c0 2024-02-26T06:22:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.961+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b4100d40 con 0x7f09b406a2c0 2024-02-26T06:22:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.962+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b410a1e0 con 0x7f09b406a2c0 2024-02-26T06:22:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.964+0000 7f09b0c00700 1 -- 172.21.15.49:0/2029746455 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f099c020030 con 0x7f09b406a2c0 2024-02-26T06:22:04.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.965+0000 7f09b0c00700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09980a2150 0x7f09980a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:04.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.965+0000 7f09b0c00700 1 -- 172.21.15.49:0/2029746455 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f099c0cef40 con 0x7f09b406a2c0 2024-02-26T06:22:04.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.966+0000 7f09b3400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09980a2150 0x7f09980a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:04.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.966+0000 7f09b3400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09980a2150 0x7f09980a4610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f09a4005fd0 tx=0x7f09a4012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:04.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:04.967+0000 7f09b0c00700 1 -- 172.21.15.49:0/2029746455 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f099c097f30 con 0x7f09b406a2c0 2024-02-26T06:22:05.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.188+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09b40689a0 con 0x7f09980a2150 2024-02-26T06:22:05.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.225+0000 7f09b0c00700 1 -- 172.21.15.49:0/2029746455 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+342728 (secure 0 0 0) 0x7f09b40689a0 con 0x7f09980a2150 2024-02-26T06:22:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09980a2150 msgr2=0x7f09980a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09980a2150 0x7f09980a4610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f09a4005fd0 tx=0x7f09a4012040 comp rx=0 tx=0).stop 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09b406a2c0 msgr2=0x7f09b4198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09b406a2c0 0x7f09b4198770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f099c000c00 tx=0x7f099c00fa20 comp rx=0 tx=0).stop 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 shutdown_connections 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09980a2150 0x7f09980a4610 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09b4107cc0 0x7f09b419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09b4069920 0x7f09b4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 --2- 172.21.15.49:0/2029746455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09b406a2c0 0x7f09b4198770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 >> 172.21.15.49:0/2029746455 conn(0x7f09b4077aa0 msgr2=0x7f09b40ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 shutdown_connections 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:05.233+0000 7f09ba400700 1 -- 172.21.15.49:0/2029746455 wait complete. 2024-02-26T06:22:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-02-26T06:22:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:05 smithi080 ceph-mon[36206]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-26T06:22:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:05 smithi115 ceph-mon[36426]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-26T06:22:05.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:05 smithi049 ceph-mon[31696]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-26T06:22:06.202 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-02-26T06:22:03.459160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655866,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":665,"num_read_kb":453,"num_write":347,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1241721,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360764,"kb_used_data":11612,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581060,"statfs":{"total":1151940427776,"available":1151571005440,"internally_reserved":0,"allocated":11890688,"data_stored":7473018,"data_compressed":191495,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005251"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440077+0000","last_change":"2024-02-26T06:20:31.603060+0000","last_active":"2024-02-26T06:21:57.440077+0000","last_peered":"2024-02-26T06:21:57.440077+0000","last_clean":"2024-02-26T06:21:57.440077+0000","last_became_active":"2024-02-26T06:20:31.602762+0000","last_became_peered":"2024-02-26T06:20:31.602762+0000","last_unstale":"2024-02-26T06:21:57.440077+0000","last_undegraded":"2024-02-26T06:21:57.440077+0000","last_fullsized":"2024-02-26T06:21:57.440077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:19:42.465510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.437637+0000","last_change":"2024-02-26T06:20:31.595138+0000","last_active":"2024-02-26T06:21:57.437637+0000","last_peered":"2024-02-26T06:21:57.437637+0000","last_clean":"2024-02-26T06:21:57.437637+0000","last_became_active":"2024-02-26T06:20:31.594826+0000","last_became_peered":"2024-02-26T06:20:31.594826+0000","last_unstale":"2024-02-26T06:21:57.437637+0000","last_undegraded":"2024-02-26T06:21:57.437637+0000","last_fullsized":"2024-02-26T06:21:57.437637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:49.900464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.431057+0000","last_change":"2024-02-26T06:20:31.600415+0000","last_active":"2024-02-26T06:21:57.431057+0000","last_peered":"2024-02-26T06:21:57.431057+0000","last_clean":"2024-02-26T06:21:57.431057+0000","last_became_active":"2024-02-26T06:20:31.596273+0000","last_became_peered":"2024-02-26T06:20:31.596273+0000","last_unstale":"2024-02-26T06:21:57.431057+0000","last_undegraded":"2024-02-26T06:21:57.431057+0000","last_fullsized":"2024-02-26T06:21:57.431057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:15:20.626515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410770+0000","last_change":"2024-02-26T06:20:31.581149+0000","last_active":"2024-02-26T06:21:57.410770+0000","last_peered":"2024-02-26T06:21:57.410770+0000","last_clean":"2024-02-26T06:21:57.410770+0000","last_became_active":"2024-02-26T06:20:31.580831+0000","last_became_peered":"2024-02-26T06:20:31.580831+0000","last_unstale":"2024-02-26T06:21:57.410770+0000","last_undegraded":"2024-02-26T06:21:57.410770+0000","last_fullsized":"2024-02-26T06:21:57.410770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:47:12.466502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412439+0000","last_change":"2024-02-26T06:20:31.608618+0000","last_active":"2024-02-26T06:21:57.412439+0000","last_peered":"2024-02-26T06:21:57.412439+0000","last_clean":"2024-02-26T06:21:57.412439+0000","last_became_active":"2024-02-26T06:20:31.608285+0000","last_became_peered":"2024-02-26T06:20:31.608285+0000","last_unstale":"2024-02-26T06:21:57.412439+0000","last_undegraded":"2024-02-26T06:21:57.412439+0000","last_fullsized":"2024-02-26T06:21:57.412439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:34:48.400836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.418219+0000","last_change":"2024-02-26T06:21:28.905073+0000","last_active":"2024-02-26T06:21:57.418219+0000","last_peered":"2024-02-26T06:21:57.418219+0000","last_clean":"2024-02-26T06:21:57.418219+0000","last_became_active":"2024-02-26T06:21:28.904630+0000","last_became_peered":"2024-02-26T06:21:28.904630+0000","last_unstale":"2024-02-26T06:21:57.418219+0000","last_undegraded":"2024-02-26T06:21:57.418219+0000","last_fullsized":"2024-02-26T06:21:57.418219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:04.531384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.407496+0000","last_change":"2024-02-26T06:20:31.587285+0000","last_active":"2024-02-26T06:21:57.407496+0000","last_peered":"2024-02-26T06:21:57.407496+0000","last_clean":"2024-02-26T06:21:57.407496+0000","last_became_active":"2024-02-26T06:20:31.586985+0000","last_became_peered":"2024-02-26T06:20:31.586985+0000","last_unstale":"2024-02-26T06:21:57.407496+0000","last_undegraded":"2024-02-26T06:21:57.407496+0000","last_fullsized":"2024-02-26T06:21:57.407496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:55:56.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438174+0000","last_change":"2024-02-26T06:20:31.591136+0000","last_active":"2024-02-26T06:21:57.438174+0000","last_peered":"2024-02-26T06:21:57.438174+0000","last_clean":"2024-02-26T06:21:57.438174+0000","last_became_active":"2024-02-26T06:20:31.590180+0000","last_became_peered":"2024-02-26T06:20:31.590180+0000","last_unstale":"2024-02-26T06:21:57.438174+0000","last_undegraded":"2024-02-26T06:21:57.438174+0000","last_fullsized":"2024-02-26T06:21:57.438174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:02.649407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420056+0000","last_change":"2024-02-26T06:20:31.583890+0000","last_active":"2024-02-26T06:21:57.420056+0000","last_peered":"2024-02-26T06:21:57.420056+0000","last_clean":"2024-02-26T06:21:57.420056+0000","last_became_active":"2024-02-26T06:20:31.583459+0000","last_became_peered":"2024-02-26T06:20:31.583459+0000","last_unstale":"2024-02-26T06:21:57.420056+0000","last_undegraded":"2024-02-26T06:21:57.420056+0000","last_fullsized":"2024-02-26T06:21:57.420056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:14.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412599+0000","last_change":"2024-02-26T06:20:31.579041+0000","last_active":"2024-02-26T06:21:57.412599+0000","last_peered":"2024-02-26T06:21:57.412599+0000","last_clean":"2024-02-26T06:21:57.412599+0000","last_became_active":"2024-02-26T06:20:31.578725+0000","last_became_peered":"2024-02-26T06:20:31.578725+0000","last_unstale":"2024-02-26T06:21:57.412599+0000","last_undegraded":"2024-02-26T06:21:57.412599+0000","last_fullsized":"2024-02-26T06:21:57.412599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:21.083206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420414+0000","last_change":"2024-02-26T06:20:31.581146+0000","last_active":"2024-02-26T06:21:57.420414+0000","last_peered":"2024-02-26T06:21:57.420414+0000","last_clean":"2024-02-26T06:21:57.420414+0000","last_became_active":"2024-02-26T06:20:31.580774+0000","last_became_peered":"2024-02-26T06:20:31.580774+0000","last_unstale":"2024-02-26T06:21:57.420414+0000","last_undegraded":"2024-02-26T06:21:57.420414+0000","last_fullsized":"2024-02-26T06:21:57.420414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:58:36.851504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410450+0000","last_change":"2024-02-26T06:20:31.582232+0000","last_active":"2024-02-26T06:21:57.410450+0000","last_peered":"2024-02-26T06:21:57.410450+0000","last_clean":"2024-02-26T06:21:57.410450+0000","last_became_active":"2024-02-26T06:20:31.581805+0000","last_became_peered":"2024-02-26T06:20:31.581805+0000","last_unstale":"2024-02-26T06:21:57.410450+0000","last_undegraded":"2024-02-26T06:21:57.410450+0000","last_fullsized":"2024-02-26T06:21:57.410450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:04.513814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428463+0000","last_change":"2024-02-26T06:20:31.606373+0000","last_active":"2024-02-26T06:21:57.428463+0000","last_peered":"2024-02-26T06:21:57.428463+0000","last_clean":"2024-02-26T06:21:57.428463+0000","last_became_active":"2024-02-26T06:20:31.603226+0000","last_became_peered":"2024-02-26T06:20:31.603226+0000","last_unstale":"2024-02-26T06:21:57.428463+0000","last_undegraded":"2024-02-26T06:21:57.428463+0000","last_fullsized":"2024-02-26T06:21:57.428463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:29:49.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.426227+0000","last_change":"2024-02-26T06:20:31.594587+0000","last_active":"2024-02-26T06:21:57.426227+0000","last_peered":"2024-02-26T06:21:57.426227+0000","last_clean":"2024-02-26T06:21:57.426227+0000","last_became_active":"2024-02-26T06:20:31.594032+0000","last_became_peered":"2024-02-26T06:20:31.594032+0000","last_unstale":"2024-02-26T06:21:57.426227+0000","last_undegraded":"2024-02-26T06:21:57.426227+0000","last_fullsized":"2024-02-26T06:21:57.426227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:49:18.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438276+0000","last_change":"2024-02-26T06:20:31.593115+0000","last_active":"2024-02-26T06:21:57.438276+0000","last_peered":"2024-02-26T06:21:57.438276+0000","last_clean":"2024-02-26T06:21:57.438276+0000","last_became_active":"2024-02-26T06:20:31.592782+0000","last_became_peered":"2024-02-26T06:20:31.592782+0000","last_unstale":"2024-02-26T06:21:57.438276+0000","last_undegraded":"2024-02-26T06:21:57.438276+0000","last_fullsized":"2024-02-26T06:21:57.438276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:10.709124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.419515+0000","last_change":"2024-02-26T06:20:31.597938+0000","last_active":"2024-02-26T06:21:57.419515+0000","last_peered":"2024-02-26T06:21:57.419515+0000","last_clean":"2024-02-26T06:21:57.419515+0000","last_became_active":"2024-02-26T06:20:31.597600+0000","last_became_peered":"2024-02-26T06:20:31.597600+0000","last_unstale":"2024-02-26T06:21:57.419515+0000","last_undegraded":"2024-02-26T06:21:57.419515+0000","last_fullsized":"2024-02-26T06:21:57.419515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:13.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440280+0000","last_change":"2024-02-26T06:20:31.603283+0000","last_active":"2024-02-26T06:21:57.440280+0000","last_peered":"2024-02-26T06:21:57.440280+0000","last_clean":"2024-02-26T06:21:57.440280+0000","last_became_active":"2024-02-26T06:20:31.602912+0000","last_became_peered":"2024-02-26T06:20:31.602912+0000","last_unstale":"2024-02-26T06:21:57.440280+0000","last_undegraded":"2024-02-26T06:21:57.440280+0000","last_fullsized":"2024-02-26T06:21:57.440280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:32.250914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.422243+0000","last_change":"2024-02-26T06:20:31.588670+0000","last_active":"2024-02-26T06:21:57.422243+0000","last_peered":"2024-02-26T06:21:57.422243+0000","last_clean":"2024-02-26T06:21:57.422243+0000","last_became_active":"2024-02-26T06:20:31.587971+0000","last_became_peered":"2024-02-26T06:20:31.587971+0000","last_unstale":"2024-02-26T06:21:57.422243+0000","last_undegraded":"2024-02-26T06:21:57.422243+0000","last_fullsized":"2024-02-26T06:21:57.422243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:13.053456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.425013+0000","last_change":"2024-02-26T06:20:31.604266+0000","last_active":"2024-02-26T06:21:57.425013+0000","last_peered":"2024-02-26T06:21:57.425013+0000","last_clean":"2024-02-26T06:21:57.425013+0000","last_became_active":"2024-02-26T06:20:31.604027+0000","last_became_peered":"2024-02-26T06:20:31.604027+0000","last_unstale":"2024-02-26T06:21:57.425013+0000","last_undegraded":"2024-02-26T06:21:57.425013+0000","last_fullsized":"2024-02-26T06:21:57.425013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:13.345138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904450+0000","last_change":"2024-02-26T06:20:31.068090+0000","last_active":"2024-02-26T06:21:28.904450+0000","last_peered":"2024-02-26T06:21:28.904450+0000","last_clean":"2024-02-26T06:21:28.904450+0000","last_became_active":"2024-02-26T06:20:31.067731+0000","last_became_peered":"2024-02-26T06:20:31.067731+0000","last_unstale":"2024-02-26T06:21:28.904450+0000","last_undegraded":"2024-02-26T06:21:28.904450+0000","last_fullsized":"2024-02-26T06:21:28.904450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:24.734040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413823+0000","last_change":"2024-02-26T06:21:28.915741+0000","last_active":"2024-02-26T06:21:57.413823+0000","last_peered":"2024-02-26T06:21:57.413823+0000","last_clean":"2024-02-26T06:21:57.413823+0000","last_became_active":"2024-02-26T06:21:28.915443+0000","last_became_peered":"2024-02-26T06:21:28.915443+0000","last_unstale":"2024-02-26T06:21:57.413823+0000","last_undegraded":"2024-02-26T06:21:57.413823+0000","last_fullsized":"2024-02-26T06:21:57.413823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:51.216291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898840+0000","last_change":"2024-02-26T06:20:31.083236+0000","last_active":"2024-02-26T06:21:28.898840+0000","last_peered":"2024-02-26T06:21:28.898840+0000","last_clean":"2024-02-26T06:21:28.898840+0000","last_became_active":"2024-02-26T06:20:31.081640+0000","last_became_peered":"2024-02-26T06:20:31.081640+0000","last_unstale":"2024-02-26T06:21:28.898840+0000","last_undegraded":"2024-02-26T06:21:28.898840+0000","last_fullsized":"2024-02-26T06:21:28.898840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:44.607108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439841+0000","last_change":"2024-02-26T06:20:31.593807+0000","last_active":"2024-02-26T06:21:57.439841+0000","last_peered":"2024-02-26T06:21:57.439841+0000","last_clean":"2024-02-26T06:21:57.439841+0000","last_became_active":"2024-02-26T06:20:31.593435+0000","last_became_peered":"2024-02-26T06:20:31.593435+0000","last_unstale":"2024-02-26T06:21:57.439841+0000","last_undegraded":"2024-02-26T06:21:57.439841+0000","last_fullsized":"2024-02-26T06:21:57.439841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:43.193196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.908116+0000","last_change":"2024-02-26T06:20:31.061339+0000","last_active":"2024-02-26T06:21:28.908116+0000","last_peered":"2024-02-26T06:21:28.908116+0000","last_clean":"2024-02-26T06:21:28.908116+0000","last_became_active":"2024-02-26T06:20:31.060769+0000","last_became_peered":"2024-02-26T06:20:31.060769+0000","last_unstale":"2024-02-26T06:21:28.908116+0000","last_undegraded":"2024-02-26T06:21:28.908116+0000","last_fullsized":"2024-02-26T06:21:28.908116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:28:30.501375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432622+0000","last_change":"2024-02-26T06:20:31.606936+0000","last_active":"2024-02-26T06:21:57.432622+0000","last_peered":"2024-02-26T06:21:57.432622+0000","last_clean":"2024-02-26T06:21:57.432622+0000","last_became_active":"2024-02-26T06:20:31.606582+0000","last_became_peered":"2024-02-26T06:20:31.606582+0000","last_unstale":"2024-02-26T06:21:57.432622+0000","last_undegraded":"2024-02-26T06:21:57.432622+0000","last_fullsized":"2024-02-26T06:21:57.432622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:09.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907110+0000","last_change":"2024-02-26T06:20:31.063472+0000","last_active":"2024-02-26T06:21:28.907110+0000","last_peered":"2024-02-26T06:21:28.907110+0000","last_clean":"2024-02-26T06:21:28.907110+0000","last_became_active":"2024-02-26T06:20:31.063132+0000","last_became_peered":"2024-02-26T06:20:31.063132+0000","last_unstale":"2024-02-26T06:21:28.907110+0000","last_undegraded":"2024-02-26T06:21:28.907110+0000","last_fullsized":"2024-02-26T06:21:28.907110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:58.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.417983+0000","last_change":"2024-02-26T06:20:31.589238+0000","last_active":"2024-02-26T06:21:57.417983+0000","last_peered":"2024-02-26T06:21:57.417983+0000","last_clean":"2024-02-26T06:21:57.417983+0000","last_became_active":"2024-02-26T06:20:31.588929+0000","last_became_peered":"2024-02-26T06:20:31.588929+0000","last_unstale":"2024-02-26T06:21:57.417983+0000","last_undegraded":"2024-02-26T06:21:57.417983+0000","last_fullsized":"2024-02-26T06:21:57.417983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:19.282943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":111,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:01.234425+0000","last_change":"2024-02-26T06:20:31.061714+0000","last_active":"2024-02-26T06:22:01.234425+0000","last_peered":"2024-02-26T06:22:01.234425+0000","last_clean":"2024-02-26T06:22:01.234425+0000","last_became_active":"2024-02-26T06:20:31.061234+0000","last_became_peered":"2024-02-26T06:20:31.061234+0000","last_unstale":"2024-02-26T06:22:01.234425+0000","last_undegraded":"2024-02-26T06:22:01.234425+0000","last_fullsized":"2024-02-26T06:22:01.234425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:29:17.668327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.400148+0000","last_change":"2024-02-26T06:20:31.588861+0000","last_active":"2024-02-26T06:21:57.400148+0000","last_peered":"2024-02-26T06:21:57.400148+0000","last_clean":"2024-02-26T06:21:57.400148+0000","last_became_active":"2024-02-26T06:20:31.588530+0000","last_became_peered":"2024-02-26T06:20:31.588530+0000","last_unstale":"2024-02-26T06:21:57.400148+0000","last_undegraded":"2024-02-26T06:21:57.400148+0000","last_fullsized":"2024-02-26T06:21:57.400148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:12.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.900294+0000","last_change":"2024-02-26T06:20:31.077656+0000","last_active":"2024-02-26T06:21:28.900294+0000","last_peered":"2024-02-26T06:21:28.900294+0000","last_clean":"2024-02-26T06:21:28.900294+0000","last_became_active":"2024-02-26T06:20:31.077288+0000","last_became_peered":"2024-02-26T06:20:31.077288+0000","last_unstale":"2024-02-26T06:21:28.900294+0000","last_undegraded":"2024-02-26T06:21:28.900294+0000","last_fullsized":"2024-02-26T06:21:28.900294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:56:05.173751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439932+0000","last_change":"2024-02-26T06:20:31.608089+0000","last_active":"2024-02-26T06:21:57.439932+0000","last_peered":"2024-02-26T06:21:57.439932+0000","last_clean":"2024-02-26T06:21:57.439932+0000","last_became_active":"2024-02-26T06:20:31.607786+0000","last_became_peered":"2024-02-26T06:20:31.607786+0000","last_unstale":"2024-02-26T06:21:57.439932+0000","last_undegraded":"2024-02-26T06:21:57.439932+0000","last_fullsized":"2024-02-26T06:21:57.439932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:24.113286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.206874+0000","last_change":"2024-02-26T06:20:31.077063+0000","last_active":"2024-02-26T06:21:29.206874+0000","last_peered":"2024-02-26T06:21:29.206874+0000","last_clean":"2024-02-26T06:21:29.206874+0000","last_became_active":"2024-02-26T06:20:31.075147+0000","last_became_peered":"2024-02-26T06:20:31.075147+0000","last_unstale":"2024-02-26T06:21:29.206874+0000","last_undegraded":"2024-02-26T06:21:29.206874+0000","last_fullsized":"2024-02-26T06:21:29.206874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:07:13.434635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438618+0000","last_change":"2024-02-26T06:20:31.600135+0000","last_active":"2024-02-26T06:21:57.438618+0000","last_peered":"2024-02-26T06:21:57.438618+0000","last_clean":"2024-02-26T06:21:57.438618+0000","last_became_active":"2024-02-26T06:20:31.595622+0000","last_became_peered":"2024-02-26T06:20:31.595622+0000","last_unstale":"2024-02-26T06:21:57.438618+0000","last_undegraded":"2024-02-26T06:21:57.438618+0000","last_fullsized":"2024-02-26T06:21:57.438618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:55.835700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914337+0000","last_change":"2024-02-26T06:20:31.060299+0000","last_active":"2024-02-26T06:21:28.914337+0000","last_peered":"2024-02-26T06:21:28.914337+0000","last_clean":"2024-02-26T06:21:28.914337+0000","last_became_active":"2024-02-26T06:20:31.059938+0000","last_became_peered":"2024-02-26T06:20:31.059938+0000","last_unstale":"2024-02-26T06:21:28.914337+0000","last_undegraded":"2024-02-26T06:21:28.914337+0000","last_fullsized":"2024-02-26T06:21:28.914337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:24.144958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.444182+0000","last_change":"2024-02-26T06:20:31.601945+0000","last_active":"2024-02-26T06:21:57.444182+0000","last_peered":"2024-02-26T06:21:57.444182+0000","last_clean":"2024-02-26T06:21:57.444182+0000","last_became_active":"2024-02-26T06:20:31.601560+0000","last_became_peered":"2024-02-26T06:20:31.601560+0000","last_unstale":"2024-02-26T06:21:57.444182+0000","last_undegraded":"2024-02-26T06:21:57.444182+0000","last_fullsized":"2024-02-26T06:21:57.444182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:13.932623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":106,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:00.231862+0000","last_change":"2024-02-26T06:20:31.068334+0000","last_active":"2024-02-26T06:22:00.231862+0000","last_peered":"2024-02-26T06:22:00.231862+0000","last_clean":"2024-02-26T06:22:00.231862+0000","last_became_active":"2024-02-26T06:20:31.068020+0000","last_became_peered":"2024-02-26T06:20:31.068020+0000","last_unstale":"2024-02-26T06:22:00.231862+0000","last_undegraded":"2024-02-26T06:22:00.231862+0000","last_fullsized":"2024-02-26T06:22:00.231862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:23:01.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421567+0000","last_change":"2024-02-26T06:20:31.587862+0000","last_active":"2024-02-26T06:21:57.421567+0000","last_peered":"2024-02-26T06:21:57.421567+0000","last_clean":"2024-02-26T06:21:57.421567+0000","last_became_active":"2024-02-26T06:20:31.587551+0000","last_became_peered":"2024-02-26T06:20:31.587551+0000","last_unstale":"2024-02-26T06:21:57.421567+0000","last_undegraded":"2024-02-26T06:21:57.421567+0000","last_fullsized":"2024-02-26T06:21:57.421567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:07:15.933125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902499+0000","last_change":"2024-02-26T06:20:31.058378+0000","last_active":"2024-02-26T06:21:28.902499+0000","last_peered":"2024-02-26T06:21:28.902499+0000","last_clean":"2024-02-26T06:21:28.902499+0000","last_became_active":"2024-02-26T06:20:31.058040+0000","last_became_peered":"2024-02-26T06:20:31.058040+0000","last_unstale":"2024-02-26T06:21:28.902499+0000","last_undegraded":"2024-02-26T06:21:28.902499+0000","last_fullsized":"2024-02-26T06:21:28.902499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:33:40.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440116+0000","last_change":"2024-02-26T06:20:31.601837+0000","last_active":"2024-02-26T06:21:57.440116+0000","last_peered":"2024-02-26T06:21:57.440116+0000","last_clean":"2024-02-26T06:21:57.440116+0000","last_became_active":"2024-02-26T06:20:31.597310+0000","last_became_peered":"2024-02-26T06:20:31.597310+0000","last_unstale":"2024-02-26T06:21:57.440116+0000","last_undegraded":"2024-02-26T06:21:57.440116+0000","last_fullsized":"2024-02-26T06:21:57.440116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.383510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914898+0000","last_change":"2024-02-26T06:20:31.076158+0000","last_active":"2024-02-26T06:21:28.914898+0000","last_peered":"2024-02-26T06:21:28.914898+0000","last_clean":"2024-02-26T06:21:28.914898+0000","last_became_active":"2024-02-26T06:20:31.075795+0000","last_became_peered":"2024-02-26T06:20:31.075795+0000","last_unstale":"2024-02-26T06:21:28.914898+0000","last_undegraded":"2024-02-26T06:21:28.914898+0000","last_fullsized":"2024-02-26T06:21:28.914898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:38.491674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421042+0000","last_change":"2024-02-26T06:20:31.586122+0000","last_active":"2024-02-26T06:21:57.421042+0000","last_peered":"2024-02-26T06:21:57.421042+0000","last_clean":"2024-02-26T06:21:57.421042+0000","last_became_active":"2024-02-26T06:20:31.585070+0000","last_became_peered":"2024-02-26T06:20:31.585070+0000","last_unstale":"2024-02-26T06:21:57.421042+0000","last_undegraded":"2024-02-26T06:21:57.421042+0000","last_fullsized":"2024-02-26T06:21:57.421042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:11.034252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":247,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:01.965158+0000","last_change":"2024-02-26T06:21:28.913681+0000","last_active":"2024-02-26T06:22:01.965158+0000","last_peered":"2024-02-26T06:22:01.965158+0000","last_clean":"2024-02-26T06:22:01.965158+0000","last_became_active":"2024-02-26T06:21:28.905006+0000","last_became_peered":"2024-02-26T06:21:28.905006+0000","last_unstale":"2024-02-26T06:22:01.965158+0000","last_undegraded":"2024-02-26T06:22:01.965158+0000","last_fullsized":"2024-02-26T06:22:01.965158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:41.432983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61122,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":176,"num_read_kb":147,"num_write":33,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60873,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410737+0000","last_change":"2024-02-26T06:20:31.584757+0000","last_active":"2024-02-26T06:21:57.410737+0000","last_peered":"2024-02-26T06:21:57.410737+0000","last_clean":"2024-02-26T06:21:57.410737+0000","last_became_active":"2024-02-26T06:20:31.583890+0000","last_became_peered":"2024-02-26T06:20:31.583890+0000","last_unstale":"2024-02-26T06:21:57.410737+0000","last_undegraded":"2024-02-26T06:21:57.410737+0000","last_fullsized":"2024-02-26T06:21:57.410737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:10:58.280082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903234+0000","last_change":"2024-02-26T06:20:31.076044+0000","last_active":"2024-02-26T06:21:28.903234+0000","last_peered":"2024-02-26T06:21:28.903234+0000","last_clean":"2024-02-26T06:21:28.903234+0000","last_became_active":"2024-02-26T06:20:31.075744+0000","last_became_peered":"2024-02-26T06:20:31.075744+0000","last_unstale":"2024-02-26T06:21:28.903234+0000","last_undegraded":"2024-02-26T06:21:28.903234+0000","last_fullsized":"2024-02-26T06:21:28.903234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:35.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.443679+0000","last_change":"2024-02-26T06:20:31.602169+0000","last_active":"2024-02-26T06:21:57.443679+0000","last_peered":"2024-02-26T06:21:57.443679+0000","last_clean":"2024-02-26T06:21:57.443679+0000","last_became_active":"2024-02-26T06:20:31.601280+0000","last_became_peered":"2024-02-26T06:20:31.601280+0000","last_unstale":"2024-02-26T06:21:57.443679+0000","last_undegraded":"2024-02-26T06:21:57.443679+0000","last_fullsized":"2024-02-26T06:21:57.443679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:07.652549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914086+0000","last_change":"2024-02-26T06:20:31.080482+0000","last_active":"2024-02-26T06:21:28.914086+0000","last_peered":"2024-02-26T06:21:28.914086+0000","last_clean":"2024-02-26T06:21:28.914086+0000","last_became_active":"2024-02-26T06:20:31.080188+0000","last_became_peered":"2024-02-26T06:20:31.080188+0000","last_unstale":"2024-02-26T06:21:28.914086+0000","last_undegraded":"2024-02-26T06:21:28.914086+0000","last_fullsized":"2024-02-26T06:21:28.914086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:32.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.433448+0000","last_change":"2024-02-26T06:20:31.599753+0000","last_active":"2024-02-26T06:21:57.433448+0000","last_peered":"2024-02-26T06:21:57.433448+0000","last_clean":"2024-02-26T06:21:57.433448+0000","last_became_active":"2024-02-26T06:20:31.599409+0000","last_became_peered":"2024-02-26T06:20:31.599409+0000","last_unstale":"2024-02-26T06:21:57.433448+0000","last_undegraded":"2024-02-26T06:21:57.433448+0000","last_fullsized":"2024-02-26T06:21:57.433448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:43.887431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:58.261670+0000","last_change":"2024-02-26T06:20:31.059656+0000","last_active":"2024-02-26T06:21:58.261670+0000","last_peered":"2024-02-26T06:21:58.261670+0000","last_clean":"2024-02-26T06:21:58.261670+0000","last_became_active":"2024-02-26T06:20:31.059289+0000","last_became_peered":"2024-02-26T06:20:31.059289+0000","last_unstale":"2024-02-26T06:21:58.261670+0000","last_undegraded":"2024-02-26T06:21:58.261670+0000","last_fullsized":"2024-02-26T06:21:58.261670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:08:57.650876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.402208+0000","last_change":"2024-02-26T06:21:28.907778+0000","last_active":"2024-02-26T06:21:57.402208+0000","last_peered":"2024-02-26T06:21:57.402208+0000","last_clean":"2024-02-26T06:21:57.402208+0000","last_became_active":"2024-02-26T06:21:28.907185+0000","last_became_peered":"2024-02-26T06:21:28.907185+0000","last_unstale":"2024-02-26T06:21:57.402208+0000","last_undegraded":"2024-02-26T06:21:57.402208+0000","last_fullsized":"2024-02-26T06:21:57.402208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:39.572867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.943152+0000","last_change":"2024-02-26T06:21:28.943152+0000","last_active":"2024-02-26T06:21:28.943152+0000","last_peered":"2024-02-26T06:21:28.943152+0000","last_clean":"2024-02-26T06:21:28.943152+0000","last_became_active":"2024-02-26T06:21:28.908090+0000","last_became_peered":"2024-02-26T06:21:28.908090+0000","last_unstale":"2024-02-26T06:21:28.943152+0000","last_undegraded":"2024-02-26T06:21:28.943152+0000","last_fullsized":"2024-02-26T06:21:28.943152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:56.254015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414233+0000","last_change":"2024-02-26T06:20:31.586151+0000","last_active":"2024-02-26T06:21:57.414233+0000","last_peered":"2024-02-26T06:21:57.414233+0000","last_clean":"2024-02-26T06:21:57.414233+0000","last_became_active":"2024-02-26T06:20:31.585615+0000","last_became_peered":"2024-02-26T06:20:31.585615+0000","last_unstale":"2024-02-26T06:21:57.414233+0000","last_undegraded":"2024-02-26T06:21:57.414233+0000","last_fullsized":"2024-02-26T06:21:57.414233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907539+0000","last_change":"2024-02-26T06:20:31.062340+0000","last_active":"2024-02-26T06:21:28.907539+0000","last_peered":"2024-02-26T06:21:28.907539+0000","last_clean":"2024-02-26T06:21:28.907539+0000","last_became_active":"2024-02-26T06:20:31.062023+0000","last_became_peered":"2024-02-26T06:20:31.062023+0000","last_unstale":"2024-02-26T06:21:28.907539+0000","last_undegraded":"2024-02-26T06:21:28.907539+0000","last_fullsized":"2024-02-26T06:21:28.907539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:45.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413393+0000","last_change":"2024-02-26T06:20:31.585190+0000","last_active":"2024-02-26T06:21:57.413393+0000","last_peered":"2024-02-26T06:21:57.413393+0000","last_clean":"2024-02-26T06:21:57.413393+0000","last_became_active":"2024-02-26T06:20:31.584883+0000","last_became_peered":"2024-02-26T06:20:31.584883+0000","last_unstale":"2024-02-26T06:21:57.413393+0000","last_undegraded":"2024-02-26T06:21:57.413393+0000","last_fullsized":"2024-02-26T06:21:57.413393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:47:51.843824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904004+0000","last_change":"2024-02-26T06:20:31.075573+0000","last_active":"2024-02-26T06:21:28.904004+0000","last_peered":"2024-02-26T06:21:28.904004+0000","last_clean":"2024-02-26T06:21:28.904004+0000","last_became_active":"2024-02-26T06:20:31.075153+0000","last_became_peered":"2024-02-26T06:20:31.075153+0000","last_unstale":"2024-02-26T06:21:28.904004+0000","last_undegraded":"2024-02-26T06:21:28.904004+0000","last_fullsized":"2024-02-26T06:21:28.904004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:43.886858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432026+0000","last_change":"2024-02-26T06:20:31.604400+0000","last_active":"2024-02-26T06:21:57.432026+0000","last_peered":"2024-02-26T06:21:57.432026+0000","last_clean":"2024-02-26T06:21:57.432026+0000","last_became_active":"2024-02-26T06:20:31.604068+0000","last_became_peered":"2024-02-26T06:20:31.604068+0000","last_unstale":"2024-02-26T06:21:57.432026+0000","last_undegraded":"2024-02-26T06:21:57.432026+0000","last_fullsized":"2024-02-26T06:21:57.432026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:28.792297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902841+0000","last_change":"2024-02-26T06:20:31.072896+0000","last_active":"2024-02-26T06:21:28.902841+0000","last_peered":"2024-02-26T06:21:28.902841+0000","last_clean":"2024-02-26T06:21:28.902841+0000","last_became_active":"2024-02-26T06:20:31.072571+0000","last_became_peered":"2024-02-26T06:20:31.072571+0000","last_unstale":"2024-02-26T06:21:28.902841+0000","last_undegraded":"2024-02-26T06:21:28.902841+0000","last_fullsized":"2024-02-26T06:21:28.902841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:39.054737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.415829+0000","last_change":"2024-02-26T06:21:28.906840+0000","last_active":"2024-02-26T06:21:57.415829+0000","last_peered":"2024-02-26T06:21:57.415829+0000","last_clean":"2024-02-26T06:21:57.415829+0000","last_became_active":"2024-02-26T06:21:28.905881+0000","last_became_peered":"2024-02-26T06:21:28.905881+0000","last_unstale":"2024-02-26T06:21:57.415829+0000","last_undegraded":"2024-02-26T06:21:57.415829+0000","last_fullsized":"2024-02-26T06:21:57.415829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:47.754145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.406314+0000","last_change":"2024-02-26T06:20:31.588989+0000","last_active":"2024-02-26T06:21:57.406314+0000","last_peered":"2024-02-26T06:21:57.406314+0000","last_clean":"2024-02-26T06:21:57.406314+0000","last_became_active":"2024-02-26T06:20:31.588683+0000","last_became_peered":"2024-02-26T06:20:31.588683+0000","last_unstale":"2024-02-26T06:21:57.406314+0000","last_undegraded":"2024-02-26T06:21:57.406314+0000","last_fullsized":"2024-02-26T06:21:57.406314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:29.687622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:02.272155+0000","last_change":"2024-02-26T06:20:31.081111+0000","last_active":"2024-02-26T06:22:02.272155+0000","last_peered":"2024-02-26T06:22:02.272155+0000","last_clean":"2024-02-26T06:22:02.272155+0000","last_became_active":"2024-02-26T06:20:31.080788+0000","last_became_peered":"2024-02-26T06:20:31.080788+0000","last_unstale":"2024-02-26T06:22:02.272155+0000","last_undegraded":"2024-02-26T06:22:02.272155+0000","last_fullsized":"2024-02-26T06:22:02.272155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:44:34.400256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440753+0000","last_change":"2024-02-26T06:21:29.211650+0000","last_active":"2024-02-26T06:21:57.440753+0000","last_peered":"2024-02-26T06:21:57.440753+0000","last_clean":"2024-02-26T06:21:57.440753+0000","last_became_active":"2024-02-26T06:21:29.210407+0000","last_became_peered":"2024-02-26T06:21:29.210407+0000","last_unstale":"2024-02-26T06:21:57.440753+0000","last_undegraded":"2024-02-26T06:21:57.440753+0000","last_fullsized":"2024-02-26T06:21:57.440753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:22:00.359506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898303+0000","last_change":"2024-02-26T06:20:31.061702+0000","last_active":"2024-02-26T06:21:28.898303+0000","last_peered":"2024-02-26T06:21:28.898303+0000","last_clean":"2024-02-26T06:21:28.898303+0000","last_became_active":"2024-02-26T06:20:31.061358+0000","last_became_peered":"2024-02-26T06:20:31.061358+0000","last_unstale":"2024-02-26T06:21:28.898303+0000","last_undegraded":"2024-02-26T06:21:28.898303+0000","last_fullsized":"2024-02-26T06:21:28.898303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:25:41.652032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903059+0000","last_change":"2024-02-26T06:19:10.834318+0000","last_active":"2024-02-26T06:21:28.903059+0000","last_peered":"2024-02-26T06:21:28.903059+0000","last_clean":"2024-02-26T06:21:28.903059+0000","last_became_active":"2024-02-26T06:19:10.833869+0000","last_became_peered":"2024-02-26T06:19:10.833869+0000","last_unstale":"2024-02-26T06:21:28.903059+0000","last_undegraded":"2024-02-26T06:21:28.903059+0000","last_fullsized":"2024-02-26T06:21:28.903059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_clean_scrub_stamp":"2024-02-26T06:15:16.238016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:57.078533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428577+0000","last_change":"2024-02-26T06:20:31.600738+0000","last_active":"2024-02-26T06:21:57.428577+0000","last_peered":"2024-02-26T06:21:57.428577+0000","last_clean":"2024-02-26T06:21:57.428577+0000","last_became_active":"2024-02-26T06:20:31.599144+0000","last_became_peered":"2024-02-26T06:20:31.599144+0000","last_unstale":"2024-02-26T06:21:57.428577+0000","last_undegraded":"2024-02-26T06:21:57.428577+0000","last_fullsized":"2024-02-26T06:21:57.428577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:01:16.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903127+0000","last_change":"2024-02-26T06:20:31.063563+0000","last_active":"2024-02-26T06:21:28.903127+0000","last_peered":"2024-02-26T06:21:28.903127+0000","last_clean":"2024-02-26T06:21:28.903127+0000","last_became_active":"2024-02-26T06:20:31.063212+0000","last_became_peered":"2024-02-26T06:20:31.063212+0000","last_unstale":"2024-02-26T06:21:28.903127+0000","last_undegraded":"2024-02-26T06:21:28.903127+0000","last_fullsized":"2024-02-26T06:21:28.903127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:16.480553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907915+0000","last_change":"2024-02-26T06:20:31.075104+0000","last_active":"2024-02-26T06:21:28.907915+0000","last_peered":"2024-02-26T06:21:28.907915+0000","last_clean":"2024-02-26T06:21:28.907915+0000","last_became_active":"2024-02-26T06:20:31.074777+0000","last_became_peered":"2024-02-26T06:20:31.074777+0000","last_unstale":"2024-02-26T06:21:28.907915+0000","last_undegraded":"2024-02-26T06:21:28.907915+0000","last_fullsized":"2024-02-26T06:21:28.907915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:32:20.511703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.416264+0000","last_change":"2024-02-26T06:20:31.605038+0000","last_active":"2024-02-26T06:21:57.416264+0000","last_peered":"2024-02-26T06:21:57.416264+0000","last_clean":"2024-02-26T06:21:57.416264+0000","last_became_active":"2024-02-26T06:20:31.603329+0000","last_became_peered":"2024-02-26T06:20:31.603329+0000","last_unstale":"2024-02-26T06:21:57.416264+0000","last_undegraded":"2024-02-26T06:21:57.416264+0000","last_fullsized":"2024-02-26T06:21:57.416264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:45.066836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904225+0000","last_change":"2024-02-26T06:20:31.084613+0000","last_active":"2024-02-26T06:21:28.904225+0000","last_peered":"2024-02-26T06:21:28.904225+0000","last_clean":"2024-02-26T06:21:28.904225+0000","last_became_active":"2024-02-26T06:20:31.084021+0000","last_became_peered":"2024-02-26T06:20:31.084021+0000","last_unstale":"2024-02-26T06:21:28.904225+0000","last_undegraded":"2024-02-26T06:21:28.904225+0000","last_fullsized":"2024-02-26T06:21:28.904225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:52:42.096437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438877+0000","last_change":"2024-02-26T06:20:31.590367+0000","last_active":"2024-02-26T06:21:57.438877+0000","last_peered":"2024-02-26T06:21:57.438877+0000","last_clean":"2024-02-26T06:21:57.438877+0000","last_became_active":"2024-02-26T06:20:31.582914+0000","last_became_peered":"2024-02-26T06:20:31.582914+0000","last_unstale":"2024-02-26T06:21:57.438877+0000","last_undegraded":"2024-02-26T06:21:57.438877+0000","last_fullsized":"2024-02-26T06:21:57.438877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:02:47.496527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898447+0000","last_change":"2024-02-26T06:20:31.070190+0000","last_active":"2024-02-26T06:21:28.898447+0000","last_peered":"2024-02-26T06:21:28.898447+0000","last_clean":"2024-02-26T06:21:28.898447+0000","last_became_active":"2024-02-26T06:20:31.069854+0000","last_became_peered":"2024-02-26T06:20:31.069854+0000","last_unstale":"2024-02-26T06:21:28.898447+0000","last_undegraded":"2024-02-26T06:21:28.898447+0000","last_fullsized":"2024-02-26T06:21:28.898447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:57.728821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440988+0000","last_change":"2024-02-26T06:20:31.595388+0000","last_active":"2024-02-26T06:21:57.440988+0000","last_peered":"2024-02-26T06:21:57.440988+0000","last_clean":"2024-02-26T06:21:57.440988+0000","last_became_active":"2024-02-26T06:20:31.594970+0000","last_became_peered":"2024-02-26T06:20:31.594970+0000","last_unstale":"2024-02-26T06:21:57.440988+0000","last_undegraded":"2024-02-26T06:21:57.440988+0000","last_fullsized":"2024-02-26T06:21:57.440988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:12:52.182988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902824+0000","last_change":"2024-02-26T06:20:31.066256+0000","last_active":"2024-02-26T06:21:28.902824+0000","last_peered":"2024-02-26T06:21:28.902824+0000","last_clean":"2024-02-26T06:21:28.902824+0000","last_became_active":"2024-02-26T06:20:31.065956+0000","last_became_peered":"2024-02-26T06:20:31.065956+0000","last_unstale":"2024-02-26T06:21:28.902824+0000","last_undegraded":"2024-02-26T06:21:28.902824+0000","last_fullsized":"2024-02-26T06:21:28.902824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:35.405114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.379908+0000","last_change":"2024-02-26T06:20:31.585102+0000","last_active":"2024-02-26T06:21:57.379908+0000","last_peered":"2024-02-26T06:21:57.379908+0000","last_clean":"2024-02-26T06:21:57.379908+0000","last_became_active":"2024-02-26T06:20:31.584351+0000","last_became_peered":"2024-02-26T06:20:31.584351+0000","last_unstale":"2024-02-26T06:21:57.379908+0000","last_undegraded":"2024-02-26T06:21:57.379908+0000","last_fullsized":"2024-02-26T06:21:57.379908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:09.708752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.897644+0000","last_change":"2024-02-26T06:20:31.079664+0000","last_active":"2024-02-26T06:21:28.897644+0000","last_peered":"2024-02-26T06:21:28.897644+0000","last_clean":"2024-02-26T06:21:28.897644+0000","last_became_active":"2024-02-26T06:20:31.079234+0000","last_became_peered":"2024-02-26T06:20:31.079234+0000","last_unstale":"2024-02-26T06:21:28.897644+0000","last_undegraded":"2024-02-26T06:21:28.897644+0000","last_fullsized":"2024-02-26T06:21:28.897644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:37.407215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439347+0000","last_change":"2024-02-26T06:20:31.597416+0000","last_active":"2024-02-26T06:21:57.439347+0000","last_peered":"2024-02-26T06:21:57.439347+0000","last_clean":"2024-02-26T06:21:57.439347+0000","last_became_active":"2024-02-26T06:20:31.597094+0000","last_became_peered":"2024-02-26T06:20:31.597094+0000","last_unstale":"2024-02-26T06:21:57.439347+0000","last_undegraded":"2024-02-26T06:21:57.439347+0000","last_fullsized":"2024-02-26T06:21:57.439347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:24.774298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903066+0000","last_change":"2024-02-26T06:20:31.074372+0000","last_active":"2024-02-26T06:21:28.903066+0000","last_peered":"2024-02-26T06:21:28.903066+0000","last_clean":"2024-02-26T06:21:28.903066+0000","last_became_active":"2024-02-26T06:20:31.074041+0000","last_became_peered":"2024-02-26T06:20:31.074041+0000","last_unstale":"2024-02-26T06:21:28.903066+0000","last_undegraded":"2024-02-26T06:21:28.903066+0000","last_fullsized":"2024-02-26T06:21:28.903066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:01.367668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441767+0000","last_change":"2024-02-26T06:20:31.599588+0000","last_active":"2024-02-26T06:21:57.441767+0000","last_peered":"2024-02-26T06:21:57.441767+0000","last_clean":"2024-02-26T06:21:57.441767+0000","last_became_active":"2024-02-26T06:20:31.599240+0000","last_became_peered":"2024-02-26T06:20:31.599240+0000","last_unstale":"2024-02-26T06:21:57.441767+0000","last_undegraded":"2024-02-26T06:21:57.441767+0000","last_fullsized":"2024-02-26T06:21:57.441767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:07:14.799565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914009+0000","last_change":"2024-02-26T06:21:28.914009+0000","last_active":"2024-02-26T06:21:28.914009+0000","last_peered":"2024-02-26T06:21:28.914009+0000","last_clean":"2024-02-26T06:21:28.914009+0000","last_became_active":"2024-02-26T06:21:28.906712+0000","last_became_peered":"2024-02-26T06:21:28.906712+0000","last_unstale":"2024-02-26T06:21:28.914009+0000","last_undegraded":"2024-02-26T06:21:28.914009+0000","last_fullsized":"2024-02-26T06:21:28.914009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:21:36.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432745+0000","last_change":"2024-02-26T06:20:31.603955+0000","last_active":"2024-02-26T06:21:57.432745+0000","last_peered":"2024-02-26T06:21:57.432745+0000","last_clean":"2024-02-26T06:21:57.432745+0000","last_became_active":"2024-02-26T06:20:31.603635+0000","last_became_peered":"2024-02-26T06:20:31.603635+0000","last_unstale":"2024-02-26T06:21:57.432745+0000","last_undegraded":"2024-02-26T06:21:57.432745+0000","last_fullsized":"2024-02-26T06:21:57.432745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:56:59.060204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903732+0000","last_change":"2024-02-26T06:20:31.074917+0000","last_active":"2024-02-26T06:21:28.903732+0000","last_peered":"2024-02-26T06:21:28.903732+0000","last_clean":"2024-02-26T06:21:28.903732+0000","last_became_active":"2024-02-26T06:20:31.074590+0000","last_became_peered":"2024-02-26T06:20:31.074590+0000","last_unstale":"2024-02-26T06:21:28.903732+0000","last_undegraded":"2024-02-26T06:21:28.903732+0000","last_fullsized":"2024-02-26T06:21:28.903732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:43.160268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420733+0000","last_change":"2024-02-26T06:20:31.582236+0000","last_active":"2024-02-26T06:21:57.420733+0000","last_peered":"2024-02-26T06:21:57.420733+0000","last_clean":"2024-02-26T06:21:57.420733+0000","last_became_active":"2024-02-26T06:20:31.581851+0000","last_became_peered":"2024-02-26T06:20:31.581851+0000","last_unstale":"2024-02-26T06:21:57.420733+0000","last_undegraded":"2024-02-26T06:21:57.420733+0000","last_fullsized":"2024-02-26T06:21:57.420733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:53.214338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914141+0000","last_change":"2024-02-26T06:20:31.068179+0000","last_active":"2024-02-26T06:21:28.914141+0000","last_peered":"2024-02-26T06:21:28.914141+0000","last_clean":"2024-02-26T06:21:28.914141+0000","last_became_active":"2024-02-26T06:20:31.067888+0000","last_became_peered":"2024-02-26T06:20:31.067888+0000","last_unstale":"2024-02-26T06:21:28.914141+0000","last_undegraded":"2024-02-26T06:21:28.914141+0000","last_fullsized":"2024-02-26T06:21:28.914141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:47:39.223749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.424729+0000","last_change":"2024-02-26T06:20:31.598820+0000","last_active":"2024-02-26T06:21:57.424729+0000","last_peered":"2024-02-26T06:21:57.424729+0000","last_clean":"2024-02-26T06:21:57.424729+0000","last_became_active":"2024-02-26T06:20:31.598469+0000","last_became_peered":"2024-02-26T06:20:31.598469+0000","last_unstale":"2024-02-26T06:21:57.424729+0000","last_undegraded":"2024-02-26T06:21:57.424729+0000","last_fullsized":"2024-02-26T06:21:57.424729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:41:15.713748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.207033+0000","last_change":"2024-02-26T06:20:31.080988+0000","last_active":"2024-02-26T06:21:29.207033+0000","last_peered":"2024-02-26T06:21:29.207033+0000","last_clean":"2024-02-26T06:21:29.207033+0000","last_became_active":"2024-02-26T06:20:31.080654+0000","last_became_peered":"2024-02-26T06:20:31.080654+0000","last_unstale":"2024-02-26T06:21:29.207033+0000","last_undegraded":"2024-02-26T06:21:29.207033+0000","last_fullsized":"2024-02-26T06:21:29.207033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:50.315162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414395+0000","last_change":"2024-02-26T06:20:31.586057+0000","last_active":"2024-02-26T06:21:57.414395+0000","last_peered":"2024-02-26T06:21:57.414395+0000","last_clean":"2024-02-26T06:21:57.414395+0000","last_became_active":"2024-02-26T06:20:31.585756+0000","last_became_peered":"2024-02-26T06:20:31.585756+0000","last_unstale":"2024-02-26T06:21:57.414395+0000","last_undegraded":"2024-02-26T06:21:57.414395+0000","last_fullsized":"2024-02-26T06:21:57.414395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:59:07.032287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.411546+0000","last_change":"2024-02-26T06:20:31.589067+0000","last_active":"2024-02-26T06:21:57.411546+0000","last_peered":"2024-02-26T06:21:57.411546+0000","last_clean":"2024-02-26T06:21:57.411546+0000","last_became_active":"2024-02-26T06:20:31.588769+0000","last_became_peered":"2024-02-26T06:20:31.588769+0000","last_unstale":"2024-02-26T06:21:57.411546+0000","last_undegraded":"2024-02-26T06:21:57.411546+0000","last_fullsized":"2024-02-26T06:21:57.411546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:43.211127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412415+0000","last_change":"2024-02-26T06:20:31.583838+0000","last_active":"2024-02-26T06:21:57.412415+0000","last_peered":"2024-02-26T06:21:57.412415+0000","last_clean":"2024-02-26T06:21:57.412415+0000","last_became_active":"2024-02-26T06:20:31.583524+0000","last_became_peered":"2024-02-26T06:20:31.583524+0000","last_unstale":"2024-02-26T06:21:57.412415+0000","last_undegraded":"2024-02-26T06:21:57.412415+0000","last_fullsized":"2024-02-26T06:21:57.412415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:21.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413932+0000","last_change":"2024-02-26T06:20:31.597901+0000","last_active":"2024-02-26T06:21:57.413932+0000","last_peered":"2024-02-26T06:21:57.413932+0000","last_clean":"2024-02-26T06:21:57.413932+0000","last_became_active":"2024-02-26T06:20:31.597633+0000","last_became_peered":"2024-02-26T06:20:31.597633+0000","last_unstale":"2024-02-26T06:21:57.413932+0000","last_undegraded":"2024-02-26T06:21:57.413932+0000","last_fullsized":"2024-02-26T06:21:57.413932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:14.084579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438872+0000","last_change":"2024-02-26T06:21:29.210694+0000","last_active":"2024-02-26T06:21:57.438872+0000","last_peered":"2024-02-26T06:21:57.438872+0000","last_clean":"2024-02-26T06:21:57.438872+0000","last_became_active":"2024-02-26T06:21:29.209421+0000","last_became_peered":"2024-02-26T06:21:29.209421+0000","last_unstale":"2024-02-26T06:21:57.438872+0000","last_undegraded":"2024-02-26T06:21:57.438872+0000","last_fullsized":"2024-02-26T06:21:57.438872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:46.511066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.430810+0000","last_change":"2024-02-26T06:20:31.605866+0000","last_active":"2024-02-26T06:21:57.430810+0000","last_peered":"2024-02-26T06:21:57.430810+0000","last_clean":"2024-02-26T06:21:57.430810+0000","last_became_active":"2024-02-26T06:20:31.605404+0000","last_became_peered":"2024-02-26T06:20:31.605404+0000","last_unstale":"2024-02-26T06:21:57.430810+0000","last_undegraded":"2024-02-26T06:21:57.430810+0000","last_fullsized":"2024-02-26T06:21:57.430810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:03:25.055462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413581+0000","last_change":"2024-02-26T06:20:31.585214+0000","last_active":"2024-02-26T06:21:57.413581+0000","last_peered":"2024-02-26T06:21:57.413581+0000","last_clean":"2024-02-26T06:21:57.413581+0000","last_became_active":"2024-02-26T06:20:31.584895+0000","last_became_peered":"2024-02-26T06:20:31.584895+0000","last_unstale":"2024-02-26T06:21:57.413581+0000","last_undegraded":"2024-02-26T06:21:57.413581+0000","last_fullsized":"2024-02-26T06:21:57.413581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:38.507499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414981+0000","last_change":"2024-02-26T06:20:31.582189+0000","last_active":"2024-02-26T06:21:57.414981+0000","last_peered":"2024-02-26T06:21:57.414981+0000","last_clean":"2024-02-26T06:21:57.414981+0000","last_became_active":"2024-02-26T06:20:31.580729+0000","last_became_peered":"2024-02-26T06:20:31.580729+0000","last_unstale":"2024-02-26T06:21:57.414981+0000","last_undegraded":"2024-02-26T06:21:57.414981+0000","last_fullsized":"2024-02-26T06:21:57.414981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:21.337261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.435771+0000","last_change":"2024-02-26T06:20:31.588894+0000","last_active":"2024-02-26T06:21:57.435771+0000","last_peered":"2024-02-26T06:21:57.435771+0000","last_clean":"2024-02-26T06:21:57.435771+0000","last_became_active":"2024-02-26T06:20:31.586362+0000","last_became_peered":"2024-02-26T06:20:31.586362+0000","last_unstale":"2024-02-26T06:21:57.435771+0000","last_undegraded":"2024-02-26T06:21:57.435771+0000","last_fullsized":"2024-02-26T06:21:57.435771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:28:30.009901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.434220+0000","last_change":"2024-02-26T06:20:31.608264+0000","last_active":"2024-02-26T06:21:57.434220+0000","last_peered":"2024-02-26T06:21:57.434220+0000","last_clean":"2024-02-26T06:21:57.434220+0000","last_became_active":"2024-02-26T06:20:31.607855+0000","last_became_peered":"2024-02-26T06:20:31.607855+0000","last_unstale":"2024-02-26T06:21:57.434220+0000","last_undegraded":"2024-02-26T06:21:57.434220+0000","last_fullsized":"2024-02-26T06:21:57.434220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:20.169489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441525+0000","last_change":"2024-02-26T06:20:31.591000+0000","last_active":"2024-02-26T06:21:57.441525+0000","last_peered":"2024-02-26T06:21:57.441525+0000","last_clean":"2024-02-26T06:21:57.441525+0000","last_became_active":"2024-02-26T06:20:31.588201+0000","last_became_peered":"2024-02-26T06:20:31.588201+0000","last_unstale":"2024-02-26T06:21:57.441525+0000","last_undegraded":"2024-02-26T06:21:57.441525+0000","last_fullsized":"2024-02-26T06:21:57.441525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:47:59.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441183+0000","last_change":"2024-02-26T06:20:31.590398+0000","last_active":"2024-02-26T06:21:57.441183+0000","last_peered":"2024-02-26T06:21:57.441183+0000","last_clean":"2024-02-26T06:21:57.441183+0000","last_became_active":"2024-02-26T06:20:31.590042+0000","last_became_peered":"2024-02-26T06:20:31.590042+0000","last_unstale":"2024-02-26T06:21:57.441183+0000","last_undegraded":"2024-02-26T06:21:57.441183+0000","last_fullsized":"2024-02-26T06:21:57.441183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:40:01.233978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440741+0000","last_change":"2024-02-26T06:20:31.611698+0000","last_active":"2024-02-26T06:21:57.440741+0000","last_peered":"2024-02-26T06:21:57.440741+0000","last_clean":"2024-02-26T06:21:57.440741+0000","last_became_active":"2024-02-26T06:20:31.610979+0000","last_became_peered":"2024-02-26T06:20:31.610979+0000","last_unstale":"2024-02-26T06:21:57.440741+0000","last_undegraded":"2024-02-26T06:21:57.440741+0000","last_fullsized":"2024-02-26T06:21:57.440741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:44.604373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414103+0000","last_change":"2024-02-26T06:20:31.582016+0000","last_active":"2024-02-26T06:21:57.414103+0000","last_peered":"2024-02-26T06:21:57.414103+0000","last_clean":"2024-02-26T06:21:57.414103+0000","last_became_active":"2024-02-26T06:20:31.581689+0000","last_became_peered":"2024-02-26T06:20:31.581689+0000","last_unstale":"2024-02-26T06:21:57.414103+0000","last_undegraded":"2024-02-26T06:21:57.414103+0000","last_fullsized":"2024-02-26T06:21:57.414103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:04:11.030182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284221,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65264,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":529,"num_read_kb":334,"num_write":164,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":60985,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":195792,"data_compressed":13382,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459479,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90500000000000003,"5min":1.018,"15min":1.018},"last":0.749}]},{"osd":1,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.45700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68400000000000005}]},{"osd":2,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.65000000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63100000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71599999999999997}]},{"osd":5,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.311,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.091,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":0.84099999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.996,"5min":1.179,"15min":1.179},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.443,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66300000000000003},"min":{"1min":0.51300000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.89200000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:21:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.628}]},{"osd":9,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1830000000000001,"5min":1.21,"15min":1.21},"last":0.94799999999999995}]},{"osd":10,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.57899999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.93600000000000005,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.52600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.874}]}]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":1112502,"data_compressed":43797,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":6.2229999999999999,"5min":6.2229999999999999,"15min":6.2229999999999999},"last":1.0629999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32000000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.28899999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.71399999999999997}]},{"osd":3,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.81899999999999995,"15min":0.83499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86099999999999999,"5min":1.403,"15min":1.403},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88800000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34100000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.871,"15min":0.88500000000000001},"min":{"1min":0.56499999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.091,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.56799999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2629999999999999,"5min":1.284,"15min":1.284},"last":0.72499999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.379}]},{"osd":8,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.58799999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.77600000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.97799999999999998,"15min":0.98799999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.202,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.73999999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93600000000000005,"5min":0.998,"15min":0.998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.97499999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75700000000000001}]}]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":459603,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.4119999999999999,"15min":1.4690000000000001},"min":{"1min":0.73199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8939999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.355},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.3500000000000001,"15min":1.3799999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.069,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.97799999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6459999999999999,"5min":1.923,"15min":1.923},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.323,"15min":1.391},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.504,"5min":2.407,"15min":2.407},"last":1.625}]},{"osd":2,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4690000000000001,"15min":1.534},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.298,"5min":2.609,"15min":2.609},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.5600000000000001,"15min":1.607},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.2370000000000001,"5min":3.681,"15min":3.681},"last":1.512}]},{"osd":3,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.804,"15min":1.913},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.4929999999999999,"5min":3.016,"15min":3.016},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.7689999999999999,"15min":1.823},"min":{"1min":0.99199999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":2.8610000000000002,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.784,"15min":1.8320000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.6419999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.7529999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.6819999999999999,"15min":1.802},"min":{"1min":0.64000000000000001,"5min":0.624,"15min":0.624},"max":{"1min":2.5089999999999999,"5min":3.98,"15min":3.98},"last":1.125}]},{"osd":5,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.883,"15min":1.9970000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6030000000000002,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.8100000000000001,"5min":1.905,"15min":1.9910000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.2050000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":2.169,"15min":2.2749999999999999},"min":{"1min":0.83999999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":3.2240000000000002,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.2749999999999999},{"interface":"front","average":{"1min":2.1400000000000001,"5min":2.2240000000000002,"15min":2.3079999999999998},"min":{"1min":0.91900000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.1600000000000001,"5min":3.835,"15min":3.835},"last":1.889}]},{"osd":7,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.835,"5min":2.0880000000000001,"15min":2.2000000000000002},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.911,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":1.3169999999999999},{"interface":"front","average":{"1min":2.04,"5min":2.1179999999999999,"15min":2.194},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":3.3170000000000002,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":1.4119999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":2.2080000000000002,"5min":2.1869999999999998,"15min":2.2360000000000002},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":6.4669999999999996,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":1.97},{"interface":"front","average":{"1min":1.9390000000000001,"5min":2.2320000000000002,"15min":2.3530000000000002},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.9830000000000001,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":1.5649999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:21:12 2024","interfaces":[{"interface":"back","average":{"1min":1.861,"5min":2.2170000000000001,"15min":2.2770000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.8940000000000001,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.9410000000000001,"5min":2.0720000000000001,"15min":2.0939999999999999},"min":{"1min":1.087,"5min":1.087,"15min":1.087},"max":{"1min":3.0219999999999998,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":2.0289999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":459411,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.437,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99099999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98799999999999999,"5min":1.198,"15min":1.198},"last":0.84999999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.498},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70899999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.46899999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.441,"15min":0.439},"min":{"1min":0.32300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63700000000000001,"5min":0.755,"15min":0.755},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.436,"15min":0.436},"min":{"1min":0.308,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.629}]},{"osd":3,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.496,"5min":0.432,"15min":0.432},"max":{"1min":0.95699999999999996,"5min":1.379,"15min":1.379},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1919999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.35599999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.34300000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.57099999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.68999999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.075,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.0680000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.504,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.48099999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69099999999999995}]},{"osd":9,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88200000000000001,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2709999999999999,"5min":1.28,"15min":1.28},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93000000000000005,"15min":0.93100000000000005},"min":{"1min":0.63300000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.23}]},{"osd":10,"last update":"Mon Feb 26 06:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66500000000000004,"15min":0.65600000000000003},"min":{"1min":0.503,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91100000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.57599999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.78000000000000003}]},{"osd":11,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.755}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051594,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40600000000000003,"15min":0.41999999999999998},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.499,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.42499999999999999,"15min":0.433},"min":{"1min":0.29999999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58499999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.434,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53000000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80500000000000005,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1000000000000001,"5min":1.591,"15min":1.591},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.51700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99199999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.65700000000000003}]},{"osd":4,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65900000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48599999999999999,"15min":0.51200000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.38800000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.85099999999999998,"15min":0.86099999999999999},"min":{"1min":0.47799999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.077,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.238,"5min":0.93899999999999995,"15min":0.88700000000000001},"min":{"1min":0.61099999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":7.4119999999999999,"5min":7.4119999999999999,"15min":7.4119999999999999},"last":0.67400000000000004}]},{"osd":8,"last update":"Mon Feb 26 06:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82099999999999995,"5min":0.88,"15min":0.88},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84999999999999998,"5min":0.873,"15min":0.873},"last":0.78200000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.89700000000000002,"15min":0.92700000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94299999999999995,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.749},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":0.94699999999999995},"min":{"1min":0.69799999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0620000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.496},{"interface":"front","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63}]},{"osd":11,"last update":"Mon Feb 26 06:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.84399999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":458992,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2829999999999999,"15min":1.2769999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.0270000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.3080000000000001,"15min":1.248},"min":{"1min":0.89800000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.0539999999999998,"5min":2.508,"15min":2.508},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0840000000000001,"15min":1.0760000000000001},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.121,"15min":1.103},"min":{"1min":0.72799999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8160000000000001,"5min":1.944,"15min":1.944},"last":0.53300000000000003}]},{"osd":2,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2509999999999999,"15min":1.258},"min":{"1min":0.61099999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.758,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3600000000000001,"15min":1.2889999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":6.4930000000000003,"5min":6.4930000000000003,"15min":6.4930000000000003},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6539999999999999,"5min":1.661,"15min":1.591},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.1320000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.677,"15min":1.6719999999999999},"min":{"1min":0.88100000000000001,"5min":0.745,"15min":0.745},"max":{"1min":2.181,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4390000000000001,"15min":1.4259999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.198,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.671,"5min":1.6479999999999999,"15min":1.597},"min":{"1min":1.012,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.4279999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.696,"5min":1.6719999999999999,"15min":1.6319999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1949999999999998,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.77,"5min":1.734,"15min":1.72},"min":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.246,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:21:46 2024","interfaces":[{"interface":"back","average":{"1min":1.819,"5min":1.744,"15min":1.6140000000000001},"min":{"1min":1.006,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.7669999999999999,"5min":1.8120000000000001,"15min":1.7450000000000001},"min":{"1min":0.90600000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.972,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.81100000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:21:12 2024","interfaces":[{"interface":"back","average":{"1min":1.9490000000000001,"5min":1.819,"15min":1.7470000000000001},"min":{"1min":1.1020000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.52,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":2.2599999999999998,"5min":1.9319999999999999,"15min":1.843},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":6.625,"5min":6.625,"15min":6.625},"last":0.86899999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:21:46 2024","interfaces":[{"interface":"back","average":{"1min":1.974,"5min":2.1760000000000002,"15min":2.2719999999999998},"min":{"1min":1.0489999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":3.1800000000000002,"5min":3.258,"15min":3.258},"last":0.83899999999999997},{"interface":"front","average":{"1min":2.048,"5min":2.214,"15min":2.2789999999999999},"min":{"1min":1.3600000000000001,"5min":1.181,"15min":1.181},"max":{"1min":3.347,"5min":3.347,"15min":3.347},"last":0.88300000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:21:12 2024","interfaces":[{"interface":"back","average":{"1min":2.0059999999999998,"5min":2.0819999999999999,"15min":2.0939999999999999},"min":{"1min":1.202,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":2.1120000000000001,"5min":2.1920000000000002,"15min":2.206},"min":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"max":{"1min":2.7010000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.90600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":519907,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.67800000000000005},"min":{"1min":0.56200000000000006,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.054,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.498,"5min":0.97999999999999998,"15min":0.79900000000000004},"min":{"1min":0.44,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":16.690999999999999,"5min":16.690999999999999,"15min":16.690999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.72699999999999998,"15min":0.60299999999999998},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":9.2629999999999999,"15min":9.2629999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.52100000000000002,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.68200000000000005}]},{"osd":3,"last update":"Mon Feb 26 06:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77000000000000002,"15min":0.74299999999999999},"min":{"1min":0.53000000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.246,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.78400000000000003,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1020000000000001,"5min":1.391,"15min":1.391},"last":1.089}]},{"osd":4,"last update":"Mon Feb 26 06:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.39800000000000002},"min":{"1min":0.30099999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65700000000000003,"5min":0.748,"15min":0.748},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.68400000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.504}]},{"osd":5,"last update":"Mon Feb 26 06:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.78600000000000003,"15min":0.66400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":16.672999999999998,"15min":16.672999999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.877,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Feb 26 06:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80200000000000005,"15min":0.78400000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.069,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.92300000000000004,"15min":0.871},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.032,"5min":8.8729999999999993,"15min":8.8729999999999993},"last":0.93200000000000005}]},{"osd":7,"last update":"Mon Feb 26 06:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52100000000000002,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.79400000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63100000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.48599999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72999999999999998,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95299999999999996,"5min":8.9380000000000006,"15min":8.9380000000000006},"last":0.78400000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.65900000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2090000000000001,"5min":1.224,"15min":1.224},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.046,"15min":1.0149999999999999},"min":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.831,"5min":8.8979999999999997,"15min":8.8979999999999997},"last":0.94499999999999995}]},{"osd":10,"last update":"Mon Feb 26 06:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0049999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.69299999999999995,"15min":0.70299999999999996},"min":{"1min":0.40699999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93799999999999994,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.80000000000000004}]},{"osd":11,"last update":"Mon Feb 26 06:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83199999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460263,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.93400000000000005,"15min":0.81899999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.46300000000000002},"max":{"1min":1.4570000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.95999999999999996,"15min":0.84999999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.649},"last":0.745}]},{"osd":2,"last update":"Mon Feb 26 06:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.073,"15min":0.92300000000000004},"min":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.58099999999999996},"max":{"1min":1.6970000000000001,"5min":6.6070000000000002,"15min":6.6070000000000002},"last":1.468},{"interface":"front","average":{"1min":1.383,"5min":1.2210000000000001,"15min":0.999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.90400000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":1.78,"5min":1.78,"15min":1.78},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.034},{"interface":"front","average":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.2509999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2250000000000001,"15min":1.0669999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.946,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2390000000000001,"15min":1.155},"min":{"1min":0.70999999999999996,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.008,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.401}]},{"osd":5,"last update":"Mon Feb 26 06:21:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0499999999999998,"5min":1.599,"15min":1.4850000000000001},"min":{"1min":1.181,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.681,"5min":1.522,"15min":1.401},"min":{"1min":0.78800000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.5979999999999999,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":1.528}]},{"osd":6,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":1.923,"5min":1.8360000000000001,"15min":1.8100000000000001},"min":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.827,"15min":1.8420000000000001},"min":{"1min":0.66500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":1.734}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.73,"15min":1.7190000000000001},"min":{"1min":1.2150000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.45},{"interface":"front","average":{"1min":1.73,"5min":1.599,"15min":1.577},"min":{"1min":1.123,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:21:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.7749999999999999,"15min":1.7330000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.8029999999999999,"15min":1.7549999999999999},"min":{"1min":0.91600000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.2639999999999998,"5min":2.452,"15min":2.452},"last":1.6459999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:21:51 2024","interfaces":[{"interface":"back","average":{"1min":1.9319999999999999,"5min":1.9450000000000001,"15min":1.986},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":1.6919999999999999},{"interface":"front","average":{"1min":2.4910000000000001,"5min":2.141,"15min":2.1120000000000001},"min":{"1min":1.365,"5min":0.746,"15min":0.746},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":1.1950000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:21:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8759999999999999,"5min":2.0259999999999998,"15min":2.0510000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.704,"5min":1.8160000000000001,"15min":1.8340000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.3780000000000001,"5min":2.423,"15min":2.423},"last":1.5960000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:21:37 2024","interfaces":[{"interface":"back","average":{"1min":2.2890000000000001,"5min":1.964,"15min":1.9099999999999999},"min":{"1min":1.075,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":2.121,"5min":2.0369999999999999,"15min":2.0230000000000001},"min":{"1min":1.3100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.1040000000000001,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.8979999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":460023,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71099999999999997,"15min":0.68700000000000006},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91700000000000004,"5min":1.091,"15min":1.091},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.70499999999999996},"min":{"1min":0.53000000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1399999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.35699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77600000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.39800000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.71099999999999997},"min":{"1min":0.49199999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0069999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.752,"15min":0.72799999999999998},"min":{"1min":0.48899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.127,"5min":1.367,"15min":1.367},"last":0.56899999999999995}]},{"osd":4,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.61499999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.46899999999999997,"15min":0.47199999999999998},"min":{"1min":0.38,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.32700000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79300000000000004,"15min":0.76400000000000001},"min":{"1min":0.57599999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.48999999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47399999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.754}]},{"osd":9,"last update":"Mon Feb 26 06:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.94299999999999995,"15min":0.94499999999999995},"min":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.204,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98499999999999999,"15min":0.98799999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.66300000000000003}]},{"osd":10,"last update":"Mon Feb 26 06:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.60899999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.78900000000000003}]},{"osd":11,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.81100000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":459303,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.163,"15min":1.0700000000000001},"min":{"1min":0.621,"5min":0.433,"15min":0.433},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.534},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.361,"15min":1.125},"min":{"1min":0.71199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.0600000000000001,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":0.98299999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:21:00 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1220000000000001,"15min":1.018},"min":{"1min":0.76000000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.524,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.129,"15min":1.0720000000000001},"min":{"1min":0.66000000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.736,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.2310000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:21:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.282,"15min":1.177},"min":{"1min":0.81799999999999995,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7450000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.085,"15min":1.0409999999999999},"min":{"1min":0.71599999999999997,"5min":0.441,"15min":0.441},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.3}]},{"osd":4,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3440000000000001,"15min":1.2270000000000001},"min":{"1min":0.83099999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8460000000000001,"5min":2.613,"15min":2.613},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5389999999999999,"15min":1.3859999999999999},"min":{"1min":0.98899999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.2200000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.3580000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.466,"15min":1.3799999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.2149999999999999,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.052},{"interface":"front","average":{"1min":1.504,"5min":1.375,"15min":1.341},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.9329999999999998,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":1.4510000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":1.792,"5min":1.5940000000000001,"15min":1.49},"min":{"1min":0.67600000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.0619999999999998,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.7969999999999999,"15min":1.655},"min":{"1min":0.99099999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.7410000000000001,"5min":4.681,"15min":4.681},"last":1.617}]},{"osd":7,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.7709999999999999,"15min":1.7370000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.456,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.671},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.8220000000000001,"15min":1.71},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3900000000000001,"5min":3.544,"15min":3.544},"last":1.587}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.806},"min":{"1min":0.97399999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.587,"15min":1.5649999999999999},"min":{"1min":0.84999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.23,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.143}]},{"osd":9,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.99,"15min":2.0459999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.0230000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.9359999999999999},{"interface":"front","average":{"1min":1.9990000000000001,"5min":2.2989999999999999,"15min":2.427},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.698,"5min":4.266,"15min":4.266},"last":1.704}]},{"osd":10,"last update":"Mon Feb 26 06:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.772,"5min":1.988,"15min":2.024},"min":{"1min":1.083,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.3690000000000002,"5min":2.476,"15min":2.476},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.986,"5min":2.2400000000000002,"15min":2.282},"min":{"1min":1.3140000000000001,"5min":1.127,"15min":1.127},"max":{"1min":2.4700000000000002,"5min":4.0609999999999999,"15min":4.0609999999999999},"last":1.8700000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:21:33 2024","interfaces":[{"interface":"back","average":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":2.0590000000000002},{"interface":"front","average":{"1min":2.073,"5min":2.073,"15min":2.073},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.9710000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":521195,"data_compressed":15495,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.50600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.079,"5min":1.093,"15min":1.093},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.378,"15min":0.35499999999999998},"min":{"1min":0.28999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56299999999999994,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.37},"min":{"1min":0.313,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56200000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35399999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75700000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.67600000000000005,"15min":0.56899999999999995},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":12.468,"15min":12.468},"last":0.59399999999999997}]},{"osd":3,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.749,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98099999999999998,"5min":1.167,"15min":1.167},"last":0.747},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.60999999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.53900000000000003},"min":{"1min":0.53200000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.56200000000000006},"min":{"1min":0.502,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.754,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52600000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77100000000000002,"15min":0.73099999999999998},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95499999999999996,"5min":1.327,"15min":1.327},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80400000000000005,"15min":0.79100000000000004},"min":{"1min":0.53700000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0620000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.42899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.441}]},{"osd":8,"last update":"Mon Feb 26 06:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.44900000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.68100000000000005,"15min":0.65900000000000003},"min":{"1min":0.53200000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69699999999999995}]},{"osd":9,"last update":"Mon Feb 26 06:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.92900000000000005,"15min":0.94899999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.2,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.304,"5min":1.046,"15min":1},"min":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":1.0049999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81200000000000006,"5min":0.87,"15min":0.87},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.5}]},{"osd":11,"last update":"Mon Feb 26 06:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79500000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1050746,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.754,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.78300000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79000000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.504},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52600000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39400000000000002},"min":{"1min":0.29699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59299999999999997,"5min":0.624,"15min":0.624},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60699999999999998,"5min":0.75,"15min":0.75},"last":0.69999999999999996}]},{"osd":3,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78700000000000003,"15min":0.77700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1279999999999999,"5min":1.226,"15min":1.226},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94399999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82299999999999995,"15min":0.79500000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.80800000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2090000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.089}]},{"osd":7,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.85799999999999998,"5min":0.872,"15min":0.872},"last":0.73499999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83399999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70199999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.79600000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89500000000000002,"15min":0.90500000000000003},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.224,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.0089999999999999,"15min":1.0429999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1859999999999999,"5min":1.444,"15min":1.444},"last":1.004}]},{"osd":10,"last update":"Mon Feb 26 06:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.875,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.82399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61122,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":62410,"data_compressed":9688,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63232,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:22:06.210 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-26T06:22:06.709 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:07.115 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:07 smithi049 ceph-mon[31696]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:22:07.115 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:07 smithi049 ceph-mon[31696]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T06:22:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:07 smithi080 ceph-mon[36206]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:22:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:07 smithi080 ceph-mon[36206]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T06:22:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:07 smithi115 ceph-mon[36426]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:22:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:07 smithi115 ceph-mon[36426]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T06:22:09.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.136+0000 7f503ee00700 1 -- 172.21.15.49:0/3910481803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 msgr2=0x7f503810cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:09.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.136+0000 7f503ee00700 1 --2- 172.21.15.49:0/3910481803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 0x7f503810cc30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f503000b0d0 tx=0x7f503000b3e0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.137+0000 7f503ee00700 1 -- 172.21.15.49:0/3910481803 shutdown_connections 2024-02-26T06:22:09.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.137+0000 7f503ee00700 1 --2- 172.21.15.49:0/3910481803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 0x7f503810cc30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.137+0000 7f503ee00700 1 --2- 172.21.15.49:0/3910481803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5038105e10 0x7f50381061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.137+0000 7f503ee00700 1 --2- 172.21.15.49:0/3910481803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50380fffb0 0x7f5038100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.137+0000 7f503ee00700 1 -- 172.21.15.49:0/3910481803 >> 172.21.15.49:0/3910481803 conn(0x7f5038075930 msgr2=0x7f5038075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.137+0000 7f503ee00700 1 -- 172.21.15.49:0/3910481803 shutdown_connections 2024-02-26T06:22:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.137+0000 7f503ee00700 1 -- 172.21.15.49:0/3910481803 wait complete. 2024-02-26T06:22:09.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.138+0000 7f503ee00700 1 Processor -- start 2024-02-26T06:22:09.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.138+0000 7f503ee00700 1 -- start start 2024-02-26T06:22:09.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503ee00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50380fffb0 0x7f503819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:09.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503ee00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 0x7f503819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503ee00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5038105e10 0x7f503819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503ee00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503819f280 con 0x7f5038100970 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503ee00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503819f3f0 con 0x7f5038105e10 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503ee00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503819f5b0 con 0x7f50380fffb0 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503d000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 0x7f503819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503da00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50380fffb0 0x7f503819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503d000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 0x7f503819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39498/0 (socket says 172.21.15.49:39498) 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503d000700 1 -- 172.21.15.49:0/820276291 learned_addr learned my addr 172.21.15.49:0/820276291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503e400700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5038105e10 0x7f503819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503da00700 1 -- 172.21.15.49:0/820276291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5038105e10 msgr2=0x7f503819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503da00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5038105e10 0x7f503819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503da00700 1 -- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 msgr2=0x7f503819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503da00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 0x7f503819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.139+0000 7f503da00700 1 -- 172.21.15.49:0/820276291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240071c0 con 0x7f50380fffb0 2024-02-26T06:22:09.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.140+0000 7f503e400700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5038105e10 0x7f503819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:09.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.140+0000 7f503d000700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 0x7f503819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:09.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.140+0000 7f503da00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50380fffb0 0x7f503819a3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f502c00e9f0 tx=0x7f502c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:09.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.140+0000 7f5036000700 1 -- 172.21.15.49:0/820276291 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0096b0 con 0x7f50380fffb0 2024-02-26T06:22:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.140+0000 7f5036000700 1 -- 172.21.15.49:0/820276291 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f502c004d10 con 0x7f50380fffb0 2024-02-26T06:22:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.140+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5030009d00 con 0x7f50380fffb0 2024-02-26T06:22:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.140+0000 7f5036000700 1 -- 172.21.15.49:0/820276291 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c00ce60 con 0x7f50380fffb0 2024-02-26T06:22:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.140+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50381a6f20 con 0x7f50380fffb0 2024-02-26T06:22:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.141+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5038068930 con 0x7f50380fffb0 2024-02-26T06:22:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.143+0000 7f5036000700 1 -- 172.21.15.49:0/820276291 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f502c004390 con 0x7f50380fffb0 2024-02-26T06:22:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.145+0000 7f5036000700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f50200a21d0 0x7f50200a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:09.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.145+0000 7f5036000700 1 -- 172.21.15.49:0/820276291 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f502c014070 con 0x7f50380fffb0 2024-02-26T06:22:09.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.145+0000 7f503d000700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f50200a21d0 0x7f50200a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:09.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.146+0000 7f503d000700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f50200a21d0 0x7f50200a4690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50380761a0 tx=0x7f502400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:09.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.149+0000 7f5036000700 1 -- 172.21.15.49:0/820276291 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f502c0cfe60 con 0x7f50380fffb0 2024-02-26T06:22:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:09 smithi080 ceph-mon[36206]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-26T06:22:09.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:09 smithi115 ceph-mon[36426]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-26T06:22:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.371+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5038111c50 con 0x7f50200a21d0 2024-02-26T06:22:09.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:09 smithi049 ceph-mon[31696]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-26T06:22:09.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.390+0000 7f5036000700 1 -- 172.21.15.49:0/820276291 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+342516 (secure 0 0 0) 0x7f5038111c50 con 0x7f50200a21d0 2024-02-26T06:22:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f50200a21d0 msgr2=0x7f50200a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f50200a21d0 0x7f50200a4690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50380761a0 tx=0x7f502400c040 comp rx=0 tx=0).stop 2024-02-26T06:22:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50380fffb0 msgr2=0x7f503819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50380fffb0 0x7f503819a3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f502c00e9f0 tx=0x7f502c00ed00 comp rx=0 tx=0).stop 2024-02-26T06:22:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 shutdown_connections 2024-02-26T06:22:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f50200a21d0 0x7f50200a4690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5038105e10 0x7f503819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50380fffb0 0x7f503819a3a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 --2- 172.21.15.49:0/820276291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5038100970 0x7f503819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 >> 172.21.15.49:0/820276291 conn(0x7f5038075930 msgr2=0x7f503810eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 shutdown_connections 2024-02-26T06:22:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:09.398+0000 7f503ee00700 1 -- 172.21.15.49:0/820276291 wait complete. 2024-02-26T06:22:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-02-26T06:22:10.389 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-02-26T06:22:07.461376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655866,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":690,"num_read_kb":463,"num_write":347,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1241721,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360764,"kb_used_data":11612,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581060,"statfs":{"total":1151940427776,"available":1151571005440,"internally_reserved":0,"allocated":11890688,"data_stored":7473018,"data_compressed":191495,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":42,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005536"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440077+0000","last_change":"2024-02-26T06:20:31.603060+0000","last_active":"2024-02-26T06:21:57.440077+0000","last_peered":"2024-02-26T06:21:57.440077+0000","last_clean":"2024-02-26T06:21:57.440077+0000","last_became_active":"2024-02-26T06:20:31.602762+0000","last_became_peered":"2024-02-26T06:20:31.602762+0000","last_unstale":"2024-02-26T06:21:57.440077+0000","last_undegraded":"2024-02-26T06:21:57.440077+0000","last_fullsized":"2024-02-26T06:21:57.440077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:19:42.465510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.437637+0000","last_change":"2024-02-26T06:20:31.595138+0000","last_active":"2024-02-26T06:21:57.437637+0000","last_peered":"2024-02-26T06:21:57.437637+0000","last_clean":"2024-02-26T06:21:57.437637+0000","last_became_active":"2024-02-26T06:20:31.594826+0000","last_became_peered":"2024-02-26T06:20:31.594826+0000","last_unstale":"2024-02-26T06:21:57.437637+0000","last_undegraded":"2024-02-26T06:21:57.437637+0000","last_fullsized":"2024-02-26T06:21:57.437637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:49.900464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.431057+0000","last_change":"2024-02-26T06:20:31.600415+0000","last_active":"2024-02-26T06:21:57.431057+0000","last_peered":"2024-02-26T06:21:57.431057+0000","last_clean":"2024-02-26T06:21:57.431057+0000","last_became_active":"2024-02-26T06:20:31.596273+0000","last_became_peered":"2024-02-26T06:20:31.596273+0000","last_unstale":"2024-02-26T06:21:57.431057+0000","last_undegraded":"2024-02-26T06:21:57.431057+0000","last_fullsized":"2024-02-26T06:21:57.431057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:15:20.626515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410770+0000","last_change":"2024-02-26T06:20:31.581149+0000","last_active":"2024-02-26T06:21:57.410770+0000","last_peered":"2024-02-26T06:21:57.410770+0000","last_clean":"2024-02-26T06:21:57.410770+0000","last_became_active":"2024-02-26T06:20:31.580831+0000","last_became_peered":"2024-02-26T06:20:31.580831+0000","last_unstale":"2024-02-26T06:21:57.410770+0000","last_undegraded":"2024-02-26T06:21:57.410770+0000","last_fullsized":"2024-02-26T06:21:57.410770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:47:12.466502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412439+0000","last_change":"2024-02-26T06:20:31.608618+0000","last_active":"2024-02-26T06:21:57.412439+0000","last_peered":"2024-02-26T06:21:57.412439+0000","last_clean":"2024-02-26T06:21:57.412439+0000","last_became_active":"2024-02-26T06:20:31.608285+0000","last_became_peered":"2024-02-26T06:20:31.608285+0000","last_unstale":"2024-02-26T06:21:57.412439+0000","last_undegraded":"2024-02-26T06:21:57.412439+0000","last_fullsized":"2024-02-26T06:21:57.412439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:34:48.400836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.418219+0000","last_change":"2024-02-26T06:21:28.905073+0000","last_active":"2024-02-26T06:21:57.418219+0000","last_peered":"2024-02-26T06:21:57.418219+0000","last_clean":"2024-02-26T06:21:57.418219+0000","last_became_active":"2024-02-26T06:21:28.904630+0000","last_became_peered":"2024-02-26T06:21:28.904630+0000","last_unstale":"2024-02-26T06:21:57.418219+0000","last_undegraded":"2024-02-26T06:21:57.418219+0000","last_fullsized":"2024-02-26T06:21:57.418219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:04.531384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.407496+0000","last_change":"2024-02-26T06:20:31.587285+0000","last_active":"2024-02-26T06:21:57.407496+0000","last_peered":"2024-02-26T06:21:57.407496+0000","last_clean":"2024-02-26T06:21:57.407496+0000","last_became_active":"2024-02-26T06:20:31.586985+0000","last_became_peered":"2024-02-26T06:20:31.586985+0000","last_unstale":"2024-02-26T06:21:57.407496+0000","last_undegraded":"2024-02-26T06:21:57.407496+0000","last_fullsized":"2024-02-26T06:21:57.407496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:55:56.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438174+0000","last_change":"2024-02-26T06:20:31.591136+0000","last_active":"2024-02-26T06:21:57.438174+0000","last_peered":"2024-02-26T06:21:57.438174+0000","last_clean":"2024-02-26T06:21:57.438174+0000","last_became_active":"2024-02-26T06:20:31.590180+0000","last_became_peered":"2024-02-26T06:20:31.590180+0000","last_unstale":"2024-02-26T06:21:57.438174+0000","last_undegraded":"2024-02-26T06:21:57.438174+0000","last_fullsized":"2024-02-26T06:21:57.438174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:02.649407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420056+0000","last_change":"2024-02-26T06:20:31.583890+0000","last_active":"2024-02-26T06:21:57.420056+0000","last_peered":"2024-02-26T06:21:57.420056+0000","last_clean":"2024-02-26T06:21:57.420056+0000","last_became_active":"2024-02-26T06:20:31.583459+0000","last_became_peered":"2024-02-26T06:20:31.583459+0000","last_unstale":"2024-02-26T06:21:57.420056+0000","last_undegraded":"2024-02-26T06:21:57.420056+0000","last_fullsized":"2024-02-26T06:21:57.420056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:14.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412599+0000","last_change":"2024-02-26T06:20:31.579041+0000","last_active":"2024-02-26T06:21:57.412599+0000","last_peered":"2024-02-26T06:21:57.412599+0000","last_clean":"2024-02-26T06:21:57.412599+0000","last_became_active":"2024-02-26T06:20:31.578725+0000","last_became_peered":"2024-02-26T06:20:31.578725+0000","last_unstale":"2024-02-26T06:21:57.412599+0000","last_undegraded":"2024-02-26T06:21:57.412599+0000","last_fullsized":"2024-02-26T06:21:57.412599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:21.083206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420414+0000","last_change":"2024-02-26T06:20:31.581146+0000","last_active":"2024-02-26T06:21:57.420414+0000","last_peered":"2024-02-26T06:21:57.420414+0000","last_clean":"2024-02-26T06:21:57.420414+0000","last_became_active":"2024-02-26T06:20:31.580774+0000","last_became_peered":"2024-02-26T06:20:31.580774+0000","last_unstale":"2024-02-26T06:21:57.420414+0000","last_undegraded":"2024-02-26T06:21:57.420414+0000","last_fullsized":"2024-02-26T06:21:57.420414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:58:36.851504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410450+0000","last_change":"2024-02-26T06:20:31.582232+0000","last_active":"2024-02-26T06:21:57.410450+0000","last_peered":"2024-02-26T06:21:57.410450+0000","last_clean":"2024-02-26T06:21:57.410450+0000","last_became_active":"2024-02-26T06:20:31.581805+0000","last_became_peered":"2024-02-26T06:20:31.581805+0000","last_unstale":"2024-02-26T06:21:57.410450+0000","last_undegraded":"2024-02-26T06:21:57.410450+0000","last_fullsized":"2024-02-26T06:21:57.410450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:04.513814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428463+0000","last_change":"2024-02-26T06:20:31.606373+0000","last_active":"2024-02-26T06:21:57.428463+0000","last_peered":"2024-02-26T06:21:57.428463+0000","last_clean":"2024-02-26T06:21:57.428463+0000","last_became_active":"2024-02-26T06:20:31.603226+0000","last_became_peered":"2024-02-26T06:20:31.603226+0000","last_unstale":"2024-02-26T06:21:57.428463+0000","last_undegraded":"2024-02-26T06:21:57.428463+0000","last_fullsized":"2024-02-26T06:21:57.428463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:29:49.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.426227+0000","last_change":"2024-02-26T06:20:31.594587+0000","last_active":"2024-02-26T06:21:57.426227+0000","last_peered":"2024-02-26T06:21:57.426227+0000","last_clean":"2024-02-26T06:21:57.426227+0000","last_became_active":"2024-02-26T06:20:31.594032+0000","last_became_peered":"2024-02-26T06:20:31.594032+0000","last_unstale":"2024-02-26T06:21:57.426227+0000","last_undegraded":"2024-02-26T06:21:57.426227+0000","last_fullsized":"2024-02-26T06:21:57.426227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:49:18.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438276+0000","last_change":"2024-02-26T06:20:31.593115+0000","last_active":"2024-02-26T06:21:57.438276+0000","last_peered":"2024-02-26T06:21:57.438276+0000","last_clean":"2024-02-26T06:21:57.438276+0000","last_became_active":"2024-02-26T06:20:31.592782+0000","last_became_peered":"2024-02-26T06:20:31.592782+0000","last_unstale":"2024-02-26T06:21:57.438276+0000","last_undegraded":"2024-02-26T06:21:57.438276+0000","last_fullsized":"2024-02-26T06:21:57.438276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:10.709124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.419515+0000","last_change":"2024-02-26T06:20:31.597938+0000","last_active":"2024-02-26T06:21:57.419515+0000","last_peered":"2024-02-26T06:21:57.419515+0000","last_clean":"2024-02-26T06:21:57.419515+0000","last_became_active":"2024-02-26T06:20:31.597600+0000","last_became_peered":"2024-02-26T06:20:31.597600+0000","last_unstale":"2024-02-26T06:21:57.419515+0000","last_undegraded":"2024-02-26T06:21:57.419515+0000","last_fullsized":"2024-02-26T06:21:57.419515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:13.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440280+0000","last_change":"2024-02-26T06:20:31.603283+0000","last_active":"2024-02-26T06:21:57.440280+0000","last_peered":"2024-02-26T06:21:57.440280+0000","last_clean":"2024-02-26T06:21:57.440280+0000","last_became_active":"2024-02-26T06:20:31.602912+0000","last_became_peered":"2024-02-26T06:20:31.602912+0000","last_unstale":"2024-02-26T06:21:57.440280+0000","last_undegraded":"2024-02-26T06:21:57.440280+0000","last_fullsized":"2024-02-26T06:21:57.440280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:32.250914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.422243+0000","last_change":"2024-02-26T06:20:31.588670+0000","last_active":"2024-02-26T06:21:57.422243+0000","last_peered":"2024-02-26T06:21:57.422243+0000","last_clean":"2024-02-26T06:21:57.422243+0000","last_became_active":"2024-02-26T06:20:31.587971+0000","last_became_peered":"2024-02-26T06:20:31.587971+0000","last_unstale":"2024-02-26T06:21:57.422243+0000","last_undegraded":"2024-02-26T06:21:57.422243+0000","last_fullsized":"2024-02-26T06:21:57.422243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:13.053456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.425013+0000","last_change":"2024-02-26T06:20:31.604266+0000","last_active":"2024-02-26T06:21:57.425013+0000","last_peered":"2024-02-26T06:21:57.425013+0000","last_clean":"2024-02-26T06:21:57.425013+0000","last_became_active":"2024-02-26T06:20:31.604027+0000","last_became_peered":"2024-02-26T06:20:31.604027+0000","last_unstale":"2024-02-26T06:21:57.425013+0000","last_undegraded":"2024-02-26T06:21:57.425013+0000","last_fullsized":"2024-02-26T06:21:57.425013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:13.345138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904450+0000","last_change":"2024-02-26T06:20:31.068090+0000","last_active":"2024-02-26T06:21:28.904450+0000","last_peered":"2024-02-26T06:21:28.904450+0000","last_clean":"2024-02-26T06:21:28.904450+0000","last_became_active":"2024-02-26T06:20:31.067731+0000","last_became_peered":"2024-02-26T06:20:31.067731+0000","last_unstale":"2024-02-26T06:21:28.904450+0000","last_undegraded":"2024-02-26T06:21:28.904450+0000","last_fullsized":"2024-02-26T06:21:28.904450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:24.734040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413823+0000","last_change":"2024-02-26T06:21:28.915741+0000","last_active":"2024-02-26T06:21:57.413823+0000","last_peered":"2024-02-26T06:21:57.413823+0000","last_clean":"2024-02-26T06:21:57.413823+0000","last_became_active":"2024-02-26T06:21:28.915443+0000","last_became_peered":"2024-02-26T06:21:28.915443+0000","last_unstale":"2024-02-26T06:21:57.413823+0000","last_undegraded":"2024-02-26T06:21:57.413823+0000","last_fullsized":"2024-02-26T06:21:57.413823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:51.216291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898840+0000","last_change":"2024-02-26T06:20:31.083236+0000","last_active":"2024-02-26T06:21:28.898840+0000","last_peered":"2024-02-26T06:21:28.898840+0000","last_clean":"2024-02-26T06:21:28.898840+0000","last_became_active":"2024-02-26T06:20:31.081640+0000","last_became_peered":"2024-02-26T06:20:31.081640+0000","last_unstale":"2024-02-26T06:21:28.898840+0000","last_undegraded":"2024-02-26T06:21:28.898840+0000","last_fullsized":"2024-02-26T06:21:28.898840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:44.607108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439841+0000","last_change":"2024-02-26T06:20:31.593807+0000","last_active":"2024-02-26T06:21:57.439841+0000","last_peered":"2024-02-26T06:21:57.439841+0000","last_clean":"2024-02-26T06:21:57.439841+0000","last_became_active":"2024-02-26T06:20:31.593435+0000","last_became_peered":"2024-02-26T06:20:31.593435+0000","last_unstale":"2024-02-26T06:21:57.439841+0000","last_undegraded":"2024-02-26T06:21:57.439841+0000","last_fullsized":"2024-02-26T06:21:57.439841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:43.193196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.908116+0000","last_change":"2024-02-26T06:20:31.061339+0000","last_active":"2024-02-26T06:21:28.908116+0000","last_peered":"2024-02-26T06:21:28.908116+0000","last_clean":"2024-02-26T06:21:28.908116+0000","last_became_active":"2024-02-26T06:20:31.060769+0000","last_became_peered":"2024-02-26T06:20:31.060769+0000","last_unstale":"2024-02-26T06:21:28.908116+0000","last_undegraded":"2024-02-26T06:21:28.908116+0000","last_fullsized":"2024-02-26T06:21:28.908116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:28:30.501375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432622+0000","last_change":"2024-02-26T06:20:31.606936+0000","last_active":"2024-02-26T06:21:57.432622+0000","last_peered":"2024-02-26T06:21:57.432622+0000","last_clean":"2024-02-26T06:21:57.432622+0000","last_became_active":"2024-02-26T06:20:31.606582+0000","last_became_peered":"2024-02-26T06:20:31.606582+0000","last_unstale":"2024-02-26T06:21:57.432622+0000","last_undegraded":"2024-02-26T06:21:57.432622+0000","last_fullsized":"2024-02-26T06:21:57.432622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:09.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907110+0000","last_change":"2024-02-26T06:20:31.063472+0000","last_active":"2024-02-26T06:21:28.907110+0000","last_peered":"2024-02-26T06:21:28.907110+0000","last_clean":"2024-02-26T06:21:28.907110+0000","last_became_active":"2024-02-26T06:20:31.063132+0000","last_became_peered":"2024-02-26T06:20:31.063132+0000","last_unstale":"2024-02-26T06:21:28.907110+0000","last_undegraded":"2024-02-26T06:21:28.907110+0000","last_fullsized":"2024-02-26T06:21:28.907110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:58.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.417983+0000","last_change":"2024-02-26T06:20:31.589238+0000","last_active":"2024-02-26T06:21:57.417983+0000","last_peered":"2024-02-26T06:21:57.417983+0000","last_clean":"2024-02-26T06:21:57.417983+0000","last_became_active":"2024-02-26T06:20:31.588929+0000","last_became_peered":"2024-02-26T06:20:31.588929+0000","last_unstale":"2024-02-26T06:21:57.417983+0000","last_undegraded":"2024-02-26T06:21:57.417983+0000","last_fullsized":"2024-02-26T06:21:57.417983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:19.282943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:06.242016+0000","last_change":"2024-02-26T06:20:31.061714+0000","last_active":"2024-02-26T06:22:06.242016+0000","last_peered":"2024-02-26T06:22:06.242016+0000","last_clean":"2024-02-26T06:22:06.242016+0000","last_became_active":"2024-02-26T06:20:31.061234+0000","last_became_peered":"2024-02-26T06:20:31.061234+0000","last_unstale":"2024-02-26T06:22:06.242016+0000","last_undegraded":"2024-02-26T06:22:06.242016+0000","last_fullsized":"2024-02-26T06:22:06.242016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:29:17.668327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.400148+0000","last_change":"2024-02-26T06:20:31.588861+0000","last_active":"2024-02-26T06:21:57.400148+0000","last_peered":"2024-02-26T06:21:57.400148+0000","last_clean":"2024-02-26T06:21:57.400148+0000","last_became_active":"2024-02-26T06:20:31.588530+0000","last_became_peered":"2024-02-26T06:20:31.588530+0000","last_unstale":"2024-02-26T06:21:57.400148+0000","last_undegraded":"2024-02-26T06:21:57.400148+0000","last_fullsized":"2024-02-26T06:21:57.400148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:12.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.900294+0000","last_change":"2024-02-26T06:20:31.077656+0000","last_active":"2024-02-26T06:21:28.900294+0000","last_peered":"2024-02-26T06:21:28.900294+0000","last_clean":"2024-02-26T06:21:28.900294+0000","last_became_active":"2024-02-26T06:20:31.077288+0000","last_became_peered":"2024-02-26T06:20:31.077288+0000","last_unstale":"2024-02-26T06:21:28.900294+0000","last_undegraded":"2024-02-26T06:21:28.900294+0000","last_fullsized":"2024-02-26T06:21:28.900294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:56:05.173751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439932+0000","last_change":"2024-02-26T06:20:31.608089+0000","last_active":"2024-02-26T06:21:57.439932+0000","last_peered":"2024-02-26T06:21:57.439932+0000","last_clean":"2024-02-26T06:21:57.439932+0000","last_became_active":"2024-02-26T06:20:31.607786+0000","last_became_peered":"2024-02-26T06:20:31.607786+0000","last_unstale":"2024-02-26T06:21:57.439932+0000","last_undegraded":"2024-02-26T06:21:57.439932+0000","last_fullsized":"2024-02-26T06:21:57.439932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:24.113286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.206874+0000","last_change":"2024-02-26T06:20:31.077063+0000","last_active":"2024-02-26T06:21:29.206874+0000","last_peered":"2024-02-26T06:21:29.206874+0000","last_clean":"2024-02-26T06:21:29.206874+0000","last_became_active":"2024-02-26T06:20:31.075147+0000","last_became_peered":"2024-02-26T06:20:31.075147+0000","last_unstale":"2024-02-26T06:21:29.206874+0000","last_undegraded":"2024-02-26T06:21:29.206874+0000","last_fullsized":"2024-02-26T06:21:29.206874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:07:13.434635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438618+0000","last_change":"2024-02-26T06:20:31.600135+0000","last_active":"2024-02-26T06:21:57.438618+0000","last_peered":"2024-02-26T06:21:57.438618+0000","last_clean":"2024-02-26T06:21:57.438618+0000","last_became_active":"2024-02-26T06:20:31.595622+0000","last_became_peered":"2024-02-26T06:20:31.595622+0000","last_unstale":"2024-02-26T06:21:57.438618+0000","last_undegraded":"2024-02-26T06:21:57.438618+0000","last_fullsized":"2024-02-26T06:21:57.438618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:55.835700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914337+0000","last_change":"2024-02-26T06:20:31.060299+0000","last_active":"2024-02-26T06:21:28.914337+0000","last_peered":"2024-02-26T06:21:28.914337+0000","last_clean":"2024-02-26T06:21:28.914337+0000","last_became_active":"2024-02-26T06:20:31.059938+0000","last_became_peered":"2024-02-26T06:20:31.059938+0000","last_unstale":"2024-02-26T06:21:28.914337+0000","last_undegraded":"2024-02-26T06:21:28.914337+0000","last_fullsized":"2024-02-26T06:21:28.914337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:24.144958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.444182+0000","last_change":"2024-02-26T06:20:31.601945+0000","last_active":"2024-02-26T06:21:57.444182+0000","last_peered":"2024-02-26T06:21:57.444182+0000","last_clean":"2024-02-26T06:21:57.444182+0000","last_became_active":"2024-02-26T06:20:31.601560+0000","last_became_peered":"2024-02-26T06:20:31.601560+0000","last_unstale":"2024-02-26T06:21:57.444182+0000","last_undegraded":"2024-02-26T06:21:57.444182+0000","last_fullsized":"2024-02-26T06:21:57.444182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:13.932623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":111,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:05.239720+0000","last_change":"2024-02-26T06:20:31.068334+0000","last_active":"2024-02-26T06:22:05.239720+0000","last_peered":"2024-02-26T06:22:05.239720+0000","last_clean":"2024-02-26T06:22:05.239720+0000","last_became_active":"2024-02-26T06:20:31.068020+0000","last_became_peered":"2024-02-26T06:20:31.068020+0000","last_unstale":"2024-02-26T06:22:05.239720+0000","last_undegraded":"2024-02-26T06:22:05.239720+0000","last_fullsized":"2024-02-26T06:22:05.239720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:23:01.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421567+0000","last_change":"2024-02-26T06:20:31.587862+0000","last_active":"2024-02-26T06:21:57.421567+0000","last_peered":"2024-02-26T06:21:57.421567+0000","last_clean":"2024-02-26T06:21:57.421567+0000","last_became_active":"2024-02-26T06:20:31.587551+0000","last_became_peered":"2024-02-26T06:20:31.587551+0000","last_unstale":"2024-02-26T06:21:57.421567+0000","last_undegraded":"2024-02-26T06:21:57.421567+0000","last_fullsized":"2024-02-26T06:21:57.421567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:07:15.933125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902499+0000","last_change":"2024-02-26T06:20:31.058378+0000","last_active":"2024-02-26T06:21:28.902499+0000","last_peered":"2024-02-26T06:21:28.902499+0000","last_clean":"2024-02-26T06:21:28.902499+0000","last_became_active":"2024-02-26T06:20:31.058040+0000","last_became_peered":"2024-02-26T06:20:31.058040+0000","last_unstale":"2024-02-26T06:21:28.902499+0000","last_undegraded":"2024-02-26T06:21:28.902499+0000","last_fullsized":"2024-02-26T06:21:28.902499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:33:40.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440116+0000","last_change":"2024-02-26T06:20:31.601837+0000","last_active":"2024-02-26T06:21:57.440116+0000","last_peered":"2024-02-26T06:21:57.440116+0000","last_clean":"2024-02-26T06:21:57.440116+0000","last_became_active":"2024-02-26T06:20:31.597310+0000","last_became_peered":"2024-02-26T06:20:31.597310+0000","last_unstale":"2024-02-26T06:21:57.440116+0000","last_undegraded":"2024-02-26T06:21:57.440116+0000","last_fullsized":"2024-02-26T06:21:57.440116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.383510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914898+0000","last_change":"2024-02-26T06:20:31.076158+0000","last_active":"2024-02-26T06:21:28.914898+0000","last_peered":"2024-02-26T06:21:28.914898+0000","last_clean":"2024-02-26T06:21:28.914898+0000","last_became_active":"2024-02-26T06:20:31.075795+0000","last_became_peered":"2024-02-26T06:20:31.075795+0000","last_unstale":"2024-02-26T06:21:28.914898+0000","last_undegraded":"2024-02-26T06:21:28.914898+0000","last_fullsized":"2024-02-26T06:21:28.914898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:38.491674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421042+0000","last_change":"2024-02-26T06:20:31.586122+0000","last_active":"2024-02-26T06:21:57.421042+0000","last_peered":"2024-02-26T06:21:57.421042+0000","last_clean":"2024-02-26T06:21:57.421042+0000","last_became_active":"2024-02-26T06:20:31.585070+0000","last_became_peered":"2024-02-26T06:20:31.585070+0000","last_unstale":"2024-02-26T06:21:57.421042+0000","last_undegraded":"2024-02-26T06:21:57.421042+0000","last_fullsized":"2024-02-26T06:21:57.421042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:11.034252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":257,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:06.972405+0000","last_change":"2024-02-26T06:21:28.913681+0000","last_active":"2024-02-26T06:22:06.972405+0000","last_peered":"2024-02-26T06:22:06.972405+0000","last_clean":"2024-02-26T06:22:06.972405+0000","last_became_active":"2024-02-26T06:21:28.905006+0000","last_became_peered":"2024-02-26T06:21:28.905006+0000","last_unstale":"2024-02-26T06:22:06.972405+0000","last_undegraded":"2024-02-26T06:22:06.972405+0000","last_fullsized":"2024-02-26T06:22:06.972405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:41.432983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61122,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":186,"num_read_kb":152,"num_write":33,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60873,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410737+0000","last_change":"2024-02-26T06:20:31.584757+0000","last_active":"2024-02-26T06:21:57.410737+0000","last_peered":"2024-02-26T06:21:57.410737+0000","last_clean":"2024-02-26T06:21:57.410737+0000","last_became_active":"2024-02-26T06:20:31.583890+0000","last_became_peered":"2024-02-26T06:20:31.583890+0000","last_unstale":"2024-02-26T06:21:57.410737+0000","last_undegraded":"2024-02-26T06:21:57.410737+0000","last_fullsized":"2024-02-26T06:21:57.410737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:10:58.280082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903234+0000","last_change":"2024-02-26T06:20:31.076044+0000","last_active":"2024-02-26T06:21:28.903234+0000","last_peered":"2024-02-26T06:21:28.903234+0000","last_clean":"2024-02-26T06:21:28.903234+0000","last_became_active":"2024-02-26T06:20:31.075744+0000","last_became_peered":"2024-02-26T06:20:31.075744+0000","last_unstale":"2024-02-26T06:21:28.903234+0000","last_undegraded":"2024-02-26T06:21:28.903234+0000","last_fullsized":"2024-02-26T06:21:28.903234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:35.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.443679+0000","last_change":"2024-02-26T06:20:31.602169+0000","last_active":"2024-02-26T06:21:57.443679+0000","last_peered":"2024-02-26T06:21:57.443679+0000","last_clean":"2024-02-26T06:21:57.443679+0000","last_became_active":"2024-02-26T06:20:31.601280+0000","last_became_peered":"2024-02-26T06:20:31.601280+0000","last_unstale":"2024-02-26T06:21:57.443679+0000","last_undegraded":"2024-02-26T06:21:57.443679+0000","last_fullsized":"2024-02-26T06:21:57.443679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:07.652549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914086+0000","last_change":"2024-02-26T06:20:31.080482+0000","last_active":"2024-02-26T06:21:28.914086+0000","last_peered":"2024-02-26T06:21:28.914086+0000","last_clean":"2024-02-26T06:21:28.914086+0000","last_became_active":"2024-02-26T06:20:31.080188+0000","last_became_peered":"2024-02-26T06:20:31.080188+0000","last_unstale":"2024-02-26T06:21:28.914086+0000","last_undegraded":"2024-02-26T06:21:28.914086+0000","last_fullsized":"2024-02-26T06:21:28.914086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:32.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.433448+0000","last_change":"2024-02-26T06:20:31.599753+0000","last_active":"2024-02-26T06:21:57.433448+0000","last_peered":"2024-02-26T06:21:57.433448+0000","last_clean":"2024-02-26T06:21:57.433448+0000","last_became_active":"2024-02-26T06:20:31.599409+0000","last_became_peered":"2024-02-26T06:20:31.599409+0000","last_unstale":"2024-02-26T06:21:57.433448+0000","last_undegraded":"2024-02-26T06:21:57.433448+0000","last_fullsized":"2024-02-26T06:21:57.433448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:43.887431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:02.272874+0000","last_change":"2024-02-26T06:20:31.059656+0000","last_active":"2024-02-26T06:22:02.272874+0000","last_peered":"2024-02-26T06:22:02.272874+0000","last_clean":"2024-02-26T06:22:02.272874+0000","last_became_active":"2024-02-26T06:20:31.059289+0000","last_became_peered":"2024-02-26T06:20:31.059289+0000","last_unstale":"2024-02-26T06:22:02.272874+0000","last_undegraded":"2024-02-26T06:22:02.272874+0000","last_fullsized":"2024-02-26T06:22:02.272874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:08:57.650876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.402208+0000","last_change":"2024-02-26T06:21:28.907778+0000","last_active":"2024-02-26T06:21:57.402208+0000","last_peered":"2024-02-26T06:21:57.402208+0000","last_clean":"2024-02-26T06:21:57.402208+0000","last_became_active":"2024-02-26T06:21:28.907185+0000","last_became_peered":"2024-02-26T06:21:28.907185+0000","last_unstale":"2024-02-26T06:21:57.402208+0000","last_undegraded":"2024-02-26T06:21:57.402208+0000","last_fullsized":"2024-02-26T06:21:57.402208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:39.572867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.943152+0000","last_change":"2024-02-26T06:21:28.943152+0000","last_active":"2024-02-26T06:21:28.943152+0000","last_peered":"2024-02-26T06:21:28.943152+0000","last_clean":"2024-02-26T06:21:28.943152+0000","last_became_active":"2024-02-26T06:21:28.908090+0000","last_became_peered":"2024-02-26T06:21:28.908090+0000","last_unstale":"2024-02-26T06:21:28.943152+0000","last_undegraded":"2024-02-26T06:21:28.943152+0000","last_fullsized":"2024-02-26T06:21:28.943152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:56.254015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414233+0000","last_change":"2024-02-26T06:20:31.586151+0000","last_active":"2024-02-26T06:21:57.414233+0000","last_peered":"2024-02-26T06:21:57.414233+0000","last_clean":"2024-02-26T06:21:57.414233+0000","last_became_active":"2024-02-26T06:20:31.585615+0000","last_became_peered":"2024-02-26T06:20:31.585615+0000","last_unstale":"2024-02-26T06:21:57.414233+0000","last_undegraded":"2024-02-26T06:21:57.414233+0000","last_fullsized":"2024-02-26T06:21:57.414233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907539+0000","last_change":"2024-02-26T06:20:31.062340+0000","last_active":"2024-02-26T06:21:28.907539+0000","last_peered":"2024-02-26T06:21:28.907539+0000","last_clean":"2024-02-26T06:21:28.907539+0000","last_became_active":"2024-02-26T06:20:31.062023+0000","last_became_peered":"2024-02-26T06:20:31.062023+0000","last_unstale":"2024-02-26T06:21:28.907539+0000","last_undegraded":"2024-02-26T06:21:28.907539+0000","last_fullsized":"2024-02-26T06:21:28.907539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:45.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413393+0000","last_change":"2024-02-26T06:20:31.585190+0000","last_active":"2024-02-26T06:21:57.413393+0000","last_peered":"2024-02-26T06:21:57.413393+0000","last_clean":"2024-02-26T06:21:57.413393+0000","last_became_active":"2024-02-26T06:20:31.584883+0000","last_became_peered":"2024-02-26T06:20:31.584883+0000","last_unstale":"2024-02-26T06:21:57.413393+0000","last_undegraded":"2024-02-26T06:21:57.413393+0000","last_fullsized":"2024-02-26T06:21:57.413393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:47:51.843824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904004+0000","last_change":"2024-02-26T06:20:31.075573+0000","last_active":"2024-02-26T06:21:28.904004+0000","last_peered":"2024-02-26T06:21:28.904004+0000","last_clean":"2024-02-26T06:21:28.904004+0000","last_became_active":"2024-02-26T06:20:31.075153+0000","last_became_peered":"2024-02-26T06:20:31.075153+0000","last_unstale":"2024-02-26T06:21:28.904004+0000","last_undegraded":"2024-02-26T06:21:28.904004+0000","last_fullsized":"2024-02-26T06:21:28.904004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:43.886858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432026+0000","last_change":"2024-02-26T06:20:31.604400+0000","last_active":"2024-02-26T06:21:57.432026+0000","last_peered":"2024-02-26T06:21:57.432026+0000","last_clean":"2024-02-26T06:21:57.432026+0000","last_became_active":"2024-02-26T06:20:31.604068+0000","last_became_peered":"2024-02-26T06:20:31.604068+0000","last_unstale":"2024-02-26T06:21:57.432026+0000","last_undegraded":"2024-02-26T06:21:57.432026+0000","last_fullsized":"2024-02-26T06:21:57.432026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:28.792297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902841+0000","last_change":"2024-02-26T06:20:31.072896+0000","last_active":"2024-02-26T06:21:28.902841+0000","last_peered":"2024-02-26T06:21:28.902841+0000","last_clean":"2024-02-26T06:21:28.902841+0000","last_became_active":"2024-02-26T06:20:31.072571+0000","last_became_peered":"2024-02-26T06:20:31.072571+0000","last_unstale":"2024-02-26T06:21:28.902841+0000","last_undegraded":"2024-02-26T06:21:28.902841+0000","last_fullsized":"2024-02-26T06:21:28.902841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:39.054737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.415829+0000","last_change":"2024-02-26T06:21:28.906840+0000","last_active":"2024-02-26T06:21:57.415829+0000","last_peered":"2024-02-26T06:21:57.415829+0000","last_clean":"2024-02-26T06:21:57.415829+0000","last_became_active":"2024-02-26T06:21:28.905881+0000","last_became_peered":"2024-02-26T06:21:28.905881+0000","last_unstale":"2024-02-26T06:21:57.415829+0000","last_undegraded":"2024-02-26T06:21:57.415829+0000","last_fullsized":"2024-02-26T06:21:57.415829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:47.754145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.406314+0000","last_change":"2024-02-26T06:20:31.588989+0000","last_active":"2024-02-26T06:21:57.406314+0000","last_peered":"2024-02-26T06:21:57.406314+0000","last_clean":"2024-02-26T06:21:57.406314+0000","last_became_active":"2024-02-26T06:20:31.588683+0000","last_became_peered":"2024-02-26T06:20:31.588683+0000","last_unstale":"2024-02-26T06:21:57.406314+0000","last_undegraded":"2024-02-26T06:21:57.406314+0000","last_fullsized":"2024-02-26T06:21:57.406314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:29.687622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:02.272155+0000","last_change":"2024-02-26T06:20:31.081111+0000","last_active":"2024-02-26T06:22:02.272155+0000","last_peered":"2024-02-26T06:22:02.272155+0000","last_clean":"2024-02-26T06:22:02.272155+0000","last_became_active":"2024-02-26T06:20:31.080788+0000","last_became_peered":"2024-02-26T06:20:31.080788+0000","last_unstale":"2024-02-26T06:22:02.272155+0000","last_undegraded":"2024-02-26T06:22:02.272155+0000","last_fullsized":"2024-02-26T06:22:02.272155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:44:34.400256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440753+0000","last_change":"2024-02-26T06:21:29.211650+0000","last_active":"2024-02-26T06:21:57.440753+0000","last_peered":"2024-02-26T06:21:57.440753+0000","last_clean":"2024-02-26T06:21:57.440753+0000","last_became_active":"2024-02-26T06:21:29.210407+0000","last_became_peered":"2024-02-26T06:21:29.210407+0000","last_unstale":"2024-02-26T06:21:57.440753+0000","last_undegraded":"2024-02-26T06:21:57.440753+0000","last_fullsized":"2024-02-26T06:21:57.440753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:22:00.359506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898303+0000","last_change":"2024-02-26T06:20:31.061702+0000","last_active":"2024-02-26T06:21:28.898303+0000","last_peered":"2024-02-26T06:21:28.898303+0000","last_clean":"2024-02-26T06:21:28.898303+0000","last_became_active":"2024-02-26T06:20:31.061358+0000","last_became_peered":"2024-02-26T06:20:31.061358+0000","last_unstale":"2024-02-26T06:21:28.898303+0000","last_undegraded":"2024-02-26T06:21:28.898303+0000","last_fullsized":"2024-02-26T06:21:28.898303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:25:41.652032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903059+0000","last_change":"2024-02-26T06:19:10.834318+0000","last_active":"2024-02-26T06:21:28.903059+0000","last_peered":"2024-02-26T06:21:28.903059+0000","last_clean":"2024-02-26T06:21:28.903059+0000","last_became_active":"2024-02-26T06:19:10.833869+0000","last_became_peered":"2024-02-26T06:19:10.833869+0000","last_unstale":"2024-02-26T06:21:28.903059+0000","last_undegraded":"2024-02-26T06:21:28.903059+0000","last_fullsized":"2024-02-26T06:21:28.903059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_clean_scrub_stamp":"2024-02-26T06:15:16.238016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:57.078533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428577+0000","last_change":"2024-02-26T06:20:31.600738+0000","last_active":"2024-02-26T06:21:57.428577+0000","last_peered":"2024-02-26T06:21:57.428577+0000","last_clean":"2024-02-26T06:21:57.428577+0000","last_became_active":"2024-02-26T06:20:31.599144+0000","last_became_peered":"2024-02-26T06:20:31.599144+0000","last_unstale":"2024-02-26T06:21:57.428577+0000","last_undegraded":"2024-02-26T06:21:57.428577+0000","last_fullsized":"2024-02-26T06:21:57.428577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:01:16.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903127+0000","last_change":"2024-02-26T06:20:31.063563+0000","last_active":"2024-02-26T06:21:28.903127+0000","last_peered":"2024-02-26T06:21:28.903127+0000","last_clean":"2024-02-26T06:21:28.903127+0000","last_became_active":"2024-02-26T06:20:31.063212+0000","last_became_peered":"2024-02-26T06:20:31.063212+0000","last_unstale":"2024-02-26T06:21:28.903127+0000","last_undegraded":"2024-02-26T06:21:28.903127+0000","last_fullsized":"2024-02-26T06:21:28.903127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:16.480553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907915+0000","last_change":"2024-02-26T06:20:31.075104+0000","last_active":"2024-02-26T06:21:28.907915+0000","last_peered":"2024-02-26T06:21:28.907915+0000","last_clean":"2024-02-26T06:21:28.907915+0000","last_became_active":"2024-02-26T06:20:31.074777+0000","last_became_peered":"2024-02-26T06:20:31.074777+0000","last_unstale":"2024-02-26T06:21:28.907915+0000","last_undegraded":"2024-02-26T06:21:28.907915+0000","last_fullsized":"2024-02-26T06:21:28.907915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:32:20.511703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.416264+0000","last_change":"2024-02-26T06:20:31.605038+0000","last_active":"2024-02-26T06:21:57.416264+0000","last_peered":"2024-02-26T06:21:57.416264+0000","last_clean":"2024-02-26T06:21:57.416264+0000","last_became_active":"2024-02-26T06:20:31.603329+0000","last_became_peered":"2024-02-26T06:20:31.603329+0000","last_unstale":"2024-02-26T06:21:57.416264+0000","last_undegraded":"2024-02-26T06:21:57.416264+0000","last_fullsized":"2024-02-26T06:21:57.416264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:45.066836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904225+0000","last_change":"2024-02-26T06:20:31.084613+0000","last_active":"2024-02-26T06:21:28.904225+0000","last_peered":"2024-02-26T06:21:28.904225+0000","last_clean":"2024-02-26T06:21:28.904225+0000","last_became_active":"2024-02-26T06:20:31.084021+0000","last_became_peered":"2024-02-26T06:20:31.084021+0000","last_unstale":"2024-02-26T06:21:28.904225+0000","last_undegraded":"2024-02-26T06:21:28.904225+0000","last_fullsized":"2024-02-26T06:21:28.904225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:52:42.096437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438877+0000","last_change":"2024-02-26T06:20:31.590367+0000","last_active":"2024-02-26T06:21:57.438877+0000","last_peered":"2024-02-26T06:21:57.438877+0000","last_clean":"2024-02-26T06:21:57.438877+0000","last_became_active":"2024-02-26T06:20:31.582914+0000","last_became_peered":"2024-02-26T06:20:31.582914+0000","last_unstale":"2024-02-26T06:21:57.438877+0000","last_undegraded":"2024-02-26T06:21:57.438877+0000","last_fullsized":"2024-02-26T06:21:57.438877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:02:47.496527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898447+0000","last_change":"2024-02-26T06:20:31.070190+0000","last_active":"2024-02-26T06:21:28.898447+0000","last_peered":"2024-02-26T06:21:28.898447+0000","last_clean":"2024-02-26T06:21:28.898447+0000","last_became_active":"2024-02-26T06:20:31.069854+0000","last_became_peered":"2024-02-26T06:20:31.069854+0000","last_unstale":"2024-02-26T06:21:28.898447+0000","last_undegraded":"2024-02-26T06:21:28.898447+0000","last_fullsized":"2024-02-26T06:21:28.898447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:57.728821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440988+0000","last_change":"2024-02-26T06:20:31.595388+0000","last_active":"2024-02-26T06:21:57.440988+0000","last_peered":"2024-02-26T06:21:57.440988+0000","last_clean":"2024-02-26T06:21:57.440988+0000","last_became_active":"2024-02-26T06:20:31.594970+0000","last_became_peered":"2024-02-26T06:20:31.594970+0000","last_unstale":"2024-02-26T06:21:57.440988+0000","last_undegraded":"2024-02-26T06:21:57.440988+0000","last_fullsized":"2024-02-26T06:21:57.440988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:12:52.182988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902824+0000","last_change":"2024-02-26T06:20:31.066256+0000","last_active":"2024-02-26T06:21:28.902824+0000","last_peered":"2024-02-26T06:21:28.902824+0000","last_clean":"2024-02-26T06:21:28.902824+0000","last_became_active":"2024-02-26T06:20:31.065956+0000","last_became_peered":"2024-02-26T06:20:31.065956+0000","last_unstale":"2024-02-26T06:21:28.902824+0000","last_undegraded":"2024-02-26T06:21:28.902824+0000","last_fullsized":"2024-02-26T06:21:28.902824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:35.405114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.379908+0000","last_change":"2024-02-26T06:20:31.585102+0000","last_active":"2024-02-26T06:21:57.379908+0000","last_peered":"2024-02-26T06:21:57.379908+0000","last_clean":"2024-02-26T06:21:57.379908+0000","last_became_active":"2024-02-26T06:20:31.584351+0000","last_became_peered":"2024-02-26T06:20:31.584351+0000","last_unstale":"2024-02-26T06:21:57.379908+0000","last_undegraded":"2024-02-26T06:21:57.379908+0000","last_fullsized":"2024-02-26T06:21:57.379908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:09.708752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.897644+0000","last_change":"2024-02-26T06:20:31.079664+0000","last_active":"2024-02-26T06:21:28.897644+0000","last_peered":"2024-02-26T06:21:28.897644+0000","last_clean":"2024-02-26T06:21:28.897644+0000","last_became_active":"2024-02-26T06:20:31.079234+0000","last_became_peered":"2024-02-26T06:20:31.079234+0000","last_unstale":"2024-02-26T06:21:28.897644+0000","last_undegraded":"2024-02-26T06:21:28.897644+0000","last_fullsized":"2024-02-26T06:21:28.897644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:37.407215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439347+0000","last_change":"2024-02-26T06:20:31.597416+0000","last_active":"2024-02-26T06:21:57.439347+0000","last_peered":"2024-02-26T06:21:57.439347+0000","last_clean":"2024-02-26T06:21:57.439347+0000","last_became_active":"2024-02-26T06:20:31.597094+0000","last_became_peered":"2024-02-26T06:20:31.597094+0000","last_unstale":"2024-02-26T06:21:57.439347+0000","last_undegraded":"2024-02-26T06:21:57.439347+0000","last_fullsized":"2024-02-26T06:21:57.439347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:24.774298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903066+0000","last_change":"2024-02-26T06:20:31.074372+0000","last_active":"2024-02-26T06:21:28.903066+0000","last_peered":"2024-02-26T06:21:28.903066+0000","last_clean":"2024-02-26T06:21:28.903066+0000","last_became_active":"2024-02-26T06:20:31.074041+0000","last_became_peered":"2024-02-26T06:20:31.074041+0000","last_unstale":"2024-02-26T06:21:28.903066+0000","last_undegraded":"2024-02-26T06:21:28.903066+0000","last_fullsized":"2024-02-26T06:21:28.903066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:01.367668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441767+0000","last_change":"2024-02-26T06:20:31.599588+0000","last_active":"2024-02-26T06:21:57.441767+0000","last_peered":"2024-02-26T06:21:57.441767+0000","last_clean":"2024-02-26T06:21:57.441767+0000","last_became_active":"2024-02-26T06:20:31.599240+0000","last_became_peered":"2024-02-26T06:20:31.599240+0000","last_unstale":"2024-02-26T06:21:57.441767+0000","last_undegraded":"2024-02-26T06:21:57.441767+0000","last_fullsized":"2024-02-26T06:21:57.441767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:07:14.799565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914009+0000","last_change":"2024-02-26T06:21:28.914009+0000","last_active":"2024-02-26T06:21:28.914009+0000","last_peered":"2024-02-26T06:21:28.914009+0000","last_clean":"2024-02-26T06:21:28.914009+0000","last_became_active":"2024-02-26T06:21:28.906712+0000","last_became_peered":"2024-02-26T06:21:28.906712+0000","last_unstale":"2024-02-26T06:21:28.914009+0000","last_undegraded":"2024-02-26T06:21:28.914009+0000","last_fullsized":"2024-02-26T06:21:28.914009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:21:36.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432745+0000","last_change":"2024-02-26T06:20:31.603955+0000","last_active":"2024-02-26T06:21:57.432745+0000","last_peered":"2024-02-26T06:21:57.432745+0000","last_clean":"2024-02-26T06:21:57.432745+0000","last_became_active":"2024-02-26T06:20:31.603635+0000","last_became_peered":"2024-02-26T06:20:31.603635+0000","last_unstale":"2024-02-26T06:21:57.432745+0000","last_undegraded":"2024-02-26T06:21:57.432745+0000","last_fullsized":"2024-02-26T06:21:57.432745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:56:59.060204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903732+0000","last_change":"2024-02-26T06:20:31.074917+0000","last_active":"2024-02-26T06:21:28.903732+0000","last_peered":"2024-02-26T06:21:28.903732+0000","last_clean":"2024-02-26T06:21:28.903732+0000","last_became_active":"2024-02-26T06:20:31.074590+0000","last_became_peered":"2024-02-26T06:20:31.074590+0000","last_unstale":"2024-02-26T06:21:28.903732+0000","last_undegraded":"2024-02-26T06:21:28.903732+0000","last_fullsized":"2024-02-26T06:21:28.903732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:43.160268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420733+0000","last_change":"2024-02-26T06:20:31.582236+0000","last_active":"2024-02-26T06:21:57.420733+0000","last_peered":"2024-02-26T06:21:57.420733+0000","last_clean":"2024-02-26T06:21:57.420733+0000","last_became_active":"2024-02-26T06:20:31.581851+0000","last_became_peered":"2024-02-26T06:20:31.581851+0000","last_unstale":"2024-02-26T06:21:57.420733+0000","last_undegraded":"2024-02-26T06:21:57.420733+0000","last_fullsized":"2024-02-26T06:21:57.420733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:53.214338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914141+0000","last_change":"2024-02-26T06:20:31.068179+0000","last_active":"2024-02-26T06:21:28.914141+0000","last_peered":"2024-02-26T06:21:28.914141+0000","last_clean":"2024-02-26T06:21:28.914141+0000","last_became_active":"2024-02-26T06:20:31.067888+0000","last_became_peered":"2024-02-26T06:20:31.067888+0000","last_unstale":"2024-02-26T06:21:28.914141+0000","last_undegraded":"2024-02-26T06:21:28.914141+0000","last_fullsized":"2024-02-26T06:21:28.914141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:47:39.223749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.424729+0000","last_change":"2024-02-26T06:20:31.598820+0000","last_active":"2024-02-26T06:21:57.424729+0000","last_peered":"2024-02-26T06:21:57.424729+0000","last_clean":"2024-02-26T06:21:57.424729+0000","last_became_active":"2024-02-26T06:20:31.598469+0000","last_became_peered":"2024-02-26T06:20:31.598469+0000","last_unstale":"2024-02-26T06:21:57.424729+0000","last_undegraded":"2024-02-26T06:21:57.424729+0000","last_fullsized":"2024-02-26T06:21:57.424729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:41:15.713748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.207033+0000","last_change":"2024-02-26T06:20:31.080988+0000","last_active":"2024-02-26T06:21:29.207033+0000","last_peered":"2024-02-26T06:21:29.207033+0000","last_clean":"2024-02-26T06:21:29.207033+0000","last_became_active":"2024-02-26T06:20:31.080654+0000","last_became_peered":"2024-02-26T06:20:31.080654+0000","last_unstale":"2024-02-26T06:21:29.207033+0000","last_undegraded":"2024-02-26T06:21:29.207033+0000","last_fullsized":"2024-02-26T06:21:29.207033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:50.315162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414395+0000","last_change":"2024-02-26T06:20:31.586057+0000","last_active":"2024-02-26T06:21:57.414395+0000","last_peered":"2024-02-26T06:21:57.414395+0000","last_clean":"2024-02-26T06:21:57.414395+0000","last_became_active":"2024-02-26T06:20:31.585756+0000","last_became_peered":"2024-02-26T06:20:31.585756+0000","last_unstale":"2024-02-26T06:21:57.414395+0000","last_undegraded":"2024-02-26T06:21:57.414395+0000","last_fullsized":"2024-02-26T06:21:57.414395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:59:07.032287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.411546+0000","last_change":"2024-02-26T06:20:31.589067+0000","last_active":"2024-02-26T06:21:57.411546+0000","last_peered":"2024-02-26T06:21:57.411546+0000","last_clean":"2024-02-26T06:21:57.411546+0000","last_became_active":"2024-02-26T06:20:31.588769+0000","last_became_peered":"2024-02-26T06:20:31.588769+0000","last_unstale":"2024-02-26T06:21:57.411546+0000","last_undegraded":"2024-02-26T06:21:57.411546+0000","last_fullsized":"2024-02-26T06:21:57.411546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:43.211127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412415+0000","last_change":"2024-02-26T06:20:31.583838+0000","last_active":"2024-02-26T06:21:57.412415+0000","last_peered":"2024-02-26T06:21:57.412415+0000","last_clean":"2024-02-26T06:21:57.412415+0000","last_became_active":"2024-02-26T06:20:31.583524+0000","last_became_peered":"2024-02-26T06:20:31.583524+0000","last_unstale":"2024-02-26T06:21:57.412415+0000","last_undegraded":"2024-02-26T06:21:57.412415+0000","last_fullsized":"2024-02-26T06:21:57.412415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:21.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413932+0000","last_change":"2024-02-26T06:20:31.597901+0000","last_active":"2024-02-26T06:21:57.413932+0000","last_peered":"2024-02-26T06:21:57.413932+0000","last_clean":"2024-02-26T06:21:57.413932+0000","last_became_active":"2024-02-26T06:20:31.597633+0000","last_became_peered":"2024-02-26T06:20:31.597633+0000","last_unstale":"2024-02-26T06:21:57.413932+0000","last_undegraded":"2024-02-26T06:21:57.413932+0000","last_fullsized":"2024-02-26T06:21:57.413932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:14.084579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438872+0000","last_change":"2024-02-26T06:21:29.210694+0000","last_active":"2024-02-26T06:21:57.438872+0000","last_peered":"2024-02-26T06:21:57.438872+0000","last_clean":"2024-02-26T06:21:57.438872+0000","last_became_active":"2024-02-26T06:21:29.209421+0000","last_became_peered":"2024-02-26T06:21:29.209421+0000","last_unstale":"2024-02-26T06:21:57.438872+0000","last_undegraded":"2024-02-26T06:21:57.438872+0000","last_fullsized":"2024-02-26T06:21:57.438872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:46.511066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.430810+0000","last_change":"2024-02-26T06:20:31.605866+0000","last_active":"2024-02-26T06:21:57.430810+0000","last_peered":"2024-02-26T06:21:57.430810+0000","last_clean":"2024-02-26T06:21:57.430810+0000","last_became_active":"2024-02-26T06:20:31.605404+0000","last_became_peered":"2024-02-26T06:20:31.605404+0000","last_unstale":"2024-02-26T06:21:57.430810+0000","last_undegraded":"2024-02-26T06:21:57.430810+0000","last_fullsized":"2024-02-26T06:21:57.430810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:03:25.055462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413581+0000","last_change":"2024-02-26T06:20:31.585214+0000","last_active":"2024-02-26T06:21:57.413581+0000","last_peered":"2024-02-26T06:21:57.413581+0000","last_clean":"2024-02-26T06:21:57.413581+0000","last_became_active":"2024-02-26T06:20:31.584895+0000","last_became_peered":"2024-02-26T06:20:31.584895+0000","last_unstale":"2024-02-26T06:21:57.413581+0000","last_undegraded":"2024-02-26T06:21:57.413581+0000","last_fullsized":"2024-02-26T06:21:57.413581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:38.507499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414981+0000","last_change":"2024-02-26T06:20:31.582189+0000","last_active":"2024-02-26T06:21:57.414981+0000","last_peered":"2024-02-26T06:21:57.414981+0000","last_clean":"2024-02-26T06:21:57.414981+0000","last_became_active":"2024-02-26T06:20:31.580729+0000","last_became_peered":"2024-02-26T06:20:31.580729+0000","last_unstale":"2024-02-26T06:21:57.414981+0000","last_undegraded":"2024-02-26T06:21:57.414981+0000","last_fullsized":"2024-02-26T06:21:57.414981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:21.337261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.435771+0000","last_change":"2024-02-26T06:20:31.588894+0000","last_active":"2024-02-26T06:21:57.435771+0000","last_peered":"2024-02-26T06:21:57.435771+0000","last_clean":"2024-02-26T06:21:57.435771+0000","last_became_active":"2024-02-26T06:20:31.586362+0000","last_became_peered":"2024-02-26T06:20:31.586362+0000","last_unstale":"2024-02-26T06:21:57.435771+0000","last_undegraded":"2024-02-26T06:21:57.435771+0000","last_fullsized":"2024-02-26T06:21:57.435771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:28:30.009901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.434220+0000","last_change":"2024-02-26T06:20:31.608264+0000","last_active":"2024-02-26T06:21:57.434220+0000","last_peered":"2024-02-26T06:21:57.434220+0000","last_clean":"2024-02-26T06:21:57.434220+0000","last_became_active":"2024-02-26T06:20:31.607855+0000","last_became_peered":"2024-02-26T06:20:31.607855+0000","last_unstale":"2024-02-26T06:21:57.434220+0000","last_undegraded":"2024-02-26T06:21:57.434220+0000","last_fullsized":"2024-02-26T06:21:57.434220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:20.169489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441525+0000","last_change":"2024-02-26T06:20:31.591000+0000","last_active":"2024-02-26T06:21:57.441525+0000","last_peered":"2024-02-26T06:21:57.441525+0000","last_clean":"2024-02-26T06:21:57.441525+0000","last_became_active":"2024-02-26T06:20:31.588201+0000","last_became_peered":"2024-02-26T06:20:31.588201+0000","last_unstale":"2024-02-26T06:21:57.441525+0000","last_undegraded":"2024-02-26T06:21:57.441525+0000","last_fullsized":"2024-02-26T06:21:57.441525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:47:59.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441183+0000","last_change":"2024-02-26T06:20:31.590398+0000","last_active":"2024-02-26T06:21:57.441183+0000","last_peered":"2024-02-26T06:21:57.441183+0000","last_clean":"2024-02-26T06:21:57.441183+0000","last_became_active":"2024-02-26T06:20:31.590042+0000","last_became_peered":"2024-02-26T06:20:31.590042+0000","last_unstale":"2024-02-26T06:21:57.441183+0000","last_undegraded":"2024-02-26T06:21:57.441183+0000","last_fullsized":"2024-02-26T06:21:57.441183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:40:01.233978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440741+0000","last_change":"2024-02-26T06:20:31.611698+0000","last_active":"2024-02-26T06:21:57.440741+0000","last_peered":"2024-02-26T06:21:57.440741+0000","last_clean":"2024-02-26T06:21:57.440741+0000","last_became_active":"2024-02-26T06:20:31.610979+0000","last_became_peered":"2024-02-26T06:20:31.610979+0000","last_unstale":"2024-02-26T06:21:57.440741+0000","last_undegraded":"2024-02-26T06:21:57.440741+0000","last_fullsized":"2024-02-26T06:21:57.440741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:44.604373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414103+0000","last_change":"2024-02-26T06:20:31.582016+0000","last_active":"2024-02-26T06:21:57.414103+0000","last_peered":"2024-02-26T06:21:57.414103+0000","last_clean":"2024-02-26T06:21:57.414103+0000","last_became_active":"2024-02-26T06:20:31.581689+0000","last_became_peered":"2024-02-26T06:20:31.581689+0000","last_unstale":"2024-02-26T06:21:57.414103+0000","last_undegraded":"2024-02-26T06:21:57.414103+0000","last_fullsized":"2024-02-26T06:21:57.414103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:04:11.030182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284221,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65264,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":554,"num_read_kb":344,"num_write":164,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":60985,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":195792,"data_compressed":13382,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459479,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90500000000000003,"5min":1.018,"15min":1.018},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.45700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.65000000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63100000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.311,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42799999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.091,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":0.84099999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.996,"5min":1.179,"15min":1.179},"last":0.67100000000000004}]},{"osd":7,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.443,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66300000000000003},"min":{"1min":0.51300000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.76400000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:21:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.499},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54100000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1830000000000001,"5min":1.21,"15min":1.21},"last":0.749}]},{"osd":10,"last update":"Mon Feb 26 06:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.57899999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.93600000000000005,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.52600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73199999999999998}]}]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":1112502,"data_compressed":43797,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":6.2229999999999999,"5min":6.2229999999999999,"15min":6.2229999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32000000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.50600000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.71299999999999997}]},{"osd":3,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.81899999999999995,"15min":0.83499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86099999999999999,"5min":1.403,"15min":1.403},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88800000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.748}]},{"osd":4,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48999999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79200000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.871,"15min":0.88500000000000001},"min":{"1min":0.56499999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.091,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.56799999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2629999999999999,"5min":1.284,"15min":1.284},"last":0.81799999999999995}]},{"osd":7,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61199999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.58799999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70199999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.97799999999999998,"15min":0.98799999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.123},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.202,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.80800000000000005}]},{"osd":11,"last update":"Mon Feb 26 06:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93600000000000005,"5min":0.998,"15min":0.998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.97499999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.874}]}]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":459603,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.4119999999999999,"15min":1.4690000000000001},"min":{"1min":0.73199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8939999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.3500000000000001,"15min":1.3799999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.069,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":1.216}]},{"osd":1,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6459999999999999,"5min":1.923,"15min":1.923},"last":1.133},{"interface":"front","average":{"1min":1.131,"5min":1.323,"15min":1.391},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.504,"5min":2.407,"15min":2.407},"last":1.397}]},{"osd":2,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4690000000000001,"15min":1.534},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.298,"5min":2.609,"15min":2.609},"last":1.663},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.5600000000000001,"15min":1.607},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.2370000000000001,"5min":3.681,"15min":3.681},"last":1.556}]},{"osd":3,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.804,"15min":1.913},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.4929999999999999,"5min":3.016,"15min":3.016},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.7689999999999999,"15min":1.823},"min":{"1min":0.99199999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":2.8610000000000002,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.8129999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.784,"15min":1.8320000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.6419999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":2.0089999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.6819999999999999,"15min":1.802},"min":{"1min":0.64000000000000001,"5min":0.624,"15min":0.624},"max":{"1min":2.5089999999999999,"5min":3.98,"15min":3.98},"last":2.1040000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.883,"15min":1.9970000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6030000000000002,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":2.379},{"interface":"front","average":{"1min":1.8100000000000001,"5min":1.905,"15min":1.9910000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.952}]},{"osd":6,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":2.169,"15min":2.2749999999999999},"min":{"1min":0.83999999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":3.2240000000000002,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":2.165},{"interface":"front","average":{"1min":2.1400000000000001,"5min":2.2240000000000002,"15min":2.3079999999999998},"min":{"1min":0.91900000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.1600000000000001,"5min":3.835,"15min":3.835},"last":2.3500000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.835,"5min":2.0880000000000001,"15min":2.2000000000000002},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.911,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":2.2480000000000002},{"interface":"front","average":{"1min":2.04,"5min":2.1179999999999999,"15min":2.194},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":3.3170000000000002,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":2.0550000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":2.2080000000000002,"5min":2.1869999999999998,"15min":2.2360000000000002},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":6.4669999999999996,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":2.206},{"interface":"front","average":{"1min":1.9390000000000001,"5min":2.2320000000000002,"15min":2.3530000000000002},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.9830000000000001,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":1.879}]},{"osd":10,"last update":"Mon Feb 26 06:21:12 2024","interfaces":[{"interface":"back","average":{"1min":1.861,"5min":2.2170000000000001,"15min":2.2770000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.8940000000000001,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":2.3250000000000002},{"interface":"front","average":{"1min":1.9410000000000001,"5min":2.0720000000000001,"15min":2.0939999999999999},"min":{"1min":1.087,"5min":1.087,"15min":1.087},"max":{"1min":3.0219999999999998,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":2.2879999999999998}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":459411,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.437,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99099999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98799999999999999,"5min":1.198,"15min":1.198},"last":1.1299999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.628},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70899999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64700000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.441,"15min":0.439},"min":{"1min":0.32300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63700000000000001,"5min":0.755,"15min":0.755},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.436,"15min":0.436},"min":{"1min":0.308,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.36499999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.496,"5min":0.432,"15min":0.432},"max":{"1min":0.95699999999999996,"5min":1.379,"15min":1.379},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1919999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.85399999999999998}]},{"osd":4,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.753},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.35599999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.34300000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.57099999999999995}]},{"osd":6,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.57099999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.68999999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.075,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.504,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.48099999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71299999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88200000000000001,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2709999999999999,"5min":1.28,"15min":1.28},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93000000000000005,"15min":0.93100000000000005},"min":{"1min":0.63300000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.73899999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66500000000000004,"15min":0.65600000000000003},"min":{"1min":0.503,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91100000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.873},{"interface":"front","average":{"1min":0.754,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.57599999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.78800000000000003}]},{"osd":11,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66500000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051594,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40600000000000003,"15min":0.41999999999999998},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.499,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.42499999999999999,"15min":0.433},"min":{"1min":0.29999999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58499999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.434,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53000000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80500000000000005,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1000000000000001,"5min":1.591,"15min":1.591},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.51700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99199999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.65700000000000003}]},{"osd":4,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65900000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48599999999999999,"15min":0.51200000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.38800000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.85099999999999998,"15min":0.86099999999999999},"min":{"1min":0.47799999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.077,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.238,"5min":0.93899999999999995,"15min":0.88700000000000001},"min":{"1min":0.61099999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":7.4119999999999999,"5min":7.4119999999999999,"15min":7.4119999999999999},"last":0.67400000000000004}]},{"osd":8,"last update":"Mon Feb 26 06:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82099999999999995,"5min":0.88,"15min":0.88},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84999999999999998,"5min":0.873,"15min":0.873},"last":0.78200000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.89700000000000002,"15min":0.92700000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94299999999999995,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.749},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":0.94699999999999995},"min":{"1min":0.69799999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0620000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.496},{"interface":"front","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63}]},{"osd":11,"last update":"Mon Feb 26 06:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.84399999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":458992,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2829999999999999,"15min":1.2769999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.0270000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.3080000000000001,"15min":1.248},"min":{"1min":0.89800000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.0539999999999998,"5min":2.508,"15min":2.508},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0840000000000001,"15min":1.0760000000000001},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.121,"15min":1.103},"min":{"1min":0.72799999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8160000000000001,"5min":1.944,"15min":1.944},"last":0.53300000000000003}]},{"osd":2,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2509999999999999,"15min":1.258},"min":{"1min":0.61099999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.758,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3600000000000001,"15min":1.2889999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":6.4930000000000003,"5min":6.4930000000000003,"15min":6.4930000000000003},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6539999999999999,"5min":1.661,"15min":1.591},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.1320000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.677,"15min":1.6719999999999999},"min":{"1min":0.88100000000000001,"5min":0.745,"15min":0.745},"max":{"1min":2.181,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4390000000000001,"15min":1.4259999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.198,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.671,"5min":1.6479999999999999,"15min":1.597},"min":{"1min":1.012,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.4279999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.696,"5min":1.6719999999999999,"15min":1.6319999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1949999999999998,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.77,"5min":1.734,"15min":1.72},"min":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.246,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:21:46 2024","interfaces":[{"interface":"back","average":{"1min":1.819,"5min":1.744,"15min":1.6140000000000001},"min":{"1min":1.006,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.7669999999999999,"5min":1.8120000000000001,"15min":1.7450000000000001},"min":{"1min":0.90600000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.972,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.81100000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:21:12 2024","interfaces":[{"interface":"back","average":{"1min":1.9490000000000001,"5min":1.819,"15min":1.7470000000000001},"min":{"1min":1.1020000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.52,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":2.2599999999999998,"5min":1.9319999999999999,"15min":1.843},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":6.625,"5min":6.625,"15min":6.625},"last":0.86899999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:21:46 2024","interfaces":[{"interface":"back","average":{"1min":1.974,"5min":2.1760000000000002,"15min":2.2719999999999998},"min":{"1min":1.0489999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":3.1800000000000002,"5min":3.258,"15min":3.258},"last":0.83899999999999997},{"interface":"front","average":{"1min":2.048,"5min":2.214,"15min":2.2789999999999999},"min":{"1min":1.3600000000000001,"5min":1.181,"15min":1.181},"max":{"1min":3.347,"5min":3.347,"15min":3.347},"last":0.88300000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:21:12 2024","interfaces":[{"interface":"back","average":{"1min":2.0059999999999998,"5min":2.0819999999999999,"15min":2.0939999999999999},"min":{"1min":1.202,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":2.1120000000000001,"5min":2.1920000000000002,"15min":2.206},"min":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"max":{"1min":2.7010000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.90600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":519907,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.67800000000000005},"min":{"1min":0.56200000000000006,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.054,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.498,"5min":0.97999999999999998,"15min":0.79900000000000004},"min":{"1min":0.44,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":16.690999999999999,"5min":16.690999999999999,"15min":16.690999999999999},"last":0.78400000000000003}]},{"osd":2,"last update":"Mon Feb 26 06:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.72699999999999998,"15min":0.60299999999999998},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":9.2629999999999999,"15min":9.2629999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.52100000000000002,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58999999999999997}]},{"osd":3,"last update":"Mon Feb 26 06:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77000000000000002,"15min":0.74299999999999999},"min":{"1min":0.53000000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.246,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.78400000000000003,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1020000000000001,"5min":1.391,"15min":1.391},"last":0.73899999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.39800000000000002},"min":{"1min":0.30099999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65700000000000003,"5min":0.748,"15min":0.748},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.68400000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.436}]},{"osd":5,"last update":"Mon Feb 26 06:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.78600000000000003,"15min":0.66400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":16.672999999999998,"15min":16.672999999999998},"last":0.622},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.877,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.71799999999999997}]},{"osd":6,"last update":"Mon Feb 26 06:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80200000000000005,"15min":0.78400000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.069,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.755},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.92300000000000004,"15min":0.871},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.032,"5min":8.8729999999999993,"15min":8.8729999999999993},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Feb 26 06:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52100000000000002,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.79400000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57099999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.48599999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72999999999999998,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95299999999999996,"5min":8.9380000000000006,"15min":8.9380000000000006},"last":0.80100000000000005}]},{"osd":9,"last update":"Mon Feb 26 06:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.65900000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2090000000000001,"5min":1.224,"15min":1.224},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.046,"15min":1.0149999999999999},"min":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.831,"5min":8.8979999999999997,"15min":8.8979999999999997},"last":0.85099999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0049999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.69299999999999995,"15min":0.70299999999999996},"min":{"1min":0.40699999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93799999999999994,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.83799999999999997}]},{"osd":11,"last update":"Mon Feb 26 06:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.879}]}]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460263,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:22:03 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.004,"15min":0.84599999999999997},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.46300000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.05,"5min":1.01,"15min":0.86699999999999999},"min":{"1min":0.624,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":1.4390000000000001,"5min":1.498,"15min":1.649},"last":0.624}]},{"osd":2,"last update":"Mon Feb 26 06:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.073,"15min":0.92300000000000004},"min":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.58099999999999996},"max":{"1min":1.6970000000000001,"5min":6.6070000000000002,"15min":6.6070000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.383,"5min":1.2210000000000001,"15min":0.999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.72299999999999998}]},{"osd":3,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":1.78,"5min":1.78,"15min":1.78},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2250000000000001,"15min":1.0669999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.946,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2390000000000001,"15min":1.155},"min":{"1min":0.70999999999999996,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.008,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Mon Feb 26 06:21:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0499999999999998,"5min":1.599,"15min":1.4850000000000001},"min":{"1min":1.181,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.681,"5min":1.522,"15min":1.401},"min":{"1min":0.78800000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.5979999999999999,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.54200000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":1.923,"5min":1.8360000000000001,"15min":1.8100000000000001},"min":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.827,"15min":1.8420000000000001},"min":{"1min":0.66500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.73,"15min":1.7190000000000001},"min":{"1min":1.2150000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.45},{"interface":"front","average":{"1min":1.73,"5min":1.599,"15min":1.577},"min":{"1min":1.123,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:21:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.7749999999999999,"15min":1.7330000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":1.012},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.8029999999999999,"15min":1.7549999999999999},"min":{"1min":0.91600000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.2639999999999998,"5min":2.452,"15min":2.452},"last":0.78200000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:21:51 2024","interfaces":[{"interface":"back","average":{"1min":1.9319999999999999,"5min":1.9450000000000001,"15min":1.986},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":0.751},{"interface":"front","average":{"1min":2.4910000000000001,"5min":2.141,"15min":2.1120000000000001},"min":{"1min":1.365,"5min":0.746,"15min":0.746},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":1.0369999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:21:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8759999999999999,"5min":2.0259999999999998,"15min":2.0510000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.704,"5min":1.8160000000000001,"15min":1.8340000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.3780000000000001,"5min":2.423,"15min":2.423},"last":0.84899999999999998}]},{"osd":11,"last update":"Mon Feb 26 06:21:37 2024","interfaces":[{"interface":"back","average":{"1min":2.2890000000000001,"5min":1.964,"15min":1.9099999999999999},"min":{"1min":1.075,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":2.121,"5min":2.0369999999999999,"15min":2.0230000000000001},"min":{"1min":1.3100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.1040000000000001,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":0.96199999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":460023,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71099999999999997,"15min":0.68700000000000006},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91700000000000004,"5min":1.091,"15min":1.091},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.70499999999999996},"min":{"1min":0.53000000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1399999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.74099999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.35699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77600000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.39200000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:22:02 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.75600000000000001,"15min":0.71899999999999997},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76100000000000001,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.119,"5min":1.367,"15min":1.367},"last":0.95499999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.46899999999999997,"15min":0.47199999999999998},"min":{"1min":0.38,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.32700000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.34599999999999997}]},{"osd":6,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79300000000000004,"15min":0.76400000000000001},"min":{"1min":0.57599999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.48999999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47399999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54900000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.94299999999999995,"15min":0.94499999999999995},"min":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.204,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98499999999999999,"15min":0.98799999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.82099999999999995}]},{"osd":10,"last update":"Mon Feb 26 06:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.60899999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63500000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58699999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":459303,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.173,"15min":1.073},"min":{"1min":0.53200000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.7010000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.345,"5min":1.4279999999999999,"15min":1.1479999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.21,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":1.024}]},{"osd":1,"last update":"Mon Feb 26 06:22:02 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.177,"15min":1.0369999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.1519999999999999,"15min":1.0800000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.60399999999999998},"max":{"1min":1.718,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.3400000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:22:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3260000000000001,"15min":1.1919999999999999},"min":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9510000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.093,"15min":1.0429999999999999},"min":{"1min":0.58199999999999996,"5min":0.441,"15min":0.441},"max":{"1min":1.4530000000000001,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.0840000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3440000000000001,"15min":1.2270000000000001},"min":{"1min":0.83099999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8460000000000001,"5min":2.613,"15min":2.613},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5389999999999999,"15min":1.3859999999999999},"min":{"1min":0.98899999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.2200000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6819999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.466,"15min":1.3799999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.2149999999999999,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.375,"15min":1.341},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.9329999999999998,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":2.0089999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:21:19 2024","interfaces":[{"interface":"back","average":{"1min":1.792,"5min":1.5940000000000001,"15min":1.49},"min":{"1min":0.67600000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.0619999999999998,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":2.1000000000000001},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.7969999999999999,"15min":1.655},"min":{"1min":0.99099999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.7410000000000001,"5min":4.681,"15min":4.681},"last":1.8919999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.7709999999999999,"15min":1.7370000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.456,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.8420000000000001},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.8220000000000001,"15min":1.71},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3900000000000001,"5min":3.544,"15min":3.544},"last":1.804}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.806},"min":{"1min":0.97399999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.587,"15min":1.5649999999999999},"min":{"1min":0.84999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.23,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.143}]},{"osd":9,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.99,"15min":2.0459999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.0230000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":2.1629999999999998},{"interface":"front","average":{"1min":1.9990000000000001,"5min":2.2989999999999999,"15min":2.427},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.698,"5min":4.266,"15min":4.266},"last":2.1779999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.772,"5min":1.988,"15min":2.024},"min":{"1min":1.083,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.3690000000000002,"5min":2.476,"15min":2.476},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.986,"5min":2.2400000000000002,"15min":2.282},"min":{"1min":1.3140000000000001,"5min":1.127,"15min":1.127},"max":{"1min":2.4700000000000002,"5min":4.0609999999999999,"15min":4.0609999999999999},"last":1.9059999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:21:33 2024","interfaces":[{"interface":"back","average":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":2.2210000000000001},{"interface":"front","average":{"1min":2.073,"5min":2.073,"15min":2.073},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":2.0609999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":521195,"data_compressed":15495,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.50600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.079,"5min":1.093,"15min":1.093},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.378,"15min":0.35499999999999998},"min":{"1min":0.28999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56299999999999994,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.37},"min":{"1min":0.313,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56200000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75700000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.67600000000000005,"15min":0.56899999999999995},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":12.468,"15min":12.468},"last":0.77300000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.749,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98099999999999998,"5min":1.167,"15min":1.167},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.53900000000000003},"min":{"1min":0.53200000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.56200000000000006},"min":{"1min":0.502,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.754,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.72699999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77100000000000002,"15min":0.73099999999999998},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95499999999999996,"5min":1.327,"15min":1.327},"last":0.628},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80400000000000005,"15min":0.79100000000000004},"min":{"1min":0.53700000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0620000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Feb 26 06:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.42899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56499999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.44900000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.75},{"interface":"front","average":{"1min":0.752,"5min":0.68100000000000005,"15min":0.65900000000000003},"min":{"1min":0.53200000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.83499999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.92900000000000005,"15min":0.94899999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.2,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.304,"5min":1.046,"15min":1},"min":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.89600000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81200000000000006,"5min":0.87,"15min":0.87},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.85899999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.879}]}]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1050746,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":1.091},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.754,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.86899999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79000000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.73799999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39400000000000002},"min":{"1min":0.29699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59299999999999997,"5min":0.624,"15min":0.624},"last":0.432},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60699999999999998,"5min":0.75,"15min":0.75},"last":0.55300000000000005}]},{"osd":3,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78700000000000003,"15min":0.77700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1279999999999999,"5min":1.226,"15min":1.226},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94399999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.84199999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82299999999999995,"15min":0.79500000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":1.135},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.80800000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2090000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.91400000000000003}]},{"osd":7,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.85799999999999998,"5min":0.872,"15min":0.872},"last":0.754}]},{"osd":8,"last update":"Mon Feb 26 06:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83399999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.501},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70199999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.52400000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89500000000000002,"15min":0.90500000000000003},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.224,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.878},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.0089999999999999,"15min":1.0429999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1859999999999999,"5min":1.444,"15min":1.444},"last":0.92800000000000005}]},{"osd":10,"last update":"Mon Feb 26 06:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.875,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61122,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":62410,"data_compressed":9688,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63232,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:22:10.395 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-26T06:22:10.395 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-26T06:22:10.395 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-26T06:22:10.396 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph health --format=json 2024-02-26T06:22:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:11.315 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:11 smithi049 ceph-mon[31696]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:22:11.316 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:11 smithi049 ceph-mon[31696]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T06:22:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:11 smithi080 ceph-mon[36206]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:22:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:11 smithi080 ceph-mon[36206]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T06:22:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:11 smithi115 ceph-mon[36426]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:22:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:11 smithi115 ceph-mon[36426]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T06:22:13.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 -- 172.21.15.49:0/3318919681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d410d490 msgr2=0x7f98d4115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:13.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 --2- 172.21.15.49:0/3318919681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d410d490 0x7f98d4115060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009f90 tx=0x7f98cc00b330 comp rx=0 tx=0).stop 2024-02-26T06:22:13.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 -- 172.21.15.49:0/3318919681 shutdown_connections 2024-02-26T06:22:13.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 --2- 172.21.15.49:0/3318919681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d410d490 0x7f98d4115060 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 --2- 172.21.15.49:0/3318919681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98d4108400 0x7f98d41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 --2- 172.21.15.49:0/3318919681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d4108db0 0x7f98d410cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 -- 172.21.15.49:0/3318919681 >> 172.21.15.49:0/3318919681 conn(0x7f98d4077910 msgr2=0x7f98d4079d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:13.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 -- 172.21.15.49:0/3318919681 shutdown_connections 2024-02-26T06:22:13.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.247+0000 7f98da000700 1 -- 172.21.15.49:0/3318919681 wait complete. 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.248+0000 7f98da000700 1 Processor -- start 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.248+0000 7f98da000700 1 -- start start 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.248+0000 7f98da000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d4108400 0x7f98d41a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.248+0000 7f98da000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d4108db0 0x7f98d41a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.248+0000 7f98da000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98d410d490 0x7f98d419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98da000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d419ec40 con 0x7f98d4108db0 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98da000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d419ed80 con 0x7f98d410d490 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98da000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d419ef40 con 0x7f98d4108400 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d8c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d4108400 0x7f98d41a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d8c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d4108400 0x7f98d41a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33504/0 (socket says 172.21.15.49:33504) 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d8c00700 1 -- 172.21.15.49:0/3365884416 learned_addr learned my addr 172.21.15.49:0/3365884416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d3e00700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d4108db0 0x7f98d41a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d9600700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98d410d490 0x7f98d419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d8c00700 1 -- 172.21.15.49:0/3365884416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98d410d490 msgr2=0x7f98d419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d8c00700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98d410d490 0x7f98d419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d8c00700 1 -- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d4108db0 msgr2=0x7f98d41a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d8c00700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d4108db0 0x7f98d41a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.249+0000 7f98d8c00700 1 -- 172.21.15.49:0/3365884416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009c40 con 0x7f98d4108400 2024-02-26T06:22:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.250+0000 7f98d8c00700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d4108400 0x7f98d41a4420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e9a0 tx=0x7f98c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.250+0000 7f98d1600700 1 -- 172.21.15.49:0/3365884416 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8009790 con 0x7f98d4108400 2024-02-26T06:22:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.250+0000 7f98d1600700 1 -- 172.21.15.49:0/3365884416 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98c8010430 con 0x7f98d4108400 2024-02-26T06:22:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.250+0000 7f98d1600700 1 -- 172.21.15.49:0/3365884416 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c800cea0 con 0x7f98d4108400 2024-02-26T06:22:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.250+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d419f220 con 0x7f98d4108400 2024-02-26T06:22:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.250+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d41ab4c0 con 0x7f98d4108400 2024-02-26T06:22:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.251+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d41107b0 con 0x7f98d4108400 2024-02-26T06:22:13.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.253+0000 7f98d1600700 1 -- 172.21.15.49:0/3365884416 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f98c8010910 con 0x7f98d4108400 2024-02-26T06:22:13.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.254+0000 7f98d1600700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f98bc0a21d0 0x7f98bc0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:13.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.255+0000 7f98d1600700 1 -- 172.21.15.49:0/3365884416 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f98c8014070 con 0x7f98d4108400 2024-02-26T06:22:13.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.255+0000 7f98d3e00700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f98bc0a21d0 0x7f98bc0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:13.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.255+0000 7f98d3e00700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f98bc0a21d0 0x7f98bc0a4690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f98d41074e0 tx=0x7f98c0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:13.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.259+0000 7f98d1600700 1 -- 172.21.15.49:0/3365884416 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f98c8052d20 con 0x7f98d4108400 2024-02-26T06:22:13.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:13 smithi080 ceph-mon[36206]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:22:13.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:13 smithi115 ceph-mon[36426]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:22:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:13 smithi049 ceph-mon[31696]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T06:22:13.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.533+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f98d419fb10 con 0x7f98d4108400 2024-02-26T06:22:13.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.534+0000 7f98d1600700 1 -- 172.21.15.49:0/3365884416 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f98c809a930 con 0x7f98d4108400 2024-02-26T06:22:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:13.535 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-26T06:22:13.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f98bc0a21d0 msgr2=0x7f98bc0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:13.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f98bc0a21d0 0x7f98bc0a4690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f98d41074e0 tx=0x7f98c0009310 comp rx=0 tx=0).stop 2024-02-26T06:22:13.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d4108400 msgr2=0x7f98d41a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:13.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d4108400 0x7f98d41a4420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e9a0 tx=0x7f98c800ecb0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 shutdown_connections 2024-02-26T06:22:13.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f98bc0a21d0 0x7f98bc0a4690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98d410d490 0x7f98d419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98d4108400 0x7f98d41a4420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 --2- 172.21.15.49:0/3365884416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d4108db0 0x7f98d41a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:13.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.537+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 >> 172.21.15.49:0/3365884416 conn(0x7f98d4077910 msgr2=0x7f98d4078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:13.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.538+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 shutdown_connections 2024-02-26T06:22:13.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:13.538+0000 7f98da000700 1 -- 172.21.15.49:0/3365884416 wait complete. 2024-02-26T06:22:14.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:14 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3365884416' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T06:22:14.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:14 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3365884416' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T06:22:14.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:14 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3365884416' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T06:22:14.387 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-26T06:22:14.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:22:14.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.627+0000 7fec4b200640 1 -- 172.21.15.49:0/2348502076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441043c0 msgr2=0x7fec44104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.627+0000 7fec4b200640 1 --2- 172.21.15.49:0/2348502076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441043c0 0x7fec44104840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec34009860 tx=0x7fec3402bb70 comp rx=0 tx=0).stop 2024-02-26T06:22:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.628+0000 7fec4b200640 1 -- 172.21.15.49:0/2348502076 shutdown_connections 2024-02-26T06:22:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.628+0000 7fec4b200640 1 --2- 172.21.15.49:0/2348502076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec44104d80 0x7fec4410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.628+0000 7fec4b200640 1 --2- 172.21.15.49:0/2348502076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441043c0 0x7fec44104840 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.628+0000 7fec4b200640 1 --2- 172.21.15.49:0/2348502076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec441031d0 0x7fec441035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.628+0000 7fec4b200640 1 -- 172.21.15.49:0/2348502076 >> 172.21.15.49:0/2348502076 conn(0x7fec440fe760 msgr2=0x7fec44100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.628+0000 7fec4b200640 1 -- 172.21.15.49:0/2348502076 shutdown_connections 2024-02-26T06:22:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.628+0000 7fec4b200640 1 -- 172.21.15.49:0/2348502076 wait complete. 2024-02-26T06:22:14.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec4b200640 1 Processor -- start 2024-02-26T06:22:14.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec4b200640 1 -- start start 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec4b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441031d0 0x7fec44199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec4b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec441043c0 0x7fec44199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec4b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec44104d80 0x7fec441a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec4b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec441a0d30 con 0x7fec441043c0 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec4b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec441a0ea0 con 0x7fec44104d80 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec4b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec441a1060 con 0x7fec441031d0 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec49e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441031d0 0x7fec44199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec49400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec441043c0 0x7fec44199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:14.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.629+0000 7fec49e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441031d0 0x7fec44199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33522/0 (socket says 172.21.15.49:33522) 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec49e00640 1 -- 172.21.15.49:0/548963799 learned_addr learned my addr 172.21.15.49:0/548963799 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec4a800640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec44104d80 0x7fec441a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec49e00640 1 -- 172.21.15.49:0/548963799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec44104d80 msgr2=0x7fec441a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec49e00640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec44104d80 0x7fec441a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec49e00640 1 -- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec441043c0 msgr2=0x7fec44199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec49e00640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec441043c0 0x7fec44199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec49e00640 1 -- 172.21.15.49:0/548963799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40009900 con 0x7fec441031d0 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec49e00640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441031d0 0x7fec44199240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec3800e9c0 tx=0x7fec3800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec3ea00640 1 -- 172.21.15.49:0/548963799 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38009700 con 0x7fec441031d0 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.630+0000 7fec3ea00640 1 -- 172.21.15.49:0/548963799 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec3800cd80 con 0x7fec441031d0 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.631+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec34009490 con 0x7fec441031d0 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.631+0000 7fec3ea00640 1 -- 172.21.15.49:0/548963799 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38010430 con 0x7fec441031d0 2024-02-26T06:22:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.631+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec441a16e0 con 0x7fec441031d0 2024-02-26T06:22:14.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.632+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4410a5c0 con 0x7fec441031d0 2024-02-26T06:22:14.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.634+0000 7fec3ea00640 1 -- 172.21.15.49:0/548963799 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fec38003680 con 0x7fec441031d0 2024-02-26T06:22:14.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.635+0000 7fec3ea00640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fec2c0a2140 0x7fec2c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:14.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.635+0000 7fec3ea00640 1 -- 172.21.15.49:0/548963799 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fec38014070 con 0x7fec441031d0 2024-02-26T06:22:14.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.636+0000 7fec49400640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fec2c0a2140 0x7fec2c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:14.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.636+0000 7fec49400640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fec2c0a2140 0x7fec2c0a4600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec34009830 tx=0x7fec34034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:14.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.637+0000 7fec3ea00640 1 -- 172.21.15.49:0/548963799 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec38096fc0 con 0x7fec441031d0 2024-02-26T06:22:14.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.842+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec4410edd0 con 0x7fec441031d0 2024-02-26T06:22:14.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.845+0000 7fec3ea00640 1 -- 172.21.15.49:0/548963799 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7fec4410edd0 con 0x7fec441031d0 2024-02-26T06:22:14.846 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:14.846 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:22:14.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.847+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fec2c0a2140 msgr2=0x7fec2c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:14.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.847+0000 7fec4b200640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fec2c0a2140 0x7fec2c0a4600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec34009830 tx=0x7fec34034be0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.847+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441031d0 msgr2=0x7fec44199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:14.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.847+0000 7fec4b200640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441031d0 0x7fec44199240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec3800e9c0 tx=0x7fec3800ee80 comp rx=0 tx=0).stop 2024-02-26T06:22:14.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.848+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 shutdown_connections 2024-02-26T06:22:14.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.848+0000 7fec4b200640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fec2c0a2140 0x7fec2c0a4600 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.848+0000 7fec4b200640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec44104d80 0x7fec441a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.848+0000 7fec4b200640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec441043c0 0x7fec44199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.848+0000 7fec4b200640 1 --2- 172.21.15.49:0/548963799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec441031d0 0x7fec44199240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:14.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.848+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 >> 172.21.15.49:0/548963799 conn(0x7fec440fe760 msgr2=0x7fec44107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:14.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.848+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 shutdown_connections 2024-02-26T06:22:14.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:14.848+0000 7fec4b200640 1 -- 172.21.15.49:0/548963799 wait complete. 2024-02-26T06:22:14.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:22:14.870 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:22:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2811682246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3f4101080 msgr2=0x7fc3f41014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2811682246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3f4101080 0x7fc3f41014a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009900 tx=0x7fc3e402bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2811682246 shutdown_connections 2024-02-26T06:22:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2811682246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4102c30 0x7fc3f41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2811682246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3f4102270 0x7fc3f41026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2811682246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3f4101080 0x7fc3f41014a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2811682246 >> 172.21.15.49:0/2811682246 conn(0x7fc3f40fc5f0 msgr2=0x7fc3f40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2811682246 shutdown_connections 2024-02-26T06:22:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.106+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2811682246 wait complete. 2024-02-26T06:22:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.107+0000 7fc3f3e00640 1 Processor -- start 2024-02-26T06:22:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.107+0000 7fc3f3e00640 1 -- start start 2024-02-26T06:22:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4101080 0x7fc3f4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3f4102270 0x7fc3f4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3f4102c30 0x7fc3f41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f3e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41a0e00 con 0x7fc3f4102c30 2024-02-26T06:22:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41a0f70 con 0x7fc3f4101080 2024-02-26T06:22:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f3e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41a1130 con 0x7fc3f4102270 2024-02-26T06:22:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4101080 0x7fc3f4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4101080 0x7fc3f4199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46578/0 (socket says 172.21.15.49:46578) 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f2a00640 1 -- 172.21.15.49:0/2627493196 learned_addr learned my addr 172.21.15.49:0/2627493196 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f3400640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3f4102c30 0x7fc3f41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.108+0000 7fc3f2000640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3f4102270 0x7fc3f4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3f2a00640 1 -- 172.21.15.49:0/2627493196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3f4102270 msgr2=0x7fc3f4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3f2a00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3f4102270 0x7fc3f4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3f2a00640 1 -- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3f4102c30 msgr2=0x7fc3f41a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3f2a00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3f4102c30 0x7fc3f41a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3f2a00640 1 -- 172.21.15.49:0/2627493196 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e4009560 con 0x7fc3f4101080 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3f2a00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4101080 0x7fc3f4199310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4000c00 tx=0x7fc3e4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3eb400640 1 -- 172.21.15.49:0/2627493196 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40043d0 con 0x7fc3f4101080 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3eb400640 1 -- 172.21.15.49:0/2627493196 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3e40052d0 con 0x7fc3f4101080 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3eb400640 1 -- 172.21.15.49:0/2627493196 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4003d30 con 0x7fc3f4101080 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f41a13b0 con 0x7fc3f4101080 2024-02-26T06:22:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.109+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f41a1870 con 0x7fc3f4101080 2024-02-26T06:22:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.110+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f410cae0 con 0x7fc3f4101080 2024-02-26T06:22:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.112+0000 7fc3eb400640 1 -- 172.21.15.49:0/2627493196 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc3e4005440 con 0x7fc3f4101080 2024-02-26T06:22:15.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.114+0000 7fc3eb400640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3d80a2140 0x7fc3d80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:15.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.114+0000 7fc3eb400640 1 -- 172.21.15.49:0/2627493196 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fc3e40f0560 con 0x7fc3f4101080 2024-02-26T06:22:15.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.114+0000 7fc3f2000640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3d80a2140 0x7fc3d80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:15.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.115+0000 7fc3f2000640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3d80a2140 0x7fc3d80a4600 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc3f419a820 tx=0x7fc3e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:15.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.118+0000 7fc3eb400640 1 -- 172.21.15.49:0/2627493196 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3e40b95e0 con 0x7fc3f4101080 2024-02-26T06:22:15.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:15 smithi049 ceph-mon[31696]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T06:22:15.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:15 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/548963799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.321+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3f41a1c90 con 0x7fc3f4101080 2024-02-26T06:22:15.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.322+0000 7fc3eb400640 1 -- 172.21.15.49:0/2627493196 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3e4043020 con 0x7fc3f4101080 2024-02-26T06:22:15.322 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:22:15.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.324+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3d80a2140 msgr2=0x7fc3d80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.324+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3d80a2140 0x7fc3d80a4600 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc3f419a820 tx=0x7fc3e0005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4101080 msgr2=0x7fc3f4199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4101080 0x7fc3f4199310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4000c00 tx=0x7fc3e4005e80 comp rx=0 tx=0).stop 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 shutdown_connections 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc3d80a2140 0x7fc3d80a4600 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3f4102c30 0x7fc3f41a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3f4102270 0x7fc3f4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 --2- 172.21.15.49:0/2627493196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4101080 0x7fc3f4199310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 >> 172.21.15.49:0/2627493196 conn(0x7fc3f40fc5f0 msgr2=0x7fc3f40733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 shutdown_connections 2024-02-26T06:22:15.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.325+0000 7fc3f3e00640 1 -- 172.21.15.49:0/2627493196 wait complete. 2024-02-26T06:22:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:15 smithi080 ceph-mon[36206]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T06:22:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:15 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/548963799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:15 smithi115 ceph-mon[36426]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T06:22:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:15 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/548963799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:15.347 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:22:15.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 -- 172.21.15.49:0/229759992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c102270 msgr2=0x7fee5c1026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 --2- 172.21.15.49:0/229759992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c102270 0x7fee5c1026f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee48009900 tx=0x7fee4802bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:15.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 -- 172.21.15.49:0/229759992 shutdown_connections 2024-02-26T06:22:15.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 --2- 172.21.15.49:0/229759992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee5c102c30 0x7fee5c1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 --2- 172.21.15.49:0/229759992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c102270 0x7fee5c1026f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 --2- 172.21.15.49:0/229759992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee5c101080 0x7fee5c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 -- 172.21.15.49:0/229759992 >> 172.21.15.49:0/229759992 conn(0x7fee5c0fc5f0 msgr2=0x7fee5c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:15.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 -- 172.21.15.49:0/229759992 shutdown_connections 2024-02-26T06:22:15.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.582+0000 7fee5b200640 1 -- 172.21.15.49:0/229759992 wait complete. 2024-02-26T06:22:15.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.583+0000 7fee5b200640 1 Processor -- start 2024-02-26T06:22:15.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee5b200640 1 -- start start 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee5b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c101080 0x7fee5c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee5b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee5c102270 0x7fee5c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee5b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee5c102c30 0x7fee5c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee5b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c1a0dc0 con 0x7fee5c101080 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee5b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c1a0f30 con 0x7fee5c102270 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee5b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c1a10f0 con 0x7fee5c102c30 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee59e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c101080 0x7fee5c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee59e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c101080 0x7fee5c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58834/0 (socket says 172.21.15.49:58834) 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee59e00640 1 -- 172.21.15.49:0/3207161610 learned_addr learned my addr 172.21.15.49:0/3207161610 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee59400640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee5c102270 0x7fee5c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.584+0000 7fee5a800640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee5c102c30 0x7fee5c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee59e00640 1 -- 172.21.15.49:0/3207161610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee5c102c30 msgr2=0x7fee5c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee59e00640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee5c102c30 0x7fee5c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee59e00640 1 -- 172.21.15.49:0/3207161610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee5c102270 msgr2=0x7fee5c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee59e00640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee5c102270 0x7fee5c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee59e00640 1 -- 172.21.15.49:0/3207161610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009560 con 0x7fee5c101080 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee59e00640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c101080 0x7fee5c1992d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e9c0 tx=0x7fee4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee52a00640 1 -- 172.21.15.49:0/3207161610 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c009700 con 0x7fee5c101080 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee52a00640 1 -- 172.21.15.49:0/3207161610 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee4c00cd80 con 0x7fee5c101080 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee52a00640 1 -- 172.21.15.49:0/3207161610 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c010430 con 0x7fee5c101080 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.585+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c1a13d0 con 0x7fee5c101080 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.586+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c1a1860 con 0x7fee5c101080 2024-02-26T06:22:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.587+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c10cae0 con 0x7fee5c101080 2024-02-26T06:22:15.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.589+0000 7fee52a00640 1 -- 172.21.15.49:0/3207161610 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fee4c003680 con 0x7fee5c101080 2024-02-26T06:22:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.592+0000 7fee52a00640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee400a2140 0x7fee400a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.592+0000 7fee52a00640 1 -- 172.21.15.49:0/3207161610 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fee4c014070 con 0x7fee5c101080 2024-02-26T06:22:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.592+0000 7fee59400640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee400a2140 0x7fee400a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:15.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.593+0000 7fee59400640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee400a2140 0x7fee400a4600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee48000c00 tx=0x7fee4802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:15.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.598+0000 7fee52a00640 1 -- 172.21.15.49:0/3207161610 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee4c0d1030 con 0x7fee5c101080 2024-02-26T06:22:15.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.802+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee5c1a1ae0 con 0x7fee5c101080 2024-02-26T06:22:15.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.803+0000 7fee52a00640 1 -- 172.21.15.49:0/3207161610 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee4c096f30 con 0x7fee5c101080 2024-02-26T06:22:15.804 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:22:15.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee400a2140 msgr2=0x7fee400a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee400a2140 0x7fee400a4600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee48000c00 tx=0x7fee4802c7e0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c101080 msgr2=0x7fee5c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:15.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c101080 0x7fee5c1992d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e9c0 tx=0x7fee4c00ee80 comp rx=0 tx=0).stop 2024-02-26T06:22:15.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 shutdown_connections 2024-02-26T06:22:15.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fee400a2140 0x7fee400a4600 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee5c102270 0x7fee5c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee5c102c30 0x7fee5c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 --2- 172.21.15.49:0/3207161610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee5c101080 0x7fee5c1992d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:15.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 >> 172.21.15.49:0/3207161610 conn(0x7fee5c0fc5f0 msgr2=0x7fee5c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:15.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.806+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 shutdown_connections 2024-02-26T06:22:15.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:15.807+0000 7fee5b200640 1 -- 172.21.15.49:0/3207161610 wait complete. 2024-02-26T06:22:15.826 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:22:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 -- 172.21.15.49:0/3200587206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104e50 msgr2=0x7f586810b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 --2- 172.21.15.49:0/3200587206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104e50 0x7f586810b6e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5860009ca0 tx=0x7f586002c0f0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 -- 172.21.15.49:0/3200587206 shutdown_connections 2024-02-26T06:22:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 --2- 172.21.15.49:0/3200587206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104e50 0x7f586810b6e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 --2- 172.21.15.49:0/3200587206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5868104490 0x7f5868104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 --2- 172.21.15.49:0/3200587206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58681032a0 0x7f58681036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 -- 172.21.15.49:0/3200587206 >> 172.21.15.49:0/3200587206 conn(0x7f58680fe830 msgr2=0x7f5868100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 -- 172.21.15.49:0/3200587206 shutdown_connections 2024-02-26T06:22:16.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.056+0000 7f5867200640 1 -- 172.21.15.49:0/3200587206 wait complete. 2024-02-26T06:22:16.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.057+0000 7f5867200640 1 Processor -- start 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.057+0000 7f5867200640 1 -- start start 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.057+0000 7f5867200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58681032a0 0x7f58681992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f5867200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104490 0x7f5868199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f5867200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5868104e50 0x7f58681a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f5867200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58681a0dc0 con 0x7f5868104490 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f5867200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58681a0f30 con 0x7f5868104e50 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f5867200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58681a10f0 con 0x7f58681032a0 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f585fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104490 0x7f5868199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f585fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104490 0x7f5868199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58852/0 (socket says 172.21.15.49:58852) 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f585fe00640 1 -- 172.21.15.49:0/3680129702 learned_addr learned my addr 172.21.15.49:0/3680129702 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f5866800640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5868104e50 0x7f58681a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:16.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f5865e00640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58681032a0 0x7f58681992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f585fe00640 1 -- 172.21.15.49:0/3680129702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58681032a0 msgr2=0x7f58681992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f585fe00640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58681032a0 0x7f58681992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f585fe00640 1 -- 172.21.15.49:0/3680129702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5868104e50 msgr2=0x7f58681a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f585fe00640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5868104e50 0x7f58681a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f585fe00640 1 -- 172.21.15.49:0/3680129702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858009490 con 0x7f5868104490 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.058+0000 7f5866800640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5868104e50 0x7f58681a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.059+0000 7f585fe00640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104490 0x7f5868199810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f585000e9f0 tx=0x7f585000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.059+0000 7f585ea00640 1 -- 172.21.15.49:0/3680129702 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850009720 con 0x7f5868104490 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.059+0000 7f585ea00640 1 -- 172.21.15.49:0/3680129702 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f585000cd80 con 0x7f5868104490 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.059+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5860009900 con 0x7f5868104490 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.059+0000 7f585ea00640 1 -- 172.21.15.49:0/3680129702 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850010690 con 0x7f5868104490 2024-02-26T06:22:16.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.059+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58681a1750 con 0x7f5868104490 2024-02-26T06:22:16.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.060+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586810a5c0 con 0x7f5868104490 2024-02-26T06:22:16.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.063+0000 7f585ea00640 1 -- 172.21.15.49:0/3680129702 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f58500046c0 con 0x7f5868104490 2024-02-26T06:22:16.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.064+0000 7f585ea00640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f584c0a2140 0x7f584c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:16.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.064+0000 7f585ea00640 1 -- 172.21.15.49:0/3680129702 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f5850014070 con 0x7f5868104490 2024-02-26T06:22:16.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.064+0000 7f5865e00640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f584c0a2140 0x7f584c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:16.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.065+0000 7f5865e00640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f584c0a2140 0x7f584c0a4600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58681042f0 tx=0x7f5858005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:16.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.067+0000 7f585ea00640 1 -- 172.21.15.49:0/3680129702 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5850097110 con 0x7f5868104490 2024-02-26T06:22:16.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:16 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2627493196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:16.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:16 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3207161610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:16.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.272+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f586810ef00 con 0x7f5868104490 2024-02-26T06:22:16.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.273+0000 7f585ea00640 1 -- 172.21.15.49:0/3680129702 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f585009a7d0 con 0x7f5868104490 2024-02-26T06:22:16.273 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:22:16.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.275+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f584c0a2140 msgr2=0x7f584c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.275+0000 7f5867200640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f584c0a2140 0x7f584c0a4600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58681042f0 tx=0x7f5858005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.275+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104490 msgr2=0x7f5868199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.275+0000 7f5867200640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104490 0x7f5868199810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f585000e9f0 tx=0x7f585000edf0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.276+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 shutdown_connections 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.276+0000 7f5867200640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f584c0a2140 0x7f584c0a4600 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.276+0000 7f5867200640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5868104e50 0x7f58681a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.276+0000 7f5867200640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5868104490 0x7f5868199810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.276+0000 7f5867200640 1 --2- 172.21.15.49:0/3680129702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58681032a0 0x7f58681992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.276+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 >> 172.21.15.49:0/3680129702 conn(0x7f58680fe830 msgr2=0x7f5868107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.276+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 shutdown_connections 2024-02-26T06:22:16.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.276+0000 7f5867200640 1 -- 172.21.15.49:0/3680129702 wait complete. 2024-02-26T06:22:16.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:22:16.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:16 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2627493196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:16.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:16 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3207161610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:16.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:16 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2627493196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:16.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:16 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3207161610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:16.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.533+0000 7fa6c3200640 1 -- 172.21.15.49:0/54767393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c41031d0 msgr2=0x7fa6c41035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.533+0000 7fa6c3200640 1 --2- 172.21.15.49:0/54767393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c41031d0 0x7fa6c41035f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009900 tx=0x7fa6b402bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:16.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.534+0000 7fa6c3200640 1 -- 172.21.15.49:0/54767393 shutdown_connections 2024-02-26T06:22:16.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.534+0000 7fa6c3200640 1 --2- 172.21.15.49:0/54767393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6c4104d80 0x7fa6c410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.534+0000 7fa6c3200640 1 --2- 172.21.15.49:0/54767393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41043c0 0x7fa6c4104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.534+0000 7fa6c3200640 1 --2- 172.21.15.49:0/54767393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c41031d0 0x7fa6c41035f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.534+0000 7fa6c3200640 1 -- 172.21.15.49:0/54767393 >> 172.21.15.49:0/54767393 conn(0x7fa6c40fe760 msgr2=0x7fa6c4100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:16.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.534+0000 7fa6c3200640 1 -- 172.21.15.49:0/54767393 shutdown_connections 2024-02-26T06:22:16.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.534+0000 7fa6c3200640 1 -- 172.21.15.49:0/54767393 wait complete. 2024-02-26T06:22:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.535+0000 7fa6c3200640 1 Processor -- start 2024-02-26T06:22:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.535+0000 7fa6c3200640 1 -- start start 2024-02-26T06:22:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c3200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41031d0 0x7fa6c4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c3200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c41043c0 0x7fa6c4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6c4104d80 0x7fa6c41a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c3200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41a0d20 con 0x7fa6c41031d0 2024-02-26T06:22:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41a0e90 con 0x7fa6c4104d80 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c3200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41a1050 con 0x7fa6c41043c0 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41031d0 0x7fa6c4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41031d0 0x7fa6c4199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58868/0 (socket says 172.21.15.49:58868) 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1e00640 1 -- 172.21.15.49:0/262715344 learned_addr learned my addr 172.21.15.49:0/262715344 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c2800640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6c4104d80 0x7fa6c41a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1400640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c41043c0 0x7fa6c4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1e00640 1 -- 172.21.15.49:0/262715344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c41043c0 msgr2=0x7fa6c4199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1e00640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c41043c0 0x7fa6c4199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1e00640 1 -- 172.21.15.49:0/262715344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6c4104d80 msgr2=0x7fa6c41a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1e00640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6c4104d80 0x7fa6c41a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c1e00640 1 -- 172.21.15.49:0/262715344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4009560 con 0x7fa6c41031d0 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.536+0000 7fa6c2800640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6c4104d80 0x7fa6c41a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.537+0000 7fa6c1e00640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41031d0 0x7fa6c4199230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009850 tx=0x7fa6b4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.537+0000 7fa6baa00640 1 -- 172.21.15.49:0/262715344 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40043d0 con 0x7fa6c41031d0 2024-02-26T06:22:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.537+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c41a12d0 con 0x7fa6c41031d0 2024-02-26T06:22:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.537+0000 7fa6baa00640 1 -- 172.21.15.49:0/262715344 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b40052d0 con 0x7fa6c41031d0 2024-02-26T06:22:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.537+0000 7fa6baa00640 1 -- 172.21.15.49:0/262715344 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4003d80 con 0x7fa6c41031d0 2024-02-26T06:22:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.537+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c41a1830 con 0x7fa6c41031d0 2024-02-26T06:22:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.539+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c410a490 con 0x7fa6c41031d0 2024-02-26T06:22:16.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.540+0000 7fa6baa00640 1 -- 172.21.15.49:0/262715344 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa6b4005440 con 0x7fa6c41031d0 2024-02-26T06:22:16.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.543+0000 7fa6baa00640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa6a80a20c0 0x7fa6a80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:16.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.543+0000 7fa6baa00640 1 -- 172.21.15.49:0/262715344 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fa6b40f03a0 con 0x7fa6c41031d0 2024-02-26T06:22:16.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.543+0000 7fa6c1400640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa6a80a20c0 0x7fa6a80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:16.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.544+0000 7fa6c1400640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa6a80a20c0 0x7fa6a80a4580 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6c419a740 tx=0x7fa6b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:16.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.547+0000 7fa6baa00640 1 -- 172.21.15.49:0/262715344 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa6b40b9480 con 0x7fa6c41031d0 2024-02-26T06:22:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.792+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6c402cda0 con 0x7fa6c41031d0 2024-02-26T06:22:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.793+0000 7fa6baa00640 1 -- 172.21.15.49:0/262715344 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11331 (secure 0 0 0) 0x7fa6b40bcb40 con 0x7fa6c41031d0 2024-02-26T06:22:16.794 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:16.794 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:20:38.098815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:22:16.794 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa6a80a20c0 msgr2=0x7fa6a80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa6a80a20c0 0x7fa6a80a4580 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6c419a740 tx=0x7fa6b0005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41031d0 msgr2=0x7fa6c4199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41031d0 0x7fa6c4199230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009850 tx=0x7fa6b4005e40 comp rx=0 tx=0).stop 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 shutdown_connections 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa6a80a20c0 0x7fa6a80a4580 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6c4104d80 0x7fa6c41a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c41043c0 0x7fa6c4199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 --2- 172.21.15.49:0/262715344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41031d0 0x7fa6c4199230 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 >> 172.21.15.49:0/262715344 conn(0x7fa6c40fe760 msgr2=0x7fa6c4107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:16.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 shutdown_connections 2024-02-26T06:22:16.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:16.796+0000 7fa6c3200640 1 -- 172.21.15.49:0/262715344 wait complete. 2024-02-26T06:22:16.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:22:17.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.052+0000 7f9541a00640 1 -- 172.21.15.49:0/2503367839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c100fe0 msgr2=0x7f953c101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.052+0000 7f9541a00640 1 --2- 172.21.15.49:0/2503367839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c100fe0 0x7f953c101400 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f952c009900 tx=0x7f952c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.053+0000 7f9541a00640 1 -- 172.21.15.49:0/2503367839 shutdown_connections 2024-02-26T06:22:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.053+0000 7f9541a00640 1 --2- 172.21.15.49:0/2503367839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f953c102b90 0x7f953c109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.053+0000 7f9541a00640 1 --2- 172.21.15.49:0/2503367839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f953c1021d0 0x7f953c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.053+0000 7f9541a00640 1 --2- 172.21.15.49:0/2503367839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c100fe0 0x7f953c101400 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.053+0000 7f9541a00640 1 -- 172.21.15.49:0/2503367839 >> 172.21.15.49:0/2503367839 conn(0x7f953c0fc550 msgr2=0x7f953c0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.053+0000 7f9541a00640 1 -- 172.21.15.49:0/2503367839 shutdown_connections 2024-02-26T06:22:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.053+0000 7f9541a00640 1 -- 172.21.15.49:0/2503367839 wait complete. 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.054+0000 7f9541a00640 1 Processor -- start 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.054+0000 7f9541a00640 1 -- start start 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.054+0000 7f9541a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f953c100fe0 0x7f953c074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.054+0000 7f9541a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c1021d0 0x7f953c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.054+0000 7f9541a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f953c102b90 0x7f953c073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.054+0000 7f9541a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c074210 con 0x7f953c1021d0 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.054+0000 7f9541a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c074380 con 0x7f953c102b90 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.054+0000 7f9541a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c1a10e0 con 0x7f953c100fe0 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c1021d0 0x7f953c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c1021d0 0x7f953c073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58886/0 (socket says 172.21.15.49:58886) 2024-02-26T06:22:17.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953b400640 1 -- 172.21.15.49:0/582057870 learned_addr learned my addr 172.21.15.49:0/582057870 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953be00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f953c100fe0 0x7f953c074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f9541000640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f953c102b90 0x7f953c073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953b400640 1 -- 172.21.15.49:0/582057870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f953c100fe0 msgr2=0x7f953c074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953b400640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f953c100fe0 0x7f953c074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953b400640 1 -- 172.21.15.49:0/582057870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f953c102b90 msgr2=0x7f953c073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953b400640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f953c102b90 0x7f953c073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.055+0000 7f953b400640 1 -- 172.21.15.49:0/582057870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534009840 con 0x7f953c1021d0 2024-02-26T06:22:17.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.056+0000 7f953b400640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c1021d0 0x7f953c073570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f952400ea20 tx=0x7f952400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:17.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.056+0000 7f9538c00640 1 -- 172.21.15.49:0/582057870 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524009700 con 0x7f953c1021d0 2024-02-26T06:22:17.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.056+0000 7f9538c00640 1 -- 172.21.15.49:0/582057870 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f952400ccc0 con 0x7f953c1021d0 2024-02-26T06:22:17.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.056+0000 7f9538c00640 1 -- 172.21.15.49:0/582057870 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95240187e0 con 0x7f953c1021d0 2024-02-26T06:22:17.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.056+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c009560 con 0x7f953c1021d0 2024-02-26T06:22:17.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.056+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c1a1760 con 0x7f953c1021d0 2024-02-26T06:22:17.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.057+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c108300 con 0x7f953c1021d0 2024-02-26T06:22:17.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.060+0000 7f9538c00640 1 -- 172.21.15.49:0/582057870 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9524010430 con 0x7f953c1021d0 2024-02-26T06:22:17.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.062+0000 7f9538c00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f951c0a2140 0x7f951c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:17.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.062+0000 7f9538c00640 1 -- 172.21.15.49:0/582057870 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9524014070 con 0x7f953c1021d0 2024-02-26T06:22:17.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.063+0000 7f953be00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f951c0a2140 0x7f951c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:17.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.063+0000 7f953be00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f951c0a2140 0x7f951c0a4600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f952c02c240 tx=0x7f952c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:17.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.067+0000 7f9538c00640 1 -- 172.21.15.49:0/582057870 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9524052d20 con 0x7f953c1021d0 2024-02-26T06:22:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.270+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f953c02cdd0 con 0x7f953c1021d0 2024-02-26T06:22:17.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:17 smithi115 ceph-mon[36426]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:17.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:17 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3680129702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:17.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:17 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/262715344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:17.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.274+0000 7f9538c00640 1 -- 172.21.15.49:0/582057870 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7f95240d1030 con 0x7f953c1021d0 2024-02-26T06:22:17.458 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:17.458 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:22:17.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f951c0a2140 msgr2=0x7f951c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f951c0a2140 0x7f951c0a4600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f952c02c240 tx=0x7f952c02c7e0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c1021d0 msgr2=0x7f953c073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c1021d0 0x7f953c073570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f952400ea20 tx=0x7f952400ee20 comp rx=0 tx=0).stop 2024-02-26T06:22:17.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 shutdown_connections 2024-02-26T06:22:17.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f951c0a2140 0x7f951c0a4600 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f953c102b90 0x7f953c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f953c1021d0 0x7f953c073570 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 --2- 172.21.15.49:0/582057870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f953c100fe0 0x7f953c074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.277+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 >> 172.21.15.49:0/582057870 conn(0x7f953c0fc550 msgr2=0x7f953c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:17.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.278+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 shutdown_connections 2024-02-26T06:22:17.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.278+0000 7f9541a00640 1 -- 172.21.15.49:0/582057870 wait complete. 2024-02-26T06:22:17.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:22:17.463 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:22:17.464 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:17 smithi049 ceph-mon[31696]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:17.464 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:17 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3680129702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:17.464 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:17 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/262715344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:17 smithi080 ceph-mon[36206]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:17 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3680129702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:17 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/262715344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:17.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 -- 172.21.15.49:0/3176707970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30104410 msgr2=0x7f3e30106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3176707970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30104410 0x7f3e30106800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009860 tx=0x7f3e1802bb70 comp rx=0 tx=0).stop 2024-02-26T06:22:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 -- 172.21.15.49:0/3176707970 shutdown_connections 2024-02-26T06:22:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3176707970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e30106d40 0x7f3e30109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3176707970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30104410 0x7f3e30106800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3176707970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e30101ae0 0x7f3e30103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 -- 172.21.15.49:0/3176707970 >> 172.21.15.49:0/3176707970 conn(0x7f3e300fb4f0 msgr2=0x7f3e300fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 -- 172.21.15.49:0/3176707970 shutdown_connections 2024-02-26T06:22:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.709+0000 7f3e35a00640 1 -- 172.21.15.49:0/3176707970 wait complete. 2024-02-26T06:22:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.710+0000 7f3e35a00640 1 Processor -- start 2024-02-26T06:22:17.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e35a00640 1 -- start start 2024-02-26T06:22:17.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e35a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30101ae0 0x7f3e30199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:17.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e35a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e30104410 0x7f3e30199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:17.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e35a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e30106d40 0x7f3e301a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:17.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e35a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e301a0e20 con 0x7f3e30106d40 2024-02-26T06:22:17.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e35a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e301a0f90 con 0x7f3e30104410 2024-02-26T06:22:17.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e35a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e301a1150 con 0x7f3e30101ae0 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e2fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30101ae0 0x7f3e30199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e35000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e30106d40 0x7f3e301a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e27400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e30104410 0x7f3e30199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e2fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30101ae0 0x7f3e30199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33656/0 (socket says 172.21.15.49:33656) 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.711+0000 7f3e2fe00640 1 -- 172.21.15.49:0/3988521627 learned_addr learned my addr 172.21.15.49:0/3988521627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2fe00640 1 -- 172.21.15.49:0/3988521627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e30104410 msgr2=0x7f3e30199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2fe00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e30104410 0x7f3e30199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2fe00640 1 -- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e30106d40 msgr2=0x7f3e301a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2fe00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e30106d40 0x7f3e301a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2fe00640 1 -- 172.21.15.49:0/3988521627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e20009560 con 0x7f3e30101ae0 2024-02-26T06:22:17.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e27400640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e30104410 0x7f3e30199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:17.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2fe00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30101ae0 0x7f3e30199330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000e9c0 tx=0x7f3e2000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:17.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2d600640 1 -- 172.21.15.49:0/3988521627 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2000cde0 con 0x7f3e30101ae0 2024-02-26T06:22:17.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2d600640 1 -- 172.21.15.49:0/3988521627 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e200035f0 con 0x7f3e30101ae0 2024-02-26T06:22:17.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e18009490 con 0x7f3e30101ae0 2024-02-26T06:22:17.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e2d600640 1 -- 172.21.15.49:0/3988521627 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20010670 con 0x7f3e30101ae0 2024-02-26T06:22:17.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.712+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e301a1780 con 0x7f3e30101ae0 2024-02-26T06:22:17.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.714+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e30108890 con 0x7f3e30101ae0 2024-02-26T06:22:17.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.716+0000 7f3e2d600640 1 -- 172.21.15.49:0/3988521627 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3e20003ad0 con 0x7f3e30101ae0 2024-02-26T06:22:17.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.717+0000 7f3e2d600640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e140a20c0 0x7f3e140a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:17.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.717+0000 7f3e2d600640 1 -- 172.21.15.49:0/3988521627 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3e20014070 con 0x7f3e30101ae0 2024-02-26T06:22:17.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.717+0000 7f3e27400640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e140a20c0 0x7f3e140a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:17.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.718+0000 7f3e27400640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e140a20c0 0x7f3e140a4580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009830 tx=0x7f3e18034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:17.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.719+0000 7f3e2d600640 1 -- 172.21.15.49:0/3988521627 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e20097020 con 0x7f3e30101ae0 2024-02-26T06:22:17.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.921+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e3002cda0 con 0x7f3e30101ae0 2024-02-26T06:22:17.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.922+0000 7f3e2d600640 1 -- 172.21.15.49:0/3988521627 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e20003760 con 0x7f3e30101ae0 2024-02-26T06:22:17.923 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:22:17.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.925+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e140a20c0 msgr2=0x7f3e140a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.925+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e140a20c0 0x7f3e140a4580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009830 tx=0x7f3e18034c00 comp rx=0 tx=0).stop 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.925+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30101ae0 msgr2=0x7f3e30199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.925+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30101ae0 0x7f3e30199330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000e9c0 tx=0x7f3e2000ee80 comp rx=0 tx=0).stop 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.926+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 shutdown_connections 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.926+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e140a20c0 0x7f3e140a4580 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.926+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e30104410 0x7f3e30199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.926+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e30106d40 0x7f3e301a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.926+0000 7f3e35a00640 1 --2- 172.21.15.49:0/3988521627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e30101ae0 0x7f3e30199330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.926+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 >> 172.21.15.49:0/3988521627 conn(0x7f3e300fb4f0 msgr2=0x7f3e30101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.926+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 shutdown_connections 2024-02-26T06:22:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:17.926+0000 7f3e35a00640 1 -- 172.21.15.49:0/3988521627 wait complete. 2024-02-26T06:22:17.948 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:22:18.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 -- 172.21.15.49:0/2699498557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 msgr2=0x7fed401036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 --2- 172.21.15.49:0/2699498557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 0x7fed401036a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fed34009900 tx=0x7fed3402bc40 comp rx=0 tx=0).stop 2024-02-26T06:22:18.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 -- 172.21.15.49:0/2699498557 shutdown_connections 2024-02-26T06:22:18.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 --2- 172.21.15.49:0/2699498557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed40104470 0x7fed401048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 --2- 172.21.15.49:0/2699498557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed40104e30 0x7fed4010b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 --2- 172.21.15.49:0/2699498557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 0x7fed401036a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 -- 172.21.15.49:0/2699498557 >> 172.21.15.49:0/2699498557 conn(0x7fed400fe7f0 msgr2=0x7fed40100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:18.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 -- 172.21.15.49:0/2699498557 shutdown_connections 2024-02-26T06:22:18.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.192+0000 7fed46400640 1 -- 172.21.15.49:0/2699498557 wait complete. 2024-02-26T06:22:18.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.193+0000 7fed46400640 1 Processor -- start 2024-02-26T06:22:18.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.193+0000 7fed46400640 1 -- start start 2024-02-26T06:22:18.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.193+0000 7fed46400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 0x7fed40199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:18.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.193+0000 7fed46400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed40104470 0x7fed40199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.193+0000 7fed46400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed40104e30 0x7fed401a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.193+0000 7fed46400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed401a0de0 con 0x7fed40103280 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.193+0000 7fed46400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed401a0f50 con 0x7fed40104470 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.193+0000 7fed46400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed401a1110 con 0x7fed40104e30 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 0x7fed40199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 0x7fed40199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58942/0 (socket says 172.21.15.49:58942) 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45000640 1 -- 172.21.15.49:0/892927199 learned_addr learned my addr 172.21.15.49:0/892927199 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed3fe00640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed40104470 0x7fed40199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45a00640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed40104e30 0x7fed401a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:18.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45000640 1 -- 172.21.15.49:0/892927199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed40104e30 msgr2=0x7fed401a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45000640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed40104e30 0x7fed401a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45000640 1 -- 172.21.15.49:0/892927199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed40104470 msgr2=0x7fed40199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45000640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed40104470 0x7fed40199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.194+0000 7fed45000640 1 -- 172.21.15.49:0/892927199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34009560 con 0x7fed40103280 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.195+0000 7fed45000640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 0x7fed40199340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed34000c00 tx=0x7fed340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.195+0000 7fed3d600640 1 -- 172.21.15.49:0/892927199 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340044c0 con 0x7fed40103280 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.195+0000 7fed3d600640 1 -- 172.21.15.49:0/892927199 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed340052d0 con 0x7fed40103280 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.195+0000 7fed3d600640 1 -- 172.21.15.49:0/892927199 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed34003e80 con 0x7fed40103280 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.195+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed401a1390 con 0x7fed40103280 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.195+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed401a1850 con 0x7fed40103280 2024-02-26T06:22:18.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.196+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4010a620 con 0x7fed40103280 2024-02-26T06:22:18.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.198+0000 7fed3d600640 1 -- 172.21.15.49:0/892927199 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fed34005440 con 0x7fed40103280 2024-02-26T06:22:18.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.200+0000 7fed3d600640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fed280a20c0 0x7fed280a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:18.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.200+0000 7fed3d600640 1 -- 172.21.15.49:0/892927199 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fed340f0570 con 0x7fed40103280 2024-02-26T06:22:18.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.200+0000 7fed3fe00640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fed280a20c0 0x7fed280a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:18.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.201+0000 7fed3fe00640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fed280a20c0 0x7fed280a4580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed4019a800 tx=0x7fed2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:18.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.205+0000 7fed3d600640 1 -- 172.21.15.49:0/892927199 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed340b94d0 con 0x7fed40103280 2024-02-26T06:22:18.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:18 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/582057870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:18.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:18 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3988521627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:18.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.418+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed4002cda0 con 0x7fed40103280 2024-02-26T06:22:18.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:18 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/582057870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:18.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:18 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3988521627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:18.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.419+0000 7fed3d600640 1 -- 172.21.15.49:0/892927199 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed34043020 con 0x7fed40103280 2024-02-26T06:22:18.420 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fed280a20c0 msgr2=0x7fed280a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fed280a20c0 0x7fed280a4580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed4019a800 tx=0x7fed2c005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 msgr2=0x7fed40199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 0x7fed40199340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed34000c00 tx=0x7fed340040c0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 shutdown_connections 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fed280a20c0 0x7fed280a4580 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed40104470 0x7fed40199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed40104e30 0x7fed401a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 --2- 172.21.15.49:0/892927199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed40103280 0x7fed40199340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 >> 172.21.15.49:0/892927199 conn(0x7fed400fe7f0 msgr2=0x7fed401077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 shutdown_connections 2024-02-26T06:22:18.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.422+0000 7fed46400640 1 -- 172.21.15.49:0/892927199 wait complete. 2024-02-26T06:22:18.443 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:22:18.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:18 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/582057870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:18.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:18 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3988521627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:18.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.677+0000 7fe3d2400640 1 -- 172.21.15.49:0/586005463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc104410 msgr2=0x7fe3cc106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.677+0000 7fe3d2400640 1 --2- 172.21.15.49:0/586005463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc104410 0x7fe3cc106800 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009900 tx=0x7fe3b802bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:18.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.678+0000 7fe3d2400640 1 -- 172.21.15.49:0/586005463 shutdown_connections 2024-02-26T06:22:18.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.678+0000 7fe3d2400640 1 --2- 172.21.15.49:0/586005463 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3cc106d40 0x7fe3cc109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.678+0000 7fe3d2400640 1 --2- 172.21.15.49:0/586005463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc104410 0x7fe3cc106800 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.678+0000 7fe3d2400640 1 --2- 172.21.15.49:0/586005463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc101ae0 0x7fe3cc103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.678+0000 7fe3d2400640 1 -- 172.21.15.49:0/586005463 >> 172.21.15.49:0/586005463 conn(0x7fe3cc0fb4f0 msgr2=0x7fe3cc0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:18.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.678+0000 7fe3d2400640 1 -- 172.21.15.49:0/586005463 shutdown_connections 2024-02-26T06:22:18.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.678+0000 7fe3d2400640 1 -- 172.21.15.49:0/586005463 wait complete. 2024-02-26T06:22:18.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d2400640 1 Processor -- start 2024-02-26T06:22:18.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d2400640 1 -- start start 2024-02-26T06:22:18.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d2400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3cc101ae0 0x7fe3cc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:18.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d2400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc104410 0x7fe3cc199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:18.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d2400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc106d40 0x7fe3cc1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:18.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d2400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc1a0e30 con 0x7fe3cc106d40 2024-02-26T06:22:18.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d2400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc1a0fa0 con 0x7fe3cc101ae0 2024-02-26T06:22:18.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d2400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc1a1160 con 0x7fe3cc104410 2024-02-26T06:22:18.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.679+0000 7fe3d1a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc106d40 0x7fe3cc1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:18.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc106d40 0x7fe3cc1a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58962/0 (socket says 172.21.15.49:58962) 2024-02-26T06:22:18.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1a00640 1 -- 172.21.15.49:0/835215171 learned_addr learned my addr 172.21.15.49:0/835215171 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:18.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1000640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3cc101ae0 0x7fe3cc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3cbe00640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc104410 0x7fe3cc199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1a00640 1 -- 172.21.15.49:0/835215171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc104410 msgr2=0x7fe3cc199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1a00640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc104410 0x7fe3cc199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1a00640 1 -- 172.21.15.49:0/835215171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3cc101ae0 msgr2=0x7fe3cc199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1a00640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3cc101ae0 0x7fe3cc199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1a00640 1 -- 172.21.15.49:0/835215171 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c4009840 con 0x7fe3cc106d40 2024-02-26T06:22:18.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3d1a00640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc106d40 0x7fe3cc1a08f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400efd0 tx=0x7fe3c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.680+0000 7fe3c9600640 1 -- 172.21.15.49:0/835215171 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4009b40 con 0x7fe3cc106d40 2024-02-26T06:22:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.681+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b8009560 con 0x7fe3cc106d40 2024-02-26T06:22:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.681+0000 7fe3c9600640 1 -- 172.21.15.49:0/835215171 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3c4003b70 con 0x7fe3cc106d40 2024-02-26T06:22:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.681+0000 7fe3c9600640 1 -- 172.21.15.49:0/835215171 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40108f0 con 0x7fe3cc106d40 2024-02-26T06:22:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.681+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3cc1a1740 con 0x7fe3cc106d40 2024-02-26T06:22:18.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.682+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc108890 con 0x7fe3cc106d40 2024-02-26T06:22:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.684+0000 7fe3c9600640 1 -- 172.21.15.49:0/835215171 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe3c4003ce0 con 0x7fe3cc106d40 2024-02-26T06:22:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.685+0000 7fe3c9600640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3b00a20c0 0x7fe3b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.686+0000 7fe3c9600640 1 -- 172.21.15.49:0/835215171 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe3c4014070 con 0x7fe3cc106d40 2024-02-26T06:22:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.686+0000 7fe3d1000640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3b00a20c0 0x7fe3b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.686+0000 7fe3d1000640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3b00a20c0 0x7fe3b00a4580 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009bd0 tx=0x7fe3bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:18.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.690+0000 7fe3c9600640 1 -- 172.21.15.49:0/835215171 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3c40971f0 con 0x7fe3cc106d40 2024-02-26T06:22:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.890+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3cc02cda0 con 0x7fe3cc106d40 2024-02-26T06:22:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.891+0000 7fe3c9600640 1 -- 172.21.15.49:0/835215171 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3c409a8b0 con 0x7fe3cc106d40 2024-02-26T06:22:18.892 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:22:18.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3b00a20c0 msgr2=0x7fe3b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3b00a20c0 0x7fe3b00a4580 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009bd0 tx=0x7fe3bc005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:18.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc106d40 msgr2=0x7fe3cc1a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:18.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc106d40 0x7fe3cc1a08f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400efd0 tx=0x7fe3c400c6a0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 shutdown_connections 2024-02-26T06:22:18.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe3b00a20c0 0x7fe3b00a4580 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc106d40 0x7fe3cc1a08f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc104410 0x7fe3cc199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 --2- 172.21.15.49:0/835215171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3cc101ae0 0x7fe3cc199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.894+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 >> 172.21.15.49:0/835215171 conn(0x7fe3cc0fb4f0 msgr2=0x7fe3cc1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.895+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 shutdown_connections 2024-02-26T06:22:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:18.895+0000 7fe3d2400640 1 -- 172.21.15.49:0/835215171 wait complete. 2024-02-26T06:22:18.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:22:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 -- 172.21.15.49:0/4199328000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 msgr2=0x7fb16c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 --2- 172.21.15.49:0/4199328000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 0x7fb16c0ffec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002bcc0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 -- 172.21.15.49:0/4199328000 shutdown_connections 2024-02-26T06:22:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 --2- 172.21.15.49:0/4199328000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb16c100da0 0x7fb16c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 --2- 172.21.15.49:0/4199328000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c100400 0x7fb16c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 --2- 172.21.15.49:0/4199328000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 0x7fb16c0ffec0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 -- 172.21.15.49:0/4199328000 >> 172.21.15.49:0/4199328000 conn(0x7fb16c0fb620 msgr2=0x7fb16c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 -- 172.21.15.49:0/4199328000 shutdown_connections 2024-02-26T06:22:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.152+0000 7fb174800640 1 -- 172.21.15.49:0/4199328000 wait complete. 2024-02-26T06:22:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb174800640 1 Processor -- start 2024-02-26T06:22:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb174800640 1 -- start start 2024-02-26T06:22:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb174800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 0x7fb16c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb174800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c100400 0x7fb16c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb174800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb16c100da0 0x7fb16c1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb174800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c1a0f70 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb174800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c1a10e0 con 0x7fb16c100da0 2024-02-26T06:22:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb174800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c1a12a0 con 0x7fb16c100400 2024-02-26T06:22:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.153+0000 7fb171e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 0x7fb16c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 0x7fb16c199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58988/0 (socket says 172.21.15.49:58988) 2024-02-26T06:22:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171e00640 1 -- 172.21.15.49:0/902379513 learned_addr learned my addr 172.21.15.49:0/902379513 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb172800640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb16c100da0 0x7fb16c1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171400640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c100400 0x7fb16c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171e00640 1 -- 172.21.15.49:0/902379513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c100400 msgr2=0x7fb16c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171e00640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c100400 0x7fb16c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171e00640 1 -- 172.21.15.49:0/902379513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb16c100da0 msgr2=0x7fb16c1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171e00640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb16c100da0 0x7fb16c1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171e00640 1 -- 172.21.15.49:0/902379513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb160009560 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.154+0000 7fb171e00640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 0x7fb16c199480 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb160005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.155+0000 7fb166a00640 1 -- 172.21.15.49:0/902379513 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1600043d0 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.155+0000 7fb166a00640 1 -- 172.21.15.49:0/902379513 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1600052d0 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.155+0000 7fb166a00640 1 -- 172.21.15.49:0/902379513 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160003cc0 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.155+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c1a1520 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.155+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c1a1a30 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.156+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c10a9f0 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.158+0000 7fb166a00640 1 -- 172.21.15.49:0/902379513 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb160005440 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.159+0000 7fb166a00640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb1540a20c0 0x7fb1540a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.160+0000 7fb166a00640 1 -- 172.21.15.49:0/902379513 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb1600f0670 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.160+0000 7fb171400640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb1540a20c0 0x7fb1540a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.160+0000 7fb171400640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb1540a20c0 0x7fb1540a4580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb16c19a990 tx=0x7fb15c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:19.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.164+0000 7fb166a00640 1 -- 172.21.15.49:0/902379513 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb1600b96f0 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:19 smithi049 ceph-mon[31696]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:19.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:19 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/892927199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:19.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:19 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/835215171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:19.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.406+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb16c02cdd0 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.408+0000 7fb166a00640 1 -- 172.21.15.49:0/902379513 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11331 (secure 0 0 0) 0x7fb1600bcdb0 con 0x7fb16c0ffaa0 2024-02-26T06:22:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:19.409 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:20:38.098815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:22:19.410 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-02-26T06:22:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb1540a20c0 msgr2=0x7fb1540a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb1540a20c0 0x7fb1540a4580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb16c19a990 tx=0x7fb15c005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 msgr2=0x7fb16c199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 0x7fb16c199480 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb160005e80 comp rx=0 tx=0).stop 2024-02-26T06:22:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 shutdown_connections 2024-02-26T06:22:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb1540a20c0 0x7fb1540a4580 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb16c100da0 0x7fb16c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c100400 0x7fb16c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 --2- 172.21.15.49:0/902379513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb16c0ffaa0 0x7fb16c199480 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 >> 172.21.15.49:0/902379513 conn(0x7fb16c0fb620 msgr2=0x7fb16c0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:19.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 shutdown_connections 2024-02-26T06:22:19.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.411+0000 7fb174800640 1 -- 172.21.15.49:0/902379513 wait complete. 2024-02-26T06:22:19.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:22:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:19 smithi080 ceph-mon[36206]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:19 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/892927199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:19 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/835215171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:19 smithi115 ceph-mon[36426]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:19 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/892927199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:19 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/835215171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.667+0000 7f2c6de00640 1 -- 172.21.15.49:0/1602503739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c681043c0 msgr2=0x7f2c68104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.667+0000 7f2c6de00640 1 --2- 172.21.15.49:0/1602503739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c681043c0 0x7f2c68104840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009900 tx=0x7f2c5402bc50 comp rx=0 tx=0).stop 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.668+0000 7f2c6de00640 1 -- 172.21.15.49:0/1602503739 shutdown_connections 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.668+0000 7f2c6de00640 1 --2- 172.21.15.49:0/1602503739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c68104d80 0x7f2c6810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.668+0000 7f2c6de00640 1 --2- 172.21.15.49:0/1602503739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c681043c0 0x7f2c68104840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.668+0000 7f2c6de00640 1 --2- 172.21.15.49:0/1602503739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c681031d0 0x7f2c681035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.668+0000 7f2c6de00640 1 -- 172.21.15.49:0/1602503739 >> 172.21.15.49:0/1602503739 conn(0x7f2c680fe760 msgr2=0x7f2c68100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.668+0000 7f2c6de00640 1 -- 172.21.15.49:0/1602503739 shutdown_connections 2024-02-26T06:22:19.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.668+0000 7f2c6de00640 1 -- 172.21.15.49:0/1602503739 wait complete. 2024-02-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6de00640 1 Processor -- start 2024-02-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6de00640 1 -- start start 2024-02-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c681031d0 0x7f2c68199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c681043c0 0x7f2c68199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c68104d80 0x7f2c681a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6de00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681a0d10 con 0x7f2c68104d80 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6de00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681a0e80 con 0x7f2c681031d0 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6de00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681a1040 con 0x7f2c681043c0 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.669+0000 7f2c6d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c68104d80 0x7f2c681a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c681031d0 0x7f2c68199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c68104d80 0x7f2c681a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59014/0 (socket says 172.21.15.49:59014) 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6d400640 1 -- 172.21.15.49:0/3807699906 learned_addr learned my addr 172.21.15.49:0/3807699906 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c67e00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c681043c0 0x7f2c68199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6ca00640 1 -- 172.21.15.49:0/3807699906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c681043c0 msgr2=0x7f2c68199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6ca00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c681043c0 0x7f2c68199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6ca00640 1 -- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c68104d80 msgr2=0x7f2c681a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6ca00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c68104d80 0x7f2c681a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6ca00640 1 -- 172.21.15.49:0/3807699906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54009560 con 0x7f2c681031d0 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6ca00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c681031d0 0x7f2c68199220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00e9c0 tx=0x7f2c5c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c65600640 1 -- 172.21.15.49:0/3807699906 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00ccc0 con 0x7f2c681031d0 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.670+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c681a1320 con 0x7f2c681031d0 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.671+0000 7f2c65600640 1 -- 172.21.15.49:0/3807699906 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c5c00ce20 con 0x7f2c681031d0 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.671+0000 7f2c65600640 1 -- 172.21.15.49:0/3807699906 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c010430 con 0x7f2c681031d0 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.671+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c681a1850 con 0x7f2c681031d0 2024-02-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.672+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6810a4f0 con 0x7f2c681031d0 2024-02-26T06:22:19.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.674+0000 7f2c65600640 1 -- 172.21.15.49:0/3807699906 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2c5c003f60 con 0x7f2c681031d0 2024-02-26T06:22:19.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.675+0000 7f2c65600640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2c500a20c0 0x7f2c500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:19.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.675+0000 7f2c65600640 1 -- 172.21.15.49:0/3807699906 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2c5c014070 con 0x7f2c681031d0 2024-02-26T06:22:19.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.675+0000 7f2c67e00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2c500a20c0 0x7f2c500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:19.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.676+0000 7f2c67e00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2c500a20c0 0x7f2c500a4580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c5402c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:19.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.678+0000 7f2c65600640 1 -- 172.21.15.49:0/3807699906 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c5c0970e0 con 0x7f2c681031d0 2024-02-26T06:22:19.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.880+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c6810edd0 con 0x7f2c681031d0 2024-02-26T06:22:19.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.882+0000 7f2c65600640 1 -- 172.21.15.49:0/3807699906 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7f2c6810edd0 con 0x7f2c681031d0 2024-02-26T06:22:19.883 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:19.883 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:22:19.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.884+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2c500a20c0 msgr2=0x7f2c500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.884+0000 7f2c6de00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2c500a20c0 0x7f2c500a4580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c5402c800 comp rx=0 tx=0).stop 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.884+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c681031d0 msgr2=0x7f2c68199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.884+0000 7f2c6de00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c681031d0 0x7f2c68199220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00e9c0 tx=0x7f2c5c00ee80 comp rx=0 tx=0).stop 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.885+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 shutdown_connections 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.885+0000 7f2c6de00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f2c500a20c0 0x7f2c500a4580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.885+0000 7f2c6de00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c68104d80 0x7f2c681a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.885+0000 7f2c6de00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c681043c0 0x7f2c68199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.885+0000 7f2c6de00640 1 --2- 172.21.15.49:0/3807699906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c681031d0 0x7f2c68199220 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.885+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 >> 172.21.15.49:0/3807699906 conn(0x7f2c680fe760 msgr2=0x7f2c681076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.885+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 shutdown_connections 2024-02-26T06:22:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:19.885+0000 7f2c6de00640 1 -- 172.21.15.49:0/3807699906 wait complete. 2024-02-26T06:22:19.906 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 -- 172.21.15.49:0/2739455661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181045b0 msgr2=0x7eff18104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 --2- 172.21.15.49:0/2739455661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181045b0 0x7eff18104a30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7eff00009900 tx=0x7eff0002bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 -- 172.21.15.49:0/2739455661 shutdown_connections 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 --2- 172.21.15.49:0/2739455661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff18104f70 0x7eff1810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 --2- 172.21.15.49:0/2739455661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181045b0 0x7eff18104a30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 --2- 172.21.15.49:0/2739455661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff181033c0 0x7eff181037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 -- 172.21.15.49:0/2739455661 >> 172.21.15.49:0/2739455661 conn(0x7eff180fe970 msgr2=0x7eff18100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 -- 172.21.15.49:0/2739455661 shutdown_connections 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.139+0000 7eff1f400640 1 -- 172.21.15.49:0/2739455661 wait complete. 2024-02-26T06:22:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.140+0000 7eff1f400640 1 Processor -- start 2024-02-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1f400640 1 -- start start 2024-02-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181033c0 0x7eff18199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff181045b0 0x7eff18199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff18104f70 0x7eff181a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1f400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a0ef0 con 0x7eff181033c0 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1f400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a1060 con 0x7eff181045b0 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1f400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a1220 con 0x7eff18104f70 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1ca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181033c0 0x7eff18199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1ca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181033c0 0x7eff18199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47040/0 (socket says 172.21.15.49:47040) 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1ca00640 1 -- 172.21.15.49:0/74803065 learned_addr learned my addr 172.21.15.49:0/74803065 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff1d400640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff18104f70 0x7eff181a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.141+0000 7eff0fe00640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff181045b0 0x7eff18199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff1ca00640 1 -- 172.21.15.49:0/74803065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff18104f70 msgr2=0x7eff181a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff1ca00640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff18104f70 0x7eff181a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff1ca00640 1 -- 172.21.15.49:0/74803065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff181045b0 msgr2=0x7eff18199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff1ca00640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff181045b0 0x7eff18199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff1ca00640 1 -- 172.21.15.49:0/74803065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff00009560 con 0x7eff181033c0 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff1ca00640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181033c0 0x7eff18199400 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff0800d950 tx=0x7eff0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff16000640 1 -- 172.21.15.49:0/74803065 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08009c30 con 0x7eff181033c0 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff16000640 1 -- 172.21.15.49:0/74803065 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff08009d90 con 0x7eff181033c0 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff181a1500 con 0x7eff181033c0 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.142+0000 7eff16000640 1 -- 172.21.15.49:0/74803065 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0800f850 con 0x7eff181033c0 2024-02-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.143+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff181a1a30 con 0x7eff181033c0 2024-02-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.144+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1810a760 con 0x7eff181033c0 2024-02-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.146+0000 7eff16000640 1 -- 172.21.15.49:0/74803065 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7eff0800f9b0 con 0x7eff181033c0 2024-02-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.148+0000 7eff16000640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efefc0a2140 0x7efefc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.148+0000 7eff16000640 1 -- 172.21.15.49:0/74803065 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7eff080cd8c0 con 0x7eff181033c0 2024-02-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.148+0000 7eff0fe00640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efefc0a2140 0x7efefc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.149+0000 7eff0fe00640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efefc0a2140 0x7efefc0a4600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff00009230 tx=0x7eff00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.153+0000 7eff16000640 1 -- 172.21.15.49:0/74803065 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff08096820 con 0x7eff181033c0 2024-02-26T06:22:20.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:20 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/902379513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:20.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:20 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3807699906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:20.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.395+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff1810edd0 con 0x7eff181033c0 2024-02-26T06:22:20.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.397+0000 7eff16000640 1 -- 172.21.15.49:0/74803065 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11331 (secure 0 0 0) 0x7eff08099ee0 con 0x7eff181033c0 2024-02-26T06:22:20.398 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:20.398 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:20:38.098815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:22:20.398 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-02-26T06:22:20.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efefc0a2140 msgr2=0x7efefc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efefc0a2140 0x7efefc0a4600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff00009230 tx=0x7eff00005fd0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181033c0 msgr2=0x7eff18199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181033c0 0x7eff18199400 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff0800d950 tx=0x7eff0800de10 comp rx=0 tx=0).stop 2024-02-26T06:22:20.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 shutdown_connections 2024-02-26T06:22:20.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efefc0a2140 0x7efefc0a4600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff181045b0 0x7eff18199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff18104f70 0x7eff181a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 --2- 172.21.15.49:0/74803065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff181033c0 0x7eff18199400 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 >> 172.21.15.49:0/74803065 conn(0x7eff180fe970 msgr2=0x7eff18107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 shutdown_connections 2024-02-26T06:22:20.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.400+0000 7eff1f400640 1 -- 172.21.15.49:0/74803065 wait complete. 2024-02-26T06:22:20.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-26T06:20:31.555053+0000', 'modified': '2024-02-26T06:20:38.098815+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24559, 'mds_2': 24535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.80:6835/1885591642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 1885591642}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 1885591642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.80:6837/2132318232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6836', 'nonce': 2132318232}, {'type': 'v1', 'addr': '172.21.15.80:6837', 'nonce': 2132318232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/2578799425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 2578799425}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 2578799425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.115:6837/3720601525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 3720601525}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 3720601525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24559': {'gid': 24559, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.115:6839/318392314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 318392314}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 318392314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6839/3160003047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6838', 'nonce': 3160003047}, {'type': 'v1', 'addr': '172.21.15.80:6839', 'nonce': 3160003047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T06:22:20.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-26T06:22:20.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-26T06:22:20.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:20 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/902379513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:20.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:20 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3807699906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:20.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:20 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/902379513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:20.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:20 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3807699906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 -- 172.21.15.49:0/1945899406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894104410 msgr2=0x7fb894106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 --2- 172.21.15.49:0/1945899406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894104410 0x7fb894106800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb880009900 tx=0x7fb88002bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 -- 172.21.15.49:0/1945899406 shutdown_connections 2024-02-26T06:22:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 --2- 172.21.15.49:0/1945899406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894104410 0x7fb894106800 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 --2- 172.21.15.49:0/1945899406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894106d40 0x7fb894109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 --2- 172.21.15.49:0/1945899406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb894101ae0 0x7fb894103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 -- 172.21.15.49:0/1945899406 >> 172.21.15.49:0/1945899406 conn(0x7fb8940fb4f0 msgr2=0x7fb8940fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 -- 172.21.15.49:0/1945899406 shutdown_connections 2024-02-26T06:22:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.655+0000 7fb899e00640 1 -- 172.21.15.49:0/1945899406 wait complete. 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.656+0000 7fb899e00640 1 Processor -- start 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.656+0000 7fb899e00640 1 -- start start 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.656+0000 7fb899e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894101ae0 0x7fb894101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.656+0000 7fb899e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894104410 0x7fb8940ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.656+0000 7fb899e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb894106d40 0x7fb8940fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.656+0000 7fb899e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb894100730 con 0x7fb894104410 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.656+0000 7fb899e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941008a0 con 0x7fb894101ae0 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.656+0000 7fb899e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941a11c0 con 0x7fb894106d40 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb893e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894104410 0x7fb8940ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb898a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894101ae0 0x7fb894101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb898a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894101ae0 0x7fb894101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38928/0 (socket says 172.21.15.49:38928) 2024-02-26T06:22:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb898a00640 1 -- 172.21.15.49:0/634954965 learned_addr learned my addr 172.21.15.49:0/634954965 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb893e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894104410 0x7fb8940ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47056/0 (socket says 172.21.15.49:47056) 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb899400640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb894106d40 0x7fb8940fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb893e00640 1 -- 172.21.15.49:0/634954965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb894106d40 msgr2=0x7fb8940fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb893e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb894106d40 0x7fb8940fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb893e00640 1 -- 172.21.15.49:0/634954965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894101ae0 msgr2=0x7fb894101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb893e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894101ae0 0x7fb894101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.657+0000 7fb893e00640 1 -- 172.21.15.49:0/634954965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c009840 con 0x7fb894104410 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.658+0000 7fb893e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894104410 0x7fb8940ffa90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb880036040 tx=0x7fb880005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.658+0000 7fb891600640 1 -- 172.21.15.49:0/634954965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800043d0 con 0x7fb894104410 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.658+0000 7fb891600640 1 -- 172.21.15.49:0/634954965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8800052d0 con 0x7fb894104410 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.658+0000 7fb891600640 1 -- 172.21.15.49:0/634954965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880003c60 con 0x7fb894104410 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.658+0000 7fb899e00640 1 -- 172.21.15.49:0/634954965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb880009560 con 0x7fb894104410 2024-02-26T06:22:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.658+0000 7fb899e00640 1 -- 172.21.15.49:0/634954965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8941a1740 con 0x7fb894104410 2024-02-26T06:22:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.661+0000 7fb891600640 1 -- 172.21.15.49:0/634954965 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb880005440 con 0x7fb894104410 2024-02-26T06:22:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.661+0000 7fb89e1e7740 1 Processor -- start 2024-02-26T06:22:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb89e1e7740 1 -- start start 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a8730 0x5620672a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562067217b00 con 0x5620672a8730 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562067215030 con 0x5620672ae290 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562067210810 con 0x5620672a9120 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb899400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a8730 0x5620672a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb899400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a8730 0x5620672a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47064/0 (socket says 172.21.15.49:47064) 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb899400640 1 -- 172.21.15.49:0/1949369700 learned_addr learned my addr 172.21.15.49:0/1949369700 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb893e00640 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.662+0000 7fb898a00640 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb899400640 1 -- 172.21.15.49:0/1949369700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 msgr2=0x5620672adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb899400640 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb899400640 1 -- 172.21.15.49:0/1949369700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 msgr2=0x5620672b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb899400640 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb899400640 1 -- 172.21.15.49:0/1949369700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620671d8490 con 0x5620672a8730 2024-02-26T06:22:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb893e00640 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:22:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb899400640 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a8730 0x5620672a8b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0115b0 tx=0x7fb88c011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f256b094696ed324 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb885c00640 1 -- 172.21.15.49:0/1949369700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c004d20 con 0x5620672a8730 2024-02-26T06:22:20.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb885c00640 1 -- 172.21.15.49:0/1949369700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb88c004e80 con 0x5620672a8730 2024-02-26T06:22:20.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.663+0000 7fb885c00640 1 -- 172.21.15.49:0/1949369700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00f8c0 con 0x5620672a8730 2024-02-26T06:22:20.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb891600640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb87c0a2140 0x7fb87c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb891600640 1 -- 172.21.15.49:0/634954965 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb8800efc90 con 0x7fb894104410 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb898a00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb87c0a2140 0x7fb87c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 -- 172.21.15.49:0/1949369700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a8730 msgr2=0x5620672a8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a8730 0x5620672a8b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0115b0 tx=0x7fb88c011a70 comp rx=0 tx=0).stop 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 -- 172.21.15.49:0/1949369700 shutdown_connections 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/1949369700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a8730 0x5620672a8b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 -- 172.21.15.49:0/1949369700 >> 172.21.15.49:0/1949369700 conn(0x5620672a3ca0 msgr2=0x5620672a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 -- 172.21.15.49:0/1949369700 shutdown_connections 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.664+0000 7fb89e1e7740 1 -- 172.21.15.49:0/1949369700 wait complete. 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.665+0000 7fb898a00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb87c0a2140 0x7fb87c0a4600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5620672a4c10 tx=0x7fb888005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.665+0000 7fb89e1e7740 1 -- start start 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.665+0000 7fb89e1e7740 1 Processor -- start 2024-02-26T06:22:20.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a8730 0x5620672b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a9120 0x5620672b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620671b5e70 con 0x5620672a9120 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620671b5b10 con 0x5620672ae290 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562066f58f20 con 0x5620672a8730 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb89e1e7740 10 client.0 _ll_register_callbacks cb 0x5620671c9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb898a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a8730 0x5620672b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb893e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a9120 0x5620672b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb898a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a8730 0x5620672b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34706/0 (socket says 172.21.15.49:34706) 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb898a00640 1 -- 172.21.15.49:0/3628564556 learned_addr learned my addr 172.21.15.49:0/3628564556 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb899400640 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb898a00640 1 -- 172.21.15.49:0/3628564556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 msgr2=0x5620672b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb898a00640 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb898a00640 1 -- 172.21.15.49:0/3628564556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a9120 msgr2=0x5620672b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb898a00640 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a9120 0x5620672b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.666+0000 7fb898a00640 1 -- 172.21.15.49:0/3628564556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c009e20 con 0x5620672a8730 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.667+0000 7fb898a00640 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a8730 0x5620672b2610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb88801e9d0 tx=0x7fb888004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d93b06ddffb12da server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.667+0000 7fb87a000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.667+0000 7fb87a000640 1 -- 172.21.15.49:0/3628564556 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888023070 con 0x5620672a8730 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.667+0000 7fb87a000640 1 -- 172.21.15.49:0/3628564556 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb888029420 con 0x5620672a8730 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.667+0000 7fb89e1e7740 1 -- 172.21.15.49:0/3628564556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562066f106c0 con 0x5620672a8730 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.667+0000 7fb87a000640 1 -- 172.21.15.49:0/3628564556 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb888017070 con 0x5620672a8730 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.667+0000 7fb89e1e7740 10 client.25009 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.667+0000 7fb89e1e7740 1 -- 172.21.15.49:0/3628564556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56206720ee90 con 0x5620672a8730 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.668+0000 7fb87a000640 1 -- 172.21.15.49:0/3628564556 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb888038080 con 0x5620672a8730 2024-02-26T06:22:20.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.668+0000 7fb87a000640 1 -- 172.21.15.49:0/3628564556 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb88800f580 con 0x5620672a8730 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.668+0000 7fb89e1e7740 10 client.25009 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.668+0000 7fb89e1e7740 10 client.25009 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.668+0000 7fb89e1e7740 20 client.25009 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.668+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5620672bdec0 0x5620672c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.668+0000 7fb89e1e7740 4 client.25009 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.668+0000 7fb89e1e7740 1 -- 172.21.15.49:0/3628564556 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562066f61270 con 0x5620672bdec0 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.669+0000 7fb893e00640 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5620672bdec0 0x5620672c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.669+0000 7fb893e00640 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5620672bdec0 0x5620672c2f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4b61d460da2a280 server_cookie=5003dc689ae88861 in_seq=0 out_seq=0 2024-02-26T06:22:20.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.669+0000 7fb87a000640 10 client.25009 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:22:20.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb87a000640 1 -- 172.21.15.49:0/3628564556 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562066f61270 con 0x5620672bdec0 2024-02-26T06:22:20.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb87a000640 10 client.25009 handle_command_reply: tid=0 2024-02-26T06:22:20.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 client.25009 shutdown 2024-02-26T06:22:20.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 -- 172.21.15.49:0/3628564556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a8730 msgr2=0x5620672b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a8730 0x5620672b2610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb88801e9d0 tx=0x7fb888004730 comp rx=0 tx=0).stop 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 -- 172.21.15.49:0/3628564556 shutdown_connections 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5620672bdec0 0x5620672c2f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672ae290 0x5620672b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a9120 0x5620672b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/3628564556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a8730 0x5620672b2610 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 -- 172.21.15.49:0/3628564556 >> 172.21.15.49:0/3628564556 conn(0x5620672a3ca0 msgr2=0x5620672a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb87a000640 0 client.25009 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 -- 172.21.15.49:0/3628564556 shutdown_connections 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.670+0000 7fb89e1e7740 1 -- 172.21.15.49:0/3628564556 wait complete. 2024-02-26T06:22:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.671+0000 7fb89e1e7740 20 client.25009 trim_cache size 0 max 16384 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.687+0000 7fb89e1e7740 1 Processor -- start 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.687+0000 7fb89e1e7740 1 -- start start 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.687+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.687+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.687+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620672b2ff0 con 0x5620672a3ca0 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562066f61270 con 0x5620672a8730 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620672b3790 con 0x5620672a9120 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb898a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb898a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47086/0 (socket says 172.21.15.49:47086) 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb898a00640 1 -- 172.21.15.49:0/375899239 learned_addr learned my addr 172.21.15.49:0/375899239 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb893e00640 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb899400640 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb898a00640 1 -- 172.21.15.49:0/375899239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 msgr2=0x5620672a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb898a00640 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb898a00640 1 -- 172.21.15.49:0/375899239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 msgr2=0x5620672b9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb898a00640 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.688+0000 7fb898a00640 1 -- 172.21.15.49:0/375899239 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620671ab880 con 0x5620672a3ca0 2024-02-26T06:22:20.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb898a00640 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b2ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb88800f930 tx=0x7fb888012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc3ad0bc81d9803 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb885200640 1 -- 172.21.15.49:0/375899239 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888023050 con 0x5620672a3ca0 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb885200640 1 -- 172.21.15.49:0/375899239 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562066f106c0 con 0x5620672a3ca0 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb885200640 1 -- 172.21.15.49:0/375899239 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888029960 con 0x5620672a3ca0 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb89e1e7740 1 -- 172.21.15.49:0/375899239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 msgr2=0x5620672b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b2ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb88800f930 tx=0x7fb888012f30 comp rx=0 tx=0).stop 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb89e1e7740 1 -- 172.21.15.49:0/375899239 shutdown_connections 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.689+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/375899239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b2ab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.690+0000 7fb89e1e7740 1 -- 172.21.15.49:0/375899239 >> 172.21.15.49:0/375899239 conn(0x5620672adc80 msgr2=0x5620672a2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.690+0000 7fb89e1e7740 1 -- 172.21.15.49:0/375899239 shutdown_connections 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.690+0000 7fb89e1e7740 1 -- 172.21.15.49:0/375899239 wait complete. 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.690+0000 7fb89e1e7740 1 -- start start 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.690+0000 7fb89e1e7740 1 Processor -- start 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb89e1e7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620672b8c00 con 0x5620672a3ca0 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620672b8d70 con 0x5620672a8730 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb89e1e7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620672b8f30 con 0x5620672a9120 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb89e1e7740 10 client.0 _ll_register_callbacks cb 0x7fb888009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb898a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb898a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47102/0 (socket says 172.21.15.49:47102) 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb898a00640 1 -- 172.21.15.49:0/2473481769 learned_addr learned my addr 172.21.15.49:0/2473481769 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb899400640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb893e00640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb898a00640 1 -- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 msgr2=0x5620672b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb898a00640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb898a00640 1 -- 172.21.15.49:0/2473481769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 msgr2=0x5620672b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb898a00640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb898a00640 1 -- 172.21.15.49:0/2473481769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56206720ee90 con 0x5620672a3ca0 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.691+0000 7fb899400640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb893e00640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb898a00640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b71e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb888012d20 tx=0x7fb88800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1f6aa4c0a9bcbe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb87a000640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb87a000640 1 -- 172.21.15.49:0/2473481769 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888023070 con 0x5620672a3ca0 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb87a000640 1 -- 172.21.15.49:0/2473481769 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb88800fda0 con 0x5620672a3ca0 2024-02-26T06:22:20.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb89e1e7740 1 -- 172.21.15.49:0/2473481769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620672be790 con 0x5620672a3ca0 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb87a000640 1 -- 172.21.15.49:0/2473481769 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb888005450 con 0x5620672a3ca0 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb89e1e7740 10 client.14883 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.692+0000 7fb89e1e7740 1 -- 172.21.15.49:0/2473481769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5620671d14e0 con 0x5620672a3ca0 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb87a000640 1 -- 172.21.15.49:0/2473481769 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5620672be790 con 0x5620672a3ca0 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb87a000640 1 -- 172.21.15.49:0/2473481769 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb8880298c0 con 0x5620672a3ca0 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb89e1e7740 10 client.14883 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb89e1e7740 10 client.14883 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb89e1e7740 20 client.14883 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5620672ae290 0x56206731f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb89e1e7740 4 client.14883 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb89e1e7740 1 -- 172.21.15.49:0/2473481769 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562067234c10 con 0x5620672ae290 2024-02-26T06:22:20.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.693+0000 7fb893e00640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5620672ae290 0x56206731f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb893e00640 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5620672ae290 0x56206731f680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98685d89760ace1f server_cookie=e7f49b21b81abc3f in_seq=0 out_seq=0 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb87a000640 10 client.14883 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb87a000640 1 -- 172.21.15.49:0/2473481769 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562067234c10 con 0x5620672ae290 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb87a000640 10 client.14883 handle_command_reply: tid=0 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 client.14883 shutdown 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 -- 172.21.15.49:0/2473481769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 msgr2=0x5620672b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b71e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb888012d20 tx=0x7fb88800f810 comp rx=0 tx=0).stop 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 -- 172.21.15.49:0/2473481769 shutdown_connections 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5620672ae290 0x56206731f680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5620672a8730 0x5620672b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620672a9120 0x5620672b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 --2- 172.21.15.49:0/2473481769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620672a3ca0 0x5620672b71e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb87a000640 0 client.14883 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:22:20.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 -- 172.21.15.49:0/2473481769 >> 172.21.15.49:0/2473481769 conn(0x5620672adc80 msgr2=0x5620672a2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 -- 172.21.15.49:0/2473481769 shutdown_connections 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.694+0000 7fb89e1e7740 1 -- 172.21.15.49:0/2473481769 wait complete. 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb89e1e7740 20 client.14883 trim_cache size 0 max 16384 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24535, 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-02-26T06:22:20.700 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-02-26T06:22:20.701 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 103.612161962, 2024-02-26T06:22:20.701 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-02-26T06:22:20.701 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-02-26T06:22:20.701 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-02-26T06:22:20.701 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 132.47428993899999 2024-02-26T06:22:20.701 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 -- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb87c0a2140 msgr2=0x7fb87c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb87c0a2140 0x7fb87c0a4600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5620672a4c10 tx=0x7fb888005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 -- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894104410 msgr2=0x7fb8940ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894104410 0x7fb8940ffa90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb880036040 tx=0x7fb880005e50 comp rx=0 tx=0).stop 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 -- 172.21.15.49:0/634954965 shutdown_connections 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb87c0a2140 0x7fb87c0a4600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb894106d40 0x7fb8940fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb894104410 0x7fb8940ffa90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 --2- 172.21.15.49:0/634954965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb894101ae0 0x7fb894101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 -- 172.21.15.49:0/634954965 >> 172.21.15.49:0/634954965 conn(0x7fb8940fb4f0 msgr2=0x7fb894102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 -- 172.21.15.49:0/634954965 shutdown_connections 2024-02-26T06:22:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.695+0000 7fb899e00640 1 -- 172.21.15.49:0/634954965 wait complete. 2024-02-26T06:22:20.714 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-26T06:22:20.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.938+0000 7f60e6600640 1 -- 172.21.15.49:0/1812126724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e0101060 msgr2=0x7f60e0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.938+0000 7f60e6600640 1 --2- 172.21.15.49:0/1812126724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e0101060 0x7f60e0101480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009900 tx=0x7f60d002bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.939+0000 7f60e6600640 1 -- 172.21.15.49:0/1812126724 shutdown_connections 2024-02-26T06:22:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.939+0000 7f60e6600640 1 --2- 172.21.15.49:0/1812126724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60e0102250 0x7f60e01026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.939+0000 7f60e6600640 1 --2- 172.21.15.49:0/1812126724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60e0102c10 0x7f60e01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.939+0000 7f60e6600640 1 --2- 172.21.15.49:0/1812126724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e0101060 0x7f60e0101480 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.939+0000 7f60e6600640 1 -- 172.21.15.49:0/1812126724 >> 172.21.15.49:0/1812126724 conn(0x7f60e00fc5f0 msgr2=0x7f60e00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.939+0000 7f60e6600640 1 -- 172.21.15.49:0/1812126724 shutdown_connections 2024-02-26T06:22:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.939+0000 7f60e6600640 1 -- 172.21.15.49:0/1812126724 wait complete. 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.940+0000 7f60e6600640 1 Processor -- start 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.940+0000 7f60e6600640 1 -- start start 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.940+0000 7f60e6600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60e0101060 0x7f60e0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.940+0000 7f60e6600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e0102250 0x7f60e0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e6600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60e0102c10 0x7f60e019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e6600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e019ec00 con 0x7f60e0102250 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e6600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e019ed70 con 0x7f60e0102c10 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e6600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e019ef30 con 0x7f60e0101060 2024-02-26T06:22:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60e0101060 0x7f60e0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60e0101060 0x7f60e0197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34736/0 (socket says 172.21.15.49:34736) 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5200640 1 -- 172.21.15.49:0/1315587395 learned_addr learned my addr 172.21.15.49:0/1315587395 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e0102250 0x7f60e0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5c00640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60e0102c10 0x7f60e019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5200640 1 -- 172.21.15.49:0/1315587395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60e0102c10 msgr2=0x7f60e019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5200640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60e0102c10 0x7f60e019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5200640 1 -- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e0102250 msgr2=0x7f60e0197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5200640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e0102250 0x7f60e0197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.941+0000 7f60e5200640 1 -- 172.21.15.49:0/1315587395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c8009490 con 0x7f60e0101060 2024-02-26T06:22:20.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.942+0000 7f60e5200640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60e0101060 0x7f60e0197110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60d0005ab0 tx=0x7f60d0005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.942+0000 7f60de000640 1 -- 172.21.15.49:0/1315587395 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00043d0 con 0x7f60e0101060 2024-02-26T06:22:20.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.942+0000 7f60de000640 1 -- 172.21.15.49:0/1315587395 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d00052d0 con 0x7f60e0101060 2024-02-26T06:22:20.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.942+0000 7f60de000640 1 -- 172.21.15.49:0/1315587395 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0003d40 con 0x7f60e0101060 2024-02-26T06:22:20.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.942+0000 7f60e6600640 1 -- 172.21.15.49:0/1315587395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d0009560 con 0x7f60e0101060 2024-02-26T06:22:20.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.942+0000 7f60e6600640 1 -- 172.21.15.49:0/1315587395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e019f4e0 con 0x7f60e0101060 2024-02-26T06:22:20.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.945+0000 7f60eabd5740 1 Processor -- start 2024-02-26T06:22:20.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.945+0000 7f60eabd5740 1 -- start start 2024-02-26T06:22:20.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.945+0000 7f60de000640 1 -- 172.21.15.49:0/1315587395 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f60d0005440 con 0x7f60e0101060 2024-02-26T06:22:20.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d73b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d78c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d7b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60eabd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583ce2b00 con 0x562583d73730 2024-02-26T06:22:20.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60eabd5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583ce0030 con 0x562583d79290 2024-02-26T06:22:20.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60eabd5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583cdb810 con 0x562583d74120 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60e5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d78c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60e5c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d73b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60e5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d78c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34752/0 (socket says 172.21.15.49:34752) 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60e5c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d73b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47126/0 (socket says 172.21.15.49:47126) 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60e5200640 1 -- 172.21.15.49:0/1332226210 learned_addr learned my addr 172.21.15.49:0/1332226210 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.946+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d7b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60e5200640 1 -- 172.21.15.49:0/1332226210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 msgr2=0x562583d7b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60e5200640 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d7b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60e5200640 1 -- 172.21.15.49:0/1332226210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 msgr2=0x562583d73b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60e5200640 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d73b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60e5200640 1 -- 172.21.15.49:0/1332226210 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562583ca3490 con 0x562583d74120 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60e5c00640 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d73b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d7b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60e5200640 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d78c20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f60d00f3d10 tx=0x7f60d00f3e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a86dc95edde4b49 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60de000640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f60c00a20c0 0x7f60c00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60d6000640 1 -- 172.21.15.49:0/1332226210 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00fb070 con 0x562583d74120 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60de000640 1 -- 172.21.15.49:0/1315587395 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f60d00efe30 con 0x7f60e0101060 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.947+0000 7f60e5c00640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f60c00a20c0 0x7f60c00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60d6000640 1 -- 172.21.15.49:0/1332226210 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d00b8d90 con 0x562583d74120 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60d6000640 1 -- 172.21.15.49:0/1332226210 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00bc8c0 con 0x562583d74120 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 -- 172.21.15.49:0/1332226210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 msgr2=0x562583d78c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d78c20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f60d00f3d10 tx=0x7f60d00f3e00 comp rx=0 tx=0).stop 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60e5c00640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f60c00a20c0 0x7f60c00a4580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x562583d74980 tx=0x7f60d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 -- 172.21.15.49:0/1332226210 shutdown_connections 2024-02-26T06:22:20.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d7b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d78c20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1332226210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d73b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 -- 172.21.15.49:0/1332226210 >> 172.21.15.49:0/1332226210 conn(0x562583d6eca0 msgr2=0x562583d6f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 -- 172.21.15.49:0/1332226210 shutdown_connections 2024-02-26T06:22:20.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.948+0000 7f60eabd5740 1 -- 172.21.15.49:0/1332226210 wait complete. 2024-02-26T06:22:20.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.949+0000 7f60eabd5740 1 -- start start 2024-02-26T06:22:20.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.949+0000 7f60eabd5740 1 Processor -- start 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d73730 0x562583d81720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d81c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d88ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60eabd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583c80e70 con 0x562583d74120 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60eabd5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583c80b10 con 0x562583d79290 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60eabd5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583a23f20 con 0x562583d73730 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60eabd5740 10 client.0 _ll_register_callbacks cb 0x562583c94920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d73730 0x562583d81720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d88ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60d7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d81c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d73730 0x562583d81720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34768/0 (socket says 172.21.15.49:34768) 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5200640 1 -- 172.21.15.49:0/713092250 learned_addr learned my addr 172.21.15.49:0/713092250 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5200640 1 -- 172.21.15.49:0/713092250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 msgr2=0x562583d88ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5200640 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d88ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5200640 1 -- 172.21.15.49:0/713092250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 msgr2=0x562583d81c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5200640 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d81c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60e5200640 1 -- 172.21.15.49:0/713092250 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e019f4e0 con 0x562583d73730 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.950+0000 7f60d7e00640 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d81c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60e5200640 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d73730 0x562583d81720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f60d00f67c0 tx=0x7f60d0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=859d44209954a77e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60cea00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60cea00640 1 -- 172.21.15.49:0/713092250 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00fb060 con 0x562583d73730 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60cea00640 1 -- 172.21.15.49:0/713092250 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f60d00bd1f0 con 0x562583d73730 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60eabd5740 1 -- 172.21.15.49:0/713092250 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562583cffc10 con 0x562583d73730 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60cea00640 1 -- 172.21.15.49:0/713092250 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d00f3360 con 0x562583d73730 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60eabd5740 10 client.25027 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60eabd5740 1 -- 172.21.15.49:0/713092250 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562583cd9e90 con 0x562583d73730 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60cea00640 1 -- 172.21.15.49:0/713092250 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562583cffc10 con 0x562583d73730 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.951+0000 7f60cea00640 1 -- 172.21.15.49:0/713092250 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60d003c070 con 0x562583d73730 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.952+0000 7f60eabd5740 10 client.25027 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.952+0000 7f60eabd5740 10 client.25027 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.952+0000 7f60eabd5740 20 client.25027 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.952+0000 7f60eabd5740 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562583d8d130 0x562583d91bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.952+0000 7f60eabd5740 4 client.25027 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.952+0000 7f60eabd5740 1 -- 172.21.15.49:0/713092250 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625839db6c0 con 0x562583d8d130 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.952+0000 7f60d7e00640 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562583d8d130 0x562583d91bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.953+0000 7f60d7e00640 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562583d8d130 0x562583d91bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0a58d79c74315a7 server_cookie=dadaf9c3d11eefa3 in_seq=0 out_seq=0 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.953+0000 7f60cea00640 10 client.25027 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.953+0000 7f60cea00640 1 -- 172.21.15.49:0/713092250 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625839db6c0 con 0x562583d8d130 2024-02-26T06:22:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.953+0000 7f60cea00640 10 client.25027 handle_command_reply: tid=0 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.953+0000 7f60eabd5740 1 client.25027 shutdown 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 -- 172.21.15.49:0/713092250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d73730 msgr2=0x562583d81720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d73730 0x562583d81720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f60d00f67c0 tx=0x7f60d0005fd0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 -- 172.21.15.49:0/713092250 shutdown_connections 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562583d8d130 0x562583d91bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d88ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d81c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60cea00640 0 client.25027 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 --2- 172.21.15.49:0/713092250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d73730 0x562583d81720 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 -- 172.21.15.49:0/713092250 >> 172.21.15.49:0/713092250 conn(0x562583d6eca0 msgr2=0x562583d6dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 -- 172.21.15.49:0/713092250 shutdown_connections 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 1 -- 172.21.15.49:0/713092250 wait complete. 2024-02-26T06:22:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.954+0000 7f60eabd5740 20 client.25027 trim_cache size 0 max 16384 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60eabd5740 1 Processor -- start 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60eabd5740 1 -- start start 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d73730 0x562583d81ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d88c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d79290 0x562583d6d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60eabd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583d7cb60 con 0x562583d74120 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60eabd5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583c80e70 con 0x562583d73730 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60eabd5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583d6f6f0 con 0x562583d79290 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60d7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d88c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60d7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d88c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47144/0 (socket says 172.21.15.49:47144) 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60d7e00640 1 -- 172.21.15.49:0/1155356033 learned_addr learned my addr 172.21.15.49:0/1155356033 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60e5200640 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d73730 0x562583d81ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.971+0000 7f60e5c00640 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d79290 0x562583d6d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d7e00640 1 -- 172.21.15.49:0/1155356033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d79290 msgr2=0x562583d6d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d79290 0x562583d6d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d7e00640 1 -- 172.21.15.49:0/1155356033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d73730 msgr2=0x562583d81ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d73730 0x562583d81ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d7e00640 1 -- 172.21.15.49:0/1155356033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562583c76880 con 0x562583d74120 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60e5200640 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d73730 0x562583d81ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d88c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x562583d6fc10 tx=0x7f60c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dd2209a65978ee server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d5600640 1 -- 172.21.15.49:0/1155356033 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8017040 con 0x562583d74120 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d5600640 1 -- 172.21.15.49:0/1155356033 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60c8018040 con 0x562583d74120 2024-02-26T06:22:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.972+0000 7f60d5600640 1 -- 172.21.15.49:0/1155356033 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8004a00 con 0x562583d74120 2024-02-26T06:22:20.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 -- 172.21.15.49:0/1155356033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 msgr2=0x562583d88c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d88c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x562583d6fc10 tx=0x7f60c8005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 -- 172.21.15.49:0/1155356033 shutdown_connections 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d79290 0x562583d6d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d74120 0x562583d88c30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1155356033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d73730 0x562583d81ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 -- 172.21.15.49:0/1155356033 >> 172.21.15.49:0/1155356033 conn(0x562583d6eca0 msgr2=0x562583d813d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 -- 172.21.15.49:0/1155356033 shutdown_connections 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 -- 172.21.15.49:0/1155356033 wait complete. 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 -- start start 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.973+0000 7f60eabd5740 1 Processor -- start 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d88620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d88b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60eabd5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d85110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60eabd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583d85810 con 0x562583d73730 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60eabd5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583d85980 con 0x562583d79290 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60eabd5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562583d85b40 con 0x562583d74120 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60eabd5740 10 client.0 _ll_register_callbacks cb 0x7f60d0103070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60e5200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d88620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60e5200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d88620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47156/0 (socket says 172.21.15.49:47156) 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60e5200640 1 -- 172.21.15.49:0/1593585891 learned_addr learned my addr 172.21.15.49:0/1593585891 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d88b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.974+0000 7f60e5c00640 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d85110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60e5200640 1 -- 172.21.15.49:0/1593585891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 msgr2=0x562583d88b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60e5200640 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d88b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60e5200640 1 -- 172.21.15.49:0/1593585891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 msgr2=0x562583d85110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:20.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60e5200640 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d85110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60e5200640 1 -- 172.21.15.49:0/1593585891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562583cd9e90 con 0x562583d73730 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60e5200640 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d88620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60d00b8f50 tx=0x7f60d00c5010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389af51828371ff2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60cea00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60cea00640 1 -- 172.21.15.49:0/1593585891 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00fb040 con 0x562583d73730 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60cea00640 1 -- 172.21.15.49:0/1593585891 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f60d00bd1f0 con 0x562583d73730 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60eabd5740 1 -- 172.21.15.49:0/1593585891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60d00b8d90 con 0x562583d73730 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60cea00640 1 -- 172.21.15.49:0/1593585891 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d00f3360 con 0x562583d73730 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60eabd5740 10 client.14901 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.975+0000 7f60eabd5740 1 -- 172.21.15.49:0/1593585891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f60c8012040 con 0x562583d73730 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60cea00640 1 -- 172.21.15.49:0/1593585891 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d00db020 con 0x562583d73730 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60cea00640 1 -- 172.21.15.49:0/1593585891 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60d003c040 con 0x562583d73730 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60eabd5740 10 client.14901 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60eabd5740 10 client.14901 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T06:22:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60eabd5740 20 client.14901 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562583de8420 0x562583de4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60eabd5740 4 client.14901 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60eabd5740 1 -- 172.21.15.49:0/1593585891 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562583a2c270 con 0x562583de8420 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.976+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562583de8420 0x562583de4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60d7e00640 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562583de8420 0x562583de4880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81751ac6588f5f91 server_cookie=b77e0f4e0c481ff4 in_seq=0 out_seq=0 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60cea00640 10 client.14901 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60cea00640 1 -- 172.21.15.49:0/1593585891 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x562583a2c270 con 0x562583de8420 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60cea00640 10 client.14901 handle_command_reply: tid=0 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 client.14901 shutdown 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 -- 172.21.15.49:0/1593585891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 msgr2=0x562583d88620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d88620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60d00b8f50 tx=0x7f60d00c5010 comp rx=0 tx=0).stop 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 -- 172.21.15.49:0/1593585891 shutdown_connections 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562583de8420 0x562583de4880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562583d79290 0x562583d85110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562583d74120 0x562583d88b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 --2- 172.21.15.49:0/1593585891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562583d73730 0x562583d88620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60eabd5740 1 -- 172.21.15.49:0/1593585891 >> 172.21.15.49:0/1593585891 conn(0x562583d6eca0 msgr2=0x562583d6d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.977+0000 7f60cea00640 0 client.14901 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60eabd5740 1 -- 172.21.15.49:0/1593585891 shutdown_connections 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60eabd5740 1 -- 172.21.15.49:0/1593585891 wait complete. 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60eabd5740 20 client.14901 trim_cache size 0 max 16384 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-02-26T06:22:20.983 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24559, 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 104.906747482, 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 126.795784712 2024-02-26T06:22:20.984 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:22:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 -- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f60c00a20c0 msgr2=0x7f60c00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f60c00a20c0 0x7f60c00a4580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x562583d74980 tx=0x7f60d8009500 comp rx=0 tx=0).stop 2024-02-26T06:22:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 -- 172.21.15.49:0/1315587395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60e0101060 msgr2=0x7f60e0197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60e0101060 0x7f60e0197110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60d0005ab0 tx=0x7f60d0005e50 comp rx=0 tx=0).stop 2024-02-26T06:22:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 -- 172.21.15.49:0/1315587395 shutdown_connections 2024-02-26T06:22:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f60c00a20c0 0x7f60c00a4580 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60e0102c10 0x7f60e019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e0102250 0x7f60e0197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 --2- 172.21.15.49:0/1315587395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60e0101060 0x7f60e0197110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:20.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.978+0000 7f60e6600640 1 -- 172.21.15.49:0/1315587395 >> 172.21.15.49:0/1315587395 conn(0x7f60e00fc5f0 msgr2=0x7f60e0105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:20.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.979+0000 7f60e6600640 1 -- 172.21.15.49:0/1315587395 shutdown_connections 2024-02-26T06:22:20.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:20.979+0000 7f60e6600640 1 -- 172.21.15.49:0/1315587395 wait complete. 2024-02-26T06:22:20.997 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T06:22:21.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.209+0000 7efc5f200640 1 -- 172.21.15.49:0/123838400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58109350 msgr2=0x7efc5810fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.209+0000 7efc5f200640 1 --2- 172.21.15.49:0/123838400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58109350 0x7efc5810fbe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efc50009ca0 tx=0x7efc5002c100 comp rx=0 tx=0).stop 2024-02-26T06:22:21.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.210+0000 7efc5f200640 1 -- 172.21.15.49:0/123838400 shutdown_connections 2024-02-26T06:22:21.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.210+0000 7efc5f200640 1 --2- 172.21.15.49:0/123838400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc58108990 0x7efc58108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.210+0000 7efc5f200640 1 --2- 172.21.15.49:0/123838400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58109350 0x7efc5810fbe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.210+0000 7efc5f200640 1 --2- 172.21.15.49:0/123838400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc581077a0 0x7efc58107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.210+0000 7efc5f200640 1 -- 172.21.15.49:0/123838400 >> 172.21.15.49:0/123838400 conn(0x7efc58076740 msgr2=0x7efc58078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:21.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.210+0000 7efc5f200640 1 -- 172.21.15.49:0/123838400 shutdown_connections 2024-02-26T06:22:21.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.210+0000 7efc5f200640 1 -- 172.21.15.49:0/123838400 wait complete. 2024-02-26T06:22:21.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5f200640 1 Processor -- start 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5f200640 1 -- start start 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc581077a0 0x7efc5819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc58108990 0x7efc5819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc58109350 0x7efc581a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581a52d0 con 0x7efc581077a0 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581a5440 con 0x7efc58109350 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581a5600 con 0x7efc58108990 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc57e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc581077a0 0x7efc5819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc57e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc581077a0 0x7efc5819d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47186/0 (socket says 172.21.15.49:47186) 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc57e00640 1 -- 172.21.15.49:0/1333387111 learned_addr learned my addr 172.21.15.49:0/1333387111 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc57400640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc58108990 0x7efc5819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.211+0000 7efc5d200640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc58109350 0x7efc581a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc57e00640 1 -- 172.21.15.49:0/1333387111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc58108990 msgr2=0x7efc5819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc57e00640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc58108990 0x7efc5819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc57e00640 1 -- 172.21.15.49:0/1333387111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc58109350 msgr2=0x7efc581a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc57e00640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc58109350 0x7efc581a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc57e00640 1 -- 172.21.15.49:0/1333387111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50009900 con 0x7efc581077a0 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc57400640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc58108990 0x7efc5819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc57e00640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc581077a0 0x7efc5819d7e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc4800e9c0 tx=0x7efc4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc54c00640 1 -- 172.21.15.49:0/1333387111 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc480096e0 con 0x7efc581077a0 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc54c00640 1 -- 172.21.15.49:0/1333387111 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc4800cd80 con 0x7efc581077a0 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc5f200640 1 -- 172.21.15.49:0/1333387111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc581a58e0 con 0x7efc581077a0 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.212+0000 7efc54c00640 1 -- 172.21.15.49:0/1333387111 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48010430 con 0x7efc581077a0 2024-02-26T06:22:21.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.213+0000 7efc5f200640 1 -- 172.21.15.49:0/1333387111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc581a5da0 con 0x7efc581077a0 2024-02-26T06:22:21.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.215+0000 7efc621f3740 1 Processor -- start 2024-02-26T06:22:21.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc621f3740 1 -- start start 2024-02-26T06:22:21.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc54c00640 1 -- 172.21.15.49:0/1333387111 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efc48003680 con 0x7efc581077a0 2024-02-26T06:22:21.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc621f3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc621f3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433af40 0x562ef433fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc621f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef4342540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc621f3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef42a9b00 con 0x562ef433af40 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc621f3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef42a7030 con 0x562ef433a550 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc621f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef42a2810 con 0x562ef43400b0 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc57e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433af40 0x562ef433fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc5d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc5d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39058/0 (socket says 172.21.15.49:39058) 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.216+0000 7efc5d200640 1 -- 172.21.15.49:0/2295548304 learned_addr learned my addr 172.21.15.49:0/2295548304 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.217+0000 7efc57400640 1 --2- 172.21.15.49:0/2295548304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef4342540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.217+0000 7efc5d200640 1 -- 172.21.15.49:0/2295548304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 msgr2=0x562ef4342540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.217+0000 7efc5d200640 1 --2- 172.21.15.49:0/2295548304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef4342540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.217+0000 7efc5d200640 1 -- 172.21.15.49:0/2295548304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433af40 msgr2=0x562ef433fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.217+0000 7efc5d200640 1 --2- 172.21.15.49:0/2295548304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433af40 0x562ef433fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.217+0000 7efc5d200640 1 -- 172.21.15.49:0/2295548304 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ef426a490 con 0x562ef433a550 2024-02-26T06:22:21.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.217+0000 7efc5d200640 1 --2- 172.21.15.49:0/2295548304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433a970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc50009ca0 tx=0x7efc500093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfafa75ccb6dc9b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:21.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc54c00640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc3c0a20c0 0x7efc3c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc4d600640 1 -- 172.21.15.49:0/2295548304 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500327d0 con 0x562ef433a550 2024-02-26T06:22:21.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc4d600640 1 -- 172.21.15.49:0/2295548304 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc50032930 con 0x562ef433a550 2024-02-26T06:22:21.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc4d600640 1 -- 172.21.15.49:0/2295548304 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50004040 con 0x562ef433a550 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc54c00640 1 -- 172.21.15.49:0/1333387111 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7efc48014070 con 0x7efc581077a0 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc57e00640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc3c0a20c0 0x7efc3c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc621f3740 1 -- 172.21.15.49:0/2295548304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 msgr2=0x562ef433a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc621f3740 1 --2- 172.21.15.49:0/2295548304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433a970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc50009ca0 tx=0x7efc500093e0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.218+0000 7efc57e00640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc3c0a20c0 0x7efc3c0a4580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc480ce130 tx=0x7efc480d18d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 -- 172.21.15.49:0/2295548304 shutdown_connections 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 --2- 172.21.15.49:0/2295548304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef4342540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 --2- 172.21.15.49:0/2295548304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433af40 0x562ef433fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 --2- 172.21.15.49:0/2295548304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433a970 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 -- 172.21.15.49:0/2295548304 >> 172.21.15.49:0/2295548304 conn(0x562ef4335ac0 msgr2=0x562ef4335ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 -- 172.21.15.49:0/2295548304 shutdown_connections 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 -- 172.21.15.49:0/2295548304 wait complete. 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 -- start start 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.219+0000 7efc621f3740 1 Processor -- start 2024-02-26T06:22:21.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc621f3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef4344380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc621f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 0x562ef43448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc621f3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef434b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc621f3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef4247e70 con 0x562ef43400b0 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc621f3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef4247b10 con 0x562ef433a550 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc621f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef3feaf20 con 0x562ef433af40 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc621f3740 10 client.0 _ll_register_callbacks cb 0x562ef425b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef434b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef434b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47214/0 (socket says 172.21.15.49:47214) 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 -- 172.21.15.49:0/3206296951 learned_addr learned my addr 172.21.15.49:0/3206296951 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc57400640 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 0x562ef43448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef4344380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 -- 172.21.15.49:0/3206296951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 msgr2=0x562ef43448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 0x562ef43448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 -- 172.21.15.49:0/3206296951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 msgr2=0x562ef4344380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef4344380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.220+0000 7efc5d200640 1 -- 172.21.15.49:0/3206296951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc581104e0 con 0x562ef43400b0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc5d200640 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef434b900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc50032530 tx=0x7efc5002c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fcd010c5905243 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc42000640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc42000640 1 -- 172.21.15.49:0/3206296951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500039b0 con 0x562ef43400b0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc42000640 1 -- 172.21.15.49:0/3206296951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7efc50032d90 con 0x562ef43400b0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc621f3740 1 -- 172.21.15.49:0/3206296951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ef42c6c10 con 0x562ef43400b0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc42000640 1 -- 172.21.15.49:0/3206296951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc50004180 con 0x562ef43400b0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc621f3740 10 client.14913 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc621f3740 1 -- 172.21.15.49:0/3206296951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562ef42a0e90 con 0x562ef43400b0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.221+0000 7efc42000640 1 -- 172.21.15.49:0/3206296951 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562ef42c6c10 con 0x562ef43400b0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.222+0000 7efc42000640 1 -- 172.21.15.49:0/3206296951 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efc5004aa50 con 0x562ef43400b0 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.222+0000 7efc621f3740 10 client.14913 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.222+0000 7efc621f3740 10 client.14913 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T06:22:21.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.222+0000 7efc621f3740 20 client.14913 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:21.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.222+0000 7efc621f3740 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562ef434fa60 0x562ef4354a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.222+0000 7efc621f3740 4 client.14913 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:22:21.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.222+0000 7efc621f3740 1 -- 172.21.15.49:0/3206296951 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ef3fa26c0 con 0x562ef434fa60 2024-02-26T06:22:21.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.222+0000 7efc57400640 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562ef434fa60 0x562ef4354a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.223+0000 7efc57400640 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562ef434fa60 0x562ef4354a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ed5e5a77e7eb9b server_cookie=1c3677d5871142ce in_seq=0 out_seq=0 2024-02-26T06:22:21.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.223+0000 7efc42000640 10 client.14913 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:22:21.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.223+0000 7efc42000640 1 -- 172.21.15.49:0/3206296951 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ef3fa26c0 con 0x562ef434fa60 2024-02-26T06:22:21.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc42000640 10 client.14913 handle_command_reply: tid=0 2024-02-26T06:22:21.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 client.14913 shutdown 2024-02-26T06:22:21.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 -- 172.21.15.49:0/3206296951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 msgr2=0x562ef434b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef434b900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc50032530 tx=0x7efc5002c100 comp rx=0 tx=0).stop 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 -- 172.21.15.49:0/3206296951 shutdown_connections 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562ef434fa60 0x562ef4354a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef434b900 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 0x562ef43448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 --2- 172.21.15.49:0/3206296951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef4344380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 -- 172.21.15.49:0/3206296951 >> 172.21.15.49:0/3206296951 conn(0x562ef4335ac0 msgr2=0x562ef4334de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc42000640 0 client.14913 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 -- 172.21.15.49:0/3206296951 shutdown_connections 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 1 -- 172.21.15.49:0/3206296951 wait complete. 2024-02-26T06:22:21.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.224+0000 7efc621f3740 20 client.14913 trim_cache size 0 max 16384 2024-02-26T06:22:21.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.240+0000 7efc621f3740 1 Processor -- start 2024-02-26T06:22:21.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc621f3740 1 -- start start 2024-02-26T06:22:21.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc621f3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc621f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 0x562ef434b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc621f3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef4335ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc621f3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef4331060 con 0x562ef43400b0 2024-02-26T06:22:21.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc621f3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef43426d0 con 0x562ef433a550 2024-02-26T06:22:21.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc621f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef4350e80 con 0x562ef433af40 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef4335ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef4335ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47220/0 (socket says 172.21.15.49:47220) 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 -- 172.21.15.49:0/3631150855 learned_addr learned my addr 172.21.15.49:0/3631150855 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc57400640 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 0x562ef434b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 -- 172.21.15.49:0/3631150855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 msgr2=0x562ef434b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 0x562ef434b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 -- 172.21.15.49:0/3631150855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 msgr2=0x562ef433b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.241+0000 7efc5d200640 1 -- 172.21.15.49:0/3631150855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ef423d880 con 0x562ef43400b0 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.242+0000 7efc5d200640 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef4335ea0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc50003d80 tx=0x7efc50036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a124d038eb33cc50 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.242+0000 7efc4cc00640 1 -- 172.21.15.49:0/3631150855 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50066070 con 0x562ef43400b0 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.242+0000 7efc4cc00640 1 -- 172.21.15.49:0/3631150855 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc5004f070 con 0x562ef43400b0 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.242+0000 7efc4cc00640 1 -- 172.21.15.49:0/3631150855 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50003af0 con 0x562ef43400b0 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 -- 172.21.15.49:0/3631150855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 msgr2=0x562ef4335ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef4335ea0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc50003d80 tx=0x7efc50036040 comp rx=0 tx=0).stop 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 -- 172.21.15.49:0/3631150855 shutdown_connections 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef43400b0 0x562ef4335ea0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef433af40 0x562ef434b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 --2- 172.21.15.49:0/3631150855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433a550 0x562ef433b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 -- 172.21.15.49:0/3631150855 >> 172.21.15.49:0/3631150855 conn(0x562ef4335ac0 msgr2=0x562ef4359320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 -- 172.21.15.49:0/3631150855 shutdown_connections 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 -- 172.21.15.49:0/3631150855 wait complete. 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 -- start start 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.243+0000 7efc621f3740 1 Processor -- start 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc621f3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433a550 0x562ef43485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc621f3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433af40 0x562ef4348b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc621f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef43a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc621f3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef43490c0 con 0x562ef433a550 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc621f3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef4349230 con 0x562ef433af40 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc621f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef43a6cc0 con 0x562ef43400b0 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc621f3740 10 client.0 _ll_register_callbacks cb 0x7efc480d17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc5d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433a550 0x562ef43485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc57400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433af40 0x562ef4348b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc5d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433a550 0x562ef43485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47222/0 (socket says 172.21.15.49:47222) 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc5d200640 1 -- 172.21.15.49:0/2407136880 learned_addr learned my addr 172.21.15.49:0/2407136880 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc5d200640 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef43a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc57400640 1 -- 172.21.15.49:0/2407136880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 msgr2=0x562ef43a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc57400640 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef43a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc57400640 1 -- 172.21.15.49:0/2407136880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433a550 msgr2=0x562ef43485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc57400640 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433a550 0x562ef43485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc57400640 1 -- 172.21.15.49:0/2407136880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ef42a0e90 con 0x562ef433af40 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.244+0000 7efc5d200640 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef43a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc57400640 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433af40 0x562ef4348b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x562ef4351fe0 tx=0x7efc44012c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6754c871c115b78b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc42000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc42000640 1 -- 172.21.15.49:0/2407136880 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44016040 con 0x562ef433af40 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc42000640 1 -- 172.21.15.49:0/2407136880 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7efc44005020 con 0x562ef433af40 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc621f3740 1 -- 172.21.15.49:0/2407136880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ef4350fc0 con 0x562ef433af40 2024-02-26T06:22:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc42000640 1 -- 172.21.15.49:0/2407136880 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc440051e0 con 0x562ef433af40 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc621f3740 10 client.25022 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc621f3740 1 -- 172.21.15.49:0/2407136880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562ef42634e0 con 0x562ef433af40 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.245+0000 7efc42000640 1 -- 172.21.15.49:0/2407136880 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562ef4350fc0 con 0x562ef433af40 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc42000640 1 -- 172.21.15.49:0/2407136880 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efc440314b0 con 0x562ef433af40 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc621f3740 10 client.25022 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc621f3740 10 client.25022 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc621f3740 20 client.25022 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc621f3740 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562ef43ac0a0 0x562ef43a8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc621f3740 4 client.25022 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc621f3740 1 -- 172.21.15.49:0/2407136880 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7efc50032930 con 0x562ef43ac0a0 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc5d200640 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562ef43ac0a0 0x562ef43a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc5d200640 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562ef43ac0a0 0x562ef43a8520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45c50dbe012b9ca server_cookie=da50b99b6a53a9e0 in_seq=0 out_seq=0 2024-02-26T06:22:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.246+0000 7efc42000640 10 client.25022 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc42000640 1 -- 172.21.15.49:0/2407136880 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7efc50032930 con 0x562ef43ac0a0 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc42000640 10 client.25022 handle_command_reply: tid=0 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 client.25022 shutdown 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 -- 172.21.15.49:0/2407136880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433af40 msgr2=0x562ef4348b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433af40 0x562ef4348b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x562ef4351fe0 tx=0x7efc44012c80 comp rx=0 tx=0).stop 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 -- 172.21.15.49:0/2407136880 shutdown_connections 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562ef43ac0a0 0x562ef43a8520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562ef433af40 0x562ef4348b30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc42000640 0 client.25022 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef43400b0 0x562ef43a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 --2- 172.21.15.49:0/2407136880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ef433a550 0x562ef43485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 -- 172.21.15.49:0/2407136880 >> 172.21.15.49:0/2407136880 conn(0x562ef4335ac0 msgr2=0x562ef4338d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 -- 172.21.15.49:0/2407136880 shutdown_connections 2024-02-26T06:22:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 1 -- 172.21.15.49:0/2407136880 wait complete. 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.247+0000 7efc621f3740 20 client.25022 trim_cache size 0 max 16384 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24572, 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 108.69452724600001, 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-02-26T06:22:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-02-26T06:22:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-02-26T06:22:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 115.797192568 2024-02-26T06:22:21.254 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:22:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 -- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc3c0a20c0 msgr2=0x7efc3c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc3c0a20c0 0x7efc3c0a4580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc480ce130 tx=0x7efc480d18d0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 -- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc581077a0 msgr2=0x7efc5819d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc581077a0 0x7efc5819d7e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc4800e9c0 tx=0x7efc4800ee80 comp rx=0 tx=0).stop 2024-02-26T06:22:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 -- 172.21.15.49:0/1333387111 shutdown_connections 2024-02-26T06:22:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc3c0a20c0 0x7efc3c0a4580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc58109350 0x7efc581a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc58108990 0x7efc5819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 --2- 172.21.15.49:0/1333387111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc581077a0 0x7efc5819d7e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 -- 172.21.15.49:0/1333387111 >> 172.21.15.49:0/1333387111 conn(0x7efc58076740 msgr2=0x7efc5810bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 -- 172.21.15.49:0/1333387111 shutdown_connections 2024-02-26T06:22:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:21.248+0000 7efc5f200640 1 -- 172.21.15.49:0/1333387111 wait complete. 2024-02-26T06:22:21.262 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:21 smithi049 ceph-mon[31696]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:21.262 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:21 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/74803065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:21.264 INFO:tasks.cephadm:Setup complete, yielding 2024-02-26T06:22:21.265 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-26T06:22:21.274 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:22:21.275 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- bash -c 'ceph orch status' 2024-02-26T06:22:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:21 smithi080 ceph-mon[36206]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:21 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/74803065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:21 smithi115 ceph-mon[36426]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:21 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/74803065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:21.746 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:23.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:23 smithi049 ceph-mon[31696]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:23 smithi080 ceph-mon[36206]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:23 smithi115 ceph-mon[36426]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.146+0000 7f099be00700 1 -- 172.21.15.49:0/3158774698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f099c0746d0 msgr2=0x7f099c074b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.146+0000 7f099be00700 1 --2- 172.21.15.49:0/3158774698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f099c0746d0 0x7f099c074b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0988009990 tx=0x7f0988009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.147+0000 7f099be00700 1 -- 172.21.15.49:0/3158774698 shutdown_connections 2024-02-26T06:22:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.147+0000 7f099be00700 1 --2- 172.21.15.49:0/3158774698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f099c109ab0 0x7f099c10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.147+0000 7f099be00700 1 --2- 172.21.15.49:0/3158774698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f099c075ca0 0x7f099c074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.147+0000 7f099be00700 1 --2- 172.21.15.49:0/3158774698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f099c0746d0 0x7f099c074b50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.147+0000 7f099be00700 1 -- 172.21.15.49:0/3158774698 >> 172.21.15.49:0/3158774698 conn(0x7f099c0fd180 msgr2=0x7f099c0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.147+0000 7f099be00700 1 -- 172.21.15.49:0/3158774698 shutdown_connections 2024-02-26T06:22:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.147+0000 7f099be00700 1 -- 172.21.15.49:0/3158774698 wait complete. 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.148+0000 7f099be00700 1 Processor -- start 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.148+0000 7f099be00700 1 -- start start 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.148+0000 7f099be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f099c0746d0 0x7f099c199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.148+0000 7f099be00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f099c075ca0 0x7f099c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.148+0000 7f099be00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f099c109ab0 0x7f099c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.148+0000 7f099be00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1a1900 con 0x7f099c075ca0 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.148+0000 7f099be00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1a1a40 con 0x7f099c109ab0 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.148+0000 7f099be00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1a1c00 con 0x7f099c0746d0 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f099c0746d0 0x7f099c199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099a000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f099c075ca0 0x7f099c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f099c0746d0 0x7f099c199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34834/0 (socket says 172.21.15.49:34834) 2024-02-26T06:22:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 -- 172.21.15.49:0/717525030 learned_addr learned my addr 172.21.15.49:0/717525030 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099b400700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f099c109ab0 0x7f099c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 -- 172.21.15.49:0/717525030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f099c109ab0 msgr2=0x7f099c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f099c109ab0 0x7f099c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 -- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f099c075ca0 msgr2=0x7f099c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f099c075ca0 0x7f099c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 -- 172.21.15.49:0/717525030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994009c40 con 0x7f099c0746d0 2024-02-26T06:22:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.149+0000 7f099aa00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f099c0746d0 0x7f099c199e00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f098c00e970 tx=0x7f098c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.150+0000 7f0993400700 1 -- 172.21.15.49:0/717525030 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c009790 con 0x7f099c0746d0 2024-02-26T06:22:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.150+0000 7f0993400700 1 -- 172.21.15.49:0/717525030 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f098c00c7c0 con 0x7f099c0746d0 2024-02-26T06:22:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.150+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09880095f0 con 0x7f099c0746d0 2024-02-26T06:22:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.150+0000 7f0993400700 1 -- 172.21.15.49:0/717525030 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0055c0 con 0x7f099c0746d0 2024-02-26T06:22:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.150+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c1a21a0 con 0x7f099c0746d0 2024-02-26T06:22:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.151+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c108870 con 0x7f099c0746d0 2024-02-26T06:22:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.153+0000 7f0993400700 1 -- 172.21.15.49:0/717525030 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f098c00c930 con 0x7f099c0746d0 2024-02-26T06:22:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.155+0000 7f0993400700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09800a2150 0x7f09800a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.155+0000 7f0993400700 1 -- 172.21.15.49:0/717525030 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f098c014070 con 0x7f099c0746d0 2024-02-26T06:22:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.155+0000 7f099a000700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09800a2150 0x7f09800a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.156+0000 7f099a000700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09800a2150 0x7f09800a4610 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f098800b6f0 tx=0x7f0988005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.157+0000 7f0993400700 1 -- 172.21.15.49:0/717525030 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f098c096db0 con 0x7f099c0746d0 2024-02-26T06:22:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.407+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f099c02d8d0 con 0x7f09800a2150 2024-02-26T06:22:24.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.408+0000 7f0993400700 1 -- 172.21.15.49:0/717525030 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f099c02d8d0 con 0x7f09800a2150 2024-02-26T06:22:24.409 INFO:teuthology.orchestra.run.smithi049.stdout:Backend: cephadm 2024-02-26T06:22:24.409 INFO:teuthology.orchestra.run.smithi049.stdout:Available: Yes 2024-02-26T06:22:24.409 INFO:teuthology.orchestra.run.smithi049.stdout:Paused: No 2024-02-26T06:22:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.411+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09800a2150 msgr2=0x7f09800a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.411+0000 7f099be00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09800a2150 0x7f09800a4610 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f098800b6f0 tx=0x7f0988005fb0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.411+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f099c0746d0 msgr2=0x7f099c199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.411+0000 7f099be00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f099c0746d0 0x7f099c199e00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f098c00e970 tx=0x7f098c00ec80 comp rx=0 tx=0).stop 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.412+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 shutdown_connections 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.412+0000 7f099be00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f09800a2150 0x7f09800a4610 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.412+0000 7f099be00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f099c109ab0 0x7f099c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.412+0000 7f099be00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f099c0746d0 0x7f099c199e00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.412+0000 7f099be00700 1 --2- 172.21.15.49:0/717525030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f099c075ca0 0x7f099c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.412+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 >> 172.21.15.49:0/717525030 conn(0x7f099c0fd180 msgr2=0x7f099c1059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.412+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 shutdown_connections 2024-02-26T06:22:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:24.412+0000 7f099be00700 1 -- 172.21.15.49:0/717525030 wait complete. 2024-02-26T06:22:25.275 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-26T06:22:25.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:25 smithi049 ceph-mon[31696]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:22:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:25 smithi080 ceph-mon[36206]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:22:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:25 smithi115 ceph-mon[36426]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:22:25.754 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:26.167 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:26 smithi049 ceph-mon[31696]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:26.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:26 smithi080 ceph-mon[36206]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:26.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:26 smithi115 ceph-mon[36426]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:27.276 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:27 smithi049 ceph-mon[31696]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:27.481 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:27 smithi080 ceph-mon[36206]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:27 smithi115 ceph-mon[36426]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 -- 172.21.15.49:0/1111204308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 msgr2=0x7fb350108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 --2- 172.21.15.49:0/1111204308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 0x7fb350108710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb344009990 tx=0x7fb344009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 -- 172.21.15.49:0/1111204308 shutdown_connections 2024-02-26T06:22:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 --2- 172.21.15.49:0/1111204308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350109e90 0x7fb350110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 --2- 172.21.15.49:0/1111204308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 0x7fb350108710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 --2- 172.21.15.49:0/1111204308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3501094f0 0x7fb350109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 -- 172.21.15.49:0/1111204308 >> 172.21.15.49:0/1111204308 conn(0x7fb3500772c0 msgr2=0x7fb350079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 -- 172.21.15.49:0/1111204308 shutdown_connections 2024-02-26T06:22:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.131+0000 7fb357400700 1 -- 172.21.15.49:0/1111204308 wait complete. 2024-02-26T06:22:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.132+0000 7fb357400700 1 Processor -- start 2024-02-26T06:22:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.132+0000 7fb357400700 1 -- start start 2024-02-26T06:22:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb357400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 0x7fb35019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb357400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3501094f0 0x7fb35019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb357400700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350109e90 0x7fb3501a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb357400700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3501a5e10 con 0x7fb3501094f0 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb357400700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3501a5f50 con 0x7fb350109e90 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb357400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3501a60e0 con 0x7fb3501082f0 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb34fe00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3501094f0 0x7fb35019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb354a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 0x7fb35019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb34fe00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3501094f0 0x7fb35019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47266/0 (socket says 172.21.15.49:47266) 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb34fe00700 1 -- 172.21.15.49:0/3992580979 learned_addr learned my addr 172.21.15.49:0/3992580979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.133+0000 7fb355400700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350109e90 0x7fb3501a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb354a00700 1 -- 172.21.15.49:0/3992580979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350109e90 msgr2=0x7fb3501a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb354a00700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350109e90 0x7fb3501a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb354a00700 1 -- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3501094f0 msgr2=0x7fb35019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb354a00700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3501094f0 0x7fb35019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb354a00700 1 -- 172.21.15.49:0/3992580979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c009520 con 0x7fb3501082f0 2024-02-26T06:22:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb354a00700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 0x7fb35019e280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3440098e0 tx=0x7fb344009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb34d600700 1 -- 172.21.15.49:0/3992580979 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34401d070 con 0x7fb3501082f0 2024-02-26T06:22:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb34d600700 1 -- 172.21.15.49:0/3992580979 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb344021410 con 0x7fb3501082f0 2024-02-26T06:22:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb34d600700 1 -- 172.21.15.49:0/3992580979 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344017650 con 0x7fb3501082f0 2024-02-26T06:22:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.134+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3440095f0 con 0x7fb3501082f0 2024-02-26T06:22:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.135+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3501a6590 con 0x7fb3501082f0 2024-02-26T06:22:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.136+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35010f6b0 con 0x7fb3501082f0 2024-02-26T06:22:28.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.138+0000 7fb34d600700 1 -- 172.21.15.49:0/3992580979 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb344021580 con 0x7fb3501082f0 2024-02-26T06:22:28.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.139+0000 7fb34d600700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb3380a21d0 0x7fb3380a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:28.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.139+0000 7fb34d600700 1 -- 172.21.15.49:0/3992580979 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb3440ceeb0 con 0x7fb3501082f0 2024-02-26T06:22:28.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.139+0000 7fb34fe00700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb3380a21d0 0x7fb3380a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:28.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.140+0000 7fb34fe00700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb3380a21d0 0x7fb3380a4690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009ef0 tx=0x7fb33c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:28.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.144+0000 7fb34d600700 1 -- 172.21.15.49:0/3992580979 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb344097ea0 con 0x7fb3501082f0 2024-02-26T06:22:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.378+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb35002d990 con 0x7fb3380a21d0 2024-02-26T06:22:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.417+0000 7fb34d600700 1 -- 172.21.15.49:0/3992580979 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7fb35002d990 con 0x7fb3380a21d0 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.a smithi049 running (2m) 26s ago 2m 12.2M - 19.0.0-1514-g25296a8f a4519cb49bed 5b151ea5f4ef 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.b smithi080 running (2m) 53s ago 2m 25.5M - 19.0.0-1514-g25296a8f a4519cb49bed c493e60d4f9a 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.c smithi115 running (2m) 53s ago 2m 17.6M - 19.0.0-1514-g25296a8f a4519cb49bed cd39c9578aae 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.d smithi049 running (2m) 26s ago 2m 12.3M - 19.0.0-1514-g25296a8f a4519cb49bed 7ce57060d4c9 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.e smithi080 running (2m) 53s ago 2m 16.9M - 19.0.0-1514-g25296a8f a4519cb49bed 24588b772ecb 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.f smithi115 running (2m) 53s ago 2m 20.3M - 19.0.0-1514-g25296a8f a4519cb49bed 0eae69f6ee63 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.g smithi049 running (2m) 26s ago 2m 12.2M - 19.0.0-1514-g25296a8f a4519cb49bed bde75ebc04e5 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.h smithi080 running (2m) 53s ago 2m 18.3M - 19.0.0-1514-g25296a8f a4519cb49bed 8469b681a6d0 2024-02-26T06:22:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:mds.i smithi115 running (2m) 53s ago 2m 17.6M - 19.0.0-1514-g25296a8f a4519cb49bed c07093591ce2 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mds.j smithi049 running (2m) 26s ago 2m 12.4M - 19.0.0-1514-g25296a8f a4519cb49bed 2c863d7af900 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mds.k smithi080 running (2m) 53s ago 2m 13.1M - 19.0.0-1514-g25296a8f a4519cb49bed b45a97587864 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mds.l smithi115 running (2m) 53s ago 2m 15.3M - 19.0.0-1514-g25296a8f a4519cb49bed 8283382f724e 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.x smithi049 *:9283,8765 running (11m) 26s ago 11m 515M - 19.0.0-1514-g25296a8f a4519cb49bed 03ccb02c6201 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.y smithi080 *:8443,8765 running (8m) 53s ago 8m 453M - 19.0.0-1514-g25296a8f a4519cb49bed b645b34ca027 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.z smithi115 *:8443,8765 running (8m) 53s ago 8m 451M - 19.0.0-1514-g25296a8f a4519cb49bed 4250a23f6741 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a smithi049 running (11m) 26s ago 11m 46.7M 2048M 19.0.0-1514-g25296a8f a4519cb49bed 12392c53e525 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b smithi080 running (8m) 53s ago 8m 41.8M 2048M 19.0.0-1514-g25296a8f a4519cb49bed 2b5037e56778 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c smithi115 running (9m) 53s ago 9m 43.0M 2048M 19.0.0-1514-g25296a8f a4519cb49bed a8521d9b0855 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (8m) 26s ago 8m 43.9M 4272M 19.0.0-1514-g25296a8f a4519cb49bed e0c4b2c27bef 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi080 running (7m) 53s ago 7m 57.7M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 481b50436af9 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi115 running (7m) 53s ago 7m 46.0M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 93deb98ccbce 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi049 running (6m) 26s ago 6m 44.1M 4272M 19.0.0-1514-g25296a8f a4519cb49bed 6ece9138e100 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi080 running (6m) 53s ago 6m 41.2M 4270M 19.0.0-1514-g25296a8f a4519cb49bed c8e3fca96d65 2024-02-26T06:22:28.419 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi115 running (5m) 53s ago 5m 54.8M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 1d8f07aef598 2024-02-26T06:22:28.420 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 smithi049 running (5m) 26s ago 5m 49.9M 4272M 19.0.0-1514-g25296a8f a4519cb49bed b5e6e42a9659 2024-02-26T06:22:28.420 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 smithi080 running (4m) 53s ago 4m 43.5M 4270M 19.0.0-1514-g25296a8f a4519cb49bed a7eec9527b6a 2024-02-26T06:22:28.420 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 smithi115 running (4m) 53s ago 4m 43.0M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 6783ff727ff6 2024-02-26T06:22:28.420 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 smithi049 running (3m) 26s ago 3m 51.8M 4272M 19.0.0-1514-g25296a8f a4519cb49bed 55697ca51998 2024-02-26T06:22:28.420 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 smithi080 running (3m) 53s ago 3m 53.0M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 76464ee1ac5d 2024-02-26T06:22:28.420 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 smithi115 running (3m) 53s ago 3m 50.2M 4270M 19.0.0-1514-g25296a8f a4519cb49bed ebb28a3bb5d1 2024-02-26T06:22:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.420+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb3380a21d0 msgr2=0x7fb3380a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.420+0000 7fb357400700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb3380a21d0 0x7fb3380a4690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009ef0 tx=0x7fb33c009260 comp rx=0 tx=0).stop 2024-02-26T06:22:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.420+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 msgr2=0x7fb35019e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.420+0000 7fb357400700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 0x7fb35019e280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3440098e0 tx=0x7fb344009580 comp rx=0 tx=0).stop 2024-02-26T06:22:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.421+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 shutdown_connections 2024-02-26T06:22:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.421+0000 7fb357400700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb3380a21d0 0x7fb3380a4690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.421+0000 7fb357400700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350109e90 0x7fb3501a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.421+0000 7fb357400700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3501082f0 0x7fb35019e280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.421+0000 7fb357400700 1 --2- 172.21.15.49:0/3992580979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3501094f0 0x7fb35019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.421+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 >> 172.21.15.49:0/3992580979 conn(0x7fb3500772c0 msgr2=0x7fb35010c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.421+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 shutdown_connections 2024-02-26T06:22:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:28.421+0000 7fb357400700 1 -- 172.21.15.49:0/3992580979 wait complete. 2024-02-26T06:22:29.241 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-26T06:22:29.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:29 smithi049 ceph-mon[31696]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:29.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:29 smithi080 ceph-mon[36206]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:29.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:29 smithi115 ceph-mon[36426]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:29.724 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:30.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:30 smithi049 ceph-mon[31696]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:30.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:30 smithi080 ceph-mon[36206]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:30.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:30 smithi115 ceph-mon[36426]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:31 smithi049 ceph-mon[31696]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:31.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:31 smithi080 ceph-mon[36206]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:31.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:31 smithi115 ceph-mon[36426]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.933+0000 7f8fe3c00700 1 -- 172.21.15.49:0/984166683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 msgr2=0x7f8fdc109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.933+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/984166683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 0x7f8fdc109950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009990 tx=0x7f8fc8009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.933+0000 7f8fe3c00700 1 -- 172.21.15.49:0/984166683 shutdown_connections 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.933+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/984166683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fdc109e90 0x7f8fdc110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.933+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/984166683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fdc1082f0 0x7f8fdc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.933+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/984166683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 0x7f8fdc109950 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.933+0000 7f8fe3c00700 1 -- 172.21.15.49:0/984166683 >> 172.21.15.49:0/984166683 conn(0x7f8fdc0772c0 msgr2=0x7f8fdc079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.934+0000 7f8fe3c00700 1 -- 172.21.15.49:0/984166683 shutdown_connections 2024-02-26T06:22:31.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.934+0000 7f8fe3c00700 1 -- 172.21.15.49:0/984166683 wait complete. 2024-02-26T06:22:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.935+0000 7f8fe3c00700 1 Processor -- start 2024-02-26T06:22:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.935+0000 7f8fe3c00700 1 -- start start 2024-02-26T06:22:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fe3c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fdc1082f0 0x7f8fdc19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fe3c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 0x7f8fdc19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fe3c00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fdc109e90 0x7f8fdc1a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fe3c00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1a5e00 con 0x7f8fdc1082f0 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fe3c00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1a5f40 con 0x7f8fdc109e90 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fe3c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1a60d0 con 0x7f8fdc1094f0 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fdbe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 0x7f8fdc19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fdbe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 0x7f8fdc19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46546/0 (socket says 172.21.15.49:46546) 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fdbe00700 1 -- 172.21.15.49:0/3561396898 learned_addr learned my addr 172.21.15.49:0/3561396898 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fe1200700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fdc1082f0 0x7f8fdc19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.936+0000 7f8fe1c00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fdc109e90 0x7f8fdc1a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fdbe00700 1 -- 172.21.15.49:0/3561396898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fdc109e90 msgr2=0x7f8fdc1a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fdbe00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fdc109e90 0x7f8fdc1a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fdbe00700 1 -- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fdc1082f0 msgr2=0x7f8fdc19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fdbe00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fdc1082f0 0x7f8fdc19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fdbe00700 1 -- 172.21.15.49:0/3561396898 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc80095f0 con 0x7f8fdc1094f0 2024-02-26T06:22:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fdbe00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 0x7f8fdc19e7b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800b720 tx=0x7f8fc8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fd9600700 1 -- 172.21.15.49:0/3561396898 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc801d070 con 0x7f8fdc1094f0 2024-02-26T06:22:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fd9600700 1 -- 172.21.15.49:0/3561396898 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8021410 con 0x7f8fdc1094f0 2024-02-26T06:22:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fd9600700 1 -- 172.21.15.49:0/3561396898 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8017650 con 0x7f8fdc1094f0 2024-02-26T06:22:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc1a6350 con 0x7f8fdc1094f0 2024-02-26T06:22:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.937+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc1a6710 con 0x7f8fdc1094f0 2024-02-26T06:22:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.938+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc10f650 con 0x7f8fdc1094f0 2024-02-26T06:22:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.941+0000 7f8fd9600700 1 -- 172.21.15.49:0/3561396898 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8fc8021580 con 0x7f8fdc1094f0 2024-02-26T06:22:31.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.942+0000 7f8fd9600700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8fc40a21d0 0x7f8fc40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:31.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.942+0000 7f8fd9600700 1 -- 172.21.15.49:0/3561396898 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f8fc80ceff0 con 0x7f8fdc1094f0 2024-02-26T06:22:31.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.942+0000 7f8fe1200700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8fc40a21d0 0x7f8fc40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:31.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.943+0000 7f8fe1200700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8fc40a21d0 0x7f8fc40a4690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc109350 tx=0x7f8fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:31.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:31.944+0000 7f8fd9600700 1 -- 172.21.15.49:0/3561396898 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fc8098000 con 0x7f8fdc1094f0 2024-02-26T06:22:32.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.184+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fdc02d8d0 con 0x7f8fc40a21d0 2024-02-26T06:22:32.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.201+0000 7f8fd9600700 1 -- 172.21.15.49:0/3561396898 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f8fdc02d8d0 con 0x7f8fc40a21d0 2024-02-26T06:22:32.201 INFO:teuthology.orchestra.run.smithi049.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-26T06:22:32.201 INFO:teuthology.orchestra.run.smithi049.stdout:mds.all 12/12 57s ago 2m smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi080=b;smithi080=e;smithi080=h;smithi080=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2024-02-26T06:22:32.201 INFO:teuthology.orchestra.run.smithi049.stdout:mgr 3/3 57s ago 8m smithi049=x;smithi080=y;smithi115=z;count:3 2024-02-26T06:22:32.201 INFO:teuthology.orchestra.run.smithi049.stdout:mon 3/3 57s ago 9m smithi049:172.21.15.49=a;smithi080:172.21.15.80=b;smithi115:172.21.15.115=c;count:3 2024-02-26T06:22:32.201 INFO:teuthology.orchestra.run.smithi049.stdout:osd 12 57s ago - 2024-02-26T06:22:32.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8fc40a21d0 msgr2=0x7f8fc40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:32.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8fc40a21d0 0x7f8fc40a4690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc109350 tx=0x7f8fd0009260 comp rx=0 tx=0).stop 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 msgr2=0x7f8fdc19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 0x7f8fdc19e7b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800b720 tx=0x7f8fc8004730 comp rx=0 tx=0).stop 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 shutdown_connections 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f8fc40a21d0 0x7f8fc40a4690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fdc109e90 0x7f8fdc1a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fdc1082f0 0x7f8fdc19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 --2- 172.21.15.49:0/3561396898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fdc1094f0 0x7f8fdc19e7b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 >> 172.21.15.49:0/3561396898 conn(0x7f8fdc0772c0 msgr2=0x7f8fdc10c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.204+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 shutdown_connections 2024-02-26T06:22:32.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:32.205+0000 7f8fe3c00700 1 -- 172.21.15.49:0/3561396898 wait complete. 2024-02-26T06:22:33.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-26T06:22:33.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:33 smithi049 ceph-mon[31696]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:33.577 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:33.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:33 smithi080 ceph-mon[36206]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:33.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:33 smithi115 ceph-mon[36426]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:34.268 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:34 smithi049 ceph-mon[31696]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:34.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:34 smithi080 ceph-mon[36206]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:34.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:34 smithi115 ceph-mon[36426]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:35.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:35 smithi049 ceph-mon[31696]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:22:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:35 smithi080 ceph-mon[36206]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:22:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:35 smithi115 ceph-mon[36426]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:22:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 -- 172.21.15.49:0/1982201018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c102730 msgr2=0x7fe79c104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/1982201018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c102730 0x7fe79c104b20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe790009990 tx=0x7fe790009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 -- 172.21.15.49:0/1982201018 shutdown_connections 2024-02-26T06:22:36.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/1982201018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe79c107990 0x7fe79c109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/1982201018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c102730 0x7fe79c104b20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/1982201018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe79c105060 0x7fe79c107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 -- 172.21.15.49:0/1982201018 >> 172.21.15.49:0/1982201018 conn(0x7fe79c0fc140 msgr2=0x7fe79c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:36.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 -- 172.21.15.49:0/1982201018 shutdown_connections 2024-02-26T06:22:36.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.072+0000 7fe7a3a00700 1 -- 172.21.15.49:0/1982201018 wait complete. 2024-02-26T06:22:36.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.073+0000 7fe7a3a00700 1 Processor -- start 2024-02-26T06:22:36.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.073+0000 7fe7a3a00700 1 -- start start 2024-02-26T06:22:36.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a3a00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe79c102730 0x7fe79c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:36.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a3a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c105060 0x7fe79c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:36.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a3a00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe79c107990 0x7fe79c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a3a00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1a1a60 con 0x7fe79c102730 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a3a00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1a1ba0 con 0x7fe79c107990 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a3a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1a1d30 con 0x7fe79c105060 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a1000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe79c102730 0x7fe79c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe79be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c105060 0x7fe79c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe79be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c105060 0x7fe79c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46570/0 (socket says 172.21.15.49:46570) 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe79be00700 1 -- 172.21.15.49:0/2488603030 learned_addr learned my addr 172.21.15.49:0/2488603030 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a1000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe79c102730 0x7fe79c199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45166/0 (socket says 172.21.15.49:45166) 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.074+0000 7fe7a1a00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe79c107990 0x7fe79c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.075+0000 7fe79be00700 1 -- 172.21.15.49:0/2488603030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe79c107990 msgr2=0x7fe79c1a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.075+0000 7fe79be00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe79c107990 0x7fe79c1a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.075+0000 7fe79be00700 1 -- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe79c102730 msgr2=0x7fe79c199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:36.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.075+0000 7fe79be00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe79c102730 0x7fe79c199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.075+0000 7fe79be00700 1 -- 172.21.15.49:0/2488603030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900095f0 con 0x7fe79c105060 2024-02-26T06:22:36.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.075+0000 7fe79be00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c105060 0x7fe79c19a410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe78800e9d0 tx=0x7fe78800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:36.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.075+0000 7fe799600700 1 -- 172.21.15.49:0/2488603030 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788009790 con 0x7fe79c105060 2024-02-26T06:22:36.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.075+0000 7fe799600700 1 -- 172.21.15.49:0/2488603030 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe78800ee70 con 0x7fe79c105060 2024-02-26T06:22:36.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.076+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c1a1fb0 con 0x7fe79c105060 2024-02-26T06:22:36.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.076+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c1a24d0 con 0x7fe79c105060 2024-02-26T06:22:36.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.076+0000 7fe799600700 1 -- 172.21.15.49:0/2488603030 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788018680 con 0x7fe79c105060 2024-02-26T06:22:36.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.077+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77c005330 con 0x7fe79c105060 2024-02-26T06:22:36.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.079+0000 7fe799600700 1 -- 172.21.15.49:0/2488603030 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe78800cbd0 con 0x7fe79c105060 2024-02-26T06:22:36.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.081+0000 7fe799600700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7840a21d0 0x7fe7840a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:36.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.081+0000 7fe799600700 1 -- 172.21.15.49:0/2488603030 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe788014070 con 0x7fe79c105060 2024-02-26T06:22:36.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.081+0000 7fe7a1000700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7840a21d0 0x7fe7840a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:36.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.082+0000 7fe7a1000700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7840a21d0 0x7fe7840a4690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b6f0 tx=0x7fe79000b7c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:36.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.082+0000 7fe799600700 1 -- 172.21.15.49:0/2488603030 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe78809a770 con 0x7fe79c105060 2024-02-26T06:22:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.322+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe77c000bc0 con 0x7fe7840a21d0 2024-02-26T06:22:36.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.325+0000 7fe799600700 1 -- 172.21.15.49:0/2488603030 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fe77c000bc0 con 0x7fe7840a21d0 2024-02-26T06:22:36.325 INFO:teuthology.orchestra.run.smithi049.stdout:HOST ADDR LABELS STATUS 2024-02-26T06:22:36.325 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 172.21.15.49 2024-02-26T06:22:36.325 INFO:teuthology.orchestra.run.smithi049.stdout:smithi080 172.21.15.80 2024-02-26T06:22:36.325 INFO:teuthology.orchestra.run.smithi049.stdout:smithi115 172.21.15.115 2024-02-26T06:22:36.325 INFO:teuthology.orchestra.run.smithi049.stdout:3 hosts in cluster 2024-02-26T06:22:36.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7840a21d0 msgr2=0x7fe7840a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7840a21d0 0x7fe7840a4690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b6f0 tx=0x7fe79000b7c0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c105060 msgr2=0x7fe79c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c105060 0x7fe79c19a410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe78800e9d0 tx=0x7fe78800ece0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 shutdown_connections 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7840a21d0 0x7fe7840a4690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe79c107990 0x7fe79c1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe79c102730 0x7fe79c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 --2- 172.21.15.49:0/2488603030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c105060 0x7fe79c19a410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.328+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 >> 172.21.15.49:0/2488603030 conn(0x7fe79c0fc140 msgr2=0x7fe79c101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.329+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 shutdown_connections 2024-02-26T06:22:36.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:36.329+0000 7fe7a3a00700 1 -- 172.21.15.49:0/2488603030 wait complete. 2024-02-26T06:22:36.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:36 smithi080 ceph-mon[36206]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:36.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:36 smithi115 ceph-mon[36426]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:36.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:36 smithi049 ceph-mon[31696]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:37.190 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-26T06:22:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:37 smithi049 ceph-mon[31696]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:37 smithi080 ceph-mon[36206]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:37 smithi115 ceph-mon[36426]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:38.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:38 smithi080 ceph-mon[36206]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:38.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:38 smithi115 ceph-mon[36426]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:38.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:38 smithi049 ceph-mon[31696]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 -- 172.21.15.49:0/2179751647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3701081e0 msgr2=0x7ff370108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 --2- 172.21.15.49:0/2179751647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3701081e0 0x7ff370108600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3600099c0 tx=0x7ff360009cd0 comp rx=0 tx=0).stop 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 -- 172.21.15.49:0/2179751647 shutdown_connections 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 --2- 172.21.15.49:0/2179751647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff370109d80 0x7ff370110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 --2- 172.21.15.49:0/2179751647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3701081e0 0x7ff370108600 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 --2- 172.21.15.49:0/2179751647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3701093e0 0x7ff370109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 -- 172.21.15.49:0/2179751647 >> 172.21.15.49:0/2179751647 conn(0x7ff3700771b0 msgr2=0x7ff3700795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 -- 172.21.15.49:0/2179751647 shutdown_connections 2024-02-26T06:22:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.912+0000 7ff36f200700 1 -- 172.21.15.49:0/2179751647 wait complete. 2024-02-26T06:22:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.914+0000 7ff36f200700 1 Processor -- start 2024-02-26T06:22:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.914+0000 7ff36f200700 1 -- start start 2024-02-26T06:22:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.914+0000 7ff36f200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3701081e0 0x7ff37019e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.914+0000 7ff36f200700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3701093e0 0x7ff37019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.914+0000 7ff36f200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370109d80 0x7ff3701a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.914+0000 7ff36f200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3701a5d50 con 0x7ff370109d80 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.914+0000 7ff36f200700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3701a5e90 con 0x7ff3701093e0 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.914+0000 7ff36f200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3701a6020 con 0x7ff3701081e0 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36e800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370109d80 0x7ff3701a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36e800700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370109d80 0x7ff3701a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45172/0 (socket says 172.21.15.49:45172) 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff367e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3701093e0 0x7ff37019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36e800700 1 -- 172.21.15.49:0/2057535921 learned_addr learned my addr 172.21.15.49:0/2057535921 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36de00700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3701081e0 0x7ff37019e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36de00700 1 -- 172.21.15.49:0/2057535921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3701093e0 msgr2=0x7ff37019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36de00700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3701093e0 0x7ff37019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36de00700 1 -- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370109d80 msgr2=0x7ff3701a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36de00700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370109d80 0x7ff3701a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36de00700 1 -- 172.21.15.49:0/2057535921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368009c40 con 0x7ff3701081e0 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.915+0000 7ff36e800700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370109d80 0x7ff3701a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.916+0000 7ff36de00700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3701081e0 0x7ff37019e210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3600098e0 tx=0x7ff3600055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.916+0000 7ff366a00700 1 -- 172.21.15.49:0/2057535921 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36001d070 con 0x7ff3701081e0 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.916+0000 7ff366a00700 1 -- 172.21.15.49:0/2057535921 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff36000be80 con 0x7ff3701081e0 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.916+0000 7ff366a00700 1 -- 172.21.15.49:0/2057535921 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36000fbc0 con 0x7ff3701081e0 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.916+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3600095f0 con 0x7ff3701081e0 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.917+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3701a64d0 con 0x7ff3701081e0 2024-02-26T06:22:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.918+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37010f5a0 con 0x7ff3701081e0 2024-02-26T06:22:39.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.920+0000 7ff366a00700 1 -- 172.21.15.49:0/2057535921 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff36000b9a0 con 0x7ff3701081e0 2024-02-26T06:22:39.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.921+0000 7ff366a00700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3540a2150 0x7ff3540a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:39.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.922+0000 7ff366a00700 1 -- 172.21.15.49:0/2057535921 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff3600ce9e0 con 0x7ff3701081e0 2024-02-26T06:22:39.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.922+0000 7ff367e00700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3540a2150 0x7ff3540a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:39.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.922+0000 7ff367e00700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3540a2150 0x7ff3540a4610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff358009c60 tx=0x7ff358009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:39.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:39.926+0000 7ff366a00700 1 -- 172.21.15.49:0/2057535921 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff3600979d0 con 0x7ff3701081e0 2024-02-26T06:22:40.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.162+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff37002d7d0 con 0x7ff3540a2150 2024-02-26T06:22:40.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.167+0000 7ff366a00700 1 -- 172.21.15.49:0/2057535921 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7ff37002d7d0 con 0x7ff3540a2150 2024-02-26T06:22:40.167 INFO:teuthology.orchestra.run.smithi049.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-26T06:22:40.167 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500D2480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-26T06:22:40.167 INFO:teuthology.orchestra.run.smithi049.stdout:smithi080 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204014T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-02-26T06:22:40.167 INFO:teuthology.orchestra.run.smithi049.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-02-26T06:22:40.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3540a2150 msgr2=0x7ff3540a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:40.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3540a2150 0x7ff3540a4610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff358009c60 tx=0x7ff358009260 comp rx=0 tx=0).stop 2024-02-26T06:22:40.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3701081e0 msgr2=0x7ff37019e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:40.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3701081e0 0x7ff37019e210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3600098e0 tx=0x7ff3600055f0 comp rx=0 tx=0).stop 2024-02-26T06:22:40.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 shutdown_connections 2024-02-26T06:22:40.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff3540a2150 0x7ff3540a4610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:40.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff370109d80 0x7ff3701a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:40.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3701081e0 0x7ff37019e210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:40.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 --2- 172.21.15.49:0/2057535921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3701093e0 0x7ff37019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:40.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 >> 172.21.15.49:0/2057535921 conn(0x7ff3700771b0 msgr2=0x7ff37010c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:40.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 shutdown_connections 2024-02-26T06:22:40.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:40.171+0000 7ff36f200700 1 -- 172.21.15.49:0/2057535921 wait complete. 2024-02-26T06:22:40.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:40 smithi080 ceph-mon[36206]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:40.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:40 smithi115 ceph-mon[36426]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:40.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:40 smithi049 ceph-mon[31696]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:22:41.050 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-26T06:22:41.059 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:22:41.059 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-26T06:22:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:41.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:41 smithi080 ceph-mon[36206]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:41.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:41 smithi115 ceph-mon[36426]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:41.853 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:41 smithi049 ceph-mon[31696]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:42 smithi080 ceph-mon[36206]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:42.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:42 smithi115 ceph-mon[36426]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:42.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:42 smithi049 ceph-mon[31696]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:22:43.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 -- 172.21.15.49:0/2606232605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 msgr2=0x7f1dbc10c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:43.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/2606232605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 0x7f1dbc10c040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009990 tx=0x7f1db4009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:43.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 -- 172.21.15.49:0/2606232605 shutdown_connections 2024-02-26T06:22:43.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/2606232605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 0x7f1dbc10c040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:43.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/2606232605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc075c40 0x7f1dbc0740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:43.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/2606232605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc074670 0x7f1dbc074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:43.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 -- 172.21.15.49:0/2606232605 >> 172.21.15.49:0/2606232605 conn(0x7f1dbc0fd270 msgr2=0x7f1dbc0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:43.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 -- 172.21.15.49:0/2606232605 shutdown_connections 2024-02-26T06:22:43.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.855+0000 7f1dc3e00700 1 -- 172.21.15.49:0/2606232605 wait complete. 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.856+0000 7f1dc3e00700 1 Processor -- start 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.856+0000 7f1dc3e00700 1 -- start start 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc074670 0x7f1dbc199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc3e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc075c40 0x7f1dbc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc3e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 0x7f1dbc1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc3e00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc1a1900 con 0x7f1dbc109bb0 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc3e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc1a1a40 con 0x7f1dbc075c40 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc3e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc1a1c00 con 0x7f1dbc074670 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc1400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc074670 0x7f1dbc199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:43.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc1400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc074670 0x7f1dbc199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42458/0 (socket says 172.21.15.49:42458) 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc1400700 1 -- 172.21.15.49:0/3517044041 learned_addr learned my addr 172.21.15.49:0/3517044041 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc0a00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc075c40 0x7f1dbc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.857+0000 7f1dc1e00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 0x7f1dbc1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc1400700 1 -- 172.21.15.49:0/3517044041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc075c40 msgr2=0x7f1dbc19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc1400700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc075c40 0x7f1dbc19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc1400700 1 -- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 msgr2=0x7f1dbc1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc1400700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 0x7f1dbc1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc1400700 1 -- 172.21.15.49:0/3517044041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db0009c40 con 0x7f1dbc074670 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc1e00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 0x7f1dbc1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc1400700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc074670 0x7f1dbc199db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c370 tx=0x7f1db000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dba000700 1 -- 172.21.15.49:0/3517044041 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0017070 con 0x7f1dbc074670 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db40095f0 con 0x7f1dbc074670 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dba000700 1 -- 172.21.15.49:0/3517044041 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1db000f040 con 0x7f1dbc074670 2024-02-26T06:22:43.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dba000700 1 -- 172.21.15.49:0/3517044041 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db000e050 con 0x7f1dbc074670 2024-02-26T06:22:43.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.858+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dbc1a21a0 con 0x7f1dbc074670 2024-02-26T06:22:43.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.859+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dbc1089c0 con 0x7f1dbc074670 2024-02-26T06:22:43.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.862+0000 7f1dba000700 1 -- 172.21.15.49:0/3517044041 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1db00090d0 con 0x7f1dbc074670 2024-02-26T06:22:43.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.863+0000 7f1dba000700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1da40a2150 0x7f1da40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:43.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.863+0000 7f1dba000700 1 -- 172.21.15.49:0/3517044041 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1db00cd720 con 0x7f1dbc074670 2024-02-26T06:22:43.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.863+0000 7f1dc0a00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1da40a2150 0x7f1da40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:43.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.864+0000 7f1dc0a00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1da40a2150 0x7f1da40a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009c60 tx=0x7f1da8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:43.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:43.866+0000 7f1dba000700 1 -- 172.21.15.49:0/3517044041 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1db0096680 con 0x7f1dbc074670 2024-02-26T06:22:44.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.149+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1dbc10f8e0 con 0x7f1dbc074670 2024-02-26T06:22:44.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.150+0000 7f1dba000700 1 -- 172.21.15.49:0/3517044041 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2933 (secure 0 0 0) 0x7f1db0099d40 con 0x7f1dbc074670 2024-02-26T06:22:44.152 INFO:teuthology.orchestra.run.smithi049.stdout:e23 2024-02-26T06:22:44.152 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T06:22:44.152 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T06:22:44.152 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-02-26T06:22:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 23 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-02-26T06:20:31.555053+0000 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-02-26T06:20:38.098815+0000 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 300 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 0 2024-02-26T06:22:44.153 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 3 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1,2 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=24572,1=24559,2=24535} 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data disabled 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 0 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b{0:24572} state up:active seq 3 addr [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.f{0:24523} state up:standby-replay seq 1 addr [v2:172.21.15.115:6834/2578799425,v1:172.21.15.115:6835/2578799425] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.i{1:24559} state up:active seq 7 addr [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.h{1:24521} state up:standby-replay seq 1 addr [v2:172.21.15.80:6836/2132318232,v1:172.21.15.80:6837/2132318232] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c{2:24535} state up:active seq 9 addr [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.e{2:24491} state up:standby-replay seq 1 addr [v2:172.21.15.80:6834/1885591642,v1:172.21.15.80:6835/1885591642] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.j{-1:14379} state up:standby seq 1 addr [v2:172.21.15.49:6834/180447409,v1:172.21.15.49:6835/180447409] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.g{-1:14445} state up:standby seq 1 addr [v2:172.21.15.49:6838/1053244563,v1:172.21.15.49:6839/1053244563] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a{-1:14463} state up:standby seq 1 addr [v2:172.21.15.49:6840/2984977662,v1:172.21.15.49:6841/2984977662] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.k{-1:24446} state up:standby seq 1 addr [v2:172.21.15.80:6832/914291308,v1:172.21.15.80:6833/914291308] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.l{-1:24457} state up:standby seq 1 addr [v2:172.21.15.115:6832/1451697651,v1:172.21.15.115:6833/1451697651] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.155 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d{-1:24479} state up:standby seq 1 addr [v2:172.21.15.49:6836/879820521,v1:172.21.15.49:6837/879820521] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:22:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1da40a2150 msgr2=0x7f1da40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1da40a2150 0x7f1da40a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009c60 tx=0x7f1da8009260 comp rx=0 tx=0).stop 2024-02-26T06:22:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc074670 msgr2=0x7f1dbc199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc074670 0x7f1dbc199db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c370 tx=0x7f1db000c680 comp rx=0 tx=0).stop 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 shutdown_connections 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1da40a2150 0x7f1da40a4610 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc109bb0 0x7f1dbc1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc074670 0x7f1dbc199db0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 --2- 172.21.15.49:0/3517044041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc075c40 0x7f1dbc19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.154+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 >> 172.21.15.49:0/3517044041 conn(0x7f1dbc0fd270 msgr2=0x7f1dbc105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.155+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 shutdown_connections 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:44.155+0000 7f1dc3e00700 1 -- 172.21.15.49:0/3517044041 wait complete. 2024-02-26T06:22:44.157 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-02-26T06:22:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:44 smithi080 ceph-mon[36206]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:22:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:44 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3517044041' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T06:22:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:44 smithi115 ceph-mon[36426]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:22:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:44 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3517044041' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T06:22:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:44 smithi049 ceph-mon[31696]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:22:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:44 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3517044041' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T06:22:44.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-26T06:22:45.501 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:46 smithi080 ceph-mon[36206]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T06:22:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:46 smithi115 ceph-mon[36426]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T06:22:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:46 smithi049 ceph-mon[31696]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T06:22:47.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 -- 172.21.15.49:0/2324335605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 msgr2=0x7f0c50108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:47.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 --2- 172.21.15.49:0/2324335605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 0x7f0c50108600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c440099c0 tx=0x7f0c44009cd0 comp rx=0 tx=0).stop 2024-02-26T06:22:47.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 -- 172.21.15.49:0/2324335605 shutdown_connections 2024-02-26T06:22:47.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 --2- 172.21.15.49:0/2324335605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c50109d80 0x7f0c50110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:47.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 --2- 172.21.15.49:0/2324335605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 0x7f0c50108600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:47.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 --2- 172.21.15.49:0/2324335605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c501093e0 0x7f0c50109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:47.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 -- 172.21.15.49:0/2324335605 >> 172.21.15.49:0/2324335605 conn(0x7f0c500771b0 msgr2=0x7f0c500795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:47.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 -- 172.21.15.49:0/2324335605 shutdown_connections 2024-02-26T06:22:47.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.820+0000 7f0c56000700 1 -- 172.21.15.49:0/2324335605 wait complete. 2024-02-26T06:22:47.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.822+0000 7f0c56000700 1 Processor -- start 2024-02-26T06:22:47.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.822+0000 7f0c56000700 1 -- start start 2024-02-26T06:22:47.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.822+0000 7f0c56000700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 0x7f0c5019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:47.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.822+0000 7f0c56000700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c501093e0 0x7f0c5019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.822+0000 7f0c56000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c50109d80 0x7f0c501a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.822+0000 7f0c56000700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c501a5cc0 con 0x7f0c501081e0 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.822+0000 7f0c56000700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c501a5e00 con 0x7f0c501093e0 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.822+0000 7f0c56000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c501a5f90 con 0x7f0c50109d80 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c54c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 0x7f0c5019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c54c00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 0x7f0c5019e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42236/0 (socket says 172.21.15.49:42236) 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c54c00700 1 -- 172.21.15.49:0/893729511 learned_addr learned my addr 172.21.15.49:0/893729511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c4fe00700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c501093e0 0x7f0c5019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c55600700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c50109d80 0x7f0c501a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c4fe00700 1 -- 172.21.15.49:0/893729511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c50109d80 msgr2=0x7f0c501a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c4fe00700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c50109d80 0x7f0c501a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c4fe00700 1 -- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 msgr2=0x7f0c5019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c4fe00700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 0x7f0c5019e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:47.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c4fe00700 1 -- 172.21.15.49:0/893729511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c009520 con 0x7f0c501093e0 2024-02-26T06:22:47.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.823+0000 7f0c54c00700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 0x7f0c5019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:47.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.824+0000 7f0c4fe00700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c501093e0 0x7f0c5019e670 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00e9f0 tx=0x7f0c3c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:47.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.824+0000 7f0c4d600700 1 -- 172.21.15.49:0/893729511 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c00cb70 con 0x7f0c501093e0 2024-02-26T06:22:47.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.824+0000 7f0c4d600700 1 -- 172.21.15.49:0/893729511 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c00ee90 con 0x7f0c501093e0 2024-02-26T06:22:47.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.824+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c440095f0 con 0x7f0c501093e0 2024-02-26T06:22:47.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.824+0000 7f0c4d600700 1 -- 172.21.15.49:0/893729511 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c003d80 con 0x7f0c501093e0 2024-02-26T06:22:47.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.824+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c501a6630 con 0x7f0c501093e0 2024-02-26T06:22:47.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.825+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5010f620 con 0x7f0c501093e0 2024-02-26T06:22:47.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.827+0000 7f0c4d600700 1 -- 172.21.15.49:0/893729511 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0c3c003ee0 con 0x7f0c501093e0 2024-02-26T06:22:47.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.829+0000 7f0c4d600700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0c380a21d0 0x7f0c380a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:47.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.829+0000 7f0c4d600700 1 -- 172.21.15.49:0/893729511 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0c3c014070 con 0x7f0c501093e0 2024-02-26T06:22:47.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.829+0000 7f0c54c00700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0c380a21d0 0x7f0c380a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:47.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.830+0000 7f0c54c00700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0c380a21d0 0x7f0c380a4690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0c44000c00 tx=0x7f0c44009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:47.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:47.833+0000 7f0c4d600700 1 -- 172.21.15.49:0/893729511 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c3c096f00 con 0x7f0c501093e0 2024-02-26T06:22:48.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.058+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f0c50068930 con 0x7f0c501093e0 2024-02-26T06:22:48.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.059+0000 7f0c4d600700 1 -- 172.21.15.49:0/893729511 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5308 (secure 0 0 0) 0x7f0c3c09a5c0 con 0x7f0c501093e0 2024-02-26T06:22:48.059 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 88 2024-02-26T06:22:48.060 INFO:teuthology.orchestra.run.smithi049.stdout:fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:22:48.060 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-02-26T06:11:09.086607+0000 2024-02-26T06:22:48.060 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-02-26T06:21:28.893557+0000 2024-02-26T06:22:48.060 INFO:teuthology.orchestra.run.smithi049.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-26T06:22:48.060 INFO:teuthology.orchestra.run.smithi049.stdout:crush_version 26 2024-02-26T06:22:48.060 INFO:teuthology.orchestra.run.smithi049.stdout:full_ratio 0.95 2024-02-26T06:22:48.061 INFO:teuthology.orchestra.run.smithi049.stdout:backfillfull_ratio 0.9 2024-02-26T06:22:48.061 INFO:teuthology.orchestra.run.smithi049.stdout:nearfull_ratio 0.85 2024-02-26T06:22:48.061 INFO:teuthology.orchestra.run.smithi049.stdout:require_min_compat_client luminous 2024-02-26T06:22:48.061 INFO:teuthology.orchestra.run.smithi049.stdout:min_compat_client luminous 2024-02-26T06:22:48.061 INFO:teuthology.orchestra.run.smithi049.stdout:require_osd_release squid 2024-02-26T06:22:48.061 INFO:teuthology.orchestra.run.smithi049.stdout:stretch_mode_enabled false 2024-02-26T06:22:48.061 INFO:teuthology.orchestra.run.smithi049.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:max_osd 12 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6802/3449358654,v1:172.21.15.49:6803/3449358654] [v2:172.21.15.49:6804/3449358654,v1:172.21.15.49:6805/3449358654] exists,up db966476-ec8d-4000-8d96-2944d08fecca 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6800/3019548533,v1:172.21.15.80:6801/3019548533] [v2:172.21.15.80:6802/3019548533,v1:172.21.15.80:6803/3019548533] exists,up 5cbb4214-1d89-4598-834c-d567d0d63d04 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/3987885615,v1:172.21.15.115:6801/3987885615] [v2:172.21.15.115:6802/3987885615,v1:172.21.15.115:6803/3987885615] exists,up d416d235-0756-4c38-835b-3d8a8e04d2cc 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 up in weight 1 up_from 26 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6810/2774029183,v1:172.21.15.49:6811/2774029183] [v2:172.21.15.49:6812/2774029183,v1:172.21.15.49:6813/2774029183] exists,up 674ab7f1-6ee1-4e0d-9e10-42770a30c5f8 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6808/1597953633,v1:172.21.15.80:6809/1597953633] [v2:172.21.15.80:6810/1597953633,v1:172.21.15.80:6811/1597953633] exists,up 41dcb99d-6d5c-4ee1-a461-06944881881c 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/1277586468,v1:172.21.15.115:6809/1277586468] [v2:172.21.15.115:6810/1277586468,v1:172.21.15.115:6811/1277586468] exists,up 53d273fb-162f-4ca7-b995-1387266dacb6 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6818/2967719452,v1:172.21.15.49:6819/2967719452] [v2:172.21.15.49:6820/2967719452,v1:172.21.15.49:6821/2967719452] exists,up 570e6f42-9220-477e-b01f-0a4169640324 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6816/4119828999,v1:172.21.15.80:6817/4119828999] [v2:172.21.15.80:6818/4119828999,v1:172.21.15.80:6819/4119828999] exists,up ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f 2024-02-26T06:22:48.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/2102490724,v1:172.21.15.115:6817/2102490724] [v2:172.21.15.115:6818/2102490724,v1:172.21.15.115:6819/2102490724] exists,up c86f9fc9-9e22-4686-83ef-2a7b3b0652cc 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6826/4117635859,v1:172.21.15.49:6827/4117635859] [v2:172.21.15.49:6828/4117635859,v1:172.21.15.49:6829/4117635859] exists,up 438322dd-a5d2-4d7a-b7f6-1de27c32fa94 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6824/1162870008,v1:172.21.15.80:6825/1162870008] [v2:172.21.15.80:6826/1162870008,v1:172.21.15.80:6827/1162870008] exists,up 88c87856-9714-45a8-be85-a007457ae094 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/1497318098,v1:172.21.15.115:6825/1497318098] [v2:172.21.15.115:6826/1497318098,v1:172.21.15.115:6827/1497318098] exists,up 46f79795-b13d-4997-b0d2-d89822eedf67 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.b [8,11] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.10 [2,4,8,11] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.14 [2,4] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.2 [5,6] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.3 [4,9] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.10 [5,9] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.1e [3,9] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.23 [4,8] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.3a [4,7] 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/2180727249 expires 2024-02-27T06:12:27.274814+0000 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/2102614714 expires 2024-02-27T06:12:27.274814+0000 2024-02-26T06:22:48.063 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/4089389406 expires 2024-02-27T06:12:27.274814+0000 2024-02-26T06:22:48.064 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/3504875682 expires 2024-02-27T06:12:27.274814+0000 2024-02-26T06:22:48.064 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/746416011 expires 2024-02-27T06:11:51.411945+0000 2024-02-26T06:22:48.064 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/765116076 expires 2024-02-27T06:11:51.411945+0000 2024-02-26T06:22:48.064 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/2937664554 expires 2024-02-27T06:11:51.411945+0000 2024-02-26T06:22:48.064 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/765116076 expires 2024-02-27T06:11:51.411945+0000 2024-02-26T06:22:48.064 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/2168062918 expires 2024-02-27T06:11:51.411945+0000 2024-02-26T06:22:48.064 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/4089389406 expires 2024-02-27T06:12:27.274814+0000 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0c380a21d0 msgr2=0x7f0c380a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0c380a21d0 0x7f0c380a4690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0c44000c00 tx=0x7f0c44009510 comp rx=0 tx=0).stop 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c501093e0 msgr2=0x7f0c5019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c501093e0 0x7f0c5019e670 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00e9f0 tx=0x7f0c3c00ed00 comp rx=0 tx=0).stop 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 shutdown_connections 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f0c380a21d0 0x7f0c380a4690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c50109d80 0x7f0c501a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c501081e0 0x7f0c5019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 --2- 172.21.15.49:0/893729511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c501093e0 0x7f0c5019e670 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 >> 172.21.15.49:0/893729511 conn(0x7f0c500771b0 msgr2=0x7f0c5010c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:48.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 shutdown_connections 2024-02-26T06:22:48.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:48.062+0000 7f0c56000700 1 -- 172.21.15.49:0/893729511 wait complete. 2024-02-26T06:22:48.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:48 smithi080 ceph-mon[36206]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-26T06:22:48.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:48 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/893729511' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T06:22:48.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:48 smithi115 ceph-mon[36426]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-26T06:22:48.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:48 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/893729511' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T06:22:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:48 smithi049 ceph-mon[31696]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-26T06:22:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:48 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/893729511' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T06:22:48.940 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-26T06:22:49.147 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:22:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 -- 172.21.15.49:0/348185721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820077720 msgr2=0x7fb820077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 --2- 172.21.15.49:0/348185721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820077720 0x7fb820077fc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb818009ca0 tx=0x7fb81802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 -- 172.21.15.49:0/348185721 shutdown_connections 2024-02-26T06:22:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 --2- 172.21.15.49:0/348185721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820076d80 0x7fb8200771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 --2- 172.21.15.49:0/348185721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820077720 0x7fb820077fc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 --2- 172.21.15.49:0/348185721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb820075b40 0x7fb820075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 -- 172.21.15.49:0/348185721 >> 172.21.15.49:0/348185721 conn(0x7fb8200fe930 msgr2=0x7fb820100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 -- 172.21.15.49:0/348185721 shutdown_connections 2024-02-26T06:22:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.394+0000 7fb825800640 1 -- 172.21.15.49:0/348185721 wait complete. 2024-02-26T06:22:49.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.395+0000 7fb825800640 1 Processor -- start 2024-02-26T06:22:49.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.395+0000 7fb825800640 1 -- start start 2024-02-26T06:22:49.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb825800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820075b40 0x7fb82019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:49.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb825800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb820076d80 0x7fb82019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:49.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb825800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820077720 0x7fb8201a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb825800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a5300 con 0x7fb820077720 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb825800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a5470 con 0x7fb820075b40 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb825800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201a5630 con 0x7fb820076d80 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820075b40 0x7fb82019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb820076d80 0x7fb82019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820077720 0x7fb8201a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb820076d80 0x7fb82019dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42490/0 (socket says 172.21.15.49:42490) 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81e000640 1 -- 172.21.15.49:0/165454519 learned_addr learned my addr 172.21.15.49:0/165454519 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81e000640 1 -- 172.21.15.49:0/165454519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820075b40 msgr2=0x7fb82019d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81e000640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820075b40 0x7fb82019d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81e000640 1 -- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820077720 msgr2=0x7fb8201a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81e000640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820077720 0x7fb8201a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81e000640 1 -- 172.21.15.49:0/165454519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb810009490 con 0x7fb820076d80 2024-02-26T06:22:49.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.396+0000 7fb81ea00640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820075b40 0x7fb82019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:49.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.397+0000 7fb81e000640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb820076d80 0x7fb82019dd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb81000ea60 tx=0x7fb81000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:49.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.397+0000 7fb817400640 1 -- 172.21.15.49:0/165454519 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000cde0 con 0x7fb820076d80 2024-02-26T06:22:49.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.397+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb818009900 con 0x7fb820076d80 2024-02-26T06:22:49.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.397+0000 7fb817400640 1 -- 172.21.15.49:0/165454519 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8100035f0 con 0x7fb820076d80 2024-02-26T06:22:49.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.397+0000 7fb817400640 1 -- 172.21.15.49:0/165454519 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810010690 con 0x7fb820076d80 2024-02-26T06:22:49.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.397+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8201a5c90 con 0x7fb820076d80 2024-02-26T06:22:49.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.398+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82010fef0 con 0x7fb820076d80 2024-02-26T06:22:49.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.401+0000 7fb817400640 1 -- 172.21.15.49:0/165454519 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb810003af0 con 0x7fb820076d80 2024-02-26T06:22:49.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.402+0000 7fb817400640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8040a20c0 0x7fb8040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:49.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.402+0000 7fb817400640 1 -- 172.21.15.49:0/165454519 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb810014070 con 0x7fb820076d80 2024-02-26T06:22:49.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.402+0000 7fb81ea00640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8040a20c0 0x7fb8040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:49.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.403+0000 7fb81ea00640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8040a20c0 0x7fb8040a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb808009fc0 tx=0x7fb808005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.406+0000 7fb817400640 1 -- 172.21.15.49:0/165454519 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb8100970b0 con 0x7fb820076d80 2024-02-26T06:22:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.610+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8201131c0 con 0x7fb820076d80 2024-02-26T06:22:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.612+0000 7fb817400640 1 -- 172.21.15.49:0/165454519 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7fb81009a770 con 0x7fb820076d80 2024-02-26T06:22:49.614 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:49.615 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.615+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8040a20c0 msgr2=0x7fb8040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.615+0000 7fb825800640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8040a20c0 0x7fb8040a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb808009fc0 tx=0x7fb808005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.615+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb820076d80 msgr2=0x7fb82019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.615+0000 7fb825800640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb820076d80 0x7fb82019dd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb81000ea60 tx=0x7fb81000ef20 comp rx=0 tx=0).stop 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.616+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 shutdown_connections 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.616+0000 7fb825800640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb8040a20c0 0x7fb8040a4580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.616+0000 7fb825800640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb820077720 0x7fb8201a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.616+0000 7fb825800640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb820076d80 0x7fb82019dd50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.616+0000 7fb825800640 1 --2- 172.21.15.49:0/165454519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb820075b40 0x7fb82019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.616+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 >> 172.21.15.49:0/165454519 conn(0x7fb8200fe930 msgr2=0x7fb820100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:49.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.616+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 shutdown_connections 2024-02-26T06:22:49.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.616+0000 7fb825800640 1 -- 172.21.15.49:0/165454519 wait complete. 2024-02-26T06:22:49.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:22:49.637 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 -- 172.21.15.49:0/799020291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34104410 msgr2=0x7efc34106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 --2- 172.21.15.49:0/799020291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34104410 0x7efc34106800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc24009900 tx=0x7efc2402bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 -- 172.21.15.49:0/799020291 shutdown_connections 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 --2- 172.21.15.49:0/799020291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34104410 0x7efc34106800 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 --2- 172.21.15.49:0/799020291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34106d40 0x7efc34109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 --2- 172.21.15.49:0/799020291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc34101ae0 0x7efc34103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 -- 172.21.15.49:0/799020291 >> 172.21.15.49:0/799020291 conn(0x7efc340fb4f0 msgr2=0x7efc340fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 -- 172.21.15.49:0/799020291 shutdown_connections 2024-02-26T06:22:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.883+0000 7efc3b200640 1 -- 172.21.15.49:0/799020291 wait complete. 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.884+0000 7efc3b200640 1 Processor -- start 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.884+0000 7efc3b200640 1 -- start start 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc3b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34101ae0 0x7efc34101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc3b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34104410 0x7efc340ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc3b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc34106d40 0x7efc340fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc3b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc34100730 con 0x7efc34101ae0 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc3b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc341008a0 con 0x7efc34104410 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc3b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc341a1160 con 0x7efc34106d40 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc39e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34101ae0 0x7efc34101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc39e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34101ae0 0x7efc34101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42276/0 (socket says 172.21.15.49:42276) 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc39e00640 1 -- 172.21.15.49:0/3577318822 learned_addr learned my addr 172.21.15.49:0/3577318822 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:49.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc39400640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34104410 0x7efc340ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:49.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.885+0000 7efc3a800640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc34106d40 0x7efc340fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc39e00640 1 -- 172.21.15.49:0/3577318822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc34106d40 msgr2=0x7efc340fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc39e00640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc34106d40 0x7efc340fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc39e00640 1 -- 172.21.15.49:0/3577318822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34104410 msgr2=0x7efc340ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc39e00640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34104410 0x7efc340ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc39e00640 1 -- 172.21.15.49:0/3577318822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24009560 con 0x7efc34101ae0 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc3a800640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc34106d40 0x7efc340fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc39e00640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34101ae0 0x7efc34101450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efc2800ea20 tx=0x7efc2800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc2ea00640 1 -- 172.21.15.49:0/3577318822 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28009720 con 0x7efc34101ae0 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.886+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc341a1410 con 0x7efc34101ae0 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.887+0000 7efc2ea00640 1 -- 172.21.15.49:0/3577318822 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc2800cd80 con 0x7efc34101ae0 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.887+0000 7efc2ea00640 1 -- 172.21.15.49:0/3577318822 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28010490 con 0x7efc34101ae0 2024-02-26T06:22:49.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.887+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc341a18f0 con 0x7efc34101ae0 2024-02-26T06:22:49.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.888+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc340fcfe0 con 0x7efc34101ae0 2024-02-26T06:22:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.890+0000 7efc2ea00640 1 -- 172.21.15.49:0/3577318822 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efc28005020 con 0x7efc34101ae0 2024-02-26T06:22:49.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.891+0000 7efc2ea00640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc1c0a2140 0x7efc1c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.892+0000 7efc2ea00640 1 -- 172.21.15.49:0/3577318822 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7efc28014070 con 0x7efc34101ae0 2024-02-26T06:22:49.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.892+0000 7efc39400640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc1c0a2140 0x7efc1c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.892+0000 7efc39400640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc1c0a2140 0x7efc1c0a4600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efc2402c240 tx=0x7efc2402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:49.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:49.896+0000 7efc2ea00640 1 -- 172.21.15.49:0/3577318822 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc28097070 con 0x7efc34101ae0 2024-02-26T06:22:50.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.094+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc3402cda0 con 0x7efc34101ae0 2024-02-26T06:22:50.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.095+0000 7efc2ea00640 1 -- 172.21.15.49:0/3577318822 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc2809a730 con 0x7efc34101ae0 2024-02-26T06:22:50.095 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:22:50.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc1c0a2140 msgr2=0x7efc1c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc1c0a2140 0x7efc1c0a4600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efc2402c240 tx=0x7efc2402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34101ae0 msgr2=0x7efc34101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34101ae0 0x7efc34101450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efc2800ea20 tx=0x7efc2800ee20 comp rx=0 tx=0).stop 2024-02-26T06:22:50.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 shutdown_connections 2024-02-26T06:22:50.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efc1c0a2140 0x7efc1c0a4600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34104410 0x7efc340ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc34106d40 0x7efc340fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 --2- 172.21.15.49:0/3577318822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34101ae0 0x7efc34101450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 >> 172.21.15.49:0/3577318822 conn(0x7efc340fb4f0 msgr2=0x7efc34102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:50.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 shutdown_connections 2024-02-26T06:22:50.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.097+0000 7efc3b200640 1 -- 172.21.15.49:0/3577318822 wait complete. 2024-02-26T06:22:50.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:22:50.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 -- 172.21.15.49:0/2925400260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102270 msgr2=0x7fd2801026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 --2- 172.21.15.49:0/2925400260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102270 0x7fd2801026f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd268009be0 tx=0x7fd26802bef0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 -- 172.21.15.49:0/2925400260 shutdown_connections 2024-02-26T06:22:50.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 --2- 172.21.15.49:0/2925400260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102270 0x7fd2801026f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 --2- 172.21.15.49:0/2925400260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd280102c30 0x7fd2801094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 --2- 172.21.15.49:0/2925400260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd280101080 0x7fd2801014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 -- 172.21.15.49:0/2925400260 >> 172.21.15.49:0/2925400260 conn(0x7fd2800fc5f0 msgr2=0x7fd2800fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:50.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 -- 172.21.15.49:0/2925400260 shutdown_connections 2024-02-26T06:22:50.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.346+0000 7fd27fe00640 1 -- 172.21.15.49:0/2925400260 wait complete. 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.347+0000 7fd27fe00640 1 Processor -- start 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.347+0000 7fd27fe00640 1 -- start start 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd280101080 0x7fd2801992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd280102270 0x7fd280199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102c30 0x7fd2801a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27fe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801a0de0 con 0x7fd280101080 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801a0f50 con 0x7fd280102c30 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27fe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801a1110 con 0x7fd280102270 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102c30 0x7fd2801a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102c30 0x7fd2801a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38540/0 (socket says 172.21.15.49:38540) 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27f400640 1 -- 172.21.15.49:0/1616505586 learned_addr learned my addr 172.21.15.49:0/1616505586 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27e000640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd280102270 0x7fd280199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27ea00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd280101080 0x7fd2801992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27f400640 1 -- 172.21.15.49:0/1616505586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd280102270 msgr2=0x7fd280199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27f400640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd280102270 0x7fd280199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27f400640 1 -- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd280101080 msgr2=0x7fd2801992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27f400640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd280101080 0x7fd2801992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.348+0000 7fd27f400640 1 -- 172.21.15.49:0/1616505586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268009840 con 0x7fd280102c30 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.349+0000 7fd27f400640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102c30 0x7fd2801a08a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd27800e970 tx=0x7fd27800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.349+0000 7fd277400640 1 -- 172.21.15.49:0/1616505586 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278009720 con 0x7fd280102c30 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.349+0000 7fd277400640 1 -- 172.21.15.49:0/1616505586 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd278005500 con 0x7fd280102c30 2024-02-26T06:22:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.349+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2801a13f0 con 0x7fd280102c30 2024-02-26T06:22:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.349+0000 7fd277400640 1 -- 172.21.15.49:0/1616505586 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278010430 con 0x7fd280102c30 2024-02-26T06:22:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.349+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2801a1920 con 0x7fd280102c30 2024-02-26T06:22:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.350+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28010cae0 con 0x7fd280102c30 2024-02-26T06:22:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.352+0000 7fd277400640 1 -- 172.21.15.49:0/1616505586 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd278004ef0 con 0x7fd280102c30 2024-02-26T06:22:50.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.354+0000 7fd277400640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2600a2140 0x7fd2600a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:50.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.354+0000 7fd277400640 1 -- 172.21.15.49:0/1616505586 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd278014070 con 0x7fd280102c30 2024-02-26T06:22:50.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.354+0000 7fd27ea00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2600a2140 0x7fd2600a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:50.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.355+0000 7fd27ea00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2600a2140 0x7fd2600a4600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd2801020d0 tx=0x7fd270009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:50.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.358+0000 7fd277400640 1 -- 172.21.15.49:0/1616505586 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd278096e50 con 0x7fd280102c30 2024-02-26T06:22:50.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.562+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2801a1d80 con 0x7fd280102c30 2024-02-26T06:22:50.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.565+0000 7fd277400640 1 -- 172.21.15.49:0/1616505586 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd27809a510 con 0x7fd280102c30 2024-02-26T06:22:50.565 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:22:50.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.567+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2600a2140 msgr2=0x7fd2600a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.567+0000 7fd27fe00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2600a2140 0x7fd2600a4600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd2801020d0 tx=0x7fd270009260 comp rx=0 tx=0).stop 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.567+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102c30 msgr2=0x7fd2801a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.567+0000 7fd27fe00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102c30 0x7fd2801a08a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd27800e970 tx=0x7fd27800ed70 comp rx=0 tx=0).stop 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.568+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 shutdown_connections 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.568+0000 7fd27fe00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2600a2140 0x7fd2600a4600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.568+0000 7fd27fe00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd280102c30 0x7fd2801a08a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.568+0000 7fd27fe00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd280102270 0x7fd280199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.568+0000 7fd27fe00640 1 --2- 172.21.15.49:0/1616505586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd280101080 0x7fd2801992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.568+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 >> 172.21.15.49:0/1616505586 conn(0x7fd2800fc5f0 msgr2=0x7fd2800733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.568+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 shutdown_connections 2024-02-26T06:22:50.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.568+0000 7fd27fe00640 1 -- 172.21.15.49:0/1616505586 wait complete. 2024-02-26T06:22:50.588 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:22:50.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 -- 172.21.15.49:0/2061347245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a41010e0 msgr2=0x7f78a4101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 --2- 172.21.15.49:0/2061347245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a41010e0 0x7f78a4101560 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7894009900 tx=0x7f789402bc50 comp rx=0 tx=0).stop 2024-02-26T06:22:50.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 -- 172.21.15.49:0/2061347245 shutdown_connections 2024-02-26T06:22:50.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 --2- 172.21.15.49:0/2061347245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a4101aa0 0x7f78a41083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 --2- 172.21.15.49:0/2061347245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a41010e0 0x7f78a4101560 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 --2- 172.21.15.49:0/2061347245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78a40fff10 0x7f78a4100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 -- 172.21.15.49:0/2061347245 >> 172.21.15.49:0/2061347245 conn(0x7f78a40fb4f0 msgr2=0x7f78a40fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:50.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 -- 172.21.15.49:0/2061347245 shutdown_connections 2024-02-26T06:22:50.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.824+0000 7f78a9a00640 1 -- 172.21.15.49:0/2061347245 wait complete. 2024-02-26T06:22:50.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.825+0000 7f78a9a00640 1 Processor -- start 2024-02-26T06:22:50.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.825+0000 7f78a9a00640 1 -- start start 2024-02-26T06:22:50.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a9a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a40fff10 0x7f78a4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:50.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a9a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a41010e0 0x7f78a4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:50.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a9a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78a4101aa0 0x7f78a41a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:50.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a9a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a41a0e10 con 0x7f78a40fff10 2024-02-26T06:22:50.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a9a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a41a0f80 con 0x7f78a41010e0 2024-02-26T06:22:50.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a9a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a41a1140 con 0x7f78a4101aa0 2024-02-26T06:22:50.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a40fff10 0x7f78a4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a41010e0 0x7f78a4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a41010e0 0x7f78a4199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38554/0 (socket says 172.21.15.49:38554) 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3400640 1 -- 172.21.15.49:0/1986061255 learned_addr learned my addr 172.21.15.49:0/1986061255 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a9000640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78a4101aa0 0x7f78a41a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3400640 1 -- 172.21.15.49:0/1986061255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78a4101aa0 msgr2=0x7f78a41a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3400640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78a4101aa0 0x7f78a41a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3400640 1 -- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a40fff10 msgr2=0x7f78a4199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3400640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a40fff10 0x7f78a4199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3400640 1 -- 172.21.15.49:0/1986061255 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7894009560 con 0x7f78a41010e0 2024-02-26T06:22:50.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.826+0000 7f78a3e00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a40fff10 0x7f78a4199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:50.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.827+0000 7f78a3400640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a41010e0 0x7f78a4199860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7894009230 tx=0x7f7894004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:50.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.827+0000 7f78a0c00640 1 -- 172.21.15.49:0/1986061255 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894004440 con 0x7f78a41010e0 2024-02-26T06:22:50.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.827+0000 7f78a0c00640 1 -- 172.21.15.49:0/1986061255 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78940052d0 con 0x7f78a41010e0 2024-02-26T06:22:50.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.827+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a41a13c0 con 0x7f78a41010e0 2024-02-26T06:22:50.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.827+0000 7f78a0c00640 1 -- 172.21.15.49:0/1986061255 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894003d00 con 0x7f78a41010e0 2024-02-26T06:22:50.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.827+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a41a18d0 con 0x7f78a41010e0 2024-02-26T06:22:50.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:50 smithi080 ceph-mon[36206]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T06:22:50.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:50 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/165454519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:50.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:50 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3577318822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:50.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.828+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a410b950 con 0x7f78a41010e0 2024-02-26T06:22:50.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.830+0000 7f78a0c00640 1 -- 172.21.15.49:0/1986061255 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7894005440 con 0x7f78a41010e0 2024-02-26T06:22:50.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.831+0000 7f78a0c00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78880a20c0 0x7f78880a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:50.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.832+0000 7f78a0c00640 1 -- 172.21.15.49:0/1986061255 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f78940eff10 con 0x7f78a41010e0 2024-02-26T06:22:50.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.832+0000 7f78a3e00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78880a20c0 0x7f78880a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:50.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.832+0000 7f78a3e00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78880a20c0 0x7f78880a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78a40ffc30 tx=0x7f789c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:50.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:50.836+0000 7f78a0c00640 1 -- 172.21.15.49:0/1986061255 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78940b8e70 con 0x7f78a41010e0 2024-02-26T06:22:50.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:50 smithi115 ceph-mon[36426]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T06:22:50.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:50 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/165454519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:50.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:50 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3577318822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:50.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:50 smithi049 ceph-mon[31696]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T06:22:50.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:50 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/165454519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:50.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:50 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3577318822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:51.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.040+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78a41a1cb0 con 0x7f78a41010e0 2024-02-26T06:22:51.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.041+0000 7f78a0c00640 1 -- 172.21.15.49:0/1986061255 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7894003e60 con 0x7f78a41010e0 2024-02-26T06:22:51.042 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:22:51.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78880a20c0 msgr2=0x7f78880a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78880a20c0 0x7f78880a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78a40ffc30 tx=0x7f789c009500 comp rx=0 tx=0).stop 2024-02-26T06:22:51.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a41010e0 msgr2=0x7f78a4199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a41010e0 0x7f78a4199860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7894009230 tx=0x7f7894004040 comp rx=0 tx=0).stop 2024-02-26T06:22:51.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 shutdown_connections 2024-02-26T06:22:51.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78880a20c0 0x7f78880a4580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a41010e0 0x7f78a4199860 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78a4101aa0 0x7f78a41a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 --2- 172.21.15.49:0/1986061255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a40fff10 0x7f78a4199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.043+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 >> 172.21.15.49:0/1986061255 conn(0x7f78a40fb4f0 msgr2=0x7f78a40fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:51.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.044+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 shutdown_connections 2024-02-26T06:22:51.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.044+0000 7f78a9a00640 1 -- 172.21.15.49:0/1986061255 wait complete. 2024-02-26T06:22:51.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 -- 172.21.15.49:0/905545856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 msgr2=0x7fd368107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 --2- 172.21.15.49:0/905545856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 0x7fd368107a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd358009900 tx=0x7fd35802bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 -- 172.21.15.49:0/905545856 shutdown_connections 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 --2- 172.21.15.49:0/905545856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3681087d0 0x7fd368108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 --2- 172.21.15.49:0/905545856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd368109190 0x7fd36810fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 --2- 172.21.15.49:0/905545856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 0x7fd368107a00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 -- 172.21.15.49:0/905545856 >> 172.21.15.49:0/905545856 conn(0x7fd368076590 msgr2=0x7fd3680789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 -- 172.21.15.49:0/905545856 shutdown_connections 2024-02-26T06:22:51.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.294+0000 7fd36ca00640 1 -- 172.21.15.49:0/905545856 wait complete. 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.295+0000 7fd36ca00640 1 Processor -- start 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.295+0000 7fd36ca00640 1 -- start start 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.295+0000 7fd36ca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 0x7fd36819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.295+0000 7fd36ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3681087d0 0x7fd36819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.295+0000 7fd36ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd368109190 0x7fd3681a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.295+0000 7fd36ca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681a5190 con 0x7fd3681075e0 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.295+0000 7fd36ca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681a5300 con 0x7fd3681087d0 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.295+0000 7fd36ca00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681a54c0 con 0x7fd368109190 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd366800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 0x7fd36819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd366800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 0x7fd36819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44490/0 (socket says 172.21.15.49:44490) 2024-02-26T06:22:51.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd366800640 1 -- 172.21.15.49:0/1517804197 learned_addr learned my addr 172.21.15.49:0/1517804197 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd365e00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3681087d0 0x7fd36819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd367200640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd368109190 0x7fd3681a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd366800640 1 -- 172.21.15.49:0/1517804197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd368109190 msgr2=0x7fd3681a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd366800640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd368109190 0x7fd3681a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd366800640 1 -- 172.21.15.49:0/1517804197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3681087d0 msgr2=0x7fd36819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd366800640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3681087d0 0x7fd36819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd366800640 1 -- 172.21.15.49:0/1517804197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd360009840 con 0x7fd3681075e0 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.296+0000 7fd365e00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3681087d0 0x7fd36819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.297+0000 7fd366800640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 0x7fd36819d6a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd358005ab0 tx=0x7fd358005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.297+0000 7fd35f400640 1 -- 172.21.15.49:0/1517804197 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3580043d0 con 0x7fd3681075e0 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.297+0000 7fd35f400640 1 -- 172.21.15.49:0/1517804197 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3580052d0 con 0x7fd3681075e0 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.297+0000 7fd35f400640 1 -- 172.21.15.49:0/1517804197 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358003cd0 con 0x7fd3681075e0 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.297+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd358009560 con 0x7fd3681075e0 2024-02-26T06:22:51.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.297+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3681a5ac0 con 0x7fd3681075e0 2024-02-26T06:22:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.298+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36810e920 con 0x7fd3681075e0 2024-02-26T06:22:51.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.300+0000 7fd35f400640 1 -- 172.21.15.49:0/1517804197 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd358005440 con 0x7fd3681075e0 2024-02-26T06:22:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.302+0000 7fd35f400640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd34c0a20c0 0x7fd34c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.302+0000 7fd35f400640 1 -- 172.21.15.49:0/1517804197 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd3580efe40 con 0x7fd3681075e0 2024-02-26T06:22:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.302+0000 7fd365e00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd34c0a20c0 0x7fd34c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.303+0000 7fd365e00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd34c0a20c0 0x7fd34c0a4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd36819ebb0 tx=0x7fd354005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:51.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.306+0000 7fd35f400640 1 -- 172.21.15.49:0/1517804197 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd3580b8da0 con 0x7fd3681075e0 2024-02-26T06:22:51.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.547+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd36802cdd0 con 0x7fd3681075e0 2024-02-26T06:22:51.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.548+0000 7fd35f400640 1 -- 172.21.15.49:0/1517804197 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11331 (secure 0 0 0) 0x7fd3580bc460 con 0x7fd3681075e0 2024-02-26T06:22:51.549 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:51.549 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:20:38.098815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:22:51.549 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-02-26T06:22:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd34c0a20c0 msgr2=0x7fd34c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd34c0a20c0 0x7fd34c0a4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd36819ebb0 tx=0x7fd354005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 msgr2=0x7fd36819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 0x7fd36819d6a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd358005ab0 tx=0x7fd358005e50 comp rx=0 tx=0).stop 2024-02-26T06:22:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 shutdown_connections 2024-02-26T06:22:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd34c0a20c0 0x7fd34c0a4580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3681087d0 0x7fd36819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd368109190 0x7fd3681a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 --2- 172.21.15.49:0/1517804197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3681075e0 0x7fd36819d6a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 >> 172.21.15.49:0/1517804197 conn(0x7fd368076590 msgr2=0x7fd36810baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 shutdown_connections 2024-02-26T06:22:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.551+0000 7fd36ca00640 1 -- 172.21.15.49:0/1517804197 wait complete. 2024-02-26T06:22:51.572 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:22:51.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.799+0000 7fba3b200640 1 -- 172.21.15.49:0/2501197742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba34100fc0 msgr2=0x7fba341013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.799+0000 7fba3b200640 1 --2- 172.21.15.49:0/2501197742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba34100fc0 0x7fba341013e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009ca0 tx=0x7fba2c02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.800+0000 7fba3b200640 1 -- 172.21.15.49:0/2501197742 shutdown_connections 2024-02-26T06:22:51.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.800+0000 7fba3b200640 1 --2- 172.21.15.49:0/2501197742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba34102b70 0x7fba34109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.800+0000 7fba3b200640 1 --2- 172.21.15.49:0/2501197742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba341021b0 0x7fba34102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.800+0000 7fba3b200640 1 --2- 172.21.15.49:0/2501197742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba34100fc0 0x7fba341013e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.800+0000 7fba3b200640 1 -- 172.21.15.49:0/2501197742 >> 172.21.15.49:0/2501197742 conn(0x7fba340fc510 msgr2=0x7fba340fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:51.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.800+0000 7fba3b200640 1 -- 172.21.15.49:0/2501197742 shutdown_connections 2024-02-26T06:22:51.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.800+0000 7fba3b200640 1 -- 172.21.15.49:0/2501197742 wait complete. 2024-02-26T06:22:51.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.801+0000 7fba3b200640 1 Processor -- start 2024-02-26T06:22:51.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.801+0000 7fba3b200640 1 -- start start 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba3b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba34100fc0 0x7fba341970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba3b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba341021b0 0x7fba341975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba3b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba34102b70 0x7fba3419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba3b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419eb90 con 0x7fba34100fc0 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba3b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419ed00 con 0x7fba34102b70 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba3b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419eec0 con 0x7fba341021b0 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba39e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba34100fc0 0x7fba341970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba39e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba34100fc0 0x7fba341970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44514/0 (socket says 172.21.15.49:44514) 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba39e00640 1 -- 172.21.15.49:0/967620976 learned_addr learned my addr 172.21.15.49:0/967620976 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba39400640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba341021b0 0x7fba341975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.802+0000 7fba3a800640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba34102b70 0x7fba3419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba39400640 1 -- 172.21.15.49:0/967620976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba34102b70 msgr2=0x7fba3419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba39400640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba34102b70 0x7fba3419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba39400640 1 -- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba34100fc0 msgr2=0x7fba341970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba39400640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba34100fc0 0x7fba341970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba39400640 1 -- 172.21.15.49:0/967620976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24009490 con 0x7fba341021b0 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba39e00640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba34100fc0 0x7fba341970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba39400640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba341021b0 0x7fba341975e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba2400e9f0 tx=0x7fba2400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba2aa00640 1 -- 172.21.15.49:0/967620976 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2400cd80 con 0x7fba341021b0 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c009900 con 0x7fba341021b0 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba2aa00640 1 -- 172.21.15.49:0/967620976 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba240035f0 con 0x7fba341021b0 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba2aa00640 1 -- 172.21.15.49:0/967620976 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24010690 con 0x7fba341021b0 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.803+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3419f570 con 0x7fba341021b0 2024-02-26T06:22:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.804+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba34108330 con 0x7fba341021b0 2024-02-26T06:22:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.806+0000 7fba2aa00640 1 -- 172.21.15.49:0/967620976 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fba24003af0 con 0x7fba341021b0 2024-02-26T06:22:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.808+0000 7fba2aa00640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fba200a2140 0x7fba200a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.809+0000 7fba2aa00640 1 -- 172.21.15.49:0/967620976 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fba24014070 con 0x7fba341021b0 2024-02-26T06:22:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.809+0000 7fba39e00640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fba200a2140 0x7fba200a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.809+0000 7fba39e00640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fba200a2140 0x7fba200a4600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba2c005e30 tx=0x7fba2c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:51.810+0000 7fba2aa00640 1 -- 172.21.15.49:0/967620976 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba24096fc0 con 0x7fba341021b0 2024-02-26T06:22:51.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:51 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1616505586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:51.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:51 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1986061255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:51.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:51 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1517804197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:51.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:51 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1616505586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:51.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:51 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1986061255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:51.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:51 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1517804197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:51.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:51 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1616505586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:51.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:51 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1986061255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:51.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:51 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1517804197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:52.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.014+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba3410c990 con 0x7fba341021b0 2024-02-26T06:22:52.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.016+0000 7fba2aa00640 1 -- 172.21.15.49:0/967620976 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7fba2409a680 con 0x7fba341021b0 2024-02-26T06:22:52.017 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:52.018 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fba200a2140 msgr2=0x7fba200a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fba200a2140 0x7fba200a4600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba2c005e30 tx=0x7fba2c004a40 comp rx=0 tx=0).stop 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba341021b0 msgr2=0x7fba341975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba341021b0 0x7fba341975e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba2400e9f0 tx=0x7fba2400eeb0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 shutdown_connections 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fba200a2140 0x7fba200a4600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba34102b70 0x7fba3419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba341021b0 0x7fba341975e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 --2- 172.21.15.49:0/967620976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba34100fc0 0x7fba341970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 >> 172.21.15.49:0/967620976 conn(0x7fba340fc510 msgr2=0x7fba34105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 shutdown_connections 2024-02-26T06:22:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.019+0000 7fba3b200640 1 -- 172.21.15.49:0/967620976 wait complete. 2024-02-26T06:22:52.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:22:52.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:22:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 -- 172.21.15.49:0/1001999609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78075140 msgr2=0x7eff78073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 --2- 172.21.15.49:0/1001999609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78075140 0x7eff78073590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009900 tx=0x7eff6c02bcd0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 -- 172.21.15.49:0/1001999609 shutdown_connections 2024-02-26T06:22:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 --2- 172.21.15.49:0/1001999609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff78073b60 0x7eff78073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 --2- 172.21.15.49:0/1001999609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff78108f30 0x7eff7810b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 --2- 172.21.15.49:0/1001999609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78075140 0x7eff78073590 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 -- 172.21.15.49:0/1001999609 >> 172.21.15.49:0/1001999609 conn(0x7eff780fc5f0 msgr2=0x7eff780fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:52.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 -- 172.21.15.49:0/1001999609 shutdown_connections 2024-02-26T06:22:52.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.278+0000 7eff7e800640 1 -- 172.21.15.49:0/1001999609 wait complete. 2024-02-26T06:22:52.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.279+0000 7eff7e800640 1 Processor -- start 2024-02-26T06:22:52.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.279+0000 7eff7e800640 1 -- start start 2024-02-26T06:22:52.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78073b60 0x7eff781992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:52.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff78075140 0x7eff78199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:52.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff78108f30 0x7eff781a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:52.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7e800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff781a0de0 con 0x7eff78073b60 2024-02-26T06:22:52.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7e800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff781a0f50 con 0x7eff78108f30 2024-02-26T06:22:52.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7e800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff781a1110 con 0x7eff78075140 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78073b60 0x7eff781992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78073b60 0x7eff781992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44544/0 (socket says 172.21.15.49:44544) 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7d400640 1 -- 172.21.15.49:0/1903812273 learned_addr learned my addr 172.21.15.49:0/1903812273 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7ca00640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff78075140 0x7eff78199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.280+0000 7eff7de00640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff78108f30 0x7eff781a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff7d400640 1 -- 172.21.15.49:0/1903812273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff78075140 msgr2=0x7eff78199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff7d400640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff78075140 0x7eff78199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff7d400640 1 -- 172.21.15.49:0/1903812273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff78108f30 msgr2=0x7eff781a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff7d400640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff78108f30 0x7eff781a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff7d400640 1 -- 172.21.15.49:0/1903812273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff64009490 con 0x7eff78073b60 2024-02-26T06:22:52.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff7d400640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78073b60 0x7eff781992f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff6c000c00 tx=0x7eff6c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:52.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff76000640 1 -- 172.21.15.49:0/1903812273 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c02dd20 con 0x7eff78073b60 2024-02-26T06:22:52.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff76000640 1 -- 172.21.15.49:0/1903812273 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff6c02de80 con 0x7eff78073b60 2024-02-26T06:22:52.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c009560 con 0x7eff78073b60 2024-02-26T06:22:52.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.281+0000 7eff76000640 1 -- 172.21.15.49:0/1903812273 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c003d30 con 0x7eff78073b60 2024-02-26T06:22:52.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.282+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff781a1760 con 0x7eff78073b60 2024-02-26T06:22:52.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.283+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff78107cf0 con 0x7eff78073b60 2024-02-26T06:22:52.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.285+0000 7eff76000640 1 -- 172.21.15.49:0/1903812273 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7eff6c004df0 con 0x7eff78073b60 2024-02-26T06:22:52.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.286+0000 7eff76000640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7eff600a2140 0x7eff600a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:52.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.286+0000 7eff76000640 1 -- 172.21.15.49:0/1903812273 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7eff6c0f0320 con 0x7eff78073b60 2024-02-26T06:22:52.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.287+0000 7eff7ca00640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7eff600a2140 0x7eff600a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:52.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.287+0000 7eff7ca00640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7eff600a2140 0x7eff600a4600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff7819a800 tx=0x7eff64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:52.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.290+0000 7eff76000640 1 -- 172.21.15.49:0/1903812273 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff6c0b9280 con 0x7eff78073b60 2024-02-26T06:22:52.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.492+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff7802cda0 con 0x7eff78073b60 2024-02-26T06:22:52.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.492+0000 7eff76000640 1 -- 172.21.15.49:0/1903812273 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff6c0bc940 con 0x7eff78073b60 2024-02-26T06:22:52.493 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7eff600a2140 msgr2=0x7eff600a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7eff600a2140 0x7eff600a4600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff7819a800 tx=0x7eff64005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78073b60 msgr2=0x7eff781992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78073b60 0x7eff781992f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff6c000c00 tx=0x7eff6c005e30 comp rx=0 tx=0).stop 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 shutdown_connections 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7eff600a2140 0x7eff600a4600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff78108f30 0x7eff781a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff78075140 0x7eff78199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 --2- 172.21.15.49:0/1903812273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff78073b60 0x7eff781992f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 >> 172.21.15.49:0/1903812273 conn(0x7eff780fc5f0 msgr2=0x7eff78104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:52.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 shutdown_connections 2024-02-26T06:22:52.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.495+0000 7eff7e800640 1 -- 172.21.15.49:0/1903812273 wait complete. 2024-02-26T06:22:52.514 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:22:52.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.739+0000 7f1953000640 1 -- 172.21.15.49:0/2340283064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c106d40 msgr2=0x7f194c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.739+0000 7f1953000640 1 --2- 172.21.15.49:0/2340283064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c106d40 0x7f194c109190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1944009900 tx=0x7f194402bc40 comp rx=0 tx=0).stop 2024-02-26T06:22:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.740+0000 7f1953000640 1 -- 172.21.15.49:0/2340283064 shutdown_connections 2024-02-26T06:22:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.740+0000 7f1953000640 1 --2- 172.21.15.49:0/2340283064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c106d40 0x7f194c109190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.740+0000 7f1953000640 1 --2- 172.21.15.49:0/2340283064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194c104410 0x7f194c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.740+0000 7f1953000640 1 --2- 172.21.15.49:0/2340283064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f194c101ae0 0x7f194c103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.740+0000 7f1953000640 1 -- 172.21.15.49:0/2340283064 >> 172.21.15.49:0/2340283064 conn(0x7f194c0fb4f0 msgr2=0x7f194c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.740+0000 7f1953000640 1 -- 172.21.15.49:0/2340283064 shutdown_connections 2024-02-26T06:22:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.740+0000 7f1953000640 1 -- 172.21.15.49:0/2340283064 wait complete. 2024-02-26T06:22:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.741+0000 7f1953000640 1 Processor -- start 2024-02-26T06:22:52.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.741+0000 7f1953000640 1 -- start start 2024-02-26T06:22:52.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.741+0000 7f1953000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194c101ae0 0x7f194c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:52.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1953000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c104410 0x7f194c0ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:52.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1953000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f194c106d40 0x7f194c0fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1953000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c100730 con 0x7f194c101ae0 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1953000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c1008a0 con 0x7f194c106d40 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1953000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c1a1130 con 0x7f194c104410 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1951c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194c101ae0 0x7f194c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1952600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f194c106d40 0x7f194c0fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1951200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c104410 0x7f194c0ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1952600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f194c106d40 0x7f194c0fffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38640/0 (socket says 172.21.15.49:38640) 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1951200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c104410 0x7f194c0ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44450/0 (socket says 172.21.15.49:44450) 2024-02-26T06:22:52.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1952600640 1 -- 172.21.15.49:0/573850009 learned_addr learned my addr 172.21.15.49:0/573850009 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:52.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1951200640 1 -- 172.21.15.49:0/573850009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f194c106d40 msgr2=0x7f194c0fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1951200640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f194c106d40 0x7f194c0fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1951200640 1 -- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194c101ae0 msgr2=0x7f194c101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1951200640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194c101ae0 0x7f194c101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1951200640 1 -- 172.21.15.49:0/573850009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944009560 con 0x7f194c104410 2024-02-26T06:22:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.742+0000 7f1952600640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f194c106d40 0x7f194c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.743+0000 7f1951200640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c104410 0x7f194c0ffa90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f193800d6a0 tx=0x7f193800db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.743+0000 7f194a000640 1 -- 172.21.15.49:0/573850009 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938005220 con 0x7f194c104410 2024-02-26T06:22:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.743+0000 7f194a000640 1 -- 172.21.15.49:0/573850009 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1938009d00 con 0x7f194c104410 2024-02-26T06:22:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.743+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c1a13e0 con 0x7f194c104410 2024-02-26T06:22:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.743+0000 7f194a000640 1 -- 172.21.15.49:0/573850009 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938004ea0 con 0x7f194c104410 2024-02-26T06:22:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.743+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c1a18f0 con 0x7f194c104410 2024-02-26T06:22:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.744+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c0fcfe0 con 0x7f194c104410 2024-02-26T06:22:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.746+0000 7f194a000640 1 -- 172.21.15.49:0/573850009 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1938005510 con 0x7f194c104410 2024-02-26T06:22:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.747+0000 7f194a000640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19300a20c0 0x7f19300a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.747+0000 7f194a000640 1 -- 172.21.15.49:0/573850009 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f19380cd4f0 con 0x7f194c104410 2024-02-26T06:22:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.748+0000 7f1951c00640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19300a20c0 0x7f19300a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:52.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.748+0000 7f1951c00640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19300a20c0 0x7f19300a4580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f193c005fd0 tx=0x7f193c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.752+0000 7f194a000640 1 -- 172.21.15.49:0/573850009 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1938096500 con 0x7f194c104410 2024-02-26T06:22:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:52 smithi080 ceph-mon[36206]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T06:22:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:52 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/967620976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:52 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1903812273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:52.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:52 smithi115 ceph-mon[36426]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T06:22:52.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:52 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/967620976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:52.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:52 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1903812273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:52.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:52 smithi049 ceph-mon[31696]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T06:22:52.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:52 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/967620976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:52.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:52 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1903812273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:22:52.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.952+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f194c0fca50 con 0x7f194c104410 2024-02-26T06:22:52.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.953+0000 7f194a000640 1 -- 172.21.15.49:0/573850009 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1938020020 con 0x7f194c104410 2024-02-26T06:22:52.954 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:22:52.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.956+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19300a20c0 msgr2=0x7f19300a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.956+0000 7f1953000640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19300a20c0 0x7f19300a4580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f193c005fd0 tx=0x7f193c009500 comp rx=0 tx=0).stop 2024-02-26T06:22:52.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.956+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c104410 msgr2=0x7f194c0ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:52.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.956+0000 7f1953000640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c104410 0x7f194c0ffa90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f193800d6a0 tx=0x7f193800db60 comp rx=0 tx=0).stop 2024-02-26T06:22:52.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.956+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 shutdown_connections 2024-02-26T06:22:52.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.956+0000 7f1953000640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f19300a20c0 0x7f19300a4580 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.957+0000 7f1953000640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f194c106d40 0x7f194c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.957+0000 7f1953000640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f194c104410 0x7f194c0ffa90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.957+0000 7f1953000640 1 --2- 172.21.15.49:0/573850009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194c101ae0 0x7f194c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:52.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.957+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 >> 172.21.15.49:0/573850009 conn(0x7f194c0fb4f0 msgr2=0x7f194c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:52.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.957+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 shutdown_connections 2024-02-26T06:22:52.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:52.957+0000 7f1953000640 1 -- 172.21.15.49:0/573850009 wait complete. 2024-02-26T06:22:52.976 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:22:53.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 -- 172.21.15.49:0/228808488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7c8100230 msgr2=0x7fe7c8100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 --2- 172.21.15.49:0/228808488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7c8100230 0x7fe7c8100690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009900 tx=0x7fe7b002bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:53.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 -- 172.21.15.49:0/228808488 shutdown_connections 2024-02-26T06:22:53.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 --2- 172.21.15.49:0/228808488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7c8100230 0x7fe7c8100690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 --2- 172.21.15.49:0/228808488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7c8100bd0 0x7fe7c8101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 --2- 172.21.15.49:0/228808488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7c80ff8d0 0x7fe7c80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 -- 172.21.15.49:0/228808488 >> 172.21.15.49:0/228808488 conn(0x7fe7c80fb4f0 msgr2=0x7fe7c80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:53.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 -- 172.21.15.49:0/228808488 shutdown_connections 2024-02-26T06:22:53.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.208+0000 7fe7cde00640 1 -- 172.21.15.49:0/228808488 wait complete. 2024-02-26T06:22:53.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.210+0000 7fe7cde00640 1 Processor -- start 2024-02-26T06:22:53.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.210+0000 7fe7cde00640 1 -- start start 2024-02-26T06:22:53.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.210+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7c80ff8d0 0x7fe7c81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.210+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7c8100230 0x7fe7c8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7c8100bd0 0x7fe7c81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7cde00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81a0dd0 con 0x7fe7c8100230 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7cde00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81a0f40 con 0x7fe7c80ff8d0 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7c8100230 0x7fe7c8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7cde00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81a1100 con 0x7fe7c8100bd0 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7c8100230 0x7fe7c8199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44586/0 (socket says 172.21.15.49:44586) 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7c7e00640 1 -- 172.21.15.49:0/121777170 learned_addr learned my addr 172.21.15.49:0/121777170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7cca00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7c80ff8d0 0x7fe7c81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7cd400640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7c8100bd0 0x7fe7c81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7c7e00640 1 -- 172.21.15.49:0/121777170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7c8100bd0 msgr2=0x7fe7c81a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7c7e00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7c8100bd0 0x7fe7c81a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7c7e00640 1 -- 172.21.15.49:0/121777170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7c80ff8d0 msgr2=0x7fe7c81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7c7e00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7c80ff8d0 0x7fe7c81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7c7e00640 1 -- 172.21.15.49:0/121777170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b0009560 con 0x7fe7c8100230 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.211+0000 7fe7cd400640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7c8100bd0 0x7fe7c81a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.212+0000 7fe7c7e00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7c8100230 0x7fe7c8199820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0006010 tx=0x7fe7b0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.212+0000 7fe7c5600640 1 -- 172.21.15.49:0/121777170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00043d0 con 0x7fe7c8100230 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.212+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c81a1380 con 0x7fe7c8100230 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.212+0000 7fe7c5600640 1 -- 172.21.15.49:0/121777170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7b00052d0 con 0x7fe7c8100230 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.212+0000 7fe7c5600640 1 -- 172.21.15.49:0/121777170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0003cc0 con 0x7fe7c8100230 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.212+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c81a18e0 con 0x7fe7c8100230 2024-02-26T06:22:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.213+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c810a620 con 0x7fe7c8100230 2024-02-26T06:22:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.215+0000 7fe7c5600640 1 -- 172.21.15.49:0/121777170 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe7b0005440 con 0x7fe7c8100230 2024-02-26T06:22:53.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.217+0000 7fe7c5600640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7ac0a20c0 0x7fe7ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.217+0000 7fe7c5600640 1 -- 172.21.15.49:0/121777170 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe7b00f04e0 con 0x7fe7c8100230 2024-02-26T06:22:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.217+0000 7fe7cca00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7ac0a20c0 0x7fe7ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.218+0000 7fe7cca00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7ac0a20c0 0x7fe7ac0a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005fd0 tx=0x7fe7b8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:53.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.221+0000 7fe7c5600640 1 -- 172.21.15.49:0/121777170 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe7b00b9440 con 0x7fe7c8100230 2024-02-26T06:22:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.425+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7c810eca0 con 0x7fe7c8100230 2024-02-26T06:22:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.426+0000 7fe7c5600640 1 -- 172.21.15.49:0/121777170 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7b0043020 con 0x7fe7c8100230 2024-02-26T06:22:53.427 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7ac0a20c0 msgr2=0x7fe7ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7ac0a20c0 0x7fe7ac0a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005fd0 tx=0x7fe7b8009500 comp rx=0 tx=0).stop 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7c8100230 msgr2=0x7fe7c8199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7c8100230 0x7fe7c8199820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0006010 tx=0x7fe7b0005e80 comp rx=0 tx=0).stop 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 shutdown_connections 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe7ac0a20c0 0x7fe7ac0a4580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7c8100bd0 0x7fe7c81a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7c8100230 0x7fe7c8199820 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 --2- 172.21.15.49:0/121777170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7c80ff8d0 0x7fe7c81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 >> 172.21.15.49:0/121777170 conn(0x7fe7c80fb4f0 msgr2=0x7fe7c80fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 shutdown_connections 2024-02-26T06:22:53.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.429+0000 7fe7cde00640 1 -- 172.21.15.49:0/121777170 wait complete. 2024-02-26T06:22:53.452 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:22:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 -- 172.21.15.49:0/2269957182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 msgr2=0x7fdba410b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 --2- 172.21.15.49:0/2269957182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 0x7fdba410b4e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009ca0 tx=0x7fdb9c02c180 comp rx=0 tx=0).stop 2024-02-26T06:22:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 -- 172.21.15.49:0/2269957182 shutdown_connections 2024-02-26T06:22:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 --2- 172.21.15.49:0/2269957182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 0x7fdba410b4e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 --2- 172.21.15.49:0/2269957182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba4073b60 0x7fdba4073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 --2- 172.21.15.49:0/2269957182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdba4075140 0x7fdba4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 -- 172.21.15.49:0/2269957182 >> 172.21.15.49:0/2269957182 conn(0x7fdba40fc730 msgr2=0x7fdba40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 -- 172.21.15.49:0/2269957182 shutdown_connections 2024-02-26T06:22:53.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.682+0000 7fdbaac00640 1 -- 172.21.15.49:0/2269957182 wait complete. 2024-02-26T06:22:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.683+0000 7fdbaac00640 1 Processor -- start 2024-02-26T06:22:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.683+0000 7fdbaac00640 1 -- start start 2024-02-26T06:22:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdbaac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdba4073b60 0x7fdba41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdbaac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba4075140 0x7fdba41998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdbaac00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 0x7fdba41a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdbaac00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba41a0e90 con 0x7fdba4109050 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdbaac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba41a1000 con 0x7fdba4073b60 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdbaac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba41a11c0 con 0x7fdba4075140 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdba8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 0x7fdba41a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdba8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 0x7fdba41a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44604/0 (socket says 172.21.15.49:44604) 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdba3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdba4073b60 0x7fdba41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdba8c00640 1 -- 172.21.15.49:0/1224882248 learned_addr learned my addr 172.21.15.49:0/1224882248 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdba3400640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba4075140 0x7fdba41998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdba8c00640 1 -- 172.21.15.49:0/1224882248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba4075140 msgr2=0x7fdba41998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.684+0000 7fdba8c00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba4075140 0x7fdba41998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba8c00640 1 -- 172.21.15.49:0/1224882248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdba4073b60 msgr2=0x7fdba41993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba8c00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdba4073b60 0x7fdba41993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba8c00640 1 -- 172.21.15.49:0/1224882248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94009490 con 0x7fdba4109050 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba3e00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdba4073b60 0x7fdba41993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba3400640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba4075140 0x7fdba41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:53.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba8c00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 0x7fdba41a0950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c02c6c0 tx=0x7fdb9c034e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:53.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba0c00640 1 -- 172.21.15.49:0/1224882248 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c003d80 con 0x7fdba4109050 2024-02-26T06:22:53.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba0c00640 1 -- 172.21.15.49:0/1224882248 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c032d70 con 0x7fdba4109050 2024-02-26T06:22:53.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c009900 con 0x7fdba4109050 2024-02-26T06:22:53.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.685+0000 7fdba0c00640 1 -- 172.21.15.49:0/1224882248 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c004770 con 0x7fdba4109050 2024-02-26T06:22:53.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.686+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba41a1760 con 0x7fdba4109050 2024-02-26T06:22:53.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.687+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba4107e10 con 0x7fdba4109050 2024-02-26T06:22:53.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.690+0000 7fdba0c00640 1 -- 172.21.15.49:0/1224882248 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdb9c00a040 con 0x7fdba4109050 2024-02-26T06:22:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.692+0000 7fdba0c00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdb880a20c0 0x7fdb880a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.692+0000 7fdba0c00640 1 -- 172.21.15.49:0/1224882248 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fdb9c0f0e10 con 0x7fdba4109050 2024-02-26T06:22:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.692+0000 7fdba3e00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdb880a20c0 0x7fdb880a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.692+0000 7fdba0c00640 1 -- 172.21.15.49:0/1224882248 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb9c0b9d70 con 0x7fdba4109050 2024-02-26T06:22:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.693+0000 7fdba3e00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdb880a20c0 0x7fdb880a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb94011630 tx=0x7fdb94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:53.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:53 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/573850009' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:53.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:53 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/121777170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:53.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:53 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/573850009' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:53.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:53 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/121777170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:53.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:53 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/573850009' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:22:53.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:53 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/121777170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:22:53.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.959+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdba410edc0 con 0x7fdba4109050 2024-02-26T06:22:53.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.960+0000 7fdba0c00640 1 -- 172.21.15.49:0/1224882248 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11331 (secure 0 0 0) 0x7fdb9c0bd430 con 0x7fdba4109050 2024-02-26T06:22:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:53.961 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:20:38.098815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:22:53.961 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-02-26T06:22:53.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdb880a20c0 msgr2=0x7fdb880a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdb880a20c0 0x7fdb880a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb94011630 tx=0x7fdb94005f90 comp rx=0 tx=0).stop 2024-02-26T06:22:53.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 msgr2=0x7fdba41a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 0x7fdba41a0950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c02c6c0 tx=0x7fdb9c034e20 comp rx=0 tx=0).stop 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 shutdown_connections 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdb880a20c0 0x7fdb880a4580 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdba4109050 0x7fdba41a0950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba4075140 0x7fdba41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 --2- 172.21.15.49:0/1224882248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdba4073b60 0x7fdba41993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.962+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 >> 172.21.15.49:0/1224882248 conn(0x7fdba40fc730 msgr2=0x7fdba4104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.963+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 shutdown_connections 2024-02-26T06:22:53.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:53.963+0000 7fdbaac00640 1 -- 172.21.15.49:0/1224882248 wait complete. 2024-02-26T06:22:53.983 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 -- 172.21.15.49:0/3738486670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78cc075b40 msgr2=0x7f78cc075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3738486670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78cc075b40 0x7f78cc075f60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009900 tx=0x7f78bc02bc30 comp rx=0 tx=0).stop 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 -- 172.21.15.49:0/3738486670 shutdown_connections 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3738486670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc077720 0x7f78cc077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3738486670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78cc076d80 0x7f78cc0771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3738486670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78cc075b40 0x7f78cc075f60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 -- 172.21.15.49:0/3738486670 >> 172.21.15.49:0/3738486670 conn(0x7f78cc0fe930 msgr2=0x7f78cc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 -- 172.21.15.49:0/3738486670 shutdown_connections 2024-02-26T06:22:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.193+0000 7f78d2c00640 1 -- 172.21.15.49:0/3738486670 wait complete. 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.194+0000 7f78d2c00640 1 Processor -- start 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.194+0000 7f78d2c00640 1 -- start start 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.194+0000 7f78d2c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc075b40 0x7f78cc19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.194+0000 7f78d2c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78cc076d80 0x7f78cc19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.194+0000 7f78d2c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78cc077720 0x7f78cc1a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.194+0000 7f78d2c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1a52f0 con 0x7f78cc077720 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.194+0000 7f78d2c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1a5460 con 0x7f78cc076d80 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.194+0000 7f78d2c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1a5620 con 0x7f78cc075b40 2024-02-26T06:22:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78d0c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78cc077720 0x7f78cc1a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78d0c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78cc077720 0x7f78cc1a4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44626/0 (socket says 172.21.15.49:44626) 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78d0c00640 1 -- 172.21.15.49:0/3766351595 learned_addr learned my addr 172.21.15.49:0/3766351595 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78cbe00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc075b40 0x7f78cc19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78c3400640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78cc076d80 0x7f78cc19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78cbe00640 1 -- 172.21.15.49:0/3766351595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78cc076d80 msgr2=0x7f78cc19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78cbe00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78cc076d80 0x7f78cc19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78cbe00640 1 -- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78cc077720 msgr2=0x7f78cc1a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78cbe00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78cc077720 0x7f78cc1a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.195+0000 7f78cbe00640 1 -- 172.21.15.49:0/3766351595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc009560 con 0x7f78cc075b40 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.196+0000 7f78cbe00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc075b40 0x7f78cc19d800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78bc02c170 tx=0x7f78bc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.196+0000 7f78c9600640 1 -- 172.21.15.49:0/3766351595 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0044e0 con 0x7f78cc075b40 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.196+0000 7f78c9600640 1 -- 172.21.15.49:0/3766351595 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78bc0054e0 con 0x7f78cc075b40 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.196+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc1a58a0 con 0x7f78cc075b40 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.196+0000 7f78c9600640 1 -- 172.21.15.49:0/3766351595 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc003e90 con 0x7f78cc075b40 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.196+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc1a5d80 con 0x7f78cc075b40 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.197+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc10fe80 con 0x7f78cc075b40 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.199+0000 7f78c9600640 1 -- 172.21.15.49:0/3766351595 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f78bc0035f0 con 0x7f78cc075b40 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.201+0000 7f78c9600640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78b00a20c0 0x7f78b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.201+0000 7f78c9600640 1 -- 172.21.15.49:0/3766351595 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f78bc0f0540 con 0x7f78cc075b40 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.201+0000 7f78c3400640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78b00a20c0 0x7f78b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.202+0000 7f78c3400640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78b00a20c0 0x7f78b00a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f78cc19ed10 tx=0x7f78b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.205+0000 7f78c9600640 1 -- 172.21.15.49:0/3766351595 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78bc0b9620 con 0x7f78cc075b40 2024-02-26T06:22:54.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.409+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78cc02cdd0 con 0x7f78cc075b40 2024-02-26T06:22:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.411+0000 7f78c9600640 1 -- 172.21.15.49:0/3766351595 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7f78bc0bcce0 con 0x7f78cc075b40 2024-02-26T06:22:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:22:54.413 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:22:54.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.414+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78b00a20c0 msgr2=0x7f78b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.414+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78b00a20c0 0x7f78b00a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f78cc19ed10 tx=0x7f78b400b390 comp rx=0 tx=0).stop 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.414+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc075b40 msgr2=0x7f78cc19d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.414+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc075b40 0x7f78cc19d800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78bc02c170 tx=0x7f78bc0042f0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.415+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 shutdown_connections 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.415+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f78b00a20c0 0x7f78b00a4580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.415+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78cc076d80 0x7f78cc19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.415+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78cc077720 0x7f78cc1a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.415+0000 7f78d2c00640 1 --2- 172.21.15.49:0/3766351595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc075b40 0x7f78cc19d800 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.415+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 >> 172.21.15.49:0/3766351595 conn(0x7f78cc0fe930 msgr2=0x7f78cc100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.415+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 shutdown_connections 2024-02-26T06:22:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.415+0000 7f78d2c00640 1 -- 172.21.15.49:0/3766351595 wait complete. 2024-02-26T06:22:54.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-26T06:20:31.555053+0000', 'modified': '2024-02-26T06:20:38.098815+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24559, 'mds_2': 24535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.80:6835/1885591642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 1885591642}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 1885591642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.80:6837/2132318232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6836', 'nonce': 2132318232}, {'type': 'v1', 'addr': '172.21.15.80:6837', 'nonce': 2132318232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/2578799425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 2578799425}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 2578799425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.115:6837/3720601525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 3720601525}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 3720601525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24559': {'gid': 24559, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.115:6839/318392314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 318392314}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 318392314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6839/3160003047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6838', 'nonce': 3160003047}, {'type': 'v1', 'addr': '172.21.15.80:6839', 'nonce': 3160003047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T06:22:54.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-26T06:22:54.437 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-26T06:22:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 -- 172.21.15.49:0/2819262500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698102c30 msgr2=0x7ff6981094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 --2- 172.21.15.49:0/2819262500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698102c30 0x7ff6981094c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff690009ca0 tx=0x7ff69002c0f0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 -- 172.21.15.49:0/2819262500 shutdown_connections 2024-02-26T06:22:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 --2- 172.21.15.49:0/2819262500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff698102270 0x7ff6981026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 --2- 172.21.15.49:0/2819262500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698102c30 0x7ff6981094c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 --2- 172.21.15.49:0/2819262500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff698101080 0x7ff6981014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 -- 172.21.15.49:0/2819262500 >> 172.21.15.49:0/2819262500 conn(0x7ff6980fc5f0 msgr2=0x7ff6980fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 -- 172.21.15.49:0/2819262500 shutdown_connections 2024-02-26T06:22:54.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.648+0000 7ff69d000640 1 -- 172.21.15.49:0/2819262500 wait complete. 2024-02-26T06:22:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.649+0000 7ff69d000640 1 Processor -- start 2024-02-26T06:22:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.649+0000 7ff69d000640 1 -- start start 2024-02-26T06:22:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.649+0000 7ff69d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698101080 0x7ff698106e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff69d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff698102270 0x7ff6981073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff69d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff698102c30 0x7ff6981039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff69d000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff698104170 con 0x7ff698101080 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff69d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6981042e0 con 0x7ff698102270 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff69d000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6981044a0 con 0x7ff698102c30 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698101080 0x7ff698106e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698101080 0x7ff698106e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44650/0 (socket says 172.21.15.49:44650) 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697400640 1 -- 172.21.15.49:0/2954159274 learned_addr learned my addr 172.21.15.49:0/2954159274 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff696a00640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff698102270 0x7ff6981073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697e00640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff698102c30 0x7ff6981039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697400640 1 -- 172.21.15.49:0/2954159274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff698102c30 msgr2=0x7ff6981039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697400640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff698102c30 0x7ff6981039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697400640 1 -- 172.21.15.49:0/2954159274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff698102270 msgr2=0x7ff6981073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697400640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff698102270 0x7ff6981073b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff697400640 1 -- 172.21.15.49:0/2954159274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688009490 con 0x7ff698101080 2024-02-26T06:22:54.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.650+0000 7ff696a00640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff698102270 0x7ff6981073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:54.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.651+0000 7ff697400640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698101080 0x7ff698106e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff68800e9c0 tx=0x7ff68800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.651+0000 7ff68fe00640 1 -- 172.21.15.49:0/2954159274 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68800cde0 con 0x7ff698101080 2024-02-26T06:22:54.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.651+0000 7ff68fe00640 1 -- 172.21.15.49:0/2954159274 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff688010910 con 0x7ff698101080 2024-02-26T06:22:54.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.651+0000 7ff69d000640 1 -- 172.21.15.49:0/2954159274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff684009490 con 0x7ff698101080 2024-02-26T06:22:54.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.651+0000 7ff68fe00640 1 -- 172.21.15.49:0/2954159274 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688018960 con 0x7ff698101080 2024-02-26T06:22:54.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.651+0000 7ff69d000640 1 -- 172.21.15.49:0/2954159274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff690009900 con 0x7ff698101080 2024-02-26T06:22:54.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.654+0000 7ff6a159e740 1 Processor -- start 2024-02-26T06:22:54.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.654+0000 7ff68fe00640 1 -- 172.21.15.49:0/2954159274 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff688010a80 con 0x7ff698101080 2024-02-26T06:22:54.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff6a159e740 1 -- start start 2024-02-26T06:22:54.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5d730 0x559986b5db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5e120 0x559986b62c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986b65720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff6a159e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986accb00 con 0x559986b5e120 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff6a159e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986aca030 con 0x559986b5d730 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff6a159e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986ac5810 con 0x559986b63290 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff697e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5d730 0x559986b5db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff697400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5e120 0x559986b62c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff697e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5d730 0x559986b5db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38742/0 (socket says 172.21.15.49:38742) 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.655+0000 7ff697e00640 1 -- 172.21.15.49:0/4062809928 learned_addr learned my addr 172.21.15.49:0/4062809928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff696a00640 1 --2- 172.21.15.49:0/4062809928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986b65720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff697e00640 1 -- 172.21.15.49:0/4062809928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 msgr2=0x559986b65720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff697e00640 1 --2- 172.21.15.49:0/4062809928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986b65720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff697e00640 1 -- 172.21.15.49:0/4062809928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5e120 msgr2=0x559986b62c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff697e00640 1 --2- 172.21.15.49:0/4062809928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5e120 0x559986b62c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff697e00640 1 -- 172.21.15.49:0/4062809928 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559986a8d490 con 0x559986b5d730 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff697e00640 1 --2- 172.21.15.49:0/4062809928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5d730 0x559986b5db50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff690009ca0 tx=0x7ff6900093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46efcca6370b6958 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff68fe00640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff67c0a20c0 0x7ff67c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff68cc00640 1 -- 172.21.15.49:0/4062809928 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690034440 con 0x559986b5d730 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff68cc00640 1 -- 172.21.15.49:0/4062809928 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6900345a0 con 0x559986b5d730 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.656+0000 7ff68cc00640 1 -- 172.21.15.49:0/4062809928 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690003c80 con 0x559986b5d730 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff68fe00640 1 -- 172.21.15.49:0/2954159274 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff688014070 con 0x7ff698101080 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff697400640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff67c0a20c0 0x7ff67c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 -- 172.21.15.49:0/4062809928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5d730 msgr2=0x559986b5db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4062809928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5d730 0x559986b5db50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff690009ca0 tx=0x7ff6900093e0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 -- 172.21.15.49:0/4062809928 shutdown_connections 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4062809928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986b65720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4062809928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5e120 0x559986b62c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4062809928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5d730 0x559986b5db50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff697400640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff67c0a20c0 0x7ff67c0a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff6880979a0 tx=0x7ff6880d1b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 -- 172.21.15.49:0/4062809928 >> 172.21.15.49:0/4062809928 conn(0x559986b58ca0 msgr2=0x559986b590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 -- 172.21.15.49:0/4062809928 shutdown_connections 2024-02-26T06:22:54.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.657+0000 7ff6a159e740 1 -- 172.21.15.49:0/4062809928 wait complete. 2024-02-26T06:22:54.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.658+0000 7ff6a159e740 1 -- start start 2024-02-26T06:22:54.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.658+0000 7ff6a159e740 1 Processor -- start 2024-02-26T06:22:54.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b5d730 0x559986b674c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b67a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b63290 0x559986b6ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff6a159e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986a6ae70 con 0x559986b63290 2024-02-26T06:22:54.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff6a159e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986a6ab10 con 0x559986b5e120 2024-02-26T06:22:54.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff6a159e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998680df20 con 0x559986b5d730 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff6a159e740 10 client.0 _ll_register_callbacks cb 0x559986a7e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff697e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b5d730 0x559986b674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff697e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b5d730 0x559986b674c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44528/0 (socket says 172.21.15.49:44528) 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff697e00640 1 -- 172.21.15.49:0/2455909039 learned_addr learned my addr 172.21.15.49:0/2455909039 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff697e00640 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b63290 0x559986b6ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff696a00640 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b67a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff696a00640 1 -- 172.21.15.49:0/2455909039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b5d730 msgr2=0x559986b674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff696a00640 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b5d730 0x559986b674c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff696a00640 1 -- 172.21.15.49:0/2455909039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b63290 msgr2=0x559986b6ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff696a00640 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b63290 0x559986b6ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff696a00640 1 -- 172.21.15.49:0/2455909039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff698109dc0 con 0x559986b5e120 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.659+0000 7ff697e00640 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b5d730 0x559986b674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.660+0000 7ff696a00640 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b67a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff698104df0 tx=0x7ff68400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4788ea4a3d78e75d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.660+0000 7ff681600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.660+0000 7ff681600640 1 -- 172.21.15.49:0/2455909039 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684017070 con 0x559986b5e120 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.660+0000 7ff681600640 1 -- 172.21.15.49:0/2455909039 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff68400fe20 con 0x559986b5e120 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.660+0000 7ff6a159e740 1 -- 172.21.15.49:0/2455909039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559986ae9c10 con 0x559986b5e120 2024-02-26T06:22:54.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.660+0000 7ff681600640 1 -- 172.21.15.49:0/2455909039 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff684004050 con 0x559986b5e120 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.660+0000 7ff6a159e740 10 client.25142 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.660+0000 7ff6a159e740 1 -- 172.21.15.49:0/2455909039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559986ac3e90 con 0x559986b5e120 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff681600640 1 -- 172.21.15.49:0/2455909039 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559986ae9c10 con 0x559986b5e120 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff681600640 1 -- 172.21.15.49:0/2455909039 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff684010430 con 0x559986b5e120 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff6a159e740 10 client.25142 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff6a159e740 10 client.25142 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff6a159e740 20 client.25142 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff6a159e740 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x559986b73480 0x559986b72d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff6a159e740 4 client.25142 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff6a159e740 1 -- 172.21.15.49:0/2455909039 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599867c56c0 con 0x559986b73480 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.661+0000 7ff697e00640 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x559986b73480 0x559986b72d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.662+0000 7ff697e00640 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x559986b73480 0x559986b72d80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=198995dfc9774e99 server_cookie=2ebb055c35ce53d3 in_seq=0 out_seq=0 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.662+0000 7ff681600640 10 client.25142 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff681600640 1 -- 172.21.15.49:0/2455909039 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599867c56c0 con 0x559986b73480 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff681600640 10 client.25142 handle_command_reply: tid=0 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 client.25142 shutdown 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 -- 172.21.15.49:0/2455909039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 msgr2=0x559986b67a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b67a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff698104df0 tx=0x7ff68400f790 comp rx=0 tx=0).stop 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 -- 172.21.15.49:0/2455909039 shutdown_connections 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x559986b73480 0x559986b72d80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b67a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b63290 0x559986b6ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 --2- 172.21.15.49:0/2455909039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b5d730 0x559986b674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff681600640 0 client.25142 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:22:54.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 -- 172.21.15.49:0/2455909039 >> 172.21.15.49:0/2455909039 conn(0x559986b58ca0 msgr2=0x559986b57de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 -- 172.21.15.49:0/2455909039 shutdown_connections 2024-02-26T06:22:54.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 1 -- 172.21.15.49:0/2455909039 wait complete. 2024-02-26T06:22:54.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.663+0000 7ff6a159e740 20 client.25142 trim_cache size 0 max 16384 2024-02-26T06:22:54.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff6a159e740 1 Processor -- start 2024-02-26T06:22:54.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff6a159e740 1 -- start start 2024-02-26T06:22:54.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b57550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986b57a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff6a159e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986ae9c10 con 0x559986b5d730 2024-02-26T06:22:54.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff6a159e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986b6f530 con 0x559986b5e120 2024-02-26T06:22:54.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff6a159e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986ac2470 con 0x559986b63290 2024-02-26T06:22:54.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff697e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b57550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff697e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b57550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44668/0 (socket says 172.21.15.49:44668) 2024-02-26T06:22:54.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.681+0000 7ff697e00640 1 -- 172.21.15.49:0/3592456966 learned_addr learned my addr 172.21.15.49:0/3592456966 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff696a00640 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff697e00640 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986b57a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff696a00640 1 -- 172.21.15.49:0/3592456966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 msgr2=0x559986b57a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff696a00640 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986b57a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff696a00640 1 -- 172.21.15.49:0/3592456966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 msgr2=0x559986b57550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff696a00640 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b57550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff696a00640 1 -- 172.21.15.49:0/3592456966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559986a60880 con 0x559986b5e120 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff697e00640 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b57550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff696a00640 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6ea00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff6840107e0 tx=0x7ff684009ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97ed8b4bf8f48f5a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.682+0000 7ff683e00640 1 -- 172.21.15.49:0/3592456966 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684017040 con 0x559986b5e120 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff683e00640 1 -- 172.21.15.49:0/3592456966 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff684005520 con 0x559986b5e120 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff683e00640 1 -- 172.21.15.49:0/3592456966 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684003850 con 0x559986b5e120 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 -- 172.21.15.49:0/3592456966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 msgr2=0x559986b6ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6ea00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff6840107e0 tx=0x7ff684009ce0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 -- 172.21.15.49:0/3592456966 shutdown_connections 2024-02-26T06:22:54.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6ea00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986b57a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 --2- 172.21.15.49:0/3592456966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b57550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 -- 172.21.15.49:0/3592456966 >> 172.21.15.49:0/3592456966 conn(0x559986b58ca0 msgr2=0x559986b7bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 -- 172.21.15.49:0/3592456966 shutdown_connections 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.683+0000 7ff6a159e740 1 -- 172.21.15.49:0/3592456966 wait complete. 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.684+0000 7ff6a159e740 1 -- start start 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.684+0000 7ff6a159e740 1 Processor -- start 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.684+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b6b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff6a159e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986bc96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff6a159e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986b6ec50 con 0x559986b5d730 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff6a159e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986b6edc0 con 0x559986b5e120 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff6a159e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559986bc9c00 con 0x559986b63290 2024-02-26T06:22:54.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff6a159e740 10 client.0 _ll_register_callbacks cb 0x7ff684009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b6b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b6b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44674/0 (socket says 172.21.15.49:44674) 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 -- 172.21.15.49:0/4293690754 learned_addr learned my addr 172.21.15.49:0/4293690754 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986bc96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff696a00640 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 -- 172.21.15.49:0/4293690754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 msgr2=0x559986bc96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986bc96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 -- 172.21.15.49:0/4293690754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 msgr2=0x559986b6bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff697e00640 1 -- 172.21.15.49:0/4293690754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff690053040 con 0x559986b5d730 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.685+0000 7ff696a00640 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.686+0000 7ff697e00640 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b6b6f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff690006010 tx=0x7ff690032480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4476c188905d40 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.686+0000 7ff681600640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:22:54.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.686+0000 7ff681600640 1 -- 172.21.15.49:0/4293690754 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69005e070 con 0x559986b5d730 2024-02-26T06:22:54.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.686+0000 7ff681600640 1 -- 172.21.15.49:0/4293690754 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff690032d70 con 0x559986b5d730 2024-02-26T06:22:54.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.686+0000 7ff6a159e740 1 -- 172.21.15.49:0/4293690754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559986b73e30 con 0x559986b5d730 2024-02-26T06:22:54.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.686+0000 7ff681600640 1 -- 172.21.15.49:0/4293690754 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff690034c40 con 0x559986b5d730 2024-02-26T06:22:54.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.686+0000 7ff6a159e740 10 client.15045 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:54.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.686+0000 7ff6a159e740 1 -- 172.21.15.49:0/4293690754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559986ac3e90 con 0x559986b5d730 2024-02-26T06:22:54.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff681600640 1 -- 172.21.15.49:0/4293690754 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559986b73e30 con 0x559986b5d730 2024-02-26T06:22:54.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff681600640 1 -- 172.21.15.49:0/4293690754 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff690003f20 con 0x559986b5d730 2024-02-26T06:22:54.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff6a159e740 10 client.15045 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:54.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff6a159e740 10 client.15045 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T06:22:54.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff6a159e740 20 client.15045 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x559986bcea80 0x559986bcaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff6a159e740 4 client.15045 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff6a159e740 1 -- 172.21.15.49:0/4293690754 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559986b74490 con 0x559986bcea80 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.687+0000 7ff696a00640 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x559986bcea80 0x559986bcaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff696a00640 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x559986bcea80 0x559986bcaf60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5157a2635b6f354c server_cookie=dcec5a504e8cbfdf in_seq=0 out_seq=0 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff681600640 10 client.15045 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff681600640 1 -- 172.21.15.49:0/4293690754 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559986b74490 con 0x559986bcea80 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff681600640 10 client.15045 handle_command_reply: tid=0 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 client.15045 shutdown 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 -- 172.21.15.49:0/4293690754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 msgr2=0x559986b6b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b6b6f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff690006010 tx=0x7ff690032480 comp rx=0 tx=0).stop 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 -- 172.21.15.49:0/4293690754 shutdown_connections 2024-02-26T06:22:54.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x559986bcea80 0x559986bcaf60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559986b5e120 0x559986b6bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff681600640 0 client.15045 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:22:54.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559986b63290 0x559986bc96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 --2- 172.21.15.49:0/4293690754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559986b5d730 0x559986b6b6f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 -- 172.21.15.49:0/4293690754 >> 172.21.15.49:0/4293690754 conn(0x559986b58ca0 msgr2=0x559986b57550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.688+0000 7ff6a159e740 1 -- 172.21.15.49:0/4293690754 shutdown_connections 2024-02-26T06:22:54.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff6a159e740 1 -- 172.21.15.49:0/4293690754 wait complete. 2024-02-26T06:22:54.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff6a159e740 20 client.15045 trim_cache size 0 max 16384 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24535, 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 137.60621218700001, 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-02-26T06:22:54.695 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-02-26T06:22:54.696 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 166.46834053800001 2024-02-26T06:22:54.696 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:22:54.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 -- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff67c0a20c0 msgr2=0x7ff67c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff67c0a20c0 0x7ff67c0a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff6880979a0 tx=0x7ff6880d1b50 comp rx=0 tx=0).stop 2024-02-26T06:22:54.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 -- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698101080 msgr2=0x7ff698106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698101080 0x7ff698106e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff68800e9c0 tx=0x7ff68800ee80 comp rx=0 tx=0).stop 2024-02-26T06:22:54.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 -- 172.21.15.49:0/2954159274 shutdown_connections 2024-02-26T06:22:54.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff67c0a20c0 0x7ff67c0a4580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff698102270 0x7ff6981073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff698102c30 0x7ff6981039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 --2- 172.21.15.49:0/2954159274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff698101080 0x7ff698106e70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.689+0000 7ff69d000640 1 -- 172.21.15.49:0/2954159274 >> 172.21.15.49:0/2954159274 conn(0x7ff6980fc5f0 msgr2=0x7ff6980733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.690+0000 7ff69d000640 1 -- 172.21.15.49:0/2954159274 shutdown_connections 2024-02-26T06:22:54.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.690+0000 7ff69d000640 1 -- 172.21.15.49:0/2954159274 wait complete. 2024-02-26T06:22:54.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:54 smithi049 ceph-mon[31696]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:22:54.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:54 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1224882248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:54.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:54 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3766351595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:54.709 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-26T06:22:54.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:54 smithi080 ceph-mon[36206]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:22:54.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:54 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1224882248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:54.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:54 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3766351595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:54 smithi115 ceph-mon[36426]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T06:22:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:54 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1224882248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:22:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:54 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3766351595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:22:54.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 -- 172.21.15.49:0/414409261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8075140 msgr2=0x7f7ad8073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 --2- 172.21.15.49:0/414409261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8075140 0x7f7ad8073590 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009900 tx=0x7f7ac802bcc0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 -- 172.21.15.49:0/414409261 shutdown_connections 2024-02-26T06:22:54.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 --2- 172.21.15.49:0/414409261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8109040 0x7f7ad810b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 --2- 172.21.15.49:0/414409261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ad8073b60 0x7f7ad8073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 --2- 172.21.15.49:0/414409261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8075140 0x7f7ad8073590 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 -- 172.21.15.49:0/414409261 >> 172.21.15.49:0/414409261 conn(0x7f7ad80fc720 msgr2=0x7f7ad80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 -- 172.21.15.49:0/414409261 shutdown_connections 2024-02-26T06:22:54.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.931+0000 7f7addc00640 1 -- 172.21.15.49:0/414409261 wait complete. 2024-02-26T06:22:54.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.932+0000 7f7addc00640 1 Processor -- start 2024-02-26T06:22:54.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.932+0000 7f7addc00640 1 -- start start 2024-02-26T06:22:54.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.932+0000 7f7addc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8073b60 0x7f7ad8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.932+0000 7f7addc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8075140 0x7f7ad81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.932+0000 7f7addc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ad8109040 0x7f7ad81a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.932+0000 7f7addc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad81a0f50 con 0x7f7ad8109040 2024-02-26T06:22:54.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7addc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad81a10c0 con 0x7f7ad8073b60 2024-02-26T06:22:54.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7addc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad81a1280 con 0x7f7ad8075140 2024-02-26T06:22:54.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8073b60 0x7f7ad8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ad8109040 0x7f7ad81a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8073b60 0x7f7ad8199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38794/0 (socket says 172.21.15.49:38794) 2024-02-26T06:22:54.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad6a00640 1 -- 172.21.15.49:0/2959367159 learned_addr learned my addr 172.21.15.49:0/2959367159 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7acfe00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8075140 0x7f7ad81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad6a00640 1 -- 172.21.15.49:0/2959367159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8075140 msgr2=0x7f7ad81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad6a00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8075140 0x7f7ad81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad6a00640 1 -- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ad8109040 msgr2=0x7f7ad81a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad6a00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ad8109040 0x7f7ad81a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.933+0000 7f7ad6a00640 1 -- 172.21.15.49:0/2959367159 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac8009560 con 0x7f7ad8073b60 2024-02-26T06:22:54.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.934+0000 7f7ad6a00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8073b60 0x7f7ad8199460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8006010 tx=0x7f7ac8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.934+0000 7f7acf400640 1 -- 172.21.15.49:0/2959367159 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80043d0 con 0x7f7ad8073b60 2024-02-26T06:22:54.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.934+0000 7f7acf400640 1 -- 172.21.15.49:0/2959367159 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ac80052d0 con 0x7f7ad8073b60 2024-02-26T06:22:54.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.934+0000 7f7addc00640 1 -- 172.21.15.49:0/2959367159 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad81a1500 con 0x7f7ad8073b60 2024-02-26T06:22:54.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.934+0000 7f7acf400640 1 -- 172.21.15.49:0/2959367159 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8003cc0 con 0x7f7ad8073b60 2024-02-26T06:22:54.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.934+0000 7f7addc00640 1 -- 172.21.15.49:0/2959367159 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad81a1a60 con 0x7f7ad8073b60 2024-02-26T06:22:54.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.938+0000 7f7ae0bce740 1 Processor -- start 2024-02-26T06:22:54.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.938+0000 7f7ae0bce740 1 -- start start 2024-02-26T06:22:54.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7acf400640 1 -- 172.21.15.49:0/2959367159 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7ac8005440 con 0x7f7ad8073b60 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3d550 0x55d73bc3d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc42a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc45540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bbacb00 con 0x55d73bc3df40 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bbaa030 con 0x55d73bc430b0 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bba5810 con 0x55d73bc3d550 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc42a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3d550 0x55d73bc3d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3d550 0x55d73bc3d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44590/0 (socket says 172.21.15.49:44590) 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc42a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44694/0 (socket says 172.21.15.49:44694) 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad7400640 1 -- 172.21.15.49:0/3941938584 learned_addr learned my addr 172.21.15.49:0/3941938584 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc45540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad7400640 1 -- 172.21.15.49:0/3941938584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 msgr2=0x55d73bc45540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad7400640 1 --2- 172.21.15.49:0/3941938584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc45540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad7400640 1 -- 172.21.15.49:0/3941938584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 msgr2=0x55d73bc42a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.939+0000 7f7ad7400640 1 --2- 172.21.15.49:0/3941938584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc42a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.940+0000 7f7ad7400640 1 -- 172.21.15.49:0/3941938584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d73bb6d490 con 0x55d73bc3d550 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.940+0000 7f7ad7400640 1 --2- 172.21.15.49:0/3941938584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3d550 0x55d73bc3d970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009fd0 tx=0x7f7ad00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac2b3a93e39c3ba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.940+0000 7f7ac7e00640 1 -- 172.21.15.49:0/3941938584 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0004d20 con 0x55d73bc3d550 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.940+0000 7f7ac7e00640 1 -- 172.21.15.49:0/3941938584 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ad0004e80 con 0x55d73bc3d550 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.940+0000 7f7ac7e00640 1 -- 172.21.15.49:0/3941938584 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0005260 con 0x55d73bc3d550 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7acf400640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7abc0a2140 0x7f7abc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7acf400640 1 -- 172.21.15.49:0/2959367159 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7ac80f0460 con 0x7f7ad8073b60 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ad6a00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7abc0a2140 0x7f7abc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3941938584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3d550 msgr2=0x55d73bc3d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3941938584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3d550 0x55d73bc3d970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009fd0 tx=0x7f7ad00118d0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3941938584 shutdown_connections 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3941938584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc45540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3941938584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc42a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3941938584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3d550 0x55d73bc3d970 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3941938584 >> 172.21.15.49:0/3941938584 conn(0x55d73bc38ac0 msgr2=0x55d73bc38ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.941+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3941938584 shutdown_connections 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.942+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3941938584 wait complete. 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.942+0000 7f7ad6a00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7abc0a2140 0x7f7abc0a4600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80ee2c0 tx=0x7f7ac80f3f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.942+0000 7f7ae0bce740 1 -- start start 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.942+0000 7f7ae0bce740 1 Processor -- start 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.943+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.943+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3df40 0x55d73bc47890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.943+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc4e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.943+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bb4ae70 con 0x55d73bc3d550 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.943+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bb4ab10 con 0x55d73bc430b0 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.943+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73b8edf20 con 0x55d73bc3df40 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.943+0000 7f7ae0bce740 10 client.0 _ll_register_callbacks cb 0x55d73bb5e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:54.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.943+0000 7f7ad7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7ad7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44710/0 (socket says 172.21.15.49:44710) 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7ad7400640 1 -- 172.21.15.49:0/2964780905 learned_addr learned my addr 172.21.15.49:0/2964780905 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3df40 0x55d73bc47890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7ad7400640 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc4e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7acfe00640 1 -- 172.21.15.49:0/2964780905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 msgr2=0x55d73bc4e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7acfe00640 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc4e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7acfe00640 1 -- 172.21.15.49:0/2964780905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 msgr2=0x55d73bc47350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7acfe00640 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7acfe00640 1 -- 172.21.15.49:0/2964780905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad0009c80 con 0x55d73bc3df40 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7ad7400640 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc4e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7acfe00640 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3df40 0x55d73bc47890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ad819a970 tx=0x7f7ac0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91186f6872150e15 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.944+0000 7f7ac4c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:22:54.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.945+0000 7f7ac4c00640 1 -- 172.21.15.49:0/2964780905 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0017070 con 0x55d73bc3df40 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.945+0000 7f7ac4c00640 1 -- 172.21.15.49:0/2964780905 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7ac000f8c0 con 0x55d73bc3df40 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.945+0000 7f7ae0bce740 1 -- 172.21.15.49:0/2964780905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d73b8a56c0 con 0x55d73bc3df40 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.945+0000 7f7ac4c00640 1 -- 172.21.15.49:0/2964780905 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ac000fa80 con 0x55d73bc3df40 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.945+0000 7f7ae0bce740 10 client.25189 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.945+0000 7f7ae0bce740 1 -- 172.21.15.49:0/2964780905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d73bba3e90 con 0x55d73bc3df40 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.945+0000 7f7ac4c00640 1 -- 172.21.15.49:0/2964780905 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d73b8a56c0 con 0x55d73bc3df40 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.946+0000 7f7ac4c00640 1 -- 172.21.15.49:0/2964780905 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ac0010d10 con 0x55d73bc3df40 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.946+0000 7f7ae0bce740 10 client.25189 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.946+0000 7f7ae0bce740 10 client.25189 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T06:22:54.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.946+0000 7f7ae0bce740 20 client.25189 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:54.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.946+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55d73bc52c70 0x55d73bc57d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.946+0000 7f7ae0bce740 4 client.25189 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:22:54.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.946+0000 7f7ae0bce740 1 -- 172.21.15.49:0/2964780905 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d73b8f6270 con 0x55d73bc52c70 2024-02-26T06:22:54.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.946+0000 7f7ad7400640 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55d73bc52c70 0x55d73bc57d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.947+0000 7f7ad7400640 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55d73bc52c70 0x55d73bc57d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a78a4959b184b96a server_cookie=7a69edff176a502b in_seq=0 out_seq=0 2024-02-26T06:22:54.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.947+0000 7f7ac4c00640 10 client.25189 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:22:54.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.947+0000 7f7ac4c00640 1 -- 172.21.15.49:0/2964780905 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d73b8f6270 con 0x55d73bc52c70 2024-02-26T06:22:54.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.947+0000 7f7ac4c00640 10 client.25189 handle_command_reply: tid=0 2024-02-26T06:22:54.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.947+0000 7f7ae0bce740 1 client.25189 shutdown 2024-02-26T06:22:54.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 -- 172.21.15.49:0/2964780905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3df40 msgr2=0x55d73bc47890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3df40 0x55d73bc47890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ad819a970 tx=0x7f7ac0012b60 comp rx=0 tx=0).stop 2024-02-26T06:22:54.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 -- 172.21.15.49:0/2964780905 shutdown_connections 2024-02-26T06:22:54.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55d73bc52c70 0x55d73bc57d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc430b0 0x55d73bc4e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc3df40 0x55d73bc47890 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ac4c00640 0 client.25189 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:22:54.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/2964780905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 -- 172.21.15.49:0/2964780905 >> 172.21.15.49:0/2964780905 conn(0x55d73bc38ac0 msgr2=0x55d73bc37de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 -- 172.21.15.49:0/2964780905 shutdown_connections 2024-02-26T06:22:54.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 1 -- 172.21.15.49:0/2964780905 wait complete. 2024-02-26T06:22:54.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.948+0000 7f7ae0bce740 20 client.25189 trim_cache size 0 max 16384 2024-02-26T06:22:54.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.982+0000 7f7ae0bce740 1 Processor -- start 2024-02-26T06:22:54.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.982+0000 7f7ae0bce740 1 -- start start 2024-02-26T06:22:54.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.982+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.982+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3df40 0x55d73bc4e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 0x55d73bc47890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bc45c00 con 0x55d73bc3d550 2024-02-26T06:22:54.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bc47190 con 0x55d73bc3df40 2024-02-26T06:22:54.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73b8f6270 con 0x55d73bc430b0 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44720/0 (socket says 172.21.15.49:44720) 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 -- 172.21.15.49:0/819764706 learned_addr learned my addr 172.21.15.49:0/819764706 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7acfe00640 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3df40 0x55d73bc4e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 0x55d73bc47890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 -- 172.21.15.49:0/819764706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 msgr2=0x55d73bc47890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 0x55d73bc47890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 -- 172.21.15.49:0/819764706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3df40 msgr2=0x55d73bc4e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3df40 0x55d73bc4e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7ad7400640 1 -- 172.21.15.49:0/819764706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d73bb40880 con 0x55d73bc3d550 2024-02-26T06:22:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.983+0000 7f7acfe00640 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3df40 0x55d73bc4e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.984+0000 7f7ad7400640 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0014040 tx=0x7f7ad0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6328e9fc9d67b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.984+0000 7f7ac7400640 1 -- 172.21.15.49:0/819764706 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00042b0 con 0x55d73bc3d550 2024-02-26T06:22:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.984+0000 7f7ac7400640 1 -- 172.21.15.49:0/819764706 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ad0004410 con 0x55d73bc3d550 2024-02-26T06:22:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.984+0000 7f7ac7400640 1 -- 172.21.15.49:0/819764706 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000f740 con 0x55d73bc3d550 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 -- 172.21.15.49:0/819764706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 msgr2=0x55d73bc47350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0014040 tx=0x7f7ad0009500 comp rx=0 tx=0).stop 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 -- 172.21.15.49:0/819764706 shutdown_connections 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3df40 0x55d73bc4e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 0x55d73bc47890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/819764706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3d550 0x55d73bc47350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 -- 172.21.15.49:0/819764706 >> 172.21.15.49:0/819764706 conn(0x55d73bc38ac0 msgr2=0x55d73bc42660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 -- 172.21.15.49:0/819764706 shutdown_connections 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.985+0000 7f7ae0bce740 1 -- 172.21.15.49:0/819764706 wait complete. 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 1 -- start start 2024-02-26T06:22:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 1 Processor -- start 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3d550 0x55d73bc4e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc4b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 0x55d73bc4b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bc4bb60 con 0x55d73bc3df40 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bc4bcd0 con 0x55d73bc3d550 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73bcb2200 con 0x55d73bc430b0 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.986+0000 7f7ae0bce740 10 client.0 _ll_register_callbacks cb 0x7f7ac0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc4b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc4b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44728/0 (socket says 172.21.15.49:44728) 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7acfe00640 1 -- 172.21.15.49:0/3295164886 learned_addr learned my addr 172.21.15.49:0/3295164886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7ad7400640 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3d550 0x55d73bc4e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7ad7400640 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 0x55d73bc4b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7acfe00640 1 -- 172.21.15.49:0/3295164886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 msgr2=0x55d73bc4b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7acfe00640 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 0x55d73bc4b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7acfe00640 1 -- 172.21.15.49:0/3295164886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3d550 msgr2=0x55d73bc4e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7acfe00640 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3d550 0x55d73bc4e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.987+0000 7f7acfe00640 1 -- 172.21.15.49:0/3295164886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d73bba3e90 con 0x55d73bc3df40 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.988+0000 7f7acfe00640 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc4b0e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ad819a970 tx=0x7f7ac001cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c00441bfaeec4f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.988+0000 7f7ac4c00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.988+0000 7f7ac4c00640 1 -- 172.21.15.49:0/3295164886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0017040 con 0x55d73bc3df40 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.988+0000 7f7ac4c00640 1 -- 172.21.15.49:0/3295164886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7ac0009e10 con 0x55d73bc3df40 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.988+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3295164886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d73bc53c20 con 0x55d73bc3df40 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.988+0000 7f7ac4c00640 1 -- 172.21.15.49:0/3295164886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ac000f9a0 con 0x55d73bc3df40 2024-02-26T06:22:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.988+0000 7f7ae0bce740 10 client.15063 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.988+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3295164886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d73bb6d490 con 0x55d73bc3df40 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ac4c00640 1 -- 172.21.15.49:0/3295164886 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d73bc53c20 con 0x55d73bc3df40 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ac4c00640 1 -- 172.21.15.49:0/3295164886 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ac0003710 con 0x55d73bc3df40 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ae0bce740 10 client.15063 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ae0bce740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ae0bce740 20 client.15063 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55d73bcb7480 0x55d73bcb3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ae0bce740 4 client.15063 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3295164886 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d73bc54f40 con 0x55d73bcb7480 2024-02-26T06:22:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.989+0000 7f7ad7400640 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55d73bcb7480 0x55d73bcb3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ad7400640 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55d73bcb7480 0x55d73bcb3920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81ee367b74bf015f server_cookie=afa132eec1abc76e in_seq=0 out_seq=0 2024-02-26T06:22:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ac4c00640 10 client.15063 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ac4c00640 1 -- 172.21.15.49:0/3295164886 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d73bc54f40 con 0x55d73bcb7480 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ac4c00640 10 client.15063 handle_command_reply: tid=0 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 client.15063 shutdown 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3295164886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 msgr2=0x55d73bc4b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc4b0e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ad819a970 tx=0x7f7ac001cd50 comp rx=0 tx=0).stop 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3295164886 shutdown_connections 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55d73bcb7480 0x55d73bcb3920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d73bc430b0 0x55d73bc4b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d73bc3df40 0x55d73bc4b0e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 --2- 172.21.15.49:0/3295164886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d73bc3d550 0x55d73bc4e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ac4c00640 0 client.15063 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3295164886 >> 172.21.15.49:0/3295164886 conn(0x55d73bc38ac0 msgr2=0x55d73bc42260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3295164886 shutdown_connections 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.990+0000 7f7ae0bce740 1 -- 172.21.15.49:0/3295164886 wait complete. 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7ae0bce740 20 client.15063 trim_cache size 0 max 16384 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24559, 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-02-26T06:22:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-02-26T06:22:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-02-26T06:22:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 138.91951222500001, 2024-02-26T06:22:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-02-26T06:22:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-02-26T06:22:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-02-26T06:22:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 160.80854941999999 2024-02-26T06:22:54.997 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:22:54.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 -- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7abc0a2140 msgr2=0x7f7abc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7abc0a2140 0x7f7abc0a4600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80ee2c0 tx=0x7f7ac80f3f00 comp rx=0 tx=0).stop 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 -- 172.21.15.49:0/2959367159 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8073b60 msgr2=0x7f7ad8199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8073b60 0x7f7ad8199460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8006010 tx=0x7f7ac8005e80 comp rx=0 tx=0).stop 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 -- 172.21.15.49:0/2959367159 shutdown_connections 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7abc0a2140 0x7f7abc0a4600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ad8109040 0x7f7ad81a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8075140 0x7f7ad81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 --2- 172.21.15.49:0/2959367159 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ad8073b60 0x7f7ad8199460 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 -- 172.21.15.49:0/2959367159 >> 172.21.15.49:0/2959367159 conn(0x7f7ad80fc720 msgr2=0x7f7ad8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.991+0000 7f7addc00640 1 -- 172.21.15.49:0/2959367159 shutdown_connections 2024-02-26T06:22:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:54.992+0000 7f7addc00640 1 -- 172.21.15.49:0/2959367159 wait complete. 2024-02-26T06:22:55.011 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T06:22:55.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.237+0000 7f035b800640 1 -- 172.21.15.49:0/3058183461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0354075160 msgr2=0x7f03540735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.237+0000 7f035b800640 1 --2- 172.21.15.49:0/3058183461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0354075160 0x7f03540735b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0344009ca0 tx=0x7f034402c190 comp rx=0 tx=0).stop 2024-02-26T06:22:55.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.237+0000 7f035b800640 1 -- 172.21.15.49:0/3058183461 shutdown_connections 2024-02-26T06:22:55.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.237+0000 7f035b800640 1 --2- 172.21.15.49:0/3058183461 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0354109020 0x7f035410b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.237+0000 7f035b800640 1 --2- 172.21.15.49:0/3058183461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0354073b80 0x7f0354073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.237+0000 7f035b800640 1 --2- 172.21.15.49:0/3058183461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0354075160 0x7f03540735b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.237+0000 7f035b800640 1 -- 172.21.15.49:0/3058183461 >> 172.21.15.49:0/3058183461 conn(0x7f03540fc720 msgr2=0x7f03540feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:55.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.238+0000 7f035b800640 1 -- 172.21.15.49:0/3058183461 shutdown_connections 2024-02-26T06:22:55.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.238+0000 7f035b800640 1 -- 172.21.15.49:0/3058183461 wait complete. 2024-02-26T06:22:55.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.238+0000 7f035b800640 1 Processor -- start 2024-02-26T06:22:55.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f035b800640 1 -- start start 2024-02-26T06:22:55.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f035b800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0354073b80 0x7f0354199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f035b800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0354075160 0x7f0354199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f035b800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0354109020 0x7f03541a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f035b800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03541a0f40 con 0x7f0354075160 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f035b800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03541a10b0 con 0x7f0354073b80 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f035b800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03541a1270 con 0x7f0354109020 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f0353e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0354075160 0x7f0354199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f0353e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0354075160 0x7f0354199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44734/0 (socket says 172.21.15.49:44734) 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f0353e00640 1 -- 172.21.15.49:0/1866627555 learned_addr learned my addr 172.21.15.49:0/1866627555 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f0358e00640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0354073b80 0x7f0354199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.239+0000 7f0359800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0354109020 0x7f03541a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0353e00640 1 -- 172.21.15.49:0/1866627555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0354109020 msgr2=0x7f03541a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0353e00640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0354109020 0x7f03541a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0353e00640 1 -- 172.21.15.49:0/1866627555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0354073b80 msgr2=0x7f0354199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0353e00640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0354073b80 0x7f0354199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0353e00640 1 -- 172.21.15.49:0/1866627555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0344009900 con 0x7f0354075160 2024-02-26T06:22:55.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0353e00640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0354075160 0x7f0354199990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e9f0 tx=0x7f033c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:55.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0351600640 1 -- 172.21.15.49:0/1866627555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c009700 con 0x7f0354075160 2024-02-26T06:22:55.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f035b800640 1 -- 172.21.15.49:0/1866627555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03541a1550 con 0x7f0354075160 2024-02-26T06:22:55.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0351600640 1 -- 172.21.15.49:0/1866627555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f033c00cd80 con 0x7f0354075160 2024-02-26T06:22:55.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.240+0000 7f0351600640 1 -- 172.21.15.49:0/1866627555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c018920 con 0x7f0354075160 2024-02-26T06:22:55.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.241+0000 7f035b800640 1 -- 172.21.15.49:0/1866627555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03541a1a60 con 0x7f0354075160 2024-02-26T06:22:55.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f035e789740 1 Processor -- start 2024-02-26T06:22:55.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f0351600640 1 -- 172.21.15.49:0/1866627555 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f033c018a80 con 0x7f0354075160 2024-02-26T06:22:55.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f035e789740 1 -- start start 2024-02-26T06:22:55.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f035e789740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f425970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f035e789740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425f40 0x563f0f42aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f035e789740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f42b0b0 0x563f0f42d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f035e789740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f394b00 con 0x563f0f425550 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f035e789740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f392030 con 0x563f0f42b0b0 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f035e789740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f38d810 con 0x563f0f425f40 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.244+0000 7f0359800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f425970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0358e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425f40 0x563f0f42aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0359800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f425970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44736/0 (socket says 172.21.15.49:44736) 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0359800640 1 -- 172.21.15.49:0/4224089479 learned_addr learned my addr 172.21.15.49:0/4224089479 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0353e00640 1 --2- 172.21.15.49:0/4224089479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f42b0b0 0x563f0f42d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0358e00640 1 -- 172.21.15.49:0/4224089479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f42b0b0 msgr2=0x563f0f42d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0358e00640 1 --2- 172.21.15.49:0/4224089479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f42b0b0 0x563f0f42d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0358e00640 1 -- 172.21.15.49:0/4224089479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 msgr2=0x563f0f425970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0358e00640 1 --2- 172.21.15.49:0/4224089479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f425970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0358e00640 1 -- 172.21.15.49:0/4224089479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f0f355490 con 0x563f0f425f40 2024-02-26T06:22:55.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0358e00640 1 --2- 172.21.15.49:0/4224089479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425f40 0x563f0f42aa40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0344009bf0 tx=0x7f0344005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f10408b4bbb72ef server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.245+0000 7f0351600640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f03380a20c0 0x7f03380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.246+0000 7f034a000640 1 -- 172.21.15.49:0/4224089479 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344034440 con 0x563f0f425f40 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.246+0000 7f0351600640 1 -- 172.21.15.49:0/1866627555 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f033c014070 con 0x7f0354075160 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.246+0000 7f034a000640 1 -- 172.21.15.49:0/4224089479 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0344003970 con 0x563f0f425f40 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.246+0000 7f034a000640 1 -- 172.21.15.49:0/4224089479 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344004040 con 0x563f0f425f40 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.246+0000 7f0359800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f03380a20c0 0x7f03380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.246+0000 7f035e789740 1 -- 172.21.15.49:0/4224089479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425f40 msgr2=0x563f0f42aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.246+0000 7f035e789740 1 --2- 172.21.15.49:0/4224089479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425f40 0x563f0f42aa40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0344009bf0 tx=0x7f0344005f10 comp rx=0 tx=0).stop 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.246+0000 7f0359800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f03380a20c0 0x7f03380a4580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x563f0f4267a0 tx=0x7f034c00b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 -- 172.21.15.49:0/4224089479 shutdown_connections 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 --2- 172.21.15.49:0/4224089479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f42b0b0 0x563f0f42d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 --2- 172.21.15.49:0/4224089479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425f40 0x563f0f42aa40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 --2- 172.21.15.49:0/4224089479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f425970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 -- 172.21.15.49:0/4224089479 >> 172.21.15.49:0/4224089479 conn(0x563f0f420ac0 msgr2=0x563f0f420ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 -- 172.21.15.49:0/4224089479 shutdown_connections 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 -- 172.21.15.49:0/4224089479 wait complete. 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 -- start start 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.247+0000 7f035e789740 1 Processor -- start 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f035e789740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f42f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f035e789740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425f40 0x563f0f42f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f035e789740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f42b0b0 0x563f0f4368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f035e789740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f332e70 con 0x563f0f425550 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f035e789740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f332b10 con 0x563f0f425f40 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f035e789740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f0d5f20 con 0x563f0f42b0b0 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f035e789740 10 client.0 _ll_register_callbacks cb 0x563f0f346920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f0358e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f42f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f0358e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f42f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44750/0 (socket says 172.21.15.49:44750) 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f0358e00640 1 -- 172.21.15.49:0/3636078888 learned_addr learned my addr 172.21.15.49:0/3636078888 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f0353e00640 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425f40 0x563f0f42f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.248+0000 7f0359800640 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f42b0b0 0x563f0f4368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0358e00640 1 -- 172.21.15.49:0/3636078888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f42b0b0 msgr2=0x563f0f4368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0358e00640 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f42b0b0 0x563f0f4368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0358e00640 1 -- 172.21.15.49:0/3636078888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425f40 msgr2=0x563f0f42f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0358e00640 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425f40 0x563f0f42f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0358e00640 1 -- 172.21.15.49:0/3636078888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035410c0a0 con 0x563f0f425550 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0359800640 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f42b0b0 0x563f0f4368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0358e00640 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f42f370 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0344037040 tx=0x7f03440325f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcc92e8ec5064a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0342a00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0342a00640 1 -- 172.21.15.49:0/3636078888 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344003cf0 con 0x563f0f425550 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f0342a00640 1 -- 172.21.15.49:0/3636078888 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0344034440 con 0x563f0f425550 2024-02-26T06:22:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.249+0000 7f035e789740 1 -- 172.21.15.49:0/3636078888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f0f3b1c10 con 0x563f0f425550 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.250+0000 7f0342a00640 1 -- 172.21.15.49:0/3636078888 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0344003e50 con 0x563f0f425550 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.250+0000 7f035e789740 10 client.15075 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.250+0000 7f035e789740 1 -- 172.21.15.49:0/3636078888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563f0f38be90 con 0x563f0f425550 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.250+0000 7f0342a00640 1 -- 172.21.15.49:0/3636078888 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563f0f3b1c10 con 0x563f0f425550 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.251+0000 7f0342a00640 1 -- 172.21.15.49:0/3636078888 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f034403f480 con 0x563f0f425550 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.251+0000 7f035e789740 10 client.15075 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.251+0000 7f035e789740 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.251+0000 7f035e789740 20 client.15075 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.251+0000 7f035e789740 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563f0f43b550 0x563f0f43f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.251+0000 7f035e789740 4 client.15075 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.251+0000 7f035e789740 1 -- 172.21.15.49:0/3636078888 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f0f08d6c0 con 0x563f0f43b550 2024-02-26T06:22:55.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.251+0000 7f0353e00640 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563f0f43b550 0x563f0f43f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.252+0000 7f0353e00640 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563f0f43b550 0x563f0f43f9d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd65eaa26689c9f0 server_cookie=91d680796bc1b0c2 in_seq=0 out_seq=0 2024-02-26T06:22:55.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.252+0000 7f0342a00640 10 client.15075 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f0342a00640 1 -- 172.21.15.49:0/3636078888 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563f0f08d6c0 con 0x563f0f43b550 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f0342a00640 10 client.15075 handle_command_reply: tid=0 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 client.15075 shutdown 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 -- 172.21.15.49:0/3636078888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 msgr2=0x563f0f42f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f42f370 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0344037040 tx=0x7f03440325f0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 -- 172.21.15.49:0/3636078888 shutdown_connections 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563f0f43b550 0x563f0f43f9d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425f40 0x563f0f42f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f42b0b0 0x563f0f4368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f0342a00640 0 client.15075 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:22:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 --2- 172.21.15.49:0/3636078888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425550 0x563f0f42f370 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.253+0000 7f035e789740 1 -- 172.21.15.49:0/3636078888 >> 172.21.15.49:0/3636078888 conn(0x563f0f420ac0 msgr2=0x563f0f41fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.254+0000 7f035e789740 1 -- 172.21.15.49:0/3636078888 shutdown_connections 2024-02-26T06:22:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.254+0000 7f035e789740 1 -- 172.21.15.49:0/3636078888 wait complete. 2024-02-26T06:22:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.254+0000 7f035e789740 20 client.15075 trim_cache size 0 max 16384 2024-02-26T06:22:55.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.271+0000 7f035e789740 1 Processor -- start 2024-02-26T06:22:55.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.271+0000 7f035e789740 1 -- start start 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.271+0000 7f035e789740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f420ac0 0x563f0f4439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.271+0000 7f035e789740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425550 0x563f0f4368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.271+0000 7f035e789740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f443f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.271+0000 7f035e789740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f4371b0 con 0x563f0f425f40 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.271+0000 7f035e789740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f43c440 con 0x563f0f420ac0 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.271+0000 7f035e789740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f08d6c0 con 0x563f0f425550 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0358e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f420ac0 0x563f0f4439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0358e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f420ac0 0x563f0f4439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38866/0 (socket says 172.21.15.49:38866) 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0359800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f443f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0358e00640 1 -- 172.21.15.49:0/3589254462 learned_addr learned my addr 172.21.15.49:0/3589254462 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0353e00640 1 --2- 172.21.15.49:0/3589254462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425550 0x563f0f4368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0358e00640 1 -- 172.21.15.49:0/3589254462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425550 msgr2=0x563f0f4368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0358e00640 1 --2- 172.21.15.49:0/3589254462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425550 0x563f0f4368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0358e00640 1 -- 172.21.15.49:0/3589254462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 msgr2=0x563f0f443f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:55.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0358e00640 1 --2- 172.21.15.49:0/3589254462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f443f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.272+0000 7f0358e00640 1 -- 172.21.15.49:0/3589254462 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f0f328880 con 0x563f0f420ac0 2024-02-26T06:22:55.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.273+0000 7f0358e00640 1 --2- 172.21.15.49:0/3589254462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f420ac0 0x563f0f4439f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f034403f8e0 tx=0x7f034402c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90713ecdb3e1af9b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.273+0000 7f0349600640 1 -- 172.21.15.49:0/3589254462 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034403f480 con 0x563f0f420ac0 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.273+0000 7f0349600640 1 -- 172.21.15.49:0/3589254462 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f034403f5e0 con 0x563f0f420ac0 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.273+0000 7f0349600640 1 -- 172.21.15.49:0/3589254462 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344048b80 con 0x563f0f420ac0 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.273+0000 7f035e789740 1 -- 172.21.15.49:0/3589254462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f420ac0 msgr2=0x563f0f4439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.273+0000 7f035e789740 1 --2- 172.21.15.49:0/3589254462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f420ac0 0x563f0f4439f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f034403f8e0 tx=0x7f034402c9a0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 -- 172.21.15.49:0/3589254462 shutdown_connections 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 --2- 172.21.15.49:0/3589254462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f443f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 --2- 172.21.15.49:0/3589254462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f425550 0x563f0f4368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 --2- 172.21.15.49:0/3589254462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f420ac0 0x563f0f4439f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 -- 172.21.15.49:0/3589254462 >> 172.21.15.49:0/3589254462 conn(0x563f0f421210 msgr2=0x563f0f427260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 -- 172.21.15.49:0/3589254462 shutdown_connections 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 -- 172.21.15.49:0/3589254462 wait complete. 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 -- start start 2024-02-26T06:22:55.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.274+0000 7f035e789740 1 Processor -- start 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f035e789740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f420ac0 0x563f0f436340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f035e789740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425550 0x563f0f436880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f035e789740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f432eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f035e789740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f4335b0 con 0x563f0f425f40 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f035e789740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f433720 con 0x563f0f425550 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f035e789740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0f4338e0 con 0x563f0f420ac0 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f035e789740 10 client.0 _ll_register_callbacks cb 0x7f034c004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f0359800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f432eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f0358e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f420ac0 0x563f0f436340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f0359800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f432eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44766/0 (socket says 172.21.15.49:44766) 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f0358e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f420ac0 0x563f0f436340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44660/0 (socket says 172.21.15.49:44660) 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f0359800640 1 -- 172.21.15.49:0/4005896275 learned_addr learned my addr 172.21.15.49:0/4005896275 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:55.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f0353e00640 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425550 0x563f0f436880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.275+0000 7f0359800640 1 -- 172.21.15.49:0/4005896275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f420ac0 msgr2=0x563f0f436340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:55.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0359800640 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f420ac0 0x563f0f436340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0359800640 1 -- 172.21.15.49:0/4005896275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425550 msgr2=0x563f0f436880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:22:55.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0359800640 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425550 0x563f0f436880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0359800640 1 -- 172.21.15.49:0/4005896275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f0f38be90 con 0x563f0f425f40 2024-02-26T06:22:55.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0359800640 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f432eb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f034c003d30 tx=0x7f034c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fff64c6b066c210 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:55.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0342a00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0342a00640 1 -- 172.21.15.49:0/4005896275 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c016070 con 0x563f0f425f40 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0342a00640 1 -- 172.21.15.49:0/4005896275 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f034c01c400 con 0x563f0f425f40 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f035e789740 1 -- 172.21.15.49:0/4005896275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f0f43c580 con 0x563f0f425f40 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f0342a00640 1 -- 172.21.15.49:0/4005896275 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f034c01ee70 con 0x563f0f425f40 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.276+0000 7f035e789740 10 client.15081 fetch_fsmap learned FSMap version 23 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f035e789740 1 -- 172.21.15.49:0/4005896275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563f0f34e4e0 con 0x563f0f425f40 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f0342a00640 1 -- 172.21.15.49:0/4005896275 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f034c038080 con 0x563f0f425f40 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f0342a00640 1 -- 172.21.15.49:0/4005896275 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f034c011850 con 0x563f0f425f40 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f035e789740 10 client.15081 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f035e789740 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f035e789740 20 client.15081 populate_metadata read hostname 'smithi049' 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f035e789740 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563f0f42b0b0 0x563f0f492c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f035e789740 4 client.15081 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f035e789740 1 -- 172.21.15.49:0/4005896275 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563f0f43b260 con 0x563f0f42b0b0 2024-02-26T06:22:55.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.277+0000 7f0358e00640 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563f0f42b0b0 0x563f0f492c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f0358e00640 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563f0f42b0b0 0x563f0f492c60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7d8f31bc94c8fd server_cookie=e4b40be7be26a425 in_seq=0 out_seq=0 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f0342a00640 10 client.15081 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f0342a00640 1 -- 172.21.15.49:0/4005896275 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563f0f43b260 con 0x563f0f42b0b0 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f0342a00640 10 client.15081 handle_command_reply: tid=0 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 client.15081 shutdown 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 -- 172.21.15.49:0/4005896275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 msgr2=0x563f0f432eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f432eb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f034c003d30 tx=0x7f034c004730 comp rx=0 tx=0).stop 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 -- 172.21.15.49:0/4005896275 shutdown_connections 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563f0f42b0b0 0x563f0f492c60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f0f425550 0x563f0f436880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f0f425f40 0x563f0f432eb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 --2- 172.21.15.49:0/4005896275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f0f420ac0 0x563f0f436340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f0342a00640 0 client.15081 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.278+0000 7f035e789740 1 -- 172.21.15.49:0/4005896275 >> 172.21.15.49:0/4005896275 conn(0x563f0f421210 msgr2=0x563f0f4439f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035e789740 1 -- 172.21.15.49:0/4005896275 shutdown_connections 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035e789740 1 -- 172.21.15.49:0/4005896275 wait complete. 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035e789740 20 client.15081 trim_cache size 0 max 16384 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "a9ae7f00-d46d-11ee-95c3-87774f69a715", 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24572, 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-02-26T06:22:55.283 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 142.72595907799999, 2024-02-26T06:22:55.284 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-02-26T06:22:55.284 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-02-26T06:22:55.284 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-02-26T06:22:55.284 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 149.82862414499999 2024-02-26T06:22:55.284 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035b800640 1 -- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f03380a20c0 msgr2=0x7f03380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035b800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f03380a20c0 0x7f03380a4580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x563f0f4267a0 tx=0x7f034c00b2e0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035b800640 1 -- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0354075160 msgr2=0x7f0354199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035b800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0354075160 0x7f0354199990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e9f0 tx=0x7f033c00eeb0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035b800640 1 -- 172.21.15.49:0/1866627555 shutdown_connections 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035b800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f03380a20c0 0x7f03380a4580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035b800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0354109020 0x7f03541a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.279+0000 7f035b800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0354075160 0x7f0354199990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.280+0000 7f035b800640 1 --2- 172.21.15.49:0/1866627555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0354073b80 0x7f0354199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.280+0000 7f035b800640 1 -- 172.21.15.49:0/1866627555 >> 172.21.15.49:0/1866627555 conn(0x7f03540fc720 msgr2=0x7f0354104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.280+0000 7f035b800640 1 -- 172.21.15.49:0/1866627555 shutdown_connections 2024-02-26T06:22:55.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:55.280+0000 7f035b800640 1 -- 172.21.15.49:0/1866627555 wait complete. 2024-02-26T06:22:55.300 INFO:teuthology.run_tasks:Running task kclient... 2024-02-26T06:22:55.323 INFO:tasks.kclient:Mounting kernel clients... 2024-02-26T06:22:55.323 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-26T06:22:55.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:22:55.323 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-02-26T06:22:55.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:22:55.382 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-02-26T06:22:55.495 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-02-26T06:22:55.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:22:55.499 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-02-26T06:22:55.499 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-26T06:22:55.499 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-02-26T06:22:55.500 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-02-26T06:22:55.500 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph fs ls 2024-02-26T06:22:55.987 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:56.661 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:56 smithi049 ceph-mon[31696]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:56 smithi115 ceph-mon[36426]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:56 smithi080 ceph-mon[36206]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:22:58.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 -- 172.21.15.49:0/474521354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0103fc0 msgr2=0x7fb5c01043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:58.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/474521354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0103fc0 0x7fb5c01043a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009990 tx=0x7fb5b0009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 -- 172.21.15.49:0/474521354 shutdown_connections 2024-02-26T06:22:58.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/474521354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c01090a0 0x7fb5c0110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/474521354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0103fc0 0x7fb5c01043a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/474521354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5c0104970 0x7fb5c01089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 -- 172.21.15.49:0/474521354 >> 172.21.15.49:0/474521354 conn(0x7fb5c00ff890 msgr2=0x7fb5c0101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:58.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 -- 172.21.15.49:0/474521354 shutdown_connections 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.272+0000 7fb5c4e00700 1 -- 172.21.15.49:0/474521354 wait complete. 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.274+0000 7fb5c4e00700 1 Processor -- start 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.275+0000 7fb5c4e00700 1 -- start start 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.275+0000 7fb5c4e00700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c0103fc0 0x7fb5c019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.275+0000 7fb5c4e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0104970 0x7fb5c019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.275+0000 7fb5c4e00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5c01090a0 0x7fb5c019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.275+0000 7fb5c4e00700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c019f0f0 con 0x7fb5c0103fc0 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.275+0000 7fb5c4e00700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c019f260 con 0x7fb5c01090a0 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.275+0000 7fb5c4e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c019f420 con 0x7fb5c0104970 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bf400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c0103fc0 0x7fb5c019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bf400700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c0103fc0 0x7fb5c019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44780/0 (socket says 172.21.15.49:44780) 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bfe00700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5c01090a0 0x7fb5c019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bf400700 1 -- 172.21.15.49:0/3816390944 learned_addr learned my addr 172.21.15.49:0/3816390944 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:22:58.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0104970 0x7fb5c019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bfe00700 1 -- 172.21.15.49:0/3816390944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0104970 msgr2=0x7fb5c019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bfe00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0104970 0x7fb5c019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bfe00700 1 -- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c0103fc0 msgr2=0x7fb5c019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bfe00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c0103fc0 0x7fb5c019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bfe00700 1 -- 172.21.15.49:0/3816390944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8009c40 con 0x7fb5c01090a0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.276+0000 7fb5bea00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0104970 0x7fb5c019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.277+0000 7fb5bfe00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5c01090a0 0x7fb5c019eae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800efb0 tx=0x7fb5b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.277+0000 7fb5b7e00700 1 -- 172.21.15.49:0/3816390944 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8003a00 con 0x7fb5c01090a0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.277+0000 7fb5b7e00700 1 -- 172.21.15.49:0/3816390944 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5b800f040 con 0x7fb5c01090a0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.277+0000 7fb5b7e00700 1 -- 172.21.15.49:0/3816390944 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80046a0 con 0x7fb5c01090a0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.277+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b00095f0 con 0x7fb5c01090a0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.277+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c01a6d90 con 0x7fb5c01090a0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.278+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c010c2d0 con 0x7fb5c01090a0 2024-02-26T06:22:58.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.280+0000 7fb5b7e00700 1 -- 172.21.15.49:0/3816390944 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb5b8003cf0 con 0x7fb5c01090a0 2024-02-26T06:22:58.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.282+0000 7fb5b7e00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb5a40a21d0 0x7fb5a40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:58.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.282+0000 7fb5b7e00700 1 -- 172.21.15.49:0/3816390944 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb5b80cd760 con 0x7fb5c01090a0 2024-02-26T06:22:58.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.282+0000 7fb5bf400700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb5a40a21d0 0x7fb5a40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:58.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.283+0000 7fb5bf400700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb5a40a21d0 0x7fb5a40a4690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00098e0 tx=0x7fb5b000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:58.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.286+0000 7fb5b7e00700 1 -- 172.21.15.49:0/3816390944 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5b8096750 con 0x7fb5c01090a0 2024-02-26T06:22:58.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.562+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5c0068930 con 0x7fb5c01090a0 2024-02-26T06:22:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.563+0000 7fb5b7e00700 1 -- 172.21.15.49:0/3816390944 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb5b8099e10 con 0x7fb5c01090a0 2024-02-26T06:22:58.564 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb5a40a21d0 msgr2=0x7fb5a40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb5a40a21d0 0x7fb5a40a4690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00098e0 tx=0x7fb5b000b260 comp rx=0 tx=0).stop 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5c01090a0 msgr2=0x7fb5c019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5c01090a0 0x7fb5c019eae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800efb0 tx=0x7fb5b800c3a0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 shutdown_connections 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fb5a40a21d0 0x7fb5a40a4690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5c01090a0 0x7fb5c019eae0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c0103fc0 0x7fb5c019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 --2- 172.21.15.49:0/3816390944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c0104970 0x7fb5c019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 >> 172.21.15.49:0/3816390944 conn(0x7fb5c00ff890 msgr2=0x7fb5c01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:58.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 shutdown_connections 2024-02-26T06:22:58.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:22:58.568+0000 7fb5c4e00700 1 -- 172.21.15.49:0/3816390944 wait complete. 2024-02-26T06:22:58.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:58 smithi115 ceph-mon[36426]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:58.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:22:58 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3816390944' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T06:22:58.882 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:58 smithi080 ceph-mon[36206]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:58.882 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:22:58 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3816390944' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T06:22:58.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:58 smithi049 ceph-mon[31696]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T06:22:58.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:22:58 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3816390944' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T06:22:59.429 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-02-26T06:22:59.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:22:59.430 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-02-26T06:22:59.489 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-02-26T06:22:59.490 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 41689sec preferred_lft 41689sec 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-02-26T06:22:59.491 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-02-26T06:22:59.492 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-26T06:22:59.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:22:59.493 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-02-26T06:22:59.493 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-02-26T06:22:59.493 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-02-26T06:22:59.493 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-02-26T06:22:59.493 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-02-26T06:22:59.493 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-26T06:22:59.493 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-02-26T06:23:00.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:00.025 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T06:23:00.123 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-02-26T06:23:00.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:00.127 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-02-26T06:23:00.191 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-02-26T06:23:00.191 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T06:23:00.191 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-26T06:23:00.191 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-26T06:23:00.191 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T06:23:00.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:00.192 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-02-26T06:23:00.192 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-02-26T06:23:00.192 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-26T06:23:00.192 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-26T06:23:00.192 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-26T06:23:00.192 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-02-26T06:23:00.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:00 smithi115 ceph-mon[36426]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:23:00.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:00 smithi049 ceph-mon[31696]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:23:00.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:00.965 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-02-26T06:23:01.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:01.027 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-02-26T06:23:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:00 smithi080 ceph-mon[36206]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T06:23:01.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:01.083 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-02-26T06:23:01.083 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-02-26T06:23:01.083 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--tmp-cephfs 2024-02-26T06:23:01.083 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-02-26T06:23:01.083 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-02-26T06:23:01.250 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-02-26T06:23:01.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:01.250 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-02-26T06:23:01.250 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-02-26T06:23:01.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-02-26T06:23:01.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-26T06:23:01.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-02-26T06:23:01.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-02-26T06:23:01.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-02-26T06:23:01.251 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-02-26T06:23:01.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:01.577 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-02-26T06:23:01.577 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-02-26T06:23:01.577 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-02-26T06:23:01.577 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-02-26T06:23:01.577 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-02-26T06:23:01.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T06:23:01.773 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /tmp/cephfs 2024-02-26T06:23:01.831 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/tmp/cephfs' 2024-02-26T06:23:01.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T06:23:01.831 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /tmp/cephfs 2024-02-26T06:23:01.884 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-26T06:23:01.884 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-26T06:23:01.884 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 2024-02-26T06:23:01.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:01.884 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-02-26T06:23:02.031 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-02-26T06:23:02.031 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-02-26T06:23:02.031 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-02-26T06:23:02.744 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-02-26T06:23:02.744 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-02-26T06:23:02.744 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-02-26T06:23:02.744 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.80:3300,172.21.15.115:3300" 2024-02-26T06:23:02.744 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:3300,172.21.15.80:3300,172.21.15.115:3300:/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 2024-02-26T06:23:02.744 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=a9ae7f00-d46d-11ee-95c3-87774f69a715" will pass to kernel 2024-02-26T06:23:02.753 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T06:23:02.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:02.753 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /tmp/cephfs 2024-02-26T06:23:02.847 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:02 smithi049 ceph-mon[31696]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:23:02.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:02.861 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T06:23:02.921 INFO:teuthology.orchestra.run.smithi049.stdout:client25193 2024-02-26T06:23:02.921 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:23:02.921 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T06:23:02.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:02.976 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T06:23:03.037 INFO:teuthology.orchestra.run.smithi049.stdout:client25193 2024-02-26T06:23:03.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:03.038 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T06:23:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:02 smithi080 ceph-mon[36206]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:23:03.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:02 smithi115 ceph-mon[36426]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T06:23:03.100 INFO:teuthology.orchestra.run.smithi049.stdout:client25193 2024-02-26T06:23:03.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:03.100 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715.client25193/status') 2024-02-26T06:23:03.200 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.25193 (3)172.21.15.49:0/1060719856 2024-02-26T06:23:03.200 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-02-26T06:23:03.200 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-02-26T06:23:03.200 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-02-26T06:23:03.201 INFO:teuthology.orchestra.run.smithi049.stderr:71 bytes copied, 9.6415e-05 s, 736 kB/s 2024-02-26T06:23:03.205 INFO:teuthology.run_tasks:Running task exec... 2024-02-26T06:23:03.218 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T06:23:03.218 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:23:03.218 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-02-26T06:23:03.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 -- 172.21.15.49:0/2495835546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c8c1045d0 msgr2=0x7f3c8c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:03.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 --2- 172.21.15.49:0/2495835546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c8c1045d0 0x7f3c8c104a50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009900 tx=0x7f3c7802bc40 comp rx=0 tx=0).stop 2024-02-26T06:23:03.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 -- 172.21.15.49:0/2495835546 shutdown_connections 2024-02-26T06:23:03.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 --2- 172.21.15.49:0/2495835546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c8c104f90 0x7f3c8c10b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:03.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 --2- 172.21.15.49:0/2495835546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c8c1045d0 0x7f3c8c104a50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:03.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 --2- 172.21.15.49:0/2495835546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c8c1033e0 0x7f3c8c103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:03.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 -- 172.21.15.49:0/2495835546 >> 172.21.15.49:0/2495835546 conn(0x7f3c8c0fe970 msgr2=0x7f3c8c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:03.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 -- 172.21.15.49:0/2495835546 shutdown_connections 2024-02-26T06:23:03.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.439+0000 7f3c92e00640 1 -- 172.21.15.49:0/2495835546 wait complete. 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.440+0000 7f3c92e00640 1 Processor -- start 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.440+0000 7f3c92e00640 1 -- start start 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.440+0000 7f3c92e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c8c1033e0 0x7f3c8c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.440+0000 7f3c92e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c8c1045d0 0x7f3c8c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.440+0000 7f3c92e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c8c104f90 0x7f3c8c1a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.440+0000 7f3c92e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c1a0e40 con 0x7f3c8c1033e0 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.440+0000 7f3c92e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c1a0fb0 con 0x7f3c8c1045d0 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.440+0000 7f3c92e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c1a1170 con 0x7f3c8c104f90 2024-02-26T06:23:03.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c8c1033e0 0x7f3c8c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c8c1045d0 0x7f3c8c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c8c1045d0 0x7f3c8c199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35486/0 (socket says 172.21.15.49:35486) 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c90e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c8c104f90 0x7f3c8c1a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8b400640 1 -- 172.21.15.49:0/649983526 learned_addr learned my addr 172.21.15.49:0/649983526 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8b400640 1 -- 172.21.15.49:0/649983526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c8c104f90 msgr2=0x7f3c8c1a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8b400640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c8c104f90 0x7f3c8c1a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8b400640 1 -- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c8c1033e0 msgr2=0x7f3c8c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8b400640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c8c1033e0 0x7f3c8c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c8b400640 1 -- 172.21.15.49:0/649983526 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c009490 con 0x7f3c8c1045d0 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.441+0000 7f3c90e00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c8c104f90 0x7f3c8c1a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:03.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.442+0000 7f3c8b400640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c8c1045d0 0x7f3c8c199890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3c7802c6b0 tx=0x7f3c780040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:03.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.442+0000 7f3c88c00640 1 -- 172.21.15.49:0/649983526 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c780044c0 con 0x7f3c8c1045d0 2024-02-26T06:23:03.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.442+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c78009560 con 0x7f3c8c1045d0 2024-02-26T06:23:03.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.442+0000 7f3c88c00640 1 -- 172.21.15.49:0/649983526 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c780052d0 con 0x7f3c8c1045d0 2024-02-26T06:23:03.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.442+0000 7f3c88c00640 1 -- 172.21.15.49:0/649983526 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78003e80 con 0x7f3c8c1045d0 2024-02-26T06:23:03.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.442+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c1a1770 con 0x7f3c8c1045d0 2024-02-26T06:23:03.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.443+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c04eee0 con 0x7f3c8c1045d0 2024-02-26T06:23:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.446+0000 7f3c88c00640 1 -- 172.21.15.49:0/649983526 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3c78005440 con 0x7f3c8c1045d0 2024-02-26T06:23:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.447+0000 7f3c88c00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c700a2160 0x7f3c700a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.447+0000 7f3c88c00640 1 -- 172.21.15.49:0/649983526 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3c780f0680 con 0x7f3c8c1045d0 2024-02-26T06:23:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.448+0000 7f3c8be00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c700a2160 0x7f3c700a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:03.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.448+0000 7f3c8be00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c700a2160 0x7f3c700a4620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c104430 tx=0x7f3c7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.451+0000 7f3c88c00640 1 -- 172.21.15.49:0/649983526 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c780b9700 con 0x7f3c8c1045d0 2024-02-26T06:23:03.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:03.709+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f3c8c02cda0 con 0x7f3c8c1045d0 2024-02-26T06:23:03.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:03 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:04.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:03 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:04.082 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:03 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:04.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.621+0000 7f3c88c00640 1 -- 172.21.15.49:0/649983526 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) v1 ==== 117+0+0 (secure 0 0 0) 0x7f3c780430b0 con 0x7f3c8c1045d0 2024-02-26T06:23:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.624+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c700a2160 msgr2=0x7f3c700a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.624+0000 7f3c92e00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c700a2160 0x7f3c700a4620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c104430 tx=0x7f3c7c005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.624+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c8c1045d0 msgr2=0x7f3c8c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.624+0000 7f3c92e00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c8c1045d0 0x7f3c8c199890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3c7802c6b0 tx=0x7f3c780040c0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.625+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 shutdown_connections 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.625+0000 7f3c92e00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3c700a2160 0x7f3c700a4620 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.625+0000 7f3c92e00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c8c1045d0 0x7f3c8c199890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.625+0000 7f3c92e00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c8c104f90 0x7f3c8c1a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.625+0000 7f3c92e00640 1 --2- 172.21.15.49:0/649983526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c8c1033e0 0x7f3c8c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.625+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 >> 172.21.15.49:0/649983526 conn(0x7f3c8c0fe970 msgr2=0x7f3c8c107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.625+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 shutdown_connections 2024-02-26T06:23:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.625+0000 7f3c92e00640 1 -- 172.21.15.49:0/649983526 wait complete. 2024-02-26T06:23:04.646 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-26T06:23:04.657 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-26T06:23:04.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:23:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 -- 172.21.15.49:0/2741164921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcaa4075b40 msgr2=0x7fcaa4075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 --2- 172.21.15.49:0/2741164921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcaa4075b40 0x7fcaa4075f60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca94009ca0 tx=0x7fca9402c100 comp rx=0 tx=0).stop 2024-02-26T06:23:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 -- 172.21.15.49:0/2741164921 shutdown_connections 2024-02-26T06:23:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 --2- 172.21.15.49:0/2741164921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa4077720 0x7fcaa4077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 --2- 172.21.15.49:0/2741164921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4076d80 0x7fcaa40771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 --2- 172.21.15.49:0/2741164921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcaa4075b40 0x7fcaa4075f60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 -- 172.21.15.49:0/2741164921 >> 172.21.15.49:0/2741164921 conn(0x7fcaa40fe930 msgr2=0x7fcaa4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 -- 172.21.15.49:0/2741164921 shutdown_connections 2024-02-26T06:23:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.911+0000 7fcaa9400640 1 -- 172.21.15.49:0/2741164921 wait complete. 2024-02-26T06:23:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.912+0000 7fcaa9400640 1 Processor -- start 2024-02-26T06:23:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.912+0000 7fcaa9400640 1 -- start start 2024-02-26T06:23:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa9400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4075b40 0x7fcaa419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa4076d80 0x7fcaa419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcaa4077720 0x7fcaa41a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa9400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41a5320 con 0x7fcaa4076d80 2024-02-26T06:23:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa9400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41a5490 con 0x7fcaa4077720 2024-02-26T06:23:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa9400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41a5650 con 0x7fcaa4075b40 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4075b40 0x7fcaa419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4075b40 0x7fcaa419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44638/0 (socket says 172.21.15.49:44638) 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa2800640 1 -- 172.21.15.49:0/1741001846 learned_addr learned my addr 172.21.15.49:0/1741001846 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fcaa3200640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcaa4077720 0x7fcaa41a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.913+0000 7fca99e00640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa4076d80 0x7fcaa419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fcaa2800640 1 -- 172.21.15.49:0/1741001846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcaa4077720 msgr2=0x7fcaa41a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fcaa2800640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcaa4077720 0x7fcaa41a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fcaa2800640 1 -- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa4076d80 msgr2=0x7fcaa419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fcaa2800640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa4076d80 0x7fcaa419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fcaa2800640 1 -- 172.21.15.49:0/1741001846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca94009900 con 0x7fcaa4075b40 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fca99e00640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa4076d80 0x7fcaa419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fcaa2800640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4075b40 0x7fcaa419d830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca94000f80 tx=0x7fca94032510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fca9be00640 1 -- 172.21.15.49:0/1741001846 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94003970 con 0x7fcaa4075b40 2024-02-26T06:23:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fca9be00640 1 -- 172.21.15.49:0/1741001846 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca94004040 con 0x7fcaa4075b40 2024-02-26T06:23:04.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa41a58d0 con 0x7fcaa4075b40 2024-02-26T06:23:04.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.914+0000 7fca9be00640 1 -- 172.21.15.49:0/1741001846 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca940406b0 con 0x7fcaa4075b40 2024-02-26T06:23:04.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.915+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa41a5dd0 con 0x7fcaa4075b40 2024-02-26T06:23:04.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.916+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa410fe70 con 0x7fcaa4075b40 2024-02-26T06:23:04.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.918+0000 7fca9be00640 1 -- 172.21.15.49:0/1741001846 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fca9400a040 con 0x7fcaa4075b40 2024-02-26T06:23:04.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:04 smithi049 ceph-mon[31696]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:23:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:04 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/649983526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T06:23:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:04 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T06:23:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:04 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:04.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.920+0000 7fca9be00640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca880a2140 0x7fca880a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.920+0000 7fca9be00640 1 -- 172.21.15.49:0/1741001846 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fca940f0d60 con 0x7fcaa4075b40 2024-02-26T06:23:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.920+0000 7fca99e00640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca880a2140 0x7fca880a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.921+0000 7fca99e00640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca880a2140 0x7fca880a4600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcaa419ed40 tx=0x7fca8c00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:04.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:04.927+0000 7fca9be00640 1 -- 172.21.15.49:0/1741001846 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca940b9d50 con 0x7fcaa4075b40 2024-02-26T06:23:04.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:04 smithi080 ceph-mon[36206]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:23:04.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:04 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/649983526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T06:23:04.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:04 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T06:23:04.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:04.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:04.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:04.967 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:04 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:04 smithi115 ceph-mon[36426]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T06:23:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:04 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/649983526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T06:23:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:04 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T06:23:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:05.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:05.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:04 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:05.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.154+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcaa410f8e0 con 0x7fcaa4075b40 2024-02-26T06:23:05.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.155+0000 7fca9be00640 1 -- 172.21.15.49:0/1741001846 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7fca940041b0 con 0x7fcaa4075b40 2024-02-26T06:23:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:05.155 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca880a2140 msgr2=0x7fca880a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca880a2140 0x7fca880a4600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcaa419ed40 tx=0x7fca8c00a5c0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4075b40 msgr2=0x7fcaa419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4075b40 0x7fcaa419d830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca94000f80 tx=0x7fca94032510 comp rx=0 tx=0).stop 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 shutdown_connections 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fca880a2140 0x7fca880a4600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcaa4077720 0x7fcaa41a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa4076d80 0x7fcaa419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 --2- 172.21.15.49:0/1741001846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4075b40 0x7fcaa419d830 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.158+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 >> 172.21.15.49:0/1741001846 conn(0x7fcaa40fe930 msgr2=0x7fcaa40fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:05.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.159+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 shutdown_connections 2024-02-26T06:23:05.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.159+0000 7fcaa9400640 1 -- 172.21.15.49:0/1741001846 wait complete. 2024-02-26T06:23:05.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:23:05.178 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:23:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 -- 172.21.15.49:0/1632315721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7face01045b0 msgr2=0x7face0104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 --2- 172.21.15.49:0/1632315721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7face01045b0 0x7face0104a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7facd0009900 tx=0x7facd002bc40 comp rx=0 tx=0).stop 2024-02-26T06:23:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 -- 172.21.15.49:0/1632315721 shutdown_connections 2024-02-26T06:23:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 --2- 172.21.15.49:0/1632315721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7face01045b0 0x7face0104a30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 --2- 172.21.15.49:0/1632315721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face0104f70 0x7face010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 --2- 172.21.15.49:0/1632315721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7face01033c0 0x7face01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 -- 172.21.15.49:0/1632315721 >> 172.21.15.49:0/1632315721 conn(0x7face00fe930 msgr2=0x7face0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 -- 172.21.15.49:0/1632315721 shutdown_connections 2024-02-26T06:23:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.385+0000 7face8800640 1 -- 172.21.15.49:0/1632315721 wait complete. 2024-02-26T06:23:05.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.386+0000 7face8800640 1 Processor -- start 2024-02-26T06:23:05.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.386+0000 7face8800640 1 -- start start 2024-02-26T06:23:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face8800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7face01033c0 0x7face0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face8800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7face01045b0 0x7face01999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face8800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face0104f70 0x7face01a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face8800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face01a0f60 con 0x7face0104f70 2024-02-26T06:23:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face8800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face01a10d0 con 0x7face01033c0 2024-02-26T06:23:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face8800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face01a1290 con 0x7face01045b0 2024-02-26T06:23:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7face01045b0 0x7face01999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7face01045b0 0x7face01999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44658/0 (socket says 172.21.15.49:44658) 2024-02-26T06:23:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face5400640 1 -- 172.21.15.49:0/3871659309 learned_addr learned my addr 172.21.15.49:0/3871659309 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face5e00640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7face01033c0 0x7face0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.387+0000 7face6800640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face0104f70 0x7face01a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7face5400640 1 -- 172.21.15.49:0/3871659309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7face01033c0 msgr2=0x7face0199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7face5400640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7face01033c0 0x7face0199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7face5400640 1 -- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face0104f70 msgr2=0x7face01a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7face5400640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face0104f70 0x7face01a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7face5400640 1 -- 172.21.15.49:0/3871659309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd0009560 con 0x7face01045b0 2024-02-26T06:23:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7face5400640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7face01045b0 0x7face01999b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7facd002c6b0 tx=0x7facd00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7facdaa00640 1 -- 172.21.15.49:0/3871659309 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd00044c0 con 0x7face01045b0 2024-02-26T06:23:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7facdaa00640 1 -- 172.21.15.49:0/3871659309 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facd00052d0 con 0x7face01045b0 2024-02-26T06:23:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face01a1510 con 0x7face01045b0 2024-02-26T06:23:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7facdaa00640 1 -- 172.21.15.49:0/3871659309 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0003e80 con 0x7face01045b0 2024-02-26T06:23:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.388+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face01a1a20 con 0x7face01045b0 2024-02-26T06:23:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.389+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face010a680 con 0x7face01045b0 2024-02-26T06:23:05.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.391+0000 7facdaa00640 1 -- 172.21.15.49:0/3871659309 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7facd0005440 con 0x7face01045b0 2024-02-26T06:23:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.393+0000 7facdaa00640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7facc80a20c0 0x7facc80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.393+0000 7facdaa00640 1 -- 172.21.15.49:0/3871659309 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7facd00f0620 con 0x7face01045b0 2024-02-26T06:23:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.393+0000 7face5e00640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7facc80a20c0 0x7facc80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:05.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.394+0000 7face5e00640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7facc80a20c0 0x7facc80a4580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7face0104410 tx=0x7facd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:05.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.396+0000 7facdaa00640 1 -- 172.21.15.49:0/3871659309 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7facd00b9700 con 0x7face01045b0 2024-02-26T06:23:05.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.607+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7face002cda0 con 0x7face01045b0 2024-02-26T06:23:05.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.608+0000 7facdaa00640 1 -- 172.21.15.49:0/3871659309 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7facd00430a0 con 0x7face01045b0 2024-02-26T06:23:05.608 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:23:05.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.610+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7facc80a20c0 msgr2=0x7facc80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.610+0000 7face8800640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7facc80a20c0 0x7facc80a4580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7face0104410 tx=0x7facd4005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.610+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7face01045b0 msgr2=0x7face01999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.610+0000 7face8800640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7face01045b0 0x7face01999b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7facd002c6b0 tx=0x7facd00040c0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.611+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 shutdown_connections 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.611+0000 7face8800640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7facc80a20c0 0x7facc80a4580 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.611+0000 7face8800640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face0104f70 0x7face01a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.611+0000 7face8800640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7face01045b0 0x7face01999b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.611+0000 7face8800640 1 --2- 172.21.15.49:0/3871659309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7face01033c0 0x7face0199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.611+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 >> 172.21.15.49:0/3871659309 conn(0x7face00fe930 msgr2=0x7face0107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.611+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 shutdown_connections 2024-02-26T06:23:05.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.611+0000 7face8800640 1 -- 172.21.15.49:0/3871659309 wait complete. 2024-02-26T06:23:05.631 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:23:05.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.859+0000 7f5d12800640 1 -- 172.21.15.49:0/4163552682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c102380 msgr2=0x7f5d0c102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.859+0000 7f5d12800640 1 --2- 172.21.15.49:0/4163552682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c102380 0x7f5d0c102800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009900 tx=0x7f5cf402bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.859+0000 7f5d12800640 1 -- 172.21.15.49:0/4163552682 shutdown_connections 2024-02-26T06:23:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.859+0000 7f5d12800640 1 --2- 172.21.15.49:0/4163552682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c102380 0x7f5d0c102800 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.859+0000 7f5d12800640 1 --2- 172.21.15.49:0/4163552682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c102d40 0x7f5d0c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.859+0000 7f5d12800640 1 --2- 172.21.15.49:0/4163552682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c101190 0x7f5d0c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.859+0000 7f5d12800640 1 -- 172.21.15.49:0/4163552682 >> 172.21.15.49:0/4163552682 conn(0x7f5d0c0fc720 msgr2=0x7f5d0c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.860+0000 7f5d12800640 1 -- 172.21.15.49:0/4163552682 shutdown_connections 2024-02-26T06:23:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.860+0000 7f5d12800640 1 -- 172.21.15.49:0/4163552682 wait complete. 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d12800640 1 Processor -- start 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d12800640 1 -- start start 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d12800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c101190 0x7f5d0c074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d12800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102380 0x7f5d0c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d12800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c102d40 0x7f5d0c073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d12800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c074210 con 0x7f5d0c102d40 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d12800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c074380 con 0x7f5d0c101190 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d12800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c1a12b0 con 0x7f5d0c102380 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d0b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c101190 0x7f5d0c074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d02a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102380 0x7f5d0c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.861+0000 7f5d0b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c101190 0x7f5d0c074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35546/0 (socket says 172.21.15.49:35546) 2024-02-26T06:23:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d0b400640 1 -- 172.21.15.49:0/3583977485 learned_addr learned my addr 172.21.15.49:0/3583977485 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d0be00640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c102d40 0x7f5d0c073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d0b400640 1 -- 172.21.15.49:0/3583977485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102380 msgr2=0x7f5d0c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d0b400640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102380 0x7f5d0c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d0b400640 1 -- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c102d40 msgr2=0x7f5d0c073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d0b400640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c102d40 0x7f5d0c073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d0b400640 1 -- 172.21.15.49:0/3583977485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf4009560 con 0x7f5d0c101190 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d02a00640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102380 0x7f5d0c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.862+0000 7f5d0b400640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c101190 0x7f5d0c074f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00eba0 tx=0x7f5cfc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.863+0000 7f5d08c00640 1 -- 172.21.15.49:0/3583977485 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc003b70 con 0x7f5d0c101190 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.863+0000 7f5d08c00640 1 -- 172.21.15.49:0/3583977485 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc00ced0 con 0x7f5d0c101190 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.863+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c1a1560 con 0x7f5d0c101190 2024-02-26T06:23:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.863+0000 7f5d08c00640 1 -- 172.21.15.49:0/3583977485 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0106b0 con 0x7f5d0c101190 2024-02-26T06:23:05.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.863+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c1a1a40 con 0x7f5d0c101190 2024-02-26T06:23:05.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.864+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c1084b0 con 0x7f5d0c101190 2024-02-26T06:23:05.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.866+0000 7f5d08c00640 1 -- 172.21.15.49:0/3583977485 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5cfc004020 con 0x7f5d0c101190 2024-02-26T06:23:05.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.867+0000 7f5d08c00640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5cf00a2140 0x7f5cf00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:05.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.867+0000 7f5d08c00640 1 -- 172.21.15.49:0/3583977485 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f5cfc014070 con 0x7f5d0c101190 2024-02-26T06:23:05.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.867+0000 7f5d02a00640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5cf00a2140 0x7f5cf00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:05.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.868+0000 7f5d02a00640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5cf00a2140 0x7f5cf00a4600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4000c00 tx=0x7f5cf402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:05.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:05.871+0000 7f5d08c00640 1 -- 172.21.15.49:0/3583977485 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cfc097210 con 0x7f5d0c101190 2024-02-26T06:23:05.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:05 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-26T06:23:05.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:05 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:23:05.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:05 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:05.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:05 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:05.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:05 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1741001846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:05.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:05 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3871659309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:06.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.074+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d0c10cbf0 con 0x7f5d0c101190 2024-02-26T06:23:06.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.075+0000 7f5d08c00640 1 -- 172.21.15.49:0/3583977485 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5cfc09a8d0 con 0x7f5d0c101190 2024-02-26T06:23:06.075 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:23:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.077+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5cf00a2140 msgr2=0x7f5cf00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.077+0000 7f5d12800640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5cf00a2140 0x7f5cf00a4600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4000c00 tx=0x7f5cf402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.077+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c101190 msgr2=0x7f5d0c074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c101190 0x7f5d0c074f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00eba0 tx=0x7f5cfc00c6a0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 shutdown_connections 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f5cf00a2140 0x7f5cf00a4600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c102d40 0x7f5d0c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102380 0x7f5d0c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 --2- 172.21.15.49:0/3583977485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d0c101190 0x7f5d0c074f30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 >> 172.21.15.49:0/3583977485 conn(0x7f5d0c0fc720 msgr2=0x7f5d0c105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 shutdown_connections 2024-02-26T06:23:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.078+0000 7f5d12800640 1 -- 172.21.15.49:0/3583977485 wait complete. 2024-02-26T06:23:06.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:05 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-26T06:23:06.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:05 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:23:06.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:05 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:06.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:05 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:06.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:05 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1741001846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:06.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:05 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3871659309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:06.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:05 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-26T06:23:06.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:05 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:23:06.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:05 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:06.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:05 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:06.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:05 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1741001846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:06.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:05 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3871659309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:06.095 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 -- 172.21.15.49:0/4160363297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102380 msgr2=0x7f7bf8102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4160363297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102380 0x7f7bf8102800 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009900 tx=0x7f7be402bc40 comp rx=0 tx=0).stop 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 -- 172.21.15.49:0/4160363297 shutdown_connections 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4160363297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102380 0x7f7bf8102800 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4160363297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bf8102d40 0x7f7bf81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4160363297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf8101190 0x7f7bf81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 -- 172.21.15.49:0/4160363297 >> 172.21.15.49:0/4160363297 conn(0x7f7bf80fc720 msgr2=0x7f7bf80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 -- 172.21.15.49:0/4160363297 shutdown_connections 2024-02-26T06:23:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.312+0000 7f7bfda00640 1 -- 172.21.15.49:0/4160363297 wait complete. 2024-02-26T06:23:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.313+0000 7f7bfda00640 1 Processor -- start 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.313+0000 7f7bfda00640 1 -- start start 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bfda00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf8101190 0x7f7bf8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bfda00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bf8102380 0x7f7bf8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bfda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102d40 0x7f7bf81a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bfda00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81a0f10 con 0x7f7bf8102380 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bfda00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81a1080 con 0x7f7bf8102d40 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bfda00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81a1240 con 0x7f7bf8101190 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102d40 0x7f7bf81a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102d40 0x7f7bf81a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35556/0 (socket says 172.21.15.49:35556) 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf7400640 1 -- 172.21.15.49:0/4253021018 learned_addr learned my addr 172.21.15.49:0/4253021018 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf6000640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bf8102380 0x7f7bf8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf6a00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf8101190 0x7f7bf8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf7400640 1 -- 172.21.15.49:0/4253021018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf8101190 msgr2=0x7f7bf8199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf7400640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf8101190 0x7f7bf8199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf7400640 1 -- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bf8102380 msgr2=0x7f7bf8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf7400640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bf8102380 0x7f7bf8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.314+0000 7f7bf7400640 1 -- 172.21.15.49:0/4253021018 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be8009490 con 0x7f7bf8102d40 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.315+0000 7f7bf7400640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102d40 0x7f7bf81a09d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000eb10 tx=0x7f7bf000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.315+0000 7f7bef400640 1 -- 172.21.15.49:0/4253021018 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0004b20 con 0x7f7bf8102d40 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.315+0000 7f7bef400640 1 -- 172.21.15.49:0/4253021018 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0004c80 con 0x7f7bf8102d40 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.315+0000 7f7bef400640 1 -- 172.21.15.49:0/4253021018 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0010650 con 0x7f7bf8102d40 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.315+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be4009560 con 0x7f7bf8102d40 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.315+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf81a1870 con 0x7f7bf8102d40 2024-02-26T06:23:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.317+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf810cbf0 con 0x7f7bf8102d40 2024-02-26T06:23:06.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.318+0000 7f7bef400640 1 -- 172.21.15.49:0/4253021018 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7bf0005020 con 0x7f7bf8102d40 2024-02-26T06:23:06.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.319+0000 7f7bef400640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7bdc0a2140 0x7f7bdc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:06.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.319+0000 7f7bef400640 1 -- 172.21.15.49:0/4253021018 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7bf0014070 con 0x7f7bf8102d40 2024-02-26T06:23:06.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.319+0000 7f7bf6a00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7bdc0a2140 0x7f7bdc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:06.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.321+0000 7f7bf6a00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7bdc0a2140 0x7f7bdc0a4600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7bf81021e0 tx=0x7f7be8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:06.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.323+0000 7f7bef400640 1 -- 172.21.15.49:0/4253021018 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7bf0097190 con 0x7f7bf8102d40 2024-02-26T06:23:06.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.540+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7bf810cd80 con 0x7f7bf8102d40 2024-02-26T06:23:06.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.540+0000 7f7bef400640 1 -- 172.21.15.49:0/4253021018 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7bf009a850 con 0x7f7bf8102d40 2024-02-26T06:23:06.540 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:23:06.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7bdc0a2140 msgr2=0x7f7bdc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7bdc0a2140 0x7f7bdc0a4600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7bf81021e0 tx=0x7f7be8005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:06.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102d40 msgr2=0x7f7bf81a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102d40 0x7f7bf81a09d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000eb10 tx=0x7f7bf000c2d0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 shutdown_connections 2024-02-26T06:23:06.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7bdc0a2140 0x7f7bdc0a4600 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bf8102d40 0x7f7bf81a09d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bf8102380 0x7f7bf8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.543+0000 7f7bfda00640 1 --2- 172.21.15.49:0/4253021018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf8101190 0x7f7bf8199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.544+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 >> 172.21.15.49:0/4253021018 conn(0x7f7bf80fc720 msgr2=0x7f7bf80733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:06.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.544+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 shutdown_connections 2024-02-26T06:23:06.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.544+0000 7f7bfda00640 1 -- 172.21.15.49:0/4253021018 wait complete. 2024-02-26T06:23:06.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:23:06.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 -- 172.21.15.49:0/3651336097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 msgr2=0x7f994c1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 --2- 172.21.15.49:0/3651336097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 0x7f994c1095e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9944009be0 tx=0x7f994402bef0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 -- 172.21.15.49:0/3651336097 shutdown_connections 2024-02-26T06:23:06.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 --2- 172.21.15.49:0/3651336097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 0x7f994c1095e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 --2- 172.21.15.49:0/3651336097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c102390 0x7f994c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 --2- 172.21.15.49:0/3651336097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c1011a0 0x7f994c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 -- 172.21.15.49:0/3651336097 >> 172.21.15.49:0/3651336097 conn(0x7f994c0fc730 msgr2=0x7f994c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:06.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 -- 172.21.15.49:0/3651336097 shutdown_connections 2024-02-26T06:23:06.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.773+0000 7f9954200640 1 -- 172.21.15.49:0/3651336097 wait complete. 2024-02-26T06:23:06.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.774+0000 7f9954200640 1 Processor -- start 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.774+0000 7f9954200640 1 -- start start 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.774+0000 7f9954200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c1011a0 0x7f994c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.774+0000 7f9954200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c102390 0x7f994c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.774+0000 7f9954200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 0x7f994c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.774+0000 7f9954200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1a0ee0 con 0x7f994c1011a0 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.774+0000 7f9954200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1a1050 con 0x7f994c102d50 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.774+0000 7f9954200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1a1210 con 0x7f994c102390 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9951800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c1011a0 0x7f994c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9951800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c1011a0 0x7f994c1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38024/0 (socket says 172.21.15.49:38024) 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9951800640 1 -- 172.21.15.49:0/3611982169 learned_addr learned my addr 172.21.15.49:0/3611982169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9952200640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 0x7f994c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9950e00640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c102390 0x7f994c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9951800640 1 -- 172.21.15.49:0/3611982169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c102390 msgr2=0x7f994c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9951800640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c102390 0x7f994c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9951800640 1 -- 172.21.15.49:0/3611982169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 msgr2=0x7f994c1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:06.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9951800640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 0x7f994c1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:06.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9951800640 1 -- 172.21.15.49:0/3611982169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940009490 con 0x7f994c1011a0 2024-02-26T06:23:06.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.775+0000 7f9952200640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 0x7f994c1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:06.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.776+0000 7f9951800640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c1011a0 0x7f994c1993f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f994000e9c0 tx=0x7f994000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:06.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.776+0000 7f994a000640 1 -- 172.21.15.49:0/3611982169 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994000cde0 con 0x7f994c1011a0 2024-02-26T06:23:06.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.776+0000 7f994a000640 1 -- 172.21.15.49:0/3611982169 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9940010910 con 0x7f994c1011a0 2024-02-26T06:23:06.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.776+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9938009560 con 0x7f994c1011a0 2024-02-26T06:23:06.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.776+0000 7f994a000640 1 -- 172.21.15.49:0/3611982169 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940018a10 con 0x7f994c1011a0 2024-02-26T06:23:06.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.776+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9944009840 con 0x7f994c1011a0 2024-02-26T06:23:06.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.777+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c10cbc0 con 0x7f994c1011a0 2024-02-26T06:23:06.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.779+0000 7f994a000640 1 -- 172.21.15.49:0/3611982169 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9940010a80 con 0x7f994c1011a0 2024-02-26T06:23:06.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.782+0000 7f994a000640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f99340a20c0 0x7f99340a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:06.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.782+0000 7f994a000640 1 -- 172.21.15.49:0/3611982169 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9940014070 con 0x7f994c1011a0 2024-02-26T06:23:06.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.782+0000 7f9950e00640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f99340a20c0 0x7f99340a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:06.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.783+0000 7f9950e00640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f99340a20c0 0x7f99340a4580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f994c19a900 tx=0x7f9938009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.785+0000 7f994a000640 1 -- 172.21.15.49:0/3611982169 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9940097300 con 0x7f994c1011a0 2024-02-26T06:23:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.998+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f994c10cda0 con 0x7f994c1011a0 2024-02-26T06:23:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:06.999+0000 7f994a000640 1 -- 172.21.15.49:0/3611982169 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7f994009a9c0 con 0x7f994c1011a0 2024-02-26T06:23:06.999 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:06.999 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:23:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f99340a20c0 msgr2=0x7f99340a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f99340a20c0 0x7f99340a4580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f994c19a900 tx=0x7f9938009260 comp rx=0 tx=0).stop 2024-02-26T06:23:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c1011a0 msgr2=0x7f994c1993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c1011a0 0x7f994c1993f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f994000e9c0 tx=0x7f994000ee80 comp rx=0 tx=0).stop 2024-02-26T06:23:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 shutdown_connections 2024-02-26T06:23:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f99340a20c0 0x7f99340a4580 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f994c102d50 0x7f994c1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c102390 0x7f994c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 --2- 172.21.15.49:0/3611982169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c1011a0 0x7f994c1993f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 >> 172.21.15.49:0/3611982169 conn(0x7f994c0fc730 msgr2=0x7f994c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:07.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.002+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 shutdown_connections 2024-02-26T06:23:07.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.003+0000 7f9954200640 1 -- 172.21.15.49:0/3611982169 wait complete. 2024-02-26T06:23:07.021 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:23:07.021 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:23:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:06 smithi049 ceph-mon[31696]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:23:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:06 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:06 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3583977485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:06 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4253021018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:07.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.224+0000 7f522bc00640 1 -- 172.21.15.49:0/2847289946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 msgr2=0x7f52240771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.224+0000 7f522bc00640 1 --2- 172.21.15.49:0/2847289946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 0x7f52240771e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5210009900 tx=0x7f521002bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:07.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.224+0000 7f522bc00640 1 -- 172.21.15.49:0/2847289946 shutdown_connections 2024-02-26T06:23:07.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.224+0000 7f522bc00640 1 --2- 172.21.15.49:0/2847289946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 0x7f52240771e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.224+0000 7f522bc00640 1 --2- 172.21.15.49:0/2847289946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5224077720 0x7f5224077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.224+0000 7f522bc00640 1 --2- 172.21.15.49:0/2847289946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5224075b40 0x7f5224075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.224+0000 7f522bc00640 1 -- 172.21.15.49:0/2847289946 >> 172.21.15.49:0/2847289946 conn(0x7f52240fe930 msgr2=0x7f5224100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:07.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.225+0000 7f522bc00640 1 -- 172.21.15.49:0/2847289946 shutdown_connections 2024-02-26T06:23:07.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.225+0000 7f522bc00640 1 -- 172.21.15.49:0/2847289946 wait complete. 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f522bc00640 1 Processor -- start 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f522bc00640 1 -- start start 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f522bc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5224075b40 0x7f522419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f522bc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 0x7f522419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f522bc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5224077720 0x7f52241a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f522bc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52241a53b0 con 0x7f5224077720 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f522bc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52241a5520 con 0x7f5224076d80 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f522bc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52241a56e0 con 0x7f5224075b40 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f5229c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5224077720 0x7f52241a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.226+0000 7f5223e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 0x7f522419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5224077720 0x7f52241a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38040/0 (socket says 172.21.15.49:38040) 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5223e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 0x7f522419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35584/0 (socket says 172.21.15.49:35584) 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229c00640 1 -- 172.21.15.49:0/3690128996 learned_addr learned my addr 172.21.15.49:0/3690128996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229200640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5224075b40 0x7f522419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229c00640 1 -- 172.21.15.49:0/3690128996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5224075b40 msgr2=0x7f522419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229c00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5224075b40 0x7f522419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229c00640 1 -- 172.21.15.49:0/3690128996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 msgr2=0x7f522419de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229c00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 0x7f522419de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229c00640 1 -- 172.21.15.49:0/3690128996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5210009560 con 0x7f5224077720 2024-02-26T06:23:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.227+0000 7f5229c00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5224077720 0x7f52241a4e70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f521c00eb10 tx=0x7f521c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:07.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.228+0000 7f5221600640 1 -- 172.21.15.49:0/3690128996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c004b20 con 0x7f5224077720 2024-02-26T06:23:07.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.228+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52241a59c0 con 0x7f5224077720 2024-02-26T06:23:07.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.228+0000 7f5221600640 1 -- 172.21.15.49:0/3690128996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f521c004c80 con 0x7f5224077720 2024-02-26T06:23:07.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.228+0000 7f5221600640 1 -- 172.21.15.49:0/3690128996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c010650 con 0x7f5224077720 2024-02-26T06:23:07.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.228+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52241a5ea0 con 0x7f5224077720 2024-02-26T06:23:07.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.230+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522410ff70 con 0x7f5224077720 2024-02-26T06:23:07.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.231+0000 7f5221600640 1 -- 172.21.15.49:0/3690128996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f521c005020 con 0x7f5224077720 2024-02-26T06:23:07.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.233+0000 7f5221600640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f520c0a20c0 0x7f520c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:07.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.233+0000 7f5221600640 1 -- 172.21.15.49:0/3690128996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f521c014070 con 0x7f5224077720 2024-02-26T06:23:07.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.233+0000 7f5229200640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f520c0a20c0 0x7f520c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:07.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.234+0000 7f5229200640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f520c0a20c0 0x7f520c0a4580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5218009bd0 tx=0x7f5218005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:07.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.237+0000 7f5221600640 1 -- 172.21.15.49:0/3690128996 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f521c0971f0 con 0x7f5224077720 2024-02-26T06:23:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:06 smithi080 ceph-mon[36206]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:23:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:06 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:06 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3583977485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:06 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4253021018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:06 smithi115 ceph-mon[36426]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T06:23:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:06 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:06 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3583977485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:06 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4253021018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.442+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f522402ce40 con 0x7f5224077720 2024-02-26T06:23:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.443+0000 7f5221600640 1 -- 172.21.15.49:0/3690128996 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f521c09a8b0 con 0x7f5224077720 2024-02-26T06:23:07.443 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.445+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f520c0a20c0 msgr2=0x7f520c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.445+0000 7f522bc00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f520c0a20c0 0x7f520c0a4580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5218009bd0 tx=0x7f5218005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.445+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5224077720 msgr2=0x7f52241a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.445+0000 7f522bc00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5224077720 0x7f52241a4e70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f521c00eb10 tx=0x7f521c00c2d0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.446+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 shutdown_connections 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.446+0000 7f522bc00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f520c0a20c0 0x7f520c0a4580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.446+0000 7f522bc00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224076d80 0x7f522419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.446+0000 7f522bc00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5224077720 0x7f52241a4e70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.446+0000 7f522bc00640 1 --2- 172.21.15.49:0/3690128996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5224075b40 0x7f522419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.446+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 >> 172.21.15.49:0/3690128996 conn(0x7f52240fe930 msgr2=0x7f52240fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.446+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 shutdown_connections 2024-02-26T06:23:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.446+0000 7f522bc00640 1 -- 172.21.15.49:0/3690128996 wait complete. 2024-02-26T06:23:07.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:23:07.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 -- 172.21.15.49:0/1637411712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 msgr2=0x7f9c40073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 --2- 172.21.15.49:0/1637411712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 0x7f9c40073610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009900 tx=0x7f9c3002bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:07.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 -- 172.21.15.49:0/1637411712 shutdown_connections 2024-02-26T06:23:07.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 --2- 172.21.15.49:0/1637411712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c40109050 0x7f9c4010b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 --2- 172.21.15.49:0/1637411712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c40073be0 0x7f9c40074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 --2- 172.21.15.49:0/1637411712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 0x7f9c40073610 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 -- 172.21.15.49:0/1637411712 >> 172.21.15.49:0/1637411712 conn(0x7f9c400fc700 msgr2=0x7f9c400feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:07.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 -- 172.21.15.49:0/1637411712 shutdown_connections 2024-02-26T06:23:07.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.696+0000 7f9c46c00640 1 -- 172.21.15.49:0/1637411712 wait complete. 2024-02-26T06:23:07.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.697+0000 7f9c46c00640 1 Processor -- start 2024-02-26T06:23:07.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c46c00640 1 -- start start 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c46c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c40073be0 0x7f9c401994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c46c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 0x7f9c40199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c46c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c40109050 0x7f9c401a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c46c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401a0f70 con 0x7f9c40073be0 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c46c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401a10e0 con 0x7f9c400751c0 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c46c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401a12a0 con 0x7f9c40109050 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c3fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c40073be0 0x7f9c401994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c3fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c40073be0 0x7f9c401994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38066/0 (socket says 172.21.15.49:38066) 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c37e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 0x7f9c40199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.698+0000 7f9c3fe00640 1 -- 172.21.15.49:0/2029082120 learned_addr learned my addr 172.21.15.49:0/2029082120 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c44c00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c40109050 0x7f9c401a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c37e00640 1 -- 172.21.15.49:0/2029082120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c40109050 msgr2=0x7f9c401a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c37e00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c40109050 0x7f9c401a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c37e00640 1 -- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c40073be0 msgr2=0x7f9c401994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c37e00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c40073be0 0x7f9c401994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c37e00640 1 -- 172.21.15.49:0/2029082120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c30009560 con 0x7f9c400751c0 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c37e00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 0x7f9c40199a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800e9f0 tx=0x7f9c2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c3d600640 1 -- 172.21.15.49:0/2029082120 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800ccc0 con 0x7f9c400751c0 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c3d600640 1 -- 172.21.15.49:0/2029082120 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c2800ce20 con 0x7f9c400751c0 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.699+0000 7f9c3d600640 1 -- 172.21.15.49:0/2029082120 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28018920 con 0x7f9c400751c0 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.700+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c401a1580 con 0x7f9c400751c0 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.700+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c401a1a90 con 0x7f9c400751c0 2024-02-26T06:23:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.701+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c40107d80 con 0x7f9c400751c0 2024-02-26T06:23:07.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.703+0000 7f9c3d600640 1 -- 172.21.15.49:0/2029082120 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9c28018a80 con 0x7f9c400751c0 2024-02-26T06:23:07.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.705+0000 7f9c3d600640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c240a20c0 0x7f9c240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.705+0000 7f9c3d600640 1 -- 172.21.15.49:0/2029082120 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9c28014070 con 0x7f9c400751c0 2024-02-26T06:23:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.705+0000 7f9c3fe00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c240a20c0 0x7f9c240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:07.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.706+0000 7f9c3fe00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c240a20c0 0x7f9c240a4580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9c30000c00 tx=0x7f9c3002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:07.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.708+0000 7f9c3d600640 1 -- 172.21.15.49:0/2029082120 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c28052d20 con 0x7f9c400751c0 2024-02-26T06:23:07.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.916+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c4002ce40 con 0x7f9c400751c0 2024-02-26T06:23:07.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.917+0000 7f9c3d600640 1 -- 172.21.15.49:0/2029082120 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c2809aac0 con 0x7f9c400751c0 2024-02-26T06:23:07.917 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:23:07.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.920+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c240a20c0 msgr2=0x7f9c240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.920+0000 7f9c46c00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c240a20c0 0x7f9c240a4580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9c30000c00 tx=0x7f9c3002c7e0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.920+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 msgr2=0x7f9c40199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.920+0000 7f9c46c00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 0x7f9c40199a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800e9f0 tx=0x7f9c2800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.921+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 shutdown_connections 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.921+0000 7f9c46c00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f9c240a20c0 0x7f9c240a4580 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.921+0000 7f9c46c00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c400751c0 0x7f9c40199a10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.921+0000 7f9c46c00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c40109050 0x7f9c401a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.921+0000 7f9c46c00640 1 --2- 172.21.15.49:0/2029082120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c40073be0 0x7f9c401994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.921+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 >> 172.21.15.49:0/2029082120 conn(0x7f9c400fc700 msgr2=0x7f9c40104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.921+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 shutdown_connections 2024-02-26T06:23:07.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:07.921+0000 7f9c46c00640 1 -- 172.21.15.49:0/2029082120 wait complete. 2024-02-26T06:23:07.939 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:23:08.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.165+0000 7fd2a4000640 1 -- 172.21.15.49:0/59358797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 msgr2=0x7fd29c101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.165+0000 7fd2a4000640 1 --2- 172.21.15.49:0/59358797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 0x7fd29c101700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd28802bc40 comp rx=0 tx=0).stop 2024-02-26T06:23:08.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.166+0000 7fd2a4000640 1 -- 172.21.15.49:0/59358797 shutdown_connections 2024-02-26T06:23:08.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.166+0000 7fd2a4000640 1 --2- 172.21.15.49:0/59358797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29c101c40 0x7fd29c1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.166+0000 7fd2a4000640 1 --2- 172.21.15.49:0/59358797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 0x7fd29c101700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.166+0000 7fd2a4000640 1 --2- 172.21.15.49:0/59358797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd29c100090 0x7fd29c1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.166+0000 7fd2a4000640 1 -- 172.21.15.49:0/59358797 >> 172.21.15.49:0/59358797 conn(0x7fd29c0fb620 msgr2=0x7fd29c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:08.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.166+0000 7fd2a4000640 1 -- 172.21.15.49:0/59358797 shutdown_connections 2024-02-26T06:23:08.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.166+0000 7fd2a4000640 1 -- 172.21.15.49:0/59358797 wait complete. 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.167+0000 7fd2a4000640 1 Processor -- start 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.167+0000 7fd2a4000640 1 -- start start 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.167+0000 7fd2a4000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd29c100090 0x7fd29c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.167+0000 7fd2a4000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 0x7fd29c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.167+0000 7fd2a4000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29c101c40 0x7fd29c1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.167+0000 7fd2a4000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1a0f20 con 0x7fd29c101280 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.167+0000 7fd2a4000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1a1090 con 0x7fd29c100090 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.167+0000 7fd2a4000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1a1250 con 0x7fd29c101c40 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd29c100090 0x7fd29c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a0c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 0x7fd29c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd29c100090 0x7fd29c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35622/0 (socket says 172.21.15.49:35622) 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a1600640 1 -- 172.21.15.49:0/1352836473 learned_addr learned my addr 172.21.15.49:0/1352836473 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a2000640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29c101c40 0x7fd29c1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:08.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a1600640 1 -- 172.21.15.49:0/1352836473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29c101c40 msgr2=0x7fd29c1a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a1600640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29c101c40 0x7fd29c1a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a1600640 1 -- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 msgr2=0x7fd29c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a1600640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 0x7fd29c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a1600640 1 -- 172.21.15.49:0/1352836473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288009560 con 0x7fd29c100090 2024-02-26T06:23:08.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.168+0000 7fd2a0c00640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 0x7fd29c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:08.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.169+0000 7fd2a1600640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd29c100090 0x7fd29c199430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd29000e9c0 tx=0x7fd29000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:07 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3611982169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:07 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:07 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:07 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3690128996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.169+0000 7fd29a000640 1 -- 172.21.15.49:0/1352836473 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290009700 con 0x7fd29c100090 2024-02-26T06:23:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.169+0000 7fd29a000640 1 -- 172.21.15.49:0/1352836473 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd29000cd80 con 0x7fd29c100090 2024-02-26T06:23:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.169+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c1a1530 con 0x7fd29c100090 2024-02-26T06:23:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.169+0000 7fd29a000640 1 -- 172.21.15.49:0/1352836473 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290010430 con 0x7fd29c100090 2024-02-26T06:23:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.170+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c191330 con 0x7fd29c100090 2024-02-26T06:23:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.171+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c10baf0 con 0x7fd29c100090 2024-02-26T06:23:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.172+0000 7fd29a000640 1 -- 172.21.15.49:0/1352836473 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd290003680 con 0x7fd29c100090 2024-02-26T06:23:08.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.174+0000 7fd29a000640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2840a2210 0x7fd2840a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:08.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.174+0000 7fd29a000640 1 -- 172.21.15.49:0/1352836473 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd290014070 con 0x7fd29c100090 2024-02-26T06:23:08.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.174+0000 7fd2a0c00640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2840a2210 0x7fd2840a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:08.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.176+0000 7fd2a0c00640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2840a2210 0x7fd2840a46d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd288000c00 tx=0x7fd28802c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:08.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.179+0000 7fd29a000640 1 -- 172.21.15.49:0/1352836473 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd2900d1030 con 0x7fd29c100090 2024-02-26T06:23:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:07 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3611982169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:07 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:07 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:07 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3690128996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:07 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3611982169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:07 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:07 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:07 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3690128996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:08.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.399+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd29c10bca0 con 0x7fd29c100090 2024-02-26T06:23:08.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.400+0000 7fd29a000640 1 -- 172.21.15.49:0/1352836473 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd290096f30 con 0x7fd29c100090 2024-02-26T06:23:08.399 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2840a2210 msgr2=0x7fd2840a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2840a2210 0x7fd2840a46d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd288000c00 tx=0x7fd28802c7f0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd29c100090 msgr2=0x7fd29c199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd29c100090 0x7fd29c199430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd29000e9c0 tx=0x7fd29000ee80 comp rx=0 tx=0).stop 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 shutdown_connections 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fd2840a2210 0x7fd2840a46d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29c101c40 0x7fd29c1a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29c101280 0x7fd29c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 --2- 172.21.15.49:0/1352836473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd29c100090 0x7fd29c199430 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 >> 172.21.15.49:0/1352836473 conn(0x7fd29c0fb620 msgr2=0x7fd29c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 shutdown_connections 2024-02-26T06:23:08.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.403+0000 7fd2a4000640 1 -- 172.21.15.49:0/1352836473 wait complete. 2024-02-26T06:23:08.421 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-26T06:23:08.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 -- 172.21.15.49:0/2574694692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 msgr2=0x7f963c103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 --2- 172.21.15.49:0/2574694692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 0x7f963c103e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f962c009900 tx=0x7f962c02bc50 comp rx=0 tx=0).stop 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 -- 172.21.15.49:0/2574694692 shutdown_connections 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 --2- 172.21.15.49:0/2574694692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f963c106ca0 0x7f963c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 --2- 172.21.15.49:0/2574694692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f963c104370 0x7f963c106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 --2- 172.21.15.49:0/2574694692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 0x7f963c103e30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 -- 172.21.15.49:0/2574694692 >> 172.21.15.49:0/2574694692 conn(0x7f963c0fb430 msgr2=0x7f963c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 -- 172.21.15.49:0/2574694692 shutdown_connections 2024-02-26T06:23:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.615+0000 7f963b200640 1 -- 172.21.15.49:0/2574694692 wait complete. 2024-02-26T06:23:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f963b200640 1 Processor -- start 2024-02-26T06:23:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f963b200640 1 -- start start 2024-02-26T06:23:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f963b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 0x7f963c10d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f963b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f963c104370 0x7f963c10ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f963b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f963c106ca0 0x7f963c10a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f963b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c10ab90 con 0x7f963c101a40 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f963b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c10ad00 con 0x7f963c104370 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f963b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c10aec0 con 0x7f963c106ca0 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.616+0000 7f9639e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 0x7f963c10d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 0x7f963c10d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38114/0 (socket says 172.21.15.49:38114) 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639e00640 1 -- 172.21.15.49:0/2673847670 learned_addr learned my addr 172.21.15.49:0/2673847670 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639400640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f963c104370 0x7f963c10ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f963a800640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f963c106ca0 0x7f963c10a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639e00640 1 -- 172.21.15.49:0/2673847670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f963c106ca0 msgr2=0x7f963c10a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639e00640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f963c106ca0 0x7f963c10a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639e00640 1 -- 172.21.15.49:0/2673847670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f963c104370 msgr2=0x7f963c10ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639e00640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f963c104370 0x7f963c10ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639e00640 1 -- 172.21.15.49:0/2673847670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9628009490 con 0x7f963c101a40 2024-02-26T06:23:08.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639400640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f963c104370 0x7f963c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:23:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.617+0000 7f9639e00640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 0x7f963c10d890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f962c02c6c0 tx=0x7f962c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.618+0000 7f9632a00640 1 -- 172.21.15.49:0/2673847670 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c0044c0 con 0x7f963c101a40 2024-02-26T06:23:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.618+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c009560 con 0x7f963c101a40 2024-02-26T06:23:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.618+0000 7f9632a00640 1 -- 172.21.15.49:0/2673847670 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f962c0052d0 con 0x7f963c101a40 2024-02-26T06:23:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.618+0000 7f9632a00640 1 -- 172.21.15.49:0/2673847670 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c003e80 con 0x7f963c101a40 2024-02-26T06:23:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.618+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c1a5b70 con 0x7f963c101a40 2024-02-26T06:23:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.619+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c0fcf20 con 0x7f963c101a40 2024-02-26T06:23:08.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.622+0000 7f9632a00640 1 -- 172.21.15.49:0/2673847670 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f962c005440 con 0x7f963c101a40 2024-02-26T06:23:08.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.623+0000 7f9632a00640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96200a2140 0x7f96200a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:08.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.623+0000 7f9632a00640 1 -- 172.21.15.49:0/2673847670 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f962c0f0a00 con 0x7f963c101a40 2024-02-26T06:23:08.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.624+0000 7f9639400640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96200a2140 0x7f96200a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:08.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.624+0000 7f9639400640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96200a2140 0x7f96200a4600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f963c10b810 tx=0x7f9628005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.627+0000 7f9632a00640 1 -- 172.21.15.49:0/2673847670 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f962c0b99c0 con 0x7f963c101a40 2024-02-26T06:23:08.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.870+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f963c10ec50 con 0x7f963c101a40 2024-02-26T06:23:08.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.872+0000 7f9632a00640 1 -- 172.21.15.49:0/2673847670 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11331 (secure 0 0 0) 0x7f962c0bd080 con 0x7f963c101a40 2024-02-26T06:23:08.872 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:08.872 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:23:03.712829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:23:08.873 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-02-26T06:23:08.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96200a2140 msgr2=0x7f96200a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96200a2140 0x7f96200a4600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f963c10b810 tx=0x7f9628005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 msgr2=0x7f963c10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 0x7f963c10d890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f962c02c6c0 tx=0x7f962c0040c0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 shutdown_connections 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f96200a2140 0x7f96200a4600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f963c104370 0x7f963c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f963c106ca0 0x7f963c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 --2- 172.21.15.49:0/2673847670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f963c101a40 0x7f963c10d890 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 >> 172.21.15.49:0/2673847670 conn(0x7f963c0fb430 msgr2=0x7f963c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:08.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.875+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 shutdown_connections 2024-02-26T06:23:08.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:08.876+0000 7f963b200640 1 -- 172.21.15.49:0/2673847670 wait complete. 2024-02-26T06:23:08.894 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-26T06:23:08.895 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-26T06:23:08.895 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:23:09.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 -- 172.21.15.49:0/737352487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 msgr2=0x7fdfcc0ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/737352487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 0x7fdfcc0ffc60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009ca0 tx=0x7fdfbc02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 -- 172.21.15.49:0/737352487 shutdown_connections 2024-02-26T06:23:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/737352487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfcc1001a0 0x7fdfcc100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/737352487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfcc100b40 0x7fdfcc100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/737352487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 0x7fdfcc0ffc60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 -- 172.21.15.49:0/737352487 >> 172.21.15.49:0/737352487 conn(0x7fdfcc0fb460 msgr2=0x7fdfcc0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 -- 172.21.15.49:0/737352487 shutdown_connections 2024-02-26T06:23:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.127+0000 7fdfd0a00640 1 -- 172.21.15.49:0/737352487 wait complete. 2024-02-26T06:23:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.129+0000 7fdfd0a00640 1 Processor -- start 2024-02-26T06:23:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.129+0000 7fdfd0a00640 1 -- start start 2024-02-26T06:23:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.129+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 0x7fdfcc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.129+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfcc1001a0 0x7fdfcc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfcc100b40 0x7fdfcc1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfd0a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc1a0d90 con 0x7fdfcc0ff840 2024-02-26T06:23:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfd0a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc1a0f00 con 0x7fdfcc1001a0 2024-02-26T06:23:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfd0a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc1a10c0 con 0x7fdfcc100b40 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 0x7fdfcc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 0x7fdfcc1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38132/0 (socket says 172.21.15.49:38132) 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcb400640 1 -- 172.21.15.49:0/2671744719 learned_addr learned my addr 172.21.15.49:0/2671744719 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcbe00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfcc100b40 0x7fdfcc1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfc2a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfcc1001a0 0x7fdfcc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcb400640 1 -- 172.21.15.49:0/2671744719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfcc100b40 msgr2=0x7fdfcc1a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcb400640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfcc100b40 0x7fdfcc1a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcb400640 1 -- 172.21.15.49:0/2671744719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfcc1001a0 msgr2=0x7fdfcc1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcb400640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfcc1001a0 0x7fdfcc1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.130+0000 7fdfcb400640 1 -- 172.21.15.49:0/2671744719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc009900 con 0x7fdfcc0ff840 2024-02-26T06:23:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.131+0000 7fdfcb400640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 0x7fdfcc1992a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc032710 tx=0x7fdfbc034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:09.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.131+0000 7fdfc8c00640 1 -- 172.21.15.49:0/2671744719 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc003970 con 0x7fdfcc0ff840 2024-02-26T06:23:09.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.131+0000 7fdfc8c00640 1 -- 172.21.15.49:0/2671744719 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc004040 con 0x7fdfcc0ff840 2024-02-26T06:23:09.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.131+0000 7fdfc8c00640 1 -- 172.21.15.49:0/2671744719 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0406b0 con 0x7fdfcc0ff840 2024-02-26T06:23:09.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.131+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc1a1340 con 0x7fdfcc0ff840 2024-02-26T06:23:09.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.131+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc1a1770 con 0x7fdfcc0ff840 2024-02-26T06:23:09.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.132+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc10a590 con 0x7fdfcc0ff840 2024-02-26T06:23:09.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.134+0000 7fdfc8c00640 1 -- 172.21.15.49:0/2671744719 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdfbc00a040 con 0x7fdfcc0ff840 2024-02-26T06:23:09.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.136+0000 7fdfc8c00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdfb00a2140 0x7fdfb00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:09.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.136+0000 7fdfc8c00640 1 -- 172.21.15.49:0/2671744719 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fdfbc0f08d0 con 0x7fdfcc0ff840 2024-02-26T06:23:09.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.136+0000 7fdfc2a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdfb00a2140 0x7fdfb00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:09.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.137+0000 7fdfc2a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdfb00a2140 0x7fdfb00a4600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc19a7b0 tx=0x7fdfb400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:09.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.139+0000 7fdfc8c00640 1 -- 172.21.15.49:0/2671744719 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdfbc0b9830 con 0x7fdfcc0ff840 2024-02-26T06:23:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:08 smithi049 ceph-mon[31696]: pgmap v398: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-26T06:23:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:08 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2029082120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:08 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1352836473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:08 smithi049 ceph-mon[31696]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:08 smithi080 ceph-mon[36206]: pgmap v398: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-26T06:23:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:08 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2029082120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:08 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:08 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:08 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1352836473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:08 smithi080 ceph-mon[36206]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.339+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfcc02ceb0 con 0x7fdfb00a2140 2024-02-26T06:23:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:08 smithi115 ceph-mon[36426]: pgmap v398: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-26T06:23:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:08 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2029082120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:08 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:08 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:08 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1352836473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:08 smithi115 ceph-mon[36426]: from='mgr.14152 172.21.15.49:0/3790952713' entity='mgr.x' 2024-02-26T06:23:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.375+0000 7fdfc8c00640 1 -- 172.21.15.49:0/2671744719 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+342838 (secure 0 0 0) 0x7fdfcc02ceb0 con 0x7fdfb00a2140 2024-02-26T06:23:09.378 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-02-26T06:23:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.380+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdfb00a2140 msgr2=0x7fdfb00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.380+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdfb00a2140 0x7fdfb00a4600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc19a7b0 tx=0x7fdfb400b410 comp rx=0 tx=0).stop 2024-02-26T06:23:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.380+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 msgr2=0x7fdfcc1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.380+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 0x7fdfcc1992a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc032710 tx=0x7fdfbc034cc0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.381+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 shutdown_connections 2024-02-26T06:23:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.381+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdfb00a2140 0x7fdfb00a4600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.381+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfcc1001a0 0x7fdfcc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.381+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfcc100b40 0x7fdfcc1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.381+0000 7fdfd0a00640 1 --2- 172.21.15.49:0/2671744719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfcc0ff840 0x7fdfcc1992a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.381+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 >> 172.21.15.49:0/2671744719 conn(0x7fdfcc0fb460 msgr2=0x7fdfcc0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.381+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 shutdown_connections 2024-02-26T06:23:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.381+0000 7fdfd0a00640 1 -- 172.21.15.49:0/2671744719 wait complete. 2024-02-26T06:23:09.401 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-02-26T06:23:07.485979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661859,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1052,"num_read_kb":650,"num_write":354,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1241721,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360780,"kb_used_data":11628,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581044,"statfs":{"total":1151940427776,"available":1151570989056,"internally_reserved":0,"allocated":11907072,"data_stored":7485624,"data_compressed":191495,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5993,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":48,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004101"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440192+0000","last_change":"2024-02-26T06:20:31.603060+0000","last_active":"2024-02-26T06:21:57.440192+0000","last_peered":"2024-02-26T06:21:57.440192+0000","last_clean":"2024-02-26T06:21:57.440192+0000","last_became_active":"2024-02-26T06:20:31.602762+0000","last_became_peered":"2024-02-26T06:20:31.602762+0000","last_unstale":"2024-02-26T06:21:57.440192+0000","last_undegraded":"2024-02-26T06:21:57.440192+0000","last_fullsized":"2024-02-26T06:21:57.440192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:19:42.465510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.437717+0000","last_change":"2024-02-26T06:20:31.595138+0000","last_active":"2024-02-26T06:21:57.437717+0000","last_peered":"2024-02-26T06:21:57.437717+0000","last_clean":"2024-02-26T06:21:57.437717+0000","last_became_active":"2024-02-26T06:20:31.594826+0000","last_became_peered":"2024-02-26T06:20:31.594826+0000","last_unstale":"2024-02-26T06:21:57.437717+0000","last_undegraded":"2024-02-26T06:21:57.437717+0000","last_fullsized":"2024-02-26T06:21:57.437717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:49.900464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.431160+0000","last_change":"2024-02-26T06:20:31.600415+0000","last_active":"2024-02-26T06:21:57.431160+0000","last_peered":"2024-02-26T06:21:57.431160+0000","last_clean":"2024-02-26T06:21:57.431160+0000","last_became_active":"2024-02-26T06:20:31.596273+0000","last_became_peered":"2024-02-26T06:20:31.596273+0000","last_unstale":"2024-02-26T06:21:57.431160+0000","last_undegraded":"2024-02-26T06:21:57.431160+0000","last_fullsized":"2024-02-26T06:21:57.431160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:15:20.626515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410929+0000","last_change":"2024-02-26T06:20:31.581149+0000","last_active":"2024-02-26T06:21:57.410929+0000","last_peered":"2024-02-26T06:21:57.410929+0000","last_clean":"2024-02-26T06:21:57.410929+0000","last_became_active":"2024-02-26T06:20:31.580831+0000","last_became_peered":"2024-02-26T06:20:31.580831+0000","last_unstale":"2024-02-26T06:21:57.410929+0000","last_undegraded":"2024-02-26T06:21:57.410929+0000","last_fullsized":"2024-02-26T06:21:57.410929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:47:12.466502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412577+0000","last_change":"2024-02-26T06:20:31.608618+0000","last_active":"2024-02-26T06:21:57.412577+0000","last_peered":"2024-02-26T06:21:57.412577+0000","last_clean":"2024-02-26T06:21:57.412577+0000","last_became_active":"2024-02-26T06:20:31.608285+0000","last_became_peered":"2024-02-26T06:20:31.608285+0000","last_unstale":"2024-02-26T06:21:57.412577+0000","last_undegraded":"2024-02-26T06:21:57.412577+0000","last_fullsized":"2024-02-26T06:21:57.412577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:34:48.400836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.418353+0000","last_change":"2024-02-26T06:21:28.905073+0000","last_active":"2024-02-26T06:21:57.418353+0000","last_peered":"2024-02-26T06:21:57.418353+0000","last_clean":"2024-02-26T06:21:57.418353+0000","last_became_active":"2024-02-26T06:21:28.904630+0000","last_became_peered":"2024-02-26T06:21:28.904630+0000","last_unstale":"2024-02-26T06:21:57.418353+0000","last_undegraded":"2024-02-26T06:21:57.418353+0000","last_fullsized":"2024-02-26T06:21:57.418353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:04.531384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.407619+0000","last_change":"2024-02-26T06:20:31.587285+0000","last_active":"2024-02-26T06:21:57.407619+0000","last_peered":"2024-02-26T06:21:57.407619+0000","last_clean":"2024-02-26T06:21:57.407619+0000","last_became_active":"2024-02-26T06:20:31.586985+0000","last_became_peered":"2024-02-26T06:20:31.586985+0000","last_unstale":"2024-02-26T06:21:57.407619+0000","last_undegraded":"2024-02-26T06:21:57.407619+0000","last_fullsized":"2024-02-26T06:21:57.407619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:55:56.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438247+0000","last_change":"2024-02-26T06:20:31.591136+0000","last_active":"2024-02-26T06:21:57.438247+0000","last_peered":"2024-02-26T06:21:57.438247+0000","last_clean":"2024-02-26T06:21:57.438247+0000","last_became_active":"2024-02-26T06:20:31.590180+0000","last_became_peered":"2024-02-26T06:20:31.590180+0000","last_unstale":"2024-02-26T06:21:57.438247+0000","last_undegraded":"2024-02-26T06:21:57.438247+0000","last_fullsized":"2024-02-26T06:21:57.438247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:02.649407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420898+0000","last_change":"2024-02-26T06:20:31.583890+0000","last_active":"2024-02-26T06:21:57.420898+0000","last_peered":"2024-02-26T06:21:57.420898+0000","last_clean":"2024-02-26T06:21:57.420898+0000","last_became_active":"2024-02-26T06:20:31.583459+0000","last_became_peered":"2024-02-26T06:20:31.583459+0000","last_unstale":"2024-02-26T06:21:57.420898+0000","last_undegraded":"2024-02-26T06:21:57.420898+0000","last_fullsized":"2024-02-26T06:21:57.420898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:14.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412793+0000","last_change":"2024-02-26T06:20:31.579041+0000","last_active":"2024-02-26T06:21:57.412793+0000","last_peered":"2024-02-26T06:21:57.412793+0000","last_clean":"2024-02-26T06:21:57.412793+0000","last_became_active":"2024-02-26T06:20:31.578725+0000","last_became_peered":"2024-02-26T06:20:31.578725+0000","last_unstale":"2024-02-26T06:21:57.412793+0000","last_undegraded":"2024-02-26T06:21:57.412793+0000","last_fullsized":"2024-02-26T06:21:57.412793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:21.083206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420613+0000","last_change":"2024-02-26T06:20:31.581146+0000","last_active":"2024-02-26T06:21:57.420613+0000","last_peered":"2024-02-26T06:21:57.420613+0000","last_clean":"2024-02-26T06:21:57.420613+0000","last_became_active":"2024-02-26T06:20:31.580774+0000","last_became_peered":"2024-02-26T06:20:31.580774+0000","last_unstale":"2024-02-26T06:21:57.420613+0000","last_undegraded":"2024-02-26T06:21:57.420613+0000","last_fullsized":"2024-02-26T06:21:57.420613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:58:36.851504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410598+0000","last_change":"2024-02-26T06:20:31.582232+0000","last_active":"2024-02-26T06:21:57.410598+0000","last_peered":"2024-02-26T06:21:57.410598+0000","last_clean":"2024-02-26T06:21:57.410598+0000","last_became_active":"2024-02-26T06:20:31.581805+0000","last_became_peered":"2024-02-26T06:20:31.581805+0000","last_unstale":"2024-02-26T06:21:57.410598+0000","last_undegraded":"2024-02-26T06:21:57.410598+0000","last_fullsized":"2024-02-26T06:21:57.410598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:04.513814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428668+0000","last_change":"2024-02-26T06:20:31.606373+0000","last_active":"2024-02-26T06:21:57.428668+0000","last_peered":"2024-02-26T06:21:57.428668+0000","last_clean":"2024-02-26T06:21:57.428668+0000","last_became_active":"2024-02-26T06:20:31.603226+0000","last_became_peered":"2024-02-26T06:20:31.603226+0000","last_unstale":"2024-02-26T06:21:57.428668+0000","last_undegraded":"2024-02-26T06:21:57.428668+0000","last_fullsized":"2024-02-26T06:21:57.428668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:29:49.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.426327+0000","last_change":"2024-02-26T06:20:31.594587+0000","last_active":"2024-02-26T06:21:57.426327+0000","last_peered":"2024-02-26T06:21:57.426327+0000","last_clean":"2024-02-26T06:21:57.426327+0000","last_became_active":"2024-02-26T06:20:31.594032+0000","last_became_peered":"2024-02-26T06:20:31.594032+0000","last_unstale":"2024-02-26T06:21:57.426327+0000","last_undegraded":"2024-02-26T06:21:57.426327+0000","last_fullsized":"2024-02-26T06:21:57.426327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:49:18.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438329+0000","last_change":"2024-02-26T06:20:31.593115+0000","last_active":"2024-02-26T06:21:57.438329+0000","last_peered":"2024-02-26T06:21:57.438329+0000","last_clean":"2024-02-26T06:21:57.438329+0000","last_became_active":"2024-02-26T06:20:31.592782+0000","last_became_peered":"2024-02-26T06:20:31.592782+0000","last_unstale":"2024-02-26T06:21:57.438329+0000","last_undegraded":"2024-02-26T06:21:57.438329+0000","last_fullsized":"2024-02-26T06:21:57.438329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:10.709124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.427221+0000","last_change":"2024-02-26T06:20:31.597938+0000","last_active":"2024-02-26T06:21:57.427221+0000","last_peered":"2024-02-26T06:21:57.427221+0000","last_clean":"2024-02-26T06:21:57.427221+0000","last_became_active":"2024-02-26T06:20:31.597600+0000","last_became_peered":"2024-02-26T06:20:31.597600+0000","last_unstale":"2024-02-26T06:21:57.427221+0000","last_undegraded":"2024-02-26T06:21:57.427221+0000","last_fullsized":"2024-02-26T06:21:57.427221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:13.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440351+0000","last_change":"2024-02-26T06:20:31.603283+0000","last_active":"2024-02-26T06:21:57.440351+0000","last_peered":"2024-02-26T06:21:57.440351+0000","last_clean":"2024-02-26T06:21:57.440351+0000","last_became_active":"2024-02-26T06:20:31.602912+0000","last_became_peered":"2024-02-26T06:20:31.602912+0000","last_unstale":"2024-02-26T06:21:57.440351+0000","last_undegraded":"2024-02-26T06:21:57.440351+0000","last_fullsized":"2024-02-26T06:21:57.440351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:32.250914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.422411+0000","last_change":"2024-02-26T06:20:31.588670+0000","last_active":"2024-02-26T06:21:57.422411+0000","last_peered":"2024-02-26T06:21:57.422411+0000","last_clean":"2024-02-26T06:21:57.422411+0000","last_became_active":"2024-02-26T06:20:31.587971+0000","last_became_peered":"2024-02-26T06:20:31.587971+0000","last_unstale":"2024-02-26T06:21:57.422411+0000","last_undegraded":"2024-02-26T06:21:57.422411+0000","last_fullsized":"2024-02-26T06:21:57.422411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:13.053456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.425204+0000","last_change":"2024-02-26T06:20:31.604266+0000","last_active":"2024-02-26T06:21:57.425204+0000","last_peered":"2024-02-26T06:21:57.425204+0000","last_clean":"2024-02-26T06:21:57.425204+0000","last_became_active":"2024-02-26T06:20:31.604027+0000","last_became_peered":"2024-02-26T06:20:31.604027+0000","last_unstale":"2024-02-26T06:21:57.425204+0000","last_undegraded":"2024-02-26T06:21:57.425204+0000","last_fullsized":"2024-02-26T06:21:57.425204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:13.345138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904510+0000","last_change":"2024-02-26T06:20:31.068090+0000","last_active":"2024-02-26T06:21:28.904510+0000","last_peered":"2024-02-26T06:21:28.904510+0000","last_clean":"2024-02-26T06:21:28.904510+0000","last_became_active":"2024-02-26T06:20:31.067731+0000","last_became_peered":"2024-02-26T06:20:31.067731+0000","last_unstale":"2024-02-26T06:21:28.904510+0000","last_undegraded":"2024-02-26T06:21:28.904510+0000","last_fullsized":"2024-02-26T06:21:28.904510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:24.734040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413823+0000","last_change":"2024-02-26T06:21:28.915741+0000","last_active":"2024-02-26T06:21:57.413823+0000","last_peered":"2024-02-26T06:21:57.413823+0000","last_clean":"2024-02-26T06:21:57.413823+0000","last_became_active":"2024-02-26T06:21:28.915443+0000","last_became_peered":"2024-02-26T06:21:28.915443+0000","last_unstale":"2024-02-26T06:21:57.413823+0000","last_undegraded":"2024-02-26T06:21:57.413823+0000","last_fullsized":"2024-02-26T06:21:57.413823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:51.216291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898901+0000","last_change":"2024-02-26T06:20:31.083236+0000","last_active":"2024-02-26T06:21:28.898901+0000","last_peered":"2024-02-26T06:21:28.898901+0000","last_clean":"2024-02-26T06:21:28.898901+0000","last_became_active":"2024-02-26T06:20:31.081640+0000","last_became_peered":"2024-02-26T06:20:31.081640+0000","last_unstale":"2024-02-26T06:21:28.898901+0000","last_undegraded":"2024-02-26T06:21:28.898901+0000","last_fullsized":"2024-02-26T06:21:28.898901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:44.607108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439959+0000","last_change":"2024-02-26T06:20:31.593807+0000","last_active":"2024-02-26T06:21:57.439959+0000","last_peered":"2024-02-26T06:21:57.439959+0000","last_clean":"2024-02-26T06:21:57.439959+0000","last_became_active":"2024-02-26T06:20:31.593435+0000","last_became_peered":"2024-02-26T06:20:31.593435+0000","last_unstale":"2024-02-26T06:21:57.439959+0000","last_undegraded":"2024-02-26T06:21:57.439959+0000","last_fullsized":"2024-02-26T06:21:57.439959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:43.193196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.908174+0000","last_change":"2024-02-26T06:20:31.061339+0000","last_active":"2024-02-26T06:21:28.908174+0000","last_peered":"2024-02-26T06:21:28.908174+0000","last_clean":"2024-02-26T06:21:28.908174+0000","last_became_active":"2024-02-26T06:20:31.060769+0000","last_became_peered":"2024-02-26T06:20:31.060769+0000","last_unstale":"2024-02-26T06:21:28.908174+0000","last_undegraded":"2024-02-26T06:21:28.908174+0000","last_fullsized":"2024-02-26T06:21:28.908174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:28:30.501375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432762+0000","last_change":"2024-02-26T06:20:31.606936+0000","last_active":"2024-02-26T06:21:57.432762+0000","last_peered":"2024-02-26T06:21:57.432762+0000","last_clean":"2024-02-26T06:21:57.432762+0000","last_became_active":"2024-02-26T06:20:31.606582+0000","last_became_peered":"2024-02-26T06:20:31.606582+0000","last_unstale":"2024-02-26T06:21:57.432762+0000","last_undegraded":"2024-02-26T06:21:57.432762+0000","last_fullsized":"2024-02-26T06:21:57.432762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:09.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907169+0000","last_change":"2024-02-26T06:20:31.063472+0000","last_active":"2024-02-26T06:21:28.907169+0000","last_peered":"2024-02-26T06:21:28.907169+0000","last_clean":"2024-02-26T06:21:28.907169+0000","last_became_active":"2024-02-26T06:20:31.063132+0000","last_became_peered":"2024-02-26T06:20:31.063132+0000","last_unstale":"2024-02-26T06:21:28.907169+0000","last_undegraded":"2024-02-26T06:21:28.907169+0000","last_fullsized":"2024-02-26T06:21:28.907169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:58.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.418097+0000","last_change":"2024-02-26T06:20:31.589238+0000","last_active":"2024-02-26T06:21:57.418097+0000","last_peered":"2024-02-26T06:21:57.418097+0000","last_clean":"2024-02-26T06:21:57.418097+0000","last_became_active":"2024-02-26T06:20:31.588929+0000","last_became_peered":"2024-02-26T06:20:31.588929+0000","last_unstale":"2024-02-26T06:21:57.418097+0000","last_undegraded":"2024-02-26T06:21:57.418097+0000","last_fullsized":"2024-02-26T06:21:57.418097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:19.282943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":179,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:06.333402+0000","last_change":"2024-02-26T06:20:31.061714+0000","last_active":"2024-02-26T06:23:06.333402+0000","last_peered":"2024-02-26T06:23:06.333402+0000","last_clean":"2024-02-26T06:23:06.333402+0000","last_became_active":"2024-02-26T06:20:31.061234+0000","last_became_peered":"2024-02-26T06:20:31.061234+0000","last_unstale":"2024-02-26T06:23:06.333402+0000","last_undegraded":"2024-02-26T06:23:06.333402+0000","last_fullsized":"2024-02-26T06:23:06.333402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:29:17.668327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.400272+0000","last_change":"2024-02-26T06:20:31.588861+0000","last_active":"2024-02-26T06:21:57.400272+0000","last_peered":"2024-02-26T06:21:57.400272+0000","last_clean":"2024-02-26T06:21:57.400272+0000","last_became_active":"2024-02-26T06:20:31.588530+0000","last_became_peered":"2024-02-26T06:20:31.588530+0000","last_unstale":"2024-02-26T06:21:57.400272+0000","last_undegraded":"2024-02-26T06:21:57.400272+0000","last_fullsized":"2024-02-26T06:21:57.400272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:12.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.900377+0000","last_change":"2024-02-26T06:20:31.077656+0000","last_active":"2024-02-26T06:21:28.900377+0000","last_peered":"2024-02-26T06:21:28.900377+0000","last_clean":"2024-02-26T06:21:28.900377+0000","last_became_active":"2024-02-26T06:20:31.077288+0000","last_became_peered":"2024-02-26T06:20:31.077288+0000","last_unstale":"2024-02-26T06:21:28.900377+0000","last_undegraded":"2024-02-26T06:21:28.900377+0000","last_fullsized":"2024-02-26T06:21:28.900377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:56:05.173751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440086+0000","last_change":"2024-02-26T06:20:31.608089+0000","last_active":"2024-02-26T06:21:57.440086+0000","last_peered":"2024-02-26T06:21:57.440086+0000","last_clean":"2024-02-26T06:21:57.440086+0000","last_became_active":"2024-02-26T06:20:31.607786+0000","last_became_peered":"2024-02-26T06:20:31.607786+0000","last_unstale":"2024-02-26T06:21:57.440086+0000","last_undegraded":"2024-02-26T06:21:57.440086+0000","last_fullsized":"2024-02-26T06:21:57.440086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:24.113286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.207050+0000","last_change":"2024-02-26T06:20:31.077063+0000","last_active":"2024-02-26T06:21:29.207050+0000","last_peered":"2024-02-26T06:21:29.207050+0000","last_clean":"2024-02-26T06:21:29.207050+0000","last_became_active":"2024-02-26T06:20:31.075147+0000","last_became_peered":"2024-02-26T06:20:31.075147+0000","last_unstale":"2024-02-26T06:21:29.207050+0000","last_undegraded":"2024-02-26T06:21:29.207050+0000","last_fullsized":"2024-02-26T06:21:29.207050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:07:13.434635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439933+0000","last_change":"2024-02-26T06:20:31.600135+0000","last_active":"2024-02-26T06:21:57.439933+0000","last_peered":"2024-02-26T06:21:57.439933+0000","last_clean":"2024-02-26T06:21:57.439933+0000","last_became_active":"2024-02-26T06:20:31.595622+0000","last_became_peered":"2024-02-26T06:20:31.595622+0000","last_unstale":"2024-02-26T06:21:57.439933+0000","last_undegraded":"2024-02-26T06:21:57.439933+0000","last_fullsized":"2024-02-26T06:21:57.439933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:55.835700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914398+0000","last_change":"2024-02-26T06:20:31.060299+0000","last_active":"2024-02-26T06:21:28.914398+0000","last_peered":"2024-02-26T06:21:28.914398+0000","last_clean":"2024-02-26T06:21:28.914398+0000","last_became_active":"2024-02-26T06:20:31.059938+0000","last_became_peered":"2024-02-26T06:20:31.059938+0000","last_unstale":"2024-02-26T06:21:28.914398+0000","last_undegraded":"2024-02-26T06:21:28.914398+0000","last_fullsized":"2024-02-26T06:21:28.914398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:24.144958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.444272+0000","last_change":"2024-02-26T06:20:31.601945+0000","last_active":"2024-02-26T06:21:57.444272+0000","last_peered":"2024-02-26T06:21:57.444272+0000","last_clean":"2024-02-26T06:21:57.444272+0000","last_became_active":"2024-02-26T06:20:31.601560+0000","last_became_peered":"2024-02-26T06:20:31.601560+0000","last_unstale":"2024-02-26T06:21:57.444272+0000","last_undegraded":"2024-02-26T06:21:57.444272+0000","last_fullsized":"2024-02-26T06:21:57.444272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:13.932623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":173,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:05.331162+0000","last_change":"2024-02-26T06:20:31.068334+0000","last_active":"2024-02-26T06:23:05.331162+0000","last_peered":"2024-02-26T06:23:05.331162+0000","last_clean":"2024-02-26T06:23:05.331162+0000","last_became_active":"2024-02-26T06:20:31.068020+0000","last_became_peered":"2024-02-26T06:20:31.068020+0000","last_unstale":"2024-02-26T06:23:05.331162+0000","last_undegraded":"2024-02-26T06:23:05.331162+0000","last_fullsized":"2024-02-26T06:23:05.331162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:23:01.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":149,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421658+0000","last_change":"2024-02-26T06:20:31.587862+0000","last_active":"2024-02-26T06:21:57.421658+0000","last_peered":"2024-02-26T06:21:57.421658+0000","last_clean":"2024-02-26T06:21:57.421658+0000","last_became_active":"2024-02-26T06:20:31.587551+0000","last_became_peered":"2024-02-26T06:20:31.587551+0000","last_unstale":"2024-02-26T06:21:57.421658+0000","last_undegraded":"2024-02-26T06:21:57.421658+0000","last_fullsized":"2024-02-26T06:21:57.421658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:07:15.933125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902570+0000","last_change":"2024-02-26T06:20:31.058378+0000","last_active":"2024-02-26T06:21:28.902570+0000","last_peered":"2024-02-26T06:21:28.902570+0000","last_clean":"2024-02-26T06:21:28.902570+0000","last_became_active":"2024-02-26T06:20:31.058040+0000","last_became_peered":"2024-02-26T06:20:31.058040+0000","last_unstale":"2024-02-26T06:21:28.902570+0000","last_undegraded":"2024-02-26T06:21:28.902570+0000","last_fullsized":"2024-02-26T06:21:28.902570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:33:40.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440318+0000","last_change":"2024-02-26T06:20:31.601837+0000","last_active":"2024-02-26T06:21:57.440318+0000","last_peered":"2024-02-26T06:21:57.440318+0000","last_clean":"2024-02-26T06:21:57.440318+0000","last_became_active":"2024-02-26T06:20:31.597310+0000","last_became_peered":"2024-02-26T06:20:31.597310+0000","last_unstale":"2024-02-26T06:21:57.440318+0000","last_undegraded":"2024-02-26T06:21:57.440318+0000","last_fullsized":"2024-02-26T06:21:57.440318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.383510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.915022+0000","last_change":"2024-02-26T06:20:31.076158+0000","last_active":"2024-02-26T06:21:28.915022+0000","last_peered":"2024-02-26T06:21:28.915022+0000","last_clean":"2024-02-26T06:21:28.915022+0000","last_became_active":"2024-02-26T06:20:31.075795+0000","last_became_peered":"2024-02-26T06:20:31.075795+0000","last_unstale":"2024-02-26T06:21:28.915022+0000","last_undegraded":"2024-02-26T06:21:28.915022+0000","last_fullsized":"2024-02-26T06:21:28.915022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:38.491674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421150+0000","last_change":"2024-02-26T06:20:31.586122+0000","last_active":"2024-02-26T06:21:57.421150+0000","last_peered":"2024-02-26T06:21:57.421150+0000","last_clean":"2024-02-26T06:21:57.421150+0000","last_became_active":"2024-02-26T06:20:31.585070+0000","last_became_peered":"2024-02-26T06:20:31.585070+0000","last_unstale":"2024-02-26T06:21:57.421150+0000","last_undegraded":"2024-02-26T06:21:57.421150+0000","last_fullsized":"2024-02-26T06:21:57.421150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:11.034252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":383,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:07.072497+0000","last_change":"2024-02-26T06:21:28.913681+0000","last_active":"2024-02-26T06:23:07.072497+0000","last_peered":"2024-02-26T06:23:07.072497+0000","last_clean":"2024-02-26T06:23:07.072497+0000","last_became_active":"2024-02-26T06:21:28.905006+0000","last_became_peered":"2024-02-26T06:21:28.905006+0000","last_unstale":"2024-02-26T06:23:07.072497+0000","last_undegraded":"2024-02-26T06:23:07.072497+0000","last_fullsized":"2024-02-26T06:23:07.072497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:41.432983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66805,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":308,"num_read_kb":219,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60873,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410911+0000","last_change":"2024-02-26T06:20:31.584757+0000","last_active":"2024-02-26T06:21:57.410911+0000","last_peered":"2024-02-26T06:21:57.410911+0000","last_clean":"2024-02-26T06:21:57.410911+0000","last_became_active":"2024-02-26T06:20:31.583890+0000","last_became_peered":"2024-02-26T06:20:31.583890+0000","last_unstale":"2024-02-26T06:21:57.410911+0000","last_undegraded":"2024-02-26T06:21:57.410911+0000","last_fullsized":"2024-02-26T06:21:57.410911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:10:58.280082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903296+0000","last_change":"2024-02-26T06:20:31.076044+0000","last_active":"2024-02-26T06:21:28.903296+0000","last_peered":"2024-02-26T06:21:28.903296+0000","last_clean":"2024-02-26T06:21:28.903296+0000","last_became_active":"2024-02-26T06:20:31.075744+0000","last_became_peered":"2024-02-26T06:20:31.075744+0000","last_unstale":"2024-02-26T06:21:28.903296+0000","last_undegraded":"2024-02-26T06:21:28.903296+0000","last_fullsized":"2024-02-26T06:21:28.903296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:35.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.443819+0000","last_change":"2024-02-26T06:20:31.602169+0000","last_active":"2024-02-26T06:21:57.443819+0000","last_peered":"2024-02-26T06:21:57.443819+0000","last_clean":"2024-02-26T06:21:57.443819+0000","last_became_active":"2024-02-26T06:20:31.601280+0000","last_became_peered":"2024-02-26T06:20:31.601280+0000","last_unstale":"2024-02-26T06:21:57.443819+0000","last_undegraded":"2024-02-26T06:21:57.443819+0000","last_fullsized":"2024-02-26T06:21:57.443819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:07.652549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914144+0000","last_change":"2024-02-26T06:20:31.080482+0000","last_active":"2024-02-26T06:21:28.914144+0000","last_peered":"2024-02-26T06:21:28.914144+0000","last_clean":"2024-02-26T06:21:28.914144+0000","last_became_active":"2024-02-26T06:20:31.080188+0000","last_became_peered":"2024-02-26T06:20:31.080188+0000","last_unstale":"2024-02-26T06:21:28.914144+0000","last_undegraded":"2024-02-26T06:21:28.914144+0000","last_fullsized":"2024-02-26T06:21:28.914144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:32.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.433538+0000","last_change":"2024-02-26T06:20:31.599753+0000","last_active":"2024-02-26T06:21:57.433538+0000","last_peered":"2024-02-26T06:21:57.433538+0000","last_clean":"2024-02-26T06:21:57.433538+0000","last_became_active":"2024-02-26T06:20:31.599409+0000","last_became_peered":"2024-02-26T06:20:31.599409+0000","last_unstale":"2024-02-26T06:21:57.433538+0000","last_undegraded":"2024-02-26T06:21:57.433538+0000","last_fullsized":"2024-02-26T06:21:57.433538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:43.887431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":175,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:02.411431+0000","last_change":"2024-02-26T06:20:31.059656+0000","last_active":"2024-02-26T06:23:02.411431+0000","last_peered":"2024-02-26T06:23:02.411431+0000","last_clean":"2024-02-26T06:23:02.411431+0000","last_became_active":"2024-02-26T06:20:31.059289+0000","last_became_peered":"2024-02-26T06:20:31.059289+0000","last_unstale":"2024-02-26T06:23:02.411431+0000","last_undegraded":"2024-02-26T06:23:02.411431+0000","last_fullsized":"2024-02-26T06:23:02.411431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:08:57.650876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":151,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.404146+0000","last_change":"2024-02-26T06:21:28.907778+0000","last_active":"2024-02-26T06:21:57.404146+0000","last_peered":"2024-02-26T06:21:57.404146+0000","last_clean":"2024-02-26T06:21:57.404146+0000","last_became_active":"2024-02-26T06:21:28.907185+0000","last_became_peered":"2024-02-26T06:21:28.907185+0000","last_unstale":"2024-02-26T06:21:57.404146+0000","last_undegraded":"2024-02-26T06:21:57.404146+0000","last_fullsized":"2024-02-26T06:21:57.404146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:39.572867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.943152+0000","last_change":"2024-02-26T06:21:28.943152+0000","last_active":"2024-02-26T06:21:28.943152+0000","last_peered":"2024-02-26T06:21:28.943152+0000","last_clean":"2024-02-26T06:21:28.943152+0000","last_became_active":"2024-02-26T06:21:28.908090+0000","last_became_peered":"2024-02-26T06:21:28.908090+0000","last_unstale":"2024-02-26T06:21:28.943152+0000","last_undegraded":"2024-02-26T06:21:28.943152+0000","last_fullsized":"2024-02-26T06:21:28.943152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:56.254015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.417526+0000","last_change":"2024-02-26T06:20:31.586151+0000","last_active":"2024-02-26T06:21:57.417526+0000","last_peered":"2024-02-26T06:21:57.417526+0000","last_clean":"2024-02-26T06:21:57.417526+0000","last_became_active":"2024-02-26T06:20:31.585615+0000","last_became_peered":"2024-02-26T06:20:31.585615+0000","last_unstale":"2024-02-26T06:21:57.417526+0000","last_undegraded":"2024-02-26T06:21:57.417526+0000","last_fullsized":"2024-02-26T06:21:57.417526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907609+0000","last_change":"2024-02-26T06:20:31.062340+0000","last_active":"2024-02-26T06:21:28.907609+0000","last_peered":"2024-02-26T06:21:28.907609+0000","last_clean":"2024-02-26T06:21:28.907609+0000","last_became_active":"2024-02-26T06:20:31.062023+0000","last_became_peered":"2024-02-26T06:20:31.062023+0000","last_unstale":"2024-02-26T06:21:28.907609+0000","last_undegraded":"2024-02-26T06:21:28.907609+0000","last_fullsized":"2024-02-26T06:21:28.907609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:45.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413574+0000","last_change":"2024-02-26T06:20:31.585190+0000","last_active":"2024-02-26T06:21:57.413574+0000","last_peered":"2024-02-26T06:21:57.413574+0000","last_clean":"2024-02-26T06:21:57.413574+0000","last_became_active":"2024-02-26T06:20:31.584883+0000","last_became_peered":"2024-02-26T06:20:31.584883+0000","last_unstale":"2024-02-26T06:21:57.413574+0000","last_undegraded":"2024-02-26T06:21:57.413574+0000","last_fullsized":"2024-02-26T06:21:57.413574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:47:51.843824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904061+0000","last_change":"2024-02-26T06:20:31.075573+0000","last_active":"2024-02-26T06:21:28.904061+0000","last_peered":"2024-02-26T06:21:28.904061+0000","last_clean":"2024-02-26T06:21:28.904061+0000","last_became_active":"2024-02-26T06:20:31.075153+0000","last_became_peered":"2024-02-26T06:20:31.075153+0000","last_unstale":"2024-02-26T06:21:28.904061+0000","last_undegraded":"2024-02-26T06:21:28.904061+0000","last_fullsized":"2024-02-26T06:21:28.904061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:43.886858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432306+0000","last_change":"2024-02-26T06:20:31.604400+0000","last_active":"2024-02-26T06:21:57.432306+0000","last_peered":"2024-02-26T06:21:57.432306+0000","last_clean":"2024-02-26T06:21:57.432306+0000","last_became_active":"2024-02-26T06:20:31.604068+0000","last_became_peered":"2024-02-26T06:20:31.604068+0000","last_unstale":"2024-02-26T06:21:57.432306+0000","last_undegraded":"2024-02-26T06:21:57.432306+0000","last_fullsized":"2024-02-26T06:21:57.432306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:28.792297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902928+0000","last_change":"2024-02-26T06:20:31.072896+0000","last_active":"2024-02-26T06:21:28.902928+0000","last_peered":"2024-02-26T06:21:28.902928+0000","last_clean":"2024-02-26T06:21:28.902928+0000","last_became_active":"2024-02-26T06:20:31.072571+0000","last_became_peered":"2024-02-26T06:20:31.072571+0000","last_unstale":"2024-02-26T06:21:28.902928+0000","last_undegraded":"2024-02-26T06:21:28.902928+0000","last_fullsized":"2024-02-26T06:21:28.902928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:39.054737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.415995+0000","last_change":"2024-02-26T06:21:28.906840+0000","last_active":"2024-02-26T06:21:57.415995+0000","last_peered":"2024-02-26T06:21:57.415995+0000","last_clean":"2024-02-26T06:21:57.415995+0000","last_became_active":"2024-02-26T06:21:28.905881+0000","last_became_peered":"2024-02-26T06:21:28.905881+0000","last_unstale":"2024-02-26T06:21:57.415995+0000","last_undegraded":"2024-02-26T06:21:57.415995+0000","last_fullsized":"2024-02-26T06:21:57.415995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:47.754145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.411939+0000","last_change":"2024-02-26T06:20:31.588989+0000","last_active":"2024-02-26T06:21:57.411939+0000","last_peered":"2024-02-26T06:21:57.411939+0000","last_clean":"2024-02-26T06:21:57.411939+0000","last_became_active":"2024-02-26T06:20:31.588683+0000","last_became_peered":"2024-02-26T06:20:31.588683+0000","last_unstale":"2024-02-26T06:21:57.411939+0000","last_undegraded":"2024-02-26T06:21:57.411939+0000","last_fullsized":"2024-02-26T06:21:57.411939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:29.687622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":170,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:01.408273+0000","last_change":"2024-02-26T06:20:31.081111+0000","last_active":"2024-02-26T06:23:01.408273+0000","last_peered":"2024-02-26T06:23:01.408273+0000","last_clean":"2024-02-26T06:23:01.408273+0000","last_became_active":"2024-02-26T06:20:31.080788+0000","last_became_peered":"2024-02-26T06:20:31.080788+0000","last_unstale":"2024-02-26T06:23:01.408273+0000","last_undegraded":"2024-02-26T06:23:01.408273+0000","last_fullsized":"2024-02-26T06:23:01.408273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:44:34.400256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440841+0000","last_change":"2024-02-26T06:21:29.211650+0000","last_active":"2024-02-26T06:21:57.440841+0000","last_peered":"2024-02-26T06:21:57.440841+0000","last_clean":"2024-02-26T06:21:57.440841+0000","last_became_active":"2024-02-26T06:21:29.210407+0000","last_became_peered":"2024-02-26T06:21:29.210407+0000","last_unstale":"2024-02-26T06:21:57.440841+0000","last_undegraded":"2024-02-26T06:21:57.440841+0000","last_fullsized":"2024-02-26T06:21:57.440841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:22:00.359506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898366+0000","last_change":"2024-02-26T06:20:31.061702+0000","last_active":"2024-02-26T06:21:28.898366+0000","last_peered":"2024-02-26T06:21:28.898366+0000","last_clean":"2024-02-26T06:21:28.898366+0000","last_became_active":"2024-02-26T06:20:31.061358+0000","last_became_peered":"2024-02-26T06:20:31.061358+0000","last_unstale":"2024-02-26T06:21:28.898366+0000","last_undegraded":"2024-02-26T06:21:28.898366+0000","last_fullsized":"2024-02-26T06:21:28.898366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:25:41.652032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903059+0000","last_change":"2024-02-26T06:19:10.834318+0000","last_active":"2024-02-26T06:21:28.903059+0000","last_peered":"2024-02-26T06:21:28.903059+0000","last_clean":"2024-02-26T06:21:28.903059+0000","last_became_active":"2024-02-26T06:19:10.833869+0000","last_became_peered":"2024-02-26T06:19:10.833869+0000","last_unstale":"2024-02-26T06:21:28.903059+0000","last_undegraded":"2024-02-26T06:21:28.903059+0000","last_fullsized":"2024-02-26T06:21:28.903059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_clean_scrub_stamp":"2024-02-26T06:15:16.238016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:57.078533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428655+0000","last_change":"2024-02-26T06:20:31.600738+0000","last_active":"2024-02-26T06:21:57.428655+0000","last_peered":"2024-02-26T06:21:57.428655+0000","last_clean":"2024-02-26T06:21:57.428655+0000","last_became_active":"2024-02-26T06:20:31.599144+0000","last_became_peered":"2024-02-26T06:20:31.599144+0000","last_unstale":"2024-02-26T06:21:57.428655+0000","last_undegraded":"2024-02-26T06:21:57.428655+0000","last_fullsized":"2024-02-26T06:21:57.428655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:01:16.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903180+0000","last_change":"2024-02-26T06:20:31.063563+0000","last_active":"2024-02-26T06:21:28.903180+0000","last_peered":"2024-02-26T06:21:28.903180+0000","last_clean":"2024-02-26T06:21:28.903180+0000","last_became_active":"2024-02-26T06:20:31.063212+0000","last_became_peered":"2024-02-26T06:20:31.063212+0000","last_unstale":"2024-02-26T06:21:28.903180+0000","last_undegraded":"2024-02-26T06:21:28.903180+0000","last_fullsized":"2024-02-26T06:21:28.903180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:16.480553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907970+0000","last_change":"2024-02-26T06:20:31.075104+0000","last_active":"2024-02-26T06:21:28.907970+0000","last_peered":"2024-02-26T06:21:28.907970+0000","last_clean":"2024-02-26T06:21:28.907970+0000","last_became_active":"2024-02-26T06:20:31.074777+0000","last_became_peered":"2024-02-26T06:20:31.074777+0000","last_unstale":"2024-02-26T06:21:28.907970+0000","last_undegraded":"2024-02-26T06:21:28.907970+0000","last_fullsized":"2024-02-26T06:21:28.907970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:32:20.511703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.417127+0000","last_change":"2024-02-26T06:20:31.605038+0000","last_active":"2024-02-26T06:21:57.417127+0000","last_peered":"2024-02-26T06:21:57.417127+0000","last_clean":"2024-02-26T06:21:57.417127+0000","last_became_active":"2024-02-26T06:20:31.603329+0000","last_became_peered":"2024-02-26T06:20:31.603329+0000","last_unstale":"2024-02-26T06:21:57.417127+0000","last_undegraded":"2024-02-26T06:21:57.417127+0000","last_fullsized":"2024-02-26T06:21:57.417127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:45.066836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:41.014176+0000","last_change":"2024-02-26T06:20:31.084613+0000","last_active":"2024-02-26T06:22:41.014176+0000","last_peered":"2024-02-26T06:22:41.014176+0000","last_clean":"2024-02-26T06:22:41.014176+0000","last_became_active":"2024-02-26T06:20:31.084021+0000","last_became_peered":"2024-02-26T06:20:31.084021+0000","last_unstale":"2024-02-26T06:22:41.014176+0000","last_undegraded":"2024-02-26T06:22:41.014176+0000","last_fullsized":"2024-02-26T06:22:41.014176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:52:42.096437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438954+0000","last_change":"2024-02-26T06:20:31.590367+0000","last_active":"2024-02-26T06:21:57.438954+0000","last_peered":"2024-02-26T06:21:57.438954+0000","last_clean":"2024-02-26T06:21:57.438954+0000","last_became_active":"2024-02-26T06:20:31.582914+0000","last_became_peered":"2024-02-26T06:20:31.582914+0000","last_unstale":"2024-02-26T06:21:57.438954+0000","last_undegraded":"2024-02-26T06:21:57.438954+0000","last_fullsized":"2024-02-26T06:21:57.438954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:02:47.496527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898485+0000","last_change":"2024-02-26T06:20:31.070190+0000","last_active":"2024-02-26T06:21:28.898485+0000","last_peered":"2024-02-26T06:21:28.898485+0000","last_clean":"2024-02-26T06:21:28.898485+0000","last_became_active":"2024-02-26T06:20:31.069854+0000","last_became_peered":"2024-02-26T06:20:31.069854+0000","last_unstale":"2024-02-26T06:21:28.898485+0000","last_undegraded":"2024-02-26T06:21:28.898485+0000","last_fullsized":"2024-02-26T06:21:28.898485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:57.728821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.442447+0000","last_change":"2024-02-26T06:20:31.595388+0000","last_active":"2024-02-26T06:21:57.442447+0000","last_peered":"2024-02-26T06:21:57.442447+0000","last_clean":"2024-02-26T06:21:57.442447+0000","last_became_active":"2024-02-26T06:20:31.594970+0000","last_became_peered":"2024-02-26T06:20:31.594970+0000","last_unstale":"2024-02-26T06:21:57.442447+0000","last_undegraded":"2024-02-26T06:21:57.442447+0000","last_fullsized":"2024-02-26T06:21:57.442447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:12:52.182988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902880+0000","last_change":"2024-02-26T06:20:31.066256+0000","last_active":"2024-02-26T06:21:28.902880+0000","last_peered":"2024-02-26T06:21:28.902880+0000","last_clean":"2024-02-26T06:21:28.902880+0000","last_became_active":"2024-02-26T06:20:31.065956+0000","last_became_peered":"2024-02-26T06:20:31.065956+0000","last_unstale":"2024-02-26T06:21:28.902880+0000","last_undegraded":"2024-02-26T06:21:28.902880+0000","last_fullsized":"2024-02-26T06:21:28.902880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:35.405114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.407926+0000","last_change":"2024-02-26T06:20:31.585102+0000","last_active":"2024-02-26T06:21:57.407926+0000","last_peered":"2024-02-26T06:21:57.407926+0000","last_clean":"2024-02-26T06:21:57.407926+0000","last_became_active":"2024-02-26T06:20:31.584351+0000","last_became_peered":"2024-02-26T06:20:31.584351+0000","last_unstale":"2024-02-26T06:21:57.407926+0000","last_undegraded":"2024-02-26T06:21:57.407926+0000","last_fullsized":"2024-02-26T06:21:57.407926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:09.708752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.897704+0000","last_change":"2024-02-26T06:20:31.079664+0000","last_active":"2024-02-26T06:21:28.897704+0000","last_peered":"2024-02-26T06:21:28.897704+0000","last_clean":"2024-02-26T06:21:28.897704+0000","last_became_active":"2024-02-26T06:20:31.079234+0000","last_became_peered":"2024-02-26T06:20:31.079234+0000","last_unstale":"2024-02-26T06:21:28.897704+0000","last_undegraded":"2024-02-26T06:21:28.897704+0000","last_fullsized":"2024-02-26T06:21:28.897704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:37.407215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439613+0000","last_change":"2024-02-26T06:20:31.597416+0000","last_active":"2024-02-26T06:21:57.439613+0000","last_peered":"2024-02-26T06:21:57.439613+0000","last_clean":"2024-02-26T06:21:57.439613+0000","last_became_active":"2024-02-26T06:20:31.597094+0000","last_became_peered":"2024-02-26T06:20:31.597094+0000","last_unstale":"2024-02-26T06:21:57.439613+0000","last_undegraded":"2024-02-26T06:21:57.439613+0000","last_fullsized":"2024-02-26T06:21:57.439613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:24.774298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903155+0000","last_change":"2024-02-26T06:20:31.074372+0000","last_active":"2024-02-26T06:21:28.903155+0000","last_peered":"2024-02-26T06:21:28.903155+0000","last_clean":"2024-02-26T06:21:28.903155+0000","last_became_active":"2024-02-26T06:20:31.074041+0000","last_became_peered":"2024-02-26T06:20:31.074041+0000","last_unstale":"2024-02-26T06:21:28.903155+0000","last_undegraded":"2024-02-26T06:21:28.903155+0000","last_fullsized":"2024-02-26T06:21:28.903155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:01.367668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441913+0000","last_change":"2024-02-26T06:20:31.599588+0000","last_active":"2024-02-26T06:21:57.441913+0000","last_peered":"2024-02-26T06:21:57.441913+0000","last_clean":"2024-02-26T06:21:57.441913+0000","last_became_active":"2024-02-26T06:20:31.599240+0000","last_became_peered":"2024-02-26T06:20:31.599240+0000","last_unstale":"2024-02-26T06:21:57.441913+0000","last_undegraded":"2024-02-26T06:21:57.441913+0000","last_fullsized":"2024-02-26T06:21:57.441913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:07:14.799565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914009+0000","last_change":"2024-02-26T06:21:28.914009+0000","last_active":"2024-02-26T06:21:28.914009+0000","last_peered":"2024-02-26T06:21:28.914009+0000","last_clean":"2024-02-26T06:21:28.914009+0000","last_became_active":"2024-02-26T06:21:28.906712+0000","last_became_peered":"2024-02-26T06:21:28.906712+0000","last_unstale":"2024-02-26T06:21:28.914009+0000","last_undegraded":"2024-02-26T06:21:28.914009+0000","last_fullsized":"2024-02-26T06:21:28.914009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:21:36.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432870+0000","last_change":"2024-02-26T06:20:31.603955+0000","last_active":"2024-02-26T06:21:57.432870+0000","last_peered":"2024-02-26T06:21:57.432870+0000","last_clean":"2024-02-26T06:21:57.432870+0000","last_became_active":"2024-02-26T06:20:31.603635+0000","last_became_peered":"2024-02-26T06:20:31.603635+0000","last_unstale":"2024-02-26T06:21:57.432870+0000","last_undegraded":"2024-02-26T06:21:57.432870+0000","last_fullsized":"2024-02-26T06:21:57.432870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:56:59.060204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903796+0000","last_change":"2024-02-26T06:20:31.074917+0000","last_active":"2024-02-26T06:21:28.903796+0000","last_peered":"2024-02-26T06:21:28.903796+0000","last_clean":"2024-02-26T06:21:28.903796+0000","last_became_active":"2024-02-26T06:20:31.074590+0000","last_became_peered":"2024-02-26T06:20:31.074590+0000","last_unstale":"2024-02-26T06:21:28.903796+0000","last_undegraded":"2024-02-26T06:21:28.903796+0000","last_fullsized":"2024-02-26T06:21:28.903796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:43.160268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420847+0000","last_change":"2024-02-26T06:20:31.582236+0000","last_active":"2024-02-26T06:21:57.420847+0000","last_peered":"2024-02-26T06:21:57.420847+0000","last_clean":"2024-02-26T06:21:57.420847+0000","last_became_active":"2024-02-26T06:20:31.581851+0000","last_became_peered":"2024-02-26T06:20:31.581851+0000","last_unstale":"2024-02-26T06:21:57.420847+0000","last_undegraded":"2024-02-26T06:21:57.420847+0000","last_fullsized":"2024-02-26T06:21:57.420847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:53.214338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914195+0000","last_change":"2024-02-26T06:20:31.068179+0000","last_active":"2024-02-26T06:21:28.914195+0000","last_peered":"2024-02-26T06:21:28.914195+0000","last_clean":"2024-02-26T06:21:28.914195+0000","last_became_active":"2024-02-26T06:20:31.067888+0000","last_became_peered":"2024-02-26T06:20:31.067888+0000","last_unstale":"2024-02-26T06:21:28.914195+0000","last_undegraded":"2024-02-26T06:21:28.914195+0000","last_fullsized":"2024-02-26T06:21:28.914195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:47:39.223749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441130+0000","last_change":"2024-02-26T06:20:31.598820+0000","last_active":"2024-02-26T06:21:57.441130+0000","last_peered":"2024-02-26T06:21:57.441130+0000","last_clean":"2024-02-26T06:21:57.441130+0000","last_became_active":"2024-02-26T06:20:31.598469+0000","last_became_peered":"2024-02-26T06:20:31.598469+0000","last_unstale":"2024-02-26T06:21:57.441130+0000","last_undegraded":"2024-02-26T06:21:57.441130+0000","last_fullsized":"2024-02-26T06:21:57.441130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:41:15.713748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.207193+0000","last_change":"2024-02-26T06:20:31.080988+0000","last_active":"2024-02-26T06:21:29.207193+0000","last_peered":"2024-02-26T06:21:29.207193+0000","last_clean":"2024-02-26T06:21:29.207193+0000","last_became_active":"2024-02-26T06:20:31.080654+0000","last_became_peered":"2024-02-26T06:20:31.080654+0000","last_unstale":"2024-02-26T06:21:29.207193+0000","last_undegraded":"2024-02-26T06:21:29.207193+0000","last_fullsized":"2024-02-26T06:21:29.207193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:50.315162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414613+0000","last_change":"2024-02-26T06:20:31.586057+0000","last_active":"2024-02-26T06:21:57.414613+0000","last_peered":"2024-02-26T06:21:57.414613+0000","last_clean":"2024-02-26T06:21:57.414613+0000","last_became_active":"2024-02-26T06:20:31.585756+0000","last_became_peered":"2024-02-26T06:20:31.585756+0000","last_unstale":"2024-02-26T06:21:57.414613+0000","last_undegraded":"2024-02-26T06:21:57.414613+0000","last_fullsized":"2024-02-26T06:21:57.414613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:59:07.032287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413388+0000","last_change":"2024-02-26T06:20:31.589067+0000","last_active":"2024-02-26T06:21:57.413388+0000","last_peered":"2024-02-26T06:21:57.413388+0000","last_clean":"2024-02-26T06:21:57.413388+0000","last_became_active":"2024-02-26T06:20:31.588769+0000","last_became_peered":"2024-02-26T06:20:31.588769+0000","last_unstale":"2024-02-26T06:21:57.413388+0000","last_undegraded":"2024-02-26T06:21:57.413388+0000","last_fullsized":"2024-02-26T06:21:57.413388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:43.211127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412546+0000","last_change":"2024-02-26T06:20:31.583838+0000","last_active":"2024-02-26T06:21:57.412546+0000","last_peered":"2024-02-26T06:21:57.412546+0000","last_clean":"2024-02-26T06:21:57.412546+0000","last_became_active":"2024-02-26T06:20:31.583524+0000","last_became_peered":"2024-02-26T06:20:31.583524+0000","last_unstale":"2024-02-26T06:21:57.412546+0000","last_undegraded":"2024-02-26T06:21:57.412546+0000","last_fullsized":"2024-02-26T06:21:57.412546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:21.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414078+0000","last_change":"2024-02-26T06:20:31.597901+0000","last_active":"2024-02-26T06:21:57.414078+0000","last_peered":"2024-02-26T06:21:57.414078+0000","last_clean":"2024-02-26T06:21:57.414078+0000","last_became_active":"2024-02-26T06:20:31.597633+0000","last_became_peered":"2024-02-26T06:20:31.597633+0000","last_unstale":"2024-02-26T06:21:57.414078+0000","last_undegraded":"2024-02-26T06:21:57.414078+0000","last_fullsized":"2024-02-26T06:21:57.414078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:14.084579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438958+0000","last_change":"2024-02-26T06:21:29.210694+0000","last_active":"2024-02-26T06:21:57.438958+0000","last_peered":"2024-02-26T06:21:57.438958+0000","last_clean":"2024-02-26T06:21:57.438958+0000","last_became_active":"2024-02-26T06:21:29.209421+0000","last_became_peered":"2024-02-26T06:21:29.209421+0000","last_unstale":"2024-02-26T06:21:57.438958+0000","last_undegraded":"2024-02-26T06:21:57.438958+0000","last_fullsized":"2024-02-26T06:21:57.438958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:46.511066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.430929+0000","last_change":"2024-02-26T06:20:31.605866+0000","last_active":"2024-02-26T06:21:57.430929+0000","last_peered":"2024-02-26T06:21:57.430929+0000","last_clean":"2024-02-26T06:21:57.430929+0000","last_became_active":"2024-02-26T06:20:31.605404+0000","last_became_peered":"2024-02-26T06:20:31.605404+0000","last_unstale":"2024-02-26T06:21:57.430929+0000","last_undegraded":"2024-02-26T06:21:57.430929+0000","last_fullsized":"2024-02-26T06:21:57.430929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:03:25.055462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413749+0000","last_change":"2024-02-26T06:20:31.585214+0000","last_active":"2024-02-26T06:21:57.413749+0000","last_peered":"2024-02-26T06:21:57.413749+0000","last_clean":"2024-02-26T06:21:57.413749+0000","last_became_active":"2024-02-26T06:20:31.584895+0000","last_became_peered":"2024-02-26T06:20:31.584895+0000","last_unstale":"2024-02-26T06:21:57.413749+0000","last_undegraded":"2024-02-26T06:21:57.413749+0000","last_fullsized":"2024-02-26T06:21:57.413749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:38.507499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.415101+0000","last_change":"2024-02-26T06:20:31.582189+0000","last_active":"2024-02-26T06:21:57.415101+0000","last_peered":"2024-02-26T06:21:57.415101+0000","last_clean":"2024-02-26T06:21:57.415101+0000","last_became_active":"2024-02-26T06:20:31.580729+0000","last_became_peered":"2024-02-26T06:20:31.580729+0000","last_unstale":"2024-02-26T06:21:57.415101+0000","last_undegraded":"2024-02-26T06:21:57.415101+0000","last_fullsized":"2024-02-26T06:21:57.415101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:21.337261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441676+0000","last_change":"2024-02-26T06:20:31.588894+0000","last_active":"2024-02-26T06:21:57.441676+0000","last_peered":"2024-02-26T06:21:57.441676+0000","last_clean":"2024-02-26T06:21:57.441676+0000","last_became_active":"2024-02-26T06:20:31.586362+0000","last_became_peered":"2024-02-26T06:20:31.586362+0000","last_unstale":"2024-02-26T06:21:57.441676+0000","last_undegraded":"2024-02-26T06:21:57.441676+0000","last_fullsized":"2024-02-26T06:21:57.441676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:28:30.009901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439174+0000","last_change":"2024-02-26T06:20:31.608264+0000","last_active":"2024-02-26T06:21:57.439174+0000","last_peered":"2024-02-26T06:21:57.439174+0000","last_clean":"2024-02-26T06:21:57.439174+0000","last_became_active":"2024-02-26T06:20:31.607855+0000","last_became_peered":"2024-02-26T06:20:31.607855+0000","last_unstale":"2024-02-26T06:21:57.439174+0000","last_undegraded":"2024-02-26T06:21:57.439174+0000","last_fullsized":"2024-02-26T06:21:57.439174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:20.169489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441839+0000","last_change":"2024-02-26T06:20:31.591000+0000","last_active":"2024-02-26T06:21:57.441839+0000","last_peered":"2024-02-26T06:21:57.441839+0000","last_clean":"2024-02-26T06:21:57.441839+0000","last_became_active":"2024-02-26T06:20:31.588201+0000","last_became_peered":"2024-02-26T06:20:31.588201+0000","last_unstale":"2024-02-26T06:21:57.441839+0000","last_undegraded":"2024-02-26T06:21:57.441839+0000","last_fullsized":"2024-02-26T06:21:57.441839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:47:59.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441280+0000","last_change":"2024-02-26T06:20:31.590398+0000","last_active":"2024-02-26T06:21:57.441280+0000","last_peered":"2024-02-26T06:21:57.441280+0000","last_clean":"2024-02-26T06:21:57.441280+0000","last_became_active":"2024-02-26T06:20:31.590042+0000","last_became_peered":"2024-02-26T06:20:31.590042+0000","last_unstale":"2024-02-26T06:21:57.441280+0000","last_undegraded":"2024-02-26T06:21:57.441280+0000","last_fullsized":"2024-02-26T06:21:57.441280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:40:01.233978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440874+0000","last_change":"2024-02-26T06:20:31.611698+0000","last_active":"2024-02-26T06:21:57.440874+0000","last_peered":"2024-02-26T06:21:57.440874+0000","last_clean":"2024-02-26T06:21:57.440874+0000","last_became_active":"2024-02-26T06:20:31.610979+0000","last_became_peered":"2024-02-26T06:20:31.610979+0000","last_unstale":"2024-02-26T06:21:57.440874+0000","last_undegraded":"2024-02-26T06:21:57.440874+0000","last_fullsized":"2024-02-26T06:21:57.440874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:44.604373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414255+0000","last_change":"2024-02-26T06:20:31.582016+0000","last_active":"2024-02-26T06:21:57.414255+0000","last_peered":"2024-02-26T06:21:57.414255+0000","last_clean":"2024-02-26T06:21:57.414255+0000","last_became_active":"2024-02-26T06:20:31.581689+0000","last_became_peered":"2024-02-26T06:20:31.581689+0000","last_unstale":"2024-02-26T06:21:57.414255+0000","last_undegraded":"2024-02-26T06:21:57.414255+0000","last_fullsized":"2024-02-26T06:21:57.414255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:04:11.030182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284221,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71257,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":916,"num_read_kb":531,"num_write":171,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":60985,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":208398,"data_compressed":13382,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351021,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459479,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94999999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.94599999999999995,"5min":1.018,"15min":1.018},"last":0.85699999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.61099999999999999,"15min":0.58299999999999996},"min":{"1min":0.53600000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70799999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46800000000000003,"15min":0.44700000000000001},"min":{"1min":0.40500000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.63,"15min":0.59599999999999997},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.871},{"interface":"front","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77600000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76900000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77200000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.95099999999999996,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.81100000000000005,"15min":0.83499999999999996},"min":{"1min":0.58999999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.179,"15min":1.179},"last":1.1000000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94899999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93400000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.82699999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.499,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.79500000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.91000000000000003,"15min":0.89400000000000002},"min":{"1min":0.61299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.2389999999999999,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":0.92200000000000004},"min":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.997}]},{"osd":10,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.623,"5min":0.501,"15min":0.501},"max":{"1min":0.92100000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58799999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.91500000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.84499999999999997}]}]},{"osd":10,"up_from":73,"seq":313532612659,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1118185,"data_compressed":43797,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77700000000000002,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88,"5min":1.069,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.075,"5min":6.2229999999999999,"15min":6.2229999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.45100000000000001,"15min":0.45700000000000002},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58099999999999996,"5min":0.624,"15min":0.624},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.28899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55700000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.434}]},{"osd":3,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80500000000000005,"15min":0.82999999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0129999999999999,"5min":1.403,"15min":1.403},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.57299999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66600000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.498}]},{"osd":5,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.79000000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.378},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.54000000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82199999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.85099999999999998,"15min":0.879},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0669999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91700000000000004,"15min":0.93200000000000005},"min":{"1min":0.63900000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.218,"5min":1.284,"15min":1.284},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:22:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93100000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88800000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.97199999999999998,"15min":0.98599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2190000000000001,"5min":1.266,"15min":1.266},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.63}]},{"osd":11,"last update":"Mon Feb 26 06:22:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.752,"15min":0.752},"min":{"1min":0.63600000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.94299999999999995,"5min":0.998,"15min":0.998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.58299999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.58799999999999997}]}]},{"osd":9,"up_from":68,"seq":292057776184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":459603,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.3500000000000001,"15min":1.448},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.9450000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.194,"5min":1.3100000000000001,"15min":1.3660000000000001},"min":{"1min":0.52800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.161,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2010000000000001,"15min":1.2649999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.704,"5min":1.923,"15min":1.923},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2809999999999999,"15min":1.377},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0030000000000001,"5min":2.407,"15min":2.407},"last":1.3}]},{"osd":2,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4179999999999999,"15min":1.5169999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.028,"5min":2.609,"15min":2.609},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.514,"15min":1.5920000000000001},"min":{"1min":0.72999999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.5219999999999998,"5min":3.681,"15min":3.681},"last":1.4330000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.704,"15min":1.8799999999999999},"min":{"1min":0.621,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.0659999999999998,"5min":3.016,"15min":3.016},"last":1.675},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.724,"15min":1.8080000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.3330000000000002,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.6180000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.7170000000000001,"15min":1.8100000000000001},"min":{"1min":0.78400000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.6019999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.6040000000000001,"15min":1.776},"min":{"1min":0.67500000000000004,"5min":0.624,"15min":0.624},"max":{"1min":2.1040000000000001,"5min":3.98,"15min":3.98},"last":1.798}]},{"osd":5,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":1.8049999999999999,"15min":1.9710000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.379,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":1.895},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.8360000000000001,"15min":1.968},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.3679999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.726}]},{"osd":6,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":2.085,"15min":2.2469999999999999},"min":{"1min":0.878,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6349999999999998,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.952},{"interface":"front","average":{"1min":1.8200000000000001,"5min":2.1179999999999999,"15min":2.2719999999999998},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.7480000000000002,"5min":3.835,"15min":3.835},"last":1.6000000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.8029999999999999,"5min":1.9970000000000001,"15min":2.169},"min":{"1min":0.85199999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.452,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":2.0920000000000001},{"interface":"front","average":{"1min":1.887,"5min":2.0489999999999999,"15min":2.1709999999999998},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.4929999999999999,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":1.8520000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.786,"5min":2.0920000000000001,"15min":2.2040000000000002},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":2.661,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":1.655},{"interface":"front","average":{"1min":1.8839999999999999,"5min":2.1259999999999999,"15min":2.3180000000000001},"min":{"1min":0.91100000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.5750000000000002,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":2.0049999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:14 2024","interfaces":[{"interface":"back","average":{"1min":2.1509999999999998,"5min":2.1859999999999999,"15min":2.266},"min":{"1min":1.3180000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.677,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":2.1589999999999998},{"interface":"front","average":{"1min":2.1200000000000001,"5min":2.0750000000000002,"15min":2.0950000000000002},"min":{"1min":1.091,"5min":1.087,"15min":1.087},"max":{"1min":2.7799999999999998,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":1.7709999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037823,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":459411,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.57299999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99299999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.55800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1299999999999999,"5min":1.198,"15min":1.198},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.46500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.46899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44},"min":{"1min":0.36099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65800000000000003,"5min":0.755,"15min":0.755},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.629,"5min":0.747,"15min":0.747},"last":0.70199999999999996}]},{"osd":3,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78600000000000003,"15min":0.77100000000000002},"min":{"1min":0.55900000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.22,"5min":1.379,"15min":1.379},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":0.86099999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.196,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.85999999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.60599999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2390000000000001,"5min":1.359,"15min":1.359},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.67200000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.1759999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.016}]},{"osd":7,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65100000000000002,"15min":0.64000000000000001},"min":{"1min":0.54400000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.75700000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88600000000000001,"15min":0.874},"min":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1439999999999999,"5min":1.28,"15min":1.28},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.93600000000000005},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.23,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.92500000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68500000000000005,"15min":0.66300000000000003},"min":{"1min":0.58899999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94899999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.72299999999999998,"15min":0.70099999999999996},"min":{"1min":0.61499999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.87}]},{"osd":11,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.85199999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.58499999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.89100000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051904,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40799999999999997,"15min":0.41999999999999998},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.624,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.434,"15min":0.436},"min":{"1min":0.31900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.38700000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76200000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67200000000000004}]},{"osd":3,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":0.83299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1100000000000001,"5min":1.591,"15min":1.591},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.48899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93000000000000005,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.70199999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.495,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.47999999999999998,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67900000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.45600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":0.85599999999999998},"min":{"1min":0.58999999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.1060000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.93000000000000005,"15min":0.88400000000000001},"min":{"1min":0.5,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0429999999999999,"5min":7.4119999999999999,"15min":7.4119999999999999},"last":0.68999999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.49099999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.83799999999999997,"5min":0.873,"15min":0.873},"last":0.61299999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.88,"15min":0.92100000000000004},"min":{"1min":0.57399999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.1599999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.90800000000000003,"15min":0.94199999999999995},"min":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.82399999999999995}]},{"osd":10,"last update":"Mon Feb 26 06:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63800000000000001,"15min":0.65700000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51000000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.61799999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78500000000000003,"15min":0.78000000000000003},"min":{"1min":0.65800000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80500000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462985,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":458992,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2689999999999999,"15min":1.2729999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.915,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.3180000000000001,"15min":1.2509999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.139}]},{"osd":1,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0920000000000001,"15min":1.079},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1299999999999999,"15min":1.1060000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.5580000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.855,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.492},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3520000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7829999999999999,"5min":6.4930000000000003,"15min":6.4930000000000003},"last":1.355}]},{"osd":3,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.645,"15min":1.5860000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":2.0449999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.655,"15min":1.665},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.7470000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4470000000000001,"15min":1.429},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.627,"15min":1.5900000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":2.1949999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.649,"15min":1.625},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.4769999999999999,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":2.2410000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.6950000000000001,"15min":1.7070000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.863,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":1.9099999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.7629999999999999,"15min":1.6200000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.4990000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.968},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.802,"15min":1.742},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.4350000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.8220000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:13 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.7969999999999999,"15min":1.7390000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":2.29},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.9139999999999999,"15min":1.837},"min":{"1min":0.86899999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.8050000000000002,"5min":6.625,"15min":6.625},"last":2.1269999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":2.0779999999999998,"15min":2.2400000000000002},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.536,"5min":3.258,"15min":3.258},"last":2.1659999999999999},{"interface":"front","average":{"1min":1.907,"5min":2.133,"15min":2.2519999999999998},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8109999999999999,"5min":3.347,"15min":3.347},"last":2.456}]},{"osd":10,"last update":"Mon Feb 26 06:22:13 2024","interfaces":[{"interface":"back","average":{"1min":1.873,"5min":2.036,"15min":2.0790000000000002},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":2.339},{"interface":"front","average":{"1min":1.8089999999999999,"5min":2.1120000000000001,"15min":2.1789999999999998},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":2.379}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":525590,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.76300000000000001,"15min":0.69199999999999995},"min":{"1min":0.58499999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3069999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1,"15min":0.80500000000000005},"min":{"1min":0.58899999999999997,"5min":0.44,"15min":0.40400000000000003},"max":{"1min":1.0760000000000001,"5min":16.690999999999999,"15min":16.690999999999999},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.73199999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.72399999999999998,"5min":9.2629999999999999,"15min":9.2629999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.54800000000000004,"15min":0.55500000000000005},"min":{"1min":0.46899999999999997,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40500000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78600000000000003,"15min":0.749},"min":{"1min":0.55500000000000005,"5min":0.377,"15min":0.377},"max":{"1min":1.1279999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78300000000000003,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.073,"5min":1.391,"15min":1.391},"last":0.77700000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40500000000000003},"min":{"1min":0.374,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72199999999999998,"5min":0.748,"15min":0.748},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.443,"15min":0.40899999999999997},"min":{"1min":0.32800000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.65300000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28499999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.79600000000000004,"15min":0.66700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.032,"5min":16.672999999999998,"15min":16.672999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.625,"15min":0.60699999999999998},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.44}]},{"osd":6,"last update":"Mon Feb 26 06:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.81599999999999995,"15min":0.78800000000000003},"min":{"1min":0.55300000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0429999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.92900000000000005,"15min":0.873},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.496,"5min":8.8729999999999993,"15min":8.8729999999999993},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54500000000000004,"15min":0.51000000000000001},"min":{"1min":0.40300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47199999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63300000000000001,"15min":0.61099999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.66000000000000003},"min":{"1min":0.52900000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89800000000000002,"5min":8.9380000000000006,"15min":8.9380000000000006},"last":0.45800000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:22:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91800000000000004,"15min":0.92000000000000004},"min":{"1min":0.70199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1519999999999999,"5min":1.224,"15min":1.224},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.03,"15min":1.01},"min":{"1min":0.64900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.105,"5min":8.8979999999999997,"15min":8.8979999999999997},"last":1.0509999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.49399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0369999999999999,"5min":1.04,"15min":1.04},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79500000000000004,"15min":0.79100000000000004},"min":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.53800000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460263,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:22:03 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.004,"15min":0.84599999999999997},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.46300000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.01,"15min":0.86699999999999999},"min":{"1min":0.624,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":1.4390000000000001,"5min":1.498,"15min":1.649},"last":1.347}]},{"osd":2,"last update":"Mon Feb 26 06:22:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1160000000000001,"15min":0.93799999999999994},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"max":{"1min":1.621,"5min":6.6070000000000002,"15min":6.6070000000000002},"last":0.997},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2869999999999999,"15min":1.024},"min":{"1min":0.72299999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0600000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.081}]},{"osd":3,"last update":"Mon Feb 26 06:22:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.7370000000000001,"15min":1.7649999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2280000000000002,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.9399999999999999},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.909,"15min":1.931},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.419,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":2.2400000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:22:20 2024","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.3460000000000001,"15min":1.107},"min":{"1min":0.75700000000000001,"5min":0.436,"15min":0.436},"max":{"1min":2.0339999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.486},{"interface":"front","average":{"1min":1.357,"5min":1.288,"15min":1.171},"min":{"1min":0.69799999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.131}]},{"osd":5,"last update":"Mon Feb 26 06:22:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.631,"15min":1.4950000000000001},"min":{"1min":0.80000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2639999999999998,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":2.1549999999999998},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5549999999999999,"15min":1.4119999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.1440000000000001,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":1.6120000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:22:20 2024","interfaces":[{"interface":"back","average":{"1min":2.1000000000000001,"5min":1.8959999999999999,"15min":1.8300000000000001},"min":{"1min":0.98099999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":2.3130000000000002},{"interface":"front","average":{"1min":2.0790000000000002,"5min":1.873,"15min":1.857},"min":{"1min":0.67000000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.819}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.73,"15min":1.7190000000000001},"min":{"1min":1.2150000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.45},{"interface":"front","average":{"1min":1.73,"5min":1.599,"15min":1.577},"min":{"1min":1.123,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:22:15 2024","interfaces":[{"interface":"back","average":{"1min":2.0179999999999998,"5min":1.8360000000000001,"15min":1.7529999999999999},"min":{"1min":1.012,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6589999999999998,"5min":2.698,"15min":2.698},"last":2.375},{"interface":"front","average":{"1min":1.964,"5min":1.849,"15min":1.7709999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":1.716}]},{"osd":9,"last update":"Mon Feb 26 06:22:54 2024","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.859,"15min":1.9570000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.2709999999999999,"5min":2.754,"15min":2.754},"last":1.222},{"interface":"front","average":{"1min":1.9490000000000001,"5min":2.1110000000000002,"15min":2.1019999999999999},"min":{"1min":1.0369999999999999,"5min":0.746,"15min":0.746},"max":{"1min":2.706,"5min":3.101,"15min":3.101},"last":2.4550000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:22:15 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":2.0339999999999998,"15min":2.0539999999999998},"min":{"1min":0.91500000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7679999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":2.0310000000000001},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.8180000000000001,"15min":1.835},"min":{"1min":0.84899999999999998,"5min":0.75,"15min":0.75},"max":{"1min":2.581,"5min":2.581,"15min":2.581},"last":1.2629999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:22:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0329999999999999,"5min":1.994,"15min":1.9199999999999999},"min":{"1min":1.0569999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7989999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":2.4929999999999999},{"interface":"front","average":{"1min":1.921,"5min":2.0179999999999998,"15min":2.016},"min":{"1min":0.96199999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.714,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":2.214}]}]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":460333,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73399999999999999,"15min":0.69299999999999995},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73999999999999999,"15min":0.70699999999999996},"min":{"1min":0.51000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93200000000000005,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.78700000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.35299999999999998,"15min":0.35199999999999998},"max":{"1min":0.78100000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.68000000000000005}]},{"osd":3,"last update":"Mon Feb 26 06:22:02 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.75600000000000001,"15min":0.71899999999999997},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76100000000000001,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.119,"5min":1.367,"15min":1.367},"last":0.629}]},{"osd":4,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.621},"min":{"1min":0.439,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Feb 26 06:22:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47399999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.69999999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53800000000000003,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51100000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.76700000000000002},"min":{"1min":0.58499999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1180000000000001,"5min":1.206,"15min":1.206},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.0469999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47399999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.40899999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:22:51 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.95299999999999996,"15min":0.94799999999999995},"min":{"1min":0.64400000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98399999999999999,"15min":0.98799999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.2150000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.52400000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.60099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.75600000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.72299999999999998},"min":{"1min":0.54400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.66200000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":459303,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2150000000000001,"15min":1.0900000000000001},"min":{"1min":0.54300000000000004,"5min":0.433,"15min":0.433},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4630000000000001,"15min":1.173},"min":{"1min":0.80100000000000005,"5min":0.375,"15min":0.375},"max":{"1min":2.0840000000000001,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":0.80100000000000005}]},{"osd":1,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2090000000000001,"15min":1.056},"min":{"1min":0.68500000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.853,"5min":2.036,"15min":2.036},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.252,"5min":1.1890000000000001,"15min":1.0940000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.786,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.345,"15min":1.204},"min":{"1min":0.81799999999999995,"5min":0.70099999999999996,"15min":0.66400000000000003},"max":{"1min":1.7589999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.194,"5min":1.1220000000000001,"15min":1.0549999999999999},"min":{"1min":0.84399999999999997,"5min":0.441,"15min":0.441},"max":{"1min":1.7989999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.84399999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:22:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3740000000000001,"15min":1.2370000000000001},"min":{"1min":0.83599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.863,"5min":2.613,"15min":2.613},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.577,"5min":1.5920000000000001,"15min":1.4039999999999999},"min":{"1min":0.85599999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.0990000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.751}]},{"osd":5,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.506,"15min":1.393},"min":{"1min":0.90800000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.405,"15min":1.351},"min":{"1min":0.93799999999999994,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.266,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Feb 26 06:22:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.6439999999999999,"15min":1.5069999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.488,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.8620000000000001,"5min":1.853,"15min":1.673},"min":{"1min":0.82099999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.806,"5min":4.681,"15min":4.681},"last":0.86499999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.915,"5min":1.8089999999999999,"15min":1.75},"min":{"1min":1.2869999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.6560000000000001,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.8700000000000001,"5min":1.865,"15min":1.7250000000000001},"min":{"1min":1.381,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.468,"5min":3.544,"15min":3.544},"last":0.83399999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.806},"min":{"1min":0.97399999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.587,"15min":1.5649999999999999},"min":{"1min":0.84999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.23,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.143}]},{"osd":9,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.9590000000000001,"5min":1.9670000000000001,"15min":2.0379999999999998},"min":{"1min":1.4610000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.7610000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":2.1429999999999998,"5min":2.2290000000000001,"15min":2.403},"min":{"1min":1.619,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.734,"5min":4.266,"15min":4.266},"last":0.98099999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8280000000000001,"5min":1.9450000000000001,"15min":2.0099999999999998},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.998},{"interface":"front","average":{"1min":1.9530000000000001,"5min":2.1699999999999999,"15min":2.2589999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.6509999999999998,"5min":4.0609999999999999,"15min":4.0609999999999999},"last":0.89500000000000002}]},{"osd":11,"last update":"Mon Feb 26 06:22:34 2024","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.9399999999999999,"15min":1.946},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.6150000000000002,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":2.0800000000000001,"5min":2.0739999999999998,"15min":2.073},"min":{"1min":1.0609999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.5739999999999998,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.92900000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888149,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":521505,"data_compressed":15495,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.50600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.079,"5min":1.093,"15min":1.093},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39700000000000002,"15min":0.36099999999999999},"min":{"1min":0.34100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.34999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.371,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69299999999999995,"15min":0.57499999999999996},"min":{"1min":0.48299999999999998,"5min":0.379,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":12.468,"15min":12.468},"last":0.51700000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76200000000000001,"15min":0.71199999999999997},"min":{"1min":0.59999999999999998,"5min":0.48299999999999998,"15min":0.39800000000000002},"max":{"1min":0.94399999999999995,"5min":1.167,"15min":1.167},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.746},"min":{"1min":0.49099999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1579999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.623}]},{"osd":5,"last update":"Mon Feb 26 06:22:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.54800000000000004},"min":{"1min":0.52300000000000002,"5min":0.42599999999999999,"15min":0.41099999999999998},"max":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.626,"15min":0.56999999999999995},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56799999999999995}]},{"osd":6,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79000000000000004,"15min":0.73799999999999999},"min":{"1min":0.628,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97499999999999998,"5min":1.327,"15min":1.327},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80200000000000005,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.1180000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Feb 26 06:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.42399999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.36499999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.70199999999999996,"15min":0.66600000000000004},"min":{"1min":0.59499999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73799999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1699999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.044,"15min":1},"min":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2190000000000001,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.70499999999999996}]},{"osd":10,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.61799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.5,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.439}]},{"osd":11,"last update":"Mon Feb 26 06:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.77100000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724625,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1051056,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72299999999999998,"15min":0.67700000000000005},"min":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.159,"15min":1.159},"last":0.749},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.0649999999999999,"5min":1.204,"15min":1.204},"last":0.91900000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79500000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70499999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.315,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61099999999999999,"5min":0.624,"15min":0.624},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42399999999999999,"15min":0.40600000000000003},"min":{"1min":0.34799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.75,"15min":0.75},"last":0.34200000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.80800000000000005,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.78600000000000003,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78400000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Feb 26 06:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82799999999999996,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.135,"5min":1.242,"15min":1.242},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83799999999999997,"15min":0.80900000000000005},"min":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.089,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.044}]},{"osd":7,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65600000000000003,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46899999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68300000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48299999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.64500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.381,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.369},{"interface":"front","average":{"1min":1.01,"5min":0.999,"15min":1.04},"min":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.2789999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.69999999999999996,"15min":0.68400000000000005},"min":{"1min":0.56100000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.90100000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66805,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":62720,"data_compressed":9688,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68915,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:23:09.408 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:23:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 -- 172.21.15.49:0/3609402303 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdee0073be0 msgr2=0x7fdee0074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 --2- 172.21.15.49:0/3609402303 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdee0073be0 0x7fdee0074060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009cd0 tx=0x7fdecc02c0e0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 -- 172.21.15.49:0/3609402303 shutdown_connections 2024-02-26T06:23:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 --2- 172.21.15.49:0/3609402303 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdee0073be0 0x7fdee0074060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 --2- 172.21.15.49:0/3609402303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdee010d590 0x7fdee010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 --2- 172.21.15.49:0/3609402303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdee00751c0 0x7fdee0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 -- 172.21.15.49:0/3609402303 >> 172.21.15.49:0/3609402303 conn(0x7fdee00fc510 msgr2=0x7fdee00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 -- 172.21.15.49:0/3609402303 shutdown_connections 2024-02-26T06:23:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.686+0000 7fdedfe00640 1 -- 172.21.15.49:0/3609402303 wait complete. 2024-02-26T06:23:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.687+0000 7fdedfe00640 1 Processor -- start 2024-02-26T06:23:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.687+0000 7fdedfe00640 1 -- start start 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdee0073be0 0x7fdee019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdee00751c0 0x7fdee019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdee010d590 0x7fdee01a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedfe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee01a5190 con 0x7fdee010d590 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedfe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee01a5300 con 0x7fdee0073be0 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedfe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee01a54c0 con 0x7fdee00751c0 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedf400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdee010d590 0x7fdee01a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdede000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdee00751c0 0x7fdee019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdee0073be0 0x7fdee019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedf400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdee010d590 0x7fdee01a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38152/0 (socket says 172.21.15.49:38152) 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdedf400640 1 -- 172.21.15.49:0/1242532543 learned_addr learned my addr 172.21.15.49:0/1242532543 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.688+0000 7fdede000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdee00751c0 0x7fdee019dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44838/0 (socket says 172.21.15.49:44838) 2024-02-26T06:23:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.689+0000 7fdede000640 1 -- 172.21.15.49:0/1242532543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdee0073be0 msgr2=0x7fdee019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.689+0000 7fdede000640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdee0073be0 0x7fdee019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.689+0000 7fdede000640 1 -- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdee010d590 msgr2=0x7fdee01a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.689+0000 7fdede000640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdee010d590 0x7fdee01a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.689+0000 7fdede000640 1 -- 172.21.15.49:0/1242532543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc009900 con 0x7fdee00751c0 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.689+0000 7fdede000640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdee00751c0 0x7fdee019dbe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdecc02c620 tx=0x7fdecc009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.689+0000 7fded7400640 1 -- 172.21.15.49:0/1242532543 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0329b0 con 0x7fdee00751c0 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.689+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee01a5740 con 0x7fdee00751c0 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.690+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee01a5c40 con 0x7fdee00751c0 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.690+0000 7fded7400640 1 -- 172.21.15.49:0/1242532543 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdecc003e50 con 0x7fdee00751c0 2024-02-26T06:23:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.690+0000 7fded7400640 1 -- 172.21.15.49:0/1242532543 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc004830 con 0x7fdee00751c0 2024-02-26T06:23:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.691+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec00052c0 con 0x7fdee00751c0 2024-02-26T06:23:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.693+0000 7fded7400640 1 -- 172.21.15.49:0/1242532543 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdecc00a040 con 0x7fdee00751c0 2024-02-26T06:23:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.694+0000 7fded7400640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdec40a20c0 0x7fdec40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.694+0000 7fded7400640 1 -- 172.21.15.49:0/1242532543 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fdecc0f04f0 con 0x7fdee00751c0 2024-02-26T06:23:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.695+0000 7fdedea00640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdec40a20c0 0x7fdec40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:09.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.696+0000 7fdedea00640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdec40a20c0 0x7fdec40a4580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fded0009bd0 tx=0x7fded0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:09.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.696+0000 7fded7400640 1 -- 172.21.15.49:0/1242532543 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdecc0b94e0 con 0x7fdee00751c0 2024-02-26T06:23:09.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.902+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdec0000bf0 con 0x7fdec40a20c0 2024-02-26T06:23:09.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.937+0000 7fded7400640 1 -- 172.21.15.49:0/1242532543 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+342859 (secure 0 0 0) 0x7fdec0000bf0 con 0x7fdec40a20c0 2024-02-26T06:23:09.942 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:09.944 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdec40a20c0 msgr2=0x7fdec40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdec40a20c0 0x7fdec40a4580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fded0009bd0 tx=0x7fded0005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdee00751c0 msgr2=0x7fdee019dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdee00751c0 0x7fdee019dbe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdecc02c620 tx=0x7fdecc009360 comp rx=0 tx=0).stop 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 shutdown_connections 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fdec40a20c0 0x7fdec40a4580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdee010d590 0x7fdee01a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdee00751c0 0x7fdee019dbe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 --2- 172.21.15.49:0/1242532543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdee0073be0 0x7fdee019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:09.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 >> 172.21.15.49:0/1242532543 conn(0x7fdee00fc510 msgr2=0x7fdee0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:09.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 shutdown_connections 2024-02-26T06:23:09.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:09.945+0000 7fdedfe00640 1 -- 172.21.15.49:0/1242532543 wait complete. 2024-02-26T06:23:09.971 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-02-26T06:23:09.486557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661859,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1062,"num_read_kb":655,"num_write":354,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1241721,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360788,"kb_used_data":11636,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581036,"statfs":{"total":1151940427776,"available":1151570980864,"internally_reserved":0,"allocated":11915264,"data_stored":7490997,"data_compressed":191495,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5993,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":43,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003824"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440192+0000","last_change":"2024-02-26T06:20:31.603060+0000","last_active":"2024-02-26T06:21:57.440192+0000","last_peered":"2024-02-26T06:21:57.440192+0000","last_clean":"2024-02-26T06:21:57.440192+0000","last_became_active":"2024-02-26T06:20:31.602762+0000","last_became_peered":"2024-02-26T06:20:31.602762+0000","last_unstale":"2024-02-26T06:21:57.440192+0000","last_undegraded":"2024-02-26T06:21:57.440192+0000","last_fullsized":"2024-02-26T06:21:57.440192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:19:42.465510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.437717+0000","last_change":"2024-02-26T06:20:31.595138+0000","last_active":"2024-02-26T06:21:57.437717+0000","last_peered":"2024-02-26T06:21:57.437717+0000","last_clean":"2024-02-26T06:21:57.437717+0000","last_became_active":"2024-02-26T06:20:31.594826+0000","last_became_peered":"2024-02-26T06:20:31.594826+0000","last_unstale":"2024-02-26T06:21:57.437717+0000","last_undegraded":"2024-02-26T06:21:57.437717+0000","last_fullsized":"2024-02-26T06:21:57.437717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:49.900464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.431160+0000","last_change":"2024-02-26T06:20:31.600415+0000","last_active":"2024-02-26T06:21:57.431160+0000","last_peered":"2024-02-26T06:21:57.431160+0000","last_clean":"2024-02-26T06:21:57.431160+0000","last_became_active":"2024-02-26T06:20:31.596273+0000","last_became_peered":"2024-02-26T06:20:31.596273+0000","last_unstale":"2024-02-26T06:21:57.431160+0000","last_undegraded":"2024-02-26T06:21:57.431160+0000","last_fullsized":"2024-02-26T06:21:57.431160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:15:20.626515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410929+0000","last_change":"2024-02-26T06:20:31.581149+0000","last_active":"2024-02-26T06:21:57.410929+0000","last_peered":"2024-02-26T06:21:57.410929+0000","last_clean":"2024-02-26T06:21:57.410929+0000","last_became_active":"2024-02-26T06:20:31.580831+0000","last_became_peered":"2024-02-26T06:20:31.580831+0000","last_unstale":"2024-02-26T06:21:57.410929+0000","last_undegraded":"2024-02-26T06:21:57.410929+0000","last_fullsized":"2024-02-26T06:21:57.410929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:47:12.466502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412577+0000","last_change":"2024-02-26T06:20:31.608618+0000","last_active":"2024-02-26T06:21:57.412577+0000","last_peered":"2024-02-26T06:21:57.412577+0000","last_clean":"2024-02-26T06:21:57.412577+0000","last_became_active":"2024-02-26T06:20:31.608285+0000","last_became_peered":"2024-02-26T06:20:31.608285+0000","last_unstale":"2024-02-26T06:21:57.412577+0000","last_undegraded":"2024-02-26T06:21:57.412577+0000","last_fullsized":"2024-02-26T06:21:57.412577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:34:48.400836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.418353+0000","last_change":"2024-02-26T06:21:28.905073+0000","last_active":"2024-02-26T06:21:57.418353+0000","last_peered":"2024-02-26T06:21:57.418353+0000","last_clean":"2024-02-26T06:21:57.418353+0000","last_became_active":"2024-02-26T06:21:28.904630+0000","last_became_peered":"2024-02-26T06:21:28.904630+0000","last_unstale":"2024-02-26T06:21:57.418353+0000","last_undegraded":"2024-02-26T06:21:57.418353+0000","last_fullsized":"2024-02-26T06:21:57.418353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:04.531384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.407619+0000","last_change":"2024-02-26T06:20:31.587285+0000","last_active":"2024-02-26T06:21:57.407619+0000","last_peered":"2024-02-26T06:21:57.407619+0000","last_clean":"2024-02-26T06:21:57.407619+0000","last_became_active":"2024-02-26T06:20:31.586985+0000","last_became_peered":"2024-02-26T06:20:31.586985+0000","last_unstale":"2024-02-26T06:21:57.407619+0000","last_undegraded":"2024-02-26T06:21:57.407619+0000","last_fullsized":"2024-02-26T06:21:57.407619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:55:56.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438247+0000","last_change":"2024-02-26T06:20:31.591136+0000","last_active":"2024-02-26T06:21:57.438247+0000","last_peered":"2024-02-26T06:21:57.438247+0000","last_clean":"2024-02-26T06:21:57.438247+0000","last_became_active":"2024-02-26T06:20:31.590180+0000","last_became_peered":"2024-02-26T06:20:31.590180+0000","last_unstale":"2024-02-26T06:21:57.438247+0000","last_undegraded":"2024-02-26T06:21:57.438247+0000","last_fullsized":"2024-02-26T06:21:57.438247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:02.649407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420898+0000","last_change":"2024-02-26T06:20:31.583890+0000","last_active":"2024-02-26T06:21:57.420898+0000","last_peered":"2024-02-26T06:21:57.420898+0000","last_clean":"2024-02-26T06:21:57.420898+0000","last_became_active":"2024-02-26T06:20:31.583459+0000","last_became_peered":"2024-02-26T06:20:31.583459+0000","last_unstale":"2024-02-26T06:21:57.420898+0000","last_undegraded":"2024-02-26T06:21:57.420898+0000","last_fullsized":"2024-02-26T06:21:57.420898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:14.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412793+0000","last_change":"2024-02-26T06:20:31.579041+0000","last_active":"2024-02-26T06:21:57.412793+0000","last_peered":"2024-02-26T06:21:57.412793+0000","last_clean":"2024-02-26T06:21:57.412793+0000","last_became_active":"2024-02-26T06:20:31.578725+0000","last_became_peered":"2024-02-26T06:20:31.578725+0000","last_unstale":"2024-02-26T06:21:57.412793+0000","last_undegraded":"2024-02-26T06:21:57.412793+0000","last_fullsized":"2024-02-26T06:21:57.412793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:21.083206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420613+0000","last_change":"2024-02-26T06:20:31.581146+0000","last_active":"2024-02-26T06:21:57.420613+0000","last_peered":"2024-02-26T06:21:57.420613+0000","last_clean":"2024-02-26T06:21:57.420613+0000","last_became_active":"2024-02-26T06:20:31.580774+0000","last_became_peered":"2024-02-26T06:20:31.580774+0000","last_unstale":"2024-02-26T06:21:57.420613+0000","last_undegraded":"2024-02-26T06:21:57.420613+0000","last_fullsized":"2024-02-26T06:21:57.420613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:58:36.851504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410598+0000","last_change":"2024-02-26T06:20:31.582232+0000","last_active":"2024-02-26T06:21:57.410598+0000","last_peered":"2024-02-26T06:21:57.410598+0000","last_clean":"2024-02-26T06:21:57.410598+0000","last_became_active":"2024-02-26T06:20:31.581805+0000","last_became_peered":"2024-02-26T06:20:31.581805+0000","last_unstale":"2024-02-26T06:21:57.410598+0000","last_undegraded":"2024-02-26T06:21:57.410598+0000","last_fullsized":"2024-02-26T06:21:57.410598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:04.513814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428668+0000","last_change":"2024-02-26T06:20:31.606373+0000","last_active":"2024-02-26T06:21:57.428668+0000","last_peered":"2024-02-26T06:21:57.428668+0000","last_clean":"2024-02-26T06:21:57.428668+0000","last_became_active":"2024-02-26T06:20:31.603226+0000","last_became_peered":"2024-02-26T06:20:31.603226+0000","last_unstale":"2024-02-26T06:21:57.428668+0000","last_undegraded":"2024-02-26T06:21:57.428668+0000","last_fullsized":"2024-02-26T06:21:57.428668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:29:49.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.426327+0000","last_change":"2024-02-26T06:20:31.594587+0000","last_active":"2024-02-26T06:21:57.426327+0000","last_peered":"2024-02-26T06:21:57.426327+0000","last_clean":"2024-02-26T06:21:57.426327+0000","last_became_active":"2024-02-26T06:20:31.594032+0000","last_became_peered":"2024-02-26T06:20:31.594032+0000","last_unstale":"2024-02-26T06:21:57.426327+0000","last_undegraded":"2024-02-26T06:21:57.426327+0000","last_fullsized":"2024-02-26T06:21:57.426327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:49:18.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438329+0000","last_change":"2024-02-26T06:20:31.593115+0000","last_active":"2024-02-26T06:21:57.438329+0000","last_peered":"2024-02-26T06:21:57.438329+0000","last_clean":"2024-02-26T06:21:57.438329+0000","last_became_active":"2024-02-26T06:20:31.592782+0000","last_became_peered":"2024-02-26T06:20:31.592782+0000","last_unstale":"2024-02-26T06:21:57.438329+0000","last_undegraded":"2024-02-26T06:21:57.438329+0000","last_fullsized":"2024-02-26T06:21:57.438329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:10.709124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.427221+0000","last_change":"2024-02-26T06:20:31.597938+0000","last_active":"2024-02-26T06:21:57.427221+0000","last_peered":"2024-02-26T06:21:57.427221+0000","last_clean":"2024-02-26T06:21:57.427221+0000","last_became_active":"2024-02-26T06:20:31.597600+0000","last_became_peered":"2024-02-26T06:20:31.597600+0000","last_unstale":"2024-02-26T06:21:57.427221+0000","last_undegraded":"2024-02-26T06:21:57.427221+0000","last_fullsized":"2024-02-26T06:21:57.427221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:13.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440351+0000","last_change":"2024-02-26T06:20:31.603283+0000","last_active":"2024-02-26T06:21:57.440351+0000","last_peered":"2024-02-26T06:21:57.440351+0000","last_clean":"2024-02-26T06:21:57.440351+0000","last_became_active":"2024-02-26T06:20:31.602912+0000","last_became_peered":"2024-02-26T06:20:31.602912+0000","last_unstale":"2024-02-26T06:21:57.440351+0000","last_undegraded":"2024-02-26T06:21:57.440351+0000","last_fullsized":"2024-02-26T06:21:57.440351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:32.250914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.422411+0000","last_change":"2024-02-26T06:20:31.588670+0000","last_active":"2024-02-26T06:21:57.422411+0000","last_peered":"2024-02-26T06:21:57.422411+0000","last_clean":"2024-02-26T06:21:57.422411+0000","last_became_active":"2024-02-26T06:20:31.587971+0000","last_became_peered":"2024-02-26T06:20:31.587971+0000","last_unstale":"2024-02-26T06:21:57.422411+0000","last_undegraded":"2024-02-26T06:21:57.422411+0000","last_fullsized":"2024-02-26T06:21:57.422411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:13.053456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.425204+0000","last_change":"2024-02-26T06:20:31.604266+0000","last_active":"2024-02-26T06:21:57.425204+0000","last_peered":"2024-02-26T06:21:57.425204+0000","last_clean":"2024-02-26T06:21:57.425204+0000","last_became_active":"2024-02-26T06:20:31.604027+0000","last_became_peered":"2024-02-26T06:20:31.604027+0000","last_unstale":"2024-02-26T06:21:57.425204+0000","last_undegraded":"2024-02-26T06:21:57.425204+0000","last_fullsized":"2024-02-26T06:21:57.425204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:13.345138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904510+0000","last_change":"2024-02-26T06:20:31.068090+0000","last_active":"2024-02-26T06:21:28.904510+0000","last_peered":"2024-02-26T06:21:28.904510+0000","last_clean":"2024-02-26T06:21:28.904510+0000","last_became_active":"2024-02-26T06:20:31.067731+0000","last_became_peered":"2024-02-26T06:20:31.067731+0000","last_unstale":"2024-02-26T06:21:28.904510+0000","last_undegraded":"2024-02-26T06:21:28.904510+0000","last_fullsized":"2024-02-26T06:21:28.904510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:24.734040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413823+0000","last_change":"2024-02-26T06:21:28.915741+0000","last_active":"2024-02-26T06:21:57.413823+0000","last_peered":"2024-02-26T06:21:57.413823+0000","last_clean":"2024-02-26T06:21:57.413823+0000","last_became_active":"2024-02-26T06:21:28.915443+0000","last_became_peered":"2024-02-26T06:21:28.915443+0000","last_unstale":"2024-02-26T06:21:57.413823+0000","last_undegraded":"2024-02-26T06:21:57.413823+0000","last_fullsized":"2024-02-26T06:21:57.413823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:51.216291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898901+0000","last_change":"2024-02-26T06:20:31.083236+0000","last_active":"2024-02-26T06:21:28.898901+0000","last_peered":"2024-02-26T06:21:28.898901+0000","last_clean":"2024-02-26T06:21:28.898901+0000","last_became_active":"2024-02-26T06:20:31.081640+0000","last_became_peered":"2024-02-26T06:20:31.081640+0000","last_unstale":"2024-02-26T06:21:28.898901+0000","last_undegraded":"2024-02-26T06:21:28.898901+0000","last_fullsized":"2024-02-26T06:21:28.898901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:44.607108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439959+0000","last_change":"2024-02-26T06:20:31.593807+0000","last_active":"2024-02-26T06:21:57.439959+0000","last_peered":"2024-02-26T06:21:57.439959+0000","last_clean":"2024-02-26T06:21:57.439959+0000","last_became_active":"2024-02-26T06:20:31.593435+0000","last_became_peered":"2024-02-26T06:20:31.593435+0000","last_unstale":"2024-02-26T06:21:57.439959+0000","last_undegraded":"2024-02-26T06:21:57.439959+0000","last_fullsized":"2024-02-26T06:21:57.439959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:43.193196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.908174+0000","last_change":"2024-02-26T06:20:31.061339+0000","last_active":"2024-02-26T06:21:28.908174+0000","last_peered":"2024-02-26T06:21:28.908174+0000","last_clean":"2024-02-26T06:21:28.908174+0000","last_became_active":"2024-02-26T06:20:31.060769+0000","last_became_peered":"2024-02-26T06:20:31.060769+0000","last_unstale":"2024-02-26T06:21:28.908174+0000","last_undegraded":"2024-02-26T06:21:28.908174+0000","last_fullsized":"2024-02-26T06:21:28.908174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:28:30.501375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432762+0000","last_change":"2024-02-26T06:20:31.606936+0000","last_active":"2024-02-26T06:21:57.432762+0000","last_peered":"2024-02-26T06:21:57.432762+0000","last_clean":"2024-02-26T06:21:57.432762+0000","last_became_active":"2024-02-26T06:20:31.606582+0000","last_became_peered":"2024-02-26T06:20:31.606582+0000","last_unstale":"2024-02-26T06:21:57.432762+0000","last_undegraded":"2024-02-26T06:21:57.432762+0000","last_fullsized":"2024-02-26T06:21:57.432762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:09.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907169+0000","last_change":"2024-02-26T06:20:31.063472+0000","last_active":"2024-02-26T06:21:28.907169+0000","last_peered":"2024-02-26T06:21:28.907169+0000","last_clean":"2024-02-26T06:21:28.907169+0000","last_became_active":"2024-02-26T06:20:31.063132+0000","last_became_peered":"2024-02-26T06:20:31.063132+0000","last_unstale":"2024-02-26T06:21:28.907169+0000","last_undegraded":"2024-02-26T06:21:28.907169+0000","last_fullsized":"2024-02-26T06:21:28.907169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:58.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.418097+0000","last_change":"2024-02-26T06:20:31.589238+0000","last_active":"2024-02-26T06:21:57.418097+0000","last_peered":"2024-02-26T06:21:57.418097+0000","last_clean":"2024-02-26T06:21:57.418097+0000","last_became_active":"2024-02-26T06:20:31.588929+0000","last_became_peered":"2024-02-26T06:20:31.588929+0000","last_unstale":"2024-02-26T06:21:57.418097+0000","last_undegraded":"2024-02-26T06:21:57.418097+0000","last_fullsized":"2024-02-26T06:21:57.418097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:19.282943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":179,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:06.333402+0000","last_change":"2024-02-26T06:20:31.061714+0000","last_active":"2024-02-26T06:23:06.333402+0000","last_peered":"2024-02-26T06:23:06.333402+0000","last_clean":"2024-02-26T06:23:06.333402+0000","last_became_active":"2024-02-26T06:20:31.061234+0000","last_became_peered":"2024-02-26T06:20:31.061234+0000","last_unstale":"2024-02-26T06:23:06.333402+0000","last_undegraded":"2024-02-26T06:23:06.333402+0000","last_fullsized":"2024-02-26T06:23:06.333402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:29:17.668327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.400272+0000","last_change":"2024-02-26T06:20:31.588861+0000","last_active":"2024-02-26T06:21:57.400272+0000","last_peered":"2024-02-26T06:21:57.400272+0000","last_clean":"2024-02-26T06:21:57.400272+0000","last_became_active":"2024-02-26T06:20:31.588530+0000","last_became_peered":"2024-02-26T06:20:31.588530+0000","last_unstale":"2024-02-26T06:21:57.400272+0000","last_undegraded":"2024-02-26T06:21:57.400272+0000","last_fullsized":"2024-02-26T06:21:57.400272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:12.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.900377+0000","last_change":"2024-02-26T06:20:31.077656+0000","last_active":"2024-02-26T06:21:28.900377+0000","last_peered":"2024-02-26T06:21:28.900377+0000","last_clean":"2024-02-26T06:21:28.900377+0000","last_became_active":"2024-02-26T06:20:31.077288+0000","last_became_peered":"2024-02-26T06:20:31.077288+0000","last_unstale":"2024-02-26T06:21:28.900377+0000","last_undegraded":"2024-02-26T06:21:28.900377+0000","last_fullsized":"2024-02-26T06:21:28.900377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:56:05.173751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440086+0000","last_change":"2024-02-26T06:20:31.608089+0000","last_active":"2024-02-26T06:21:57.440086+0000","last_peered":"2024-02-26T06:21:57.440086+0000","last_clean":"2024-02-26T06:21:57.440086+0000","last_became_active":"2024-02-26T06:20:31.607786+0000","last_became_peered":"2024-02-26T06:20:31.607786+0000","last_unstale":"2024-02-26T06:21:57.440086+0000","last_undegraded":"2024-02-26T06:21:57.440086+0000","last_fullsized":"2024-02-26T06:21:57.440086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:24.113286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.207050+0000","last_change":"2024-02-26T06:20:31.077063+0000","last_active":"2024-02-26T06:21:29.207050+0000","last_peered":"2024-02-26T06:21:29.207050+0000","last_clean":"2024-02-26T06:21:29.207050+0000","last_became_active":"2024-02-26T06:20:31.075147+0000","last_became_peered":"2024-02-26T06:20:31.075147+0000","last_unstale":"2024-02-26T06:21:29.207050+0000","last_undegraded":"2024-02-26T06:21:29.207050+0000","last_fullsized":"2024-02-26T06:21:29.207050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:07:13.434635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439933+0000","last_change":"2024-02-26T06:20:31.600135+0000","last_active":"2024-02-26T06:21:57.439933+0000","last_peered":"2024-02-26T06:21:57.439933+0000","last_clean":"2024-02-26T06:21:57.439933+0000","last_became_active":"2024-02-26T06:20:31.595622+0000","last_became_peered":"2024-02-26T06:20:31.595622+0000","last_unstale":"2024-02-26T06:21:57.439933+0000","last_undegraded":"2024-02-26T06:21:57.439933+0000","last_fullsized":"2024-02-26T06:21:57.439933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:55.835700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914398+0000","last_change":"2024-02-26T06:20:31.060299+0000","last_active":"2024-02-26T06:21:28.914398+0000","last_peered":"2024-02-26T06:21:28.914398+0000","last_clean":"2024-02-26T06:21:28.914398+0000","last_became_active":"2024-02-26T06:20:31.059938+0000","last_became_peered":"2024-02-26T06:20:31.059938+0000","last_unstale":"2024-02-26T06:21:28.914398+0000","last_undegraded":"2024-02-26T06:21:28.914398+0000","last_fullsized":"2024-02-26T06:21:28.914398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:24.144958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.444272+0000","last_change":"2024-02-26T06:20:31.601945+0000","last_active":"2024-02-26T06:21:57.444272+0000","last_peered":"2024-02-26T06:21:57.444272+0000","last_clean":"2024-02-26T06:21:57.444272+0000","last_became_active":"2024-02-26T06:20:31.601560+0000","last_became_peered":"2024-02-26T06:20:31.601560+0000","last_unstale":"2024-02-26T06:21:57.444272+0000","last_undegraded":"2024-02-26T06:21:57.444272+0000","last_fullsized":"2024-02-26T06:21:57.444272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:13.932623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":173,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:05.331162+0000","last_change":"2024-02-26T06:20:31.068334+0000","last_active":"2024-02-26T06:23:05.331162+0000","last_peered":"2024-02-26T06:23:05.331162+0000","last_clean":"2024-02-26T06:23:05.331162+0000","last_became_active":"2024-02-26T06:20:31.068020+0000","last_became_peered":"2024-02-26T06:20:31.068020+0000","last_unstale":"2024-02-26T06:23:05.331162+0000","last_undegraded":"2024-02-26T06:23:05.331162+0000","last_fullsized":"2024-02-26T06:23:05.331162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:23:01.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":149,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421658+0000","last_change":"2024-02-26T06:20:31.587862+0000","last_active":"2024-02-26T06:21:57.421658+0000","last_peered":"2024-02-26T06:21:57.421658+0000","last_clean":"2024-02-26T06:21:57.421658+0000","last_became_active":"2024-02-26T06:20:31.587551+0000","last_became_peered":"2024-02-26T06:20:31.587551+0000","last_unstale":"2024-02-26T06:21:57.421658+0000","last_undegraded":"2024-02-26T06:21:57.421658+0000","last_fullsized":"2024-02-26T06:21:57.421658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:07:15.933125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902570+0000","last_change":"2024-02-26T06:20:31.058378+0000","last_active":"2024-02-26T06:21:28.902570+0000","last_peered":"2024-02-26T06:21:28.902570+0000","last_clean":"2024-02-26T06:21:28.902570+0000","last_became_active":"2024-02-26T06:20:31.058040+0000","last_became_peered":"2024-02-26T06:20:31.058040+0000","last_unstale":"2024-02-26T06:21:28.902570+0000","last_undegraded":"2024-02-26T06:21:28.902570+0000","last_fullsized":"2024-02-26T06:21:28.902570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:33:40.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440318+0000","last_change":"2024-02-26T06:20:31.601837+0000","last_active":"2024-02-26T06:21:57.440318+0000","last_peered":"2024-02-26T06:21:57.440318+0000","last_clean":"2024-02-26T06:21:57.440318+0000","last_became_active":"2024-02-26T06:20:31.597310+0000","last_became_peered":"2024-02-26T06:20:31.597310+0000","last_unstale":"2024-02-26T06:21:57.440318+0000","last_undegraded":"2024-02-26T06:21:57.440318+0000","last_fullsized":"2024-02-26T06:21:57.440318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.383510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.915022+0000","last_change":"2024-02-26T06:20:31.076158+0000","last_active":"2024-02-26T06:21:28.915022+0000","last_peered":"2024-02-26T06:21:28.915022+0000","last_clean":"2024-02-26T06:21:28.915022+0000","last_became_active":"2024-02-26T06:20:31.075795+0000","last_became_peered":"2024-02-26T06:20:31.075795+0000","last_unstale":"2024-02-26T06:21:28.915022+0000","last_undegraded":"2024-02-26T06:21:28.915022+0000","last_fullsized":"2024-02-26T06:21:28.915022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:38.491674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421150+0000","last_change":"2024-02-26T06:20:31.586122+0000","last_active":"2024-02-26T06:21:57.421150+0000","last_peered":"2024-02-26T06:21:57.421150+0000","last_clean":"2024-02-26T06:21:57.421150+0000","last_became_active":"2024-02-26T06:20:31.585070+0000","last_became_peered":"2024-02-26T06:20:31.585070+0000","last_unstale":"2024-02-26T06:21:57.421150+0000","last_undegraded":"2024-02-26T06:21:57.421150+0000","last_fullsized":"2024-02-26T06:21:57.421150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:11.034252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":383,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:07.072497+0000","last_change":"2024-02-26T06:21:28.913681+0000","last_active":"2024-02-26T06:23:07.072497+0000","last_peered":"2024-02-26T06:23:07.072497+0000","last_clean":"2024-02-26T06:23:07.072497+0000","last_became_active":"2024-02-26T06:21:28.905006+0000","last_became_peered":"2024-02-26T06:21:28.905006+0000","last_unstale":"2024-02-26T06:23:07.072497+0000","last_undegraded":"2024-02-26T06:23:07.072497+0000","last_fullsized":"2024-02-26T06:23:07.072497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:41.432983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66805,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":308,"num_read_kb":219,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60873,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410911+0000","last_change":"2024-02-26T06:20:31.584757+0000","last_active":"2024-02-26T06:21:57.410911+0000","last_peered":"2024-02-26T06:21:57.410911+0000","last_clean":"2024-02-26T06:21:57.410911+0000","last_became_active":"2024-02-26T06:20:31.583890+0000","last_became_peered":"2024-02-26T06:20:31.583890+0000","last_unstale":"2024-02-26T06:21:57.410911+0000","last_undegraded":"2024-02-26T06:21:57.410911+0000","last_fullsized":"2024-02-26T06:21:57.410911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:10:58.280082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903296+0000","last_change":"2024-02-26T06:20:31.076044+0000","last_active":"2024-02-26T06:21:28.903296+0000","last_peered":"2024-02-26T06:21:28.903296+0000","last_clean":"2024-02-26T06:21:28.903296+0000","last_became_active":"2024-02-26T06:20:31.075744+0000","last_became_peered":"2024-02-26T06:20:31.075744+0000","last_unstale":"2024-02-26T06:21:28.903296+0000","last_undegraded":"2024-02-26T06:21:28.903296+0000","last_fullsized":"2024-02-26T06:21:28.903296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:35.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.443819+0000","last_change":"2024-02-26T06:20:31.602169+0000","last_active":"2024-02-26T06:21:57.443819+0000","last_peered":"2024-02-26T06:21:57.443819+0000","last_clean":"2024-02-26T06:21:57.443819+0000","last_became_active":"2024-02-26T06:20:31.601280+0000","last_became_peered":"2024-02-26T06:20:31.601280+0000","last_unstale":"2024-02-26T06:21:57.443819+0000","last_undegraded":"2024-02-26T06:21:57.443819+0000","last_fullsized":"2024-02-26T06:21:57.443819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:07.652549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914144+0000","last_change":"2024-02-26T06:20:31.080482+0000","last_active":"2024-02-26T06:21:28.914144+0000","last_peered":"2024-02-26T06:21:28.914144+0000","last_clean":"2024-02-26T06:21:28.914144+0000","last_became_active":"2024-02-26T06:20:31.080188+0000","last_became_peered":"2024-02-26T06:20:31.080188+0000","last_unstale":"2024-02-26T06:21:28.914144+0000","last_undegraded":"2024-02-26T06:21:28.914144+0000","last_fullsized":"2024-02-26T06:21:28.914144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:32.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.433538+0000","last_change":"2024-02-26T06:20:31.599753+0000","last_active":"2024-02-26T06:21:57.433538+0000","last_peered":"2024-02-26T06:21:57.433538+0000","last_clean":"2024-02-26T06:21:57.433538+0000","last_became_active":"2024-02-26T06:20:31.599409+0000","last_became_peered":"2024-02-26T06:20:31.599409+0000","last_unstale":"2024-02-26T06:21:57.433538+0000","last_undegraded":"2024-02-26T06:21:57.433538+0000","last_fullsized":"2024-02-26T06:21:57.433538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:43.887431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:07.420991+0000","last_change":"2024-02-26T06:20:31.059656+0000","last_active":"2024-02-26T06:23:07.420991+0000","last_peered":"2024-02-26T06:23:07.420991+0000","last_clean":"2024-02-26T06:23:07.420991+0000","last_became_active":"2024-02-26T06:20:31.059289+0000","last_became_peered":"2024-02-26T06:20:31.059289+0000","last_unstale":"2024-02-26T06:23:07.420991+0000","last_undegraded":"2024-02-26T06:23:07.420991+0000","last_fullsized":"2024-02-26T06:23:07.420991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:08:57.650876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.404146+0000","last_change":"2024-02-26T06:21:28.907778+0000","last_active":"2024-02-26T06:21:57.404146+0000","last_peered":"2024-02-26T06:21:57.404146+0000","last_clean":"2024-02-26T06:21:57.404146+0000","last_became_active":"2024-02-26T06:21:28.907185+0000","last_became_peered":"2024-02-26T06:21:28.907185+0000","last_unstale":"2024-02-26T06:21:57.404146+0000","last_undegraded":"2024-02-26T06:21:57.404146+0000","last_fullsized":"2024-02-26T06:21:57.404146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:39.572867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.943152+0000","last_change":"2024-02-26T06:21:28.943152+0000","last_active":"2024-02-26T06:21:28.943152+0000","last_peered":"2024-02-26T06:21:28.943152+0000","last_clean":"2024-02-26T06:21:28.943152+0000","last_became_active":"2024-02-26T06:21:28.908090+0000","last_became_peered":"2024-02-26T06:21:28.908090+0000","last_unstale":"2024-02-26T06:21:28.943152+0000","last_undegraded":"2024-02-26T06:21:28.943152+0000","last_fullsized":"2024-02-26T06:21:28.943152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:56.254015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.417526+0000","last_change":"2024-02-26T06:20:31.586151+0000","last_active":"2024-02-26T06:21:57.417526+0000","last_peered":"2024-02-26T06:21:57.417526+0000","last_clean":"2024-02-26T06:21:57.417526+0000","last_became_active":"2024-02-26T06:20:31.585615+0000","last_became_peered":"2024-02-26T06:20:31.585615+0000","last_unstale":"2024-02-26T06:21:57.417526+0000","last_undegraded":"2024-02-26T06:21:57.417526+0000","last_fullsized":"2024-02-26T06:21:57.417526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907609+0000","last_change":"2024-02-26T06:20:31.062340+0000","last_active":"2024-02-26T06:21:28.907609+0000","last_peered":"2024-02-26T06:21:28.907609+0000","last_clean":"2024-02-26T06:21:28.907609+0000","last_became_active":"2024-02-26T06:20:31.062023+0000","last_became_peered":"2024-02-26T06:20:31.062023+0000","last_unstale":"2024-02-26T06:21:28.907609+0000","last_undegraded":"2024-02-26T06:21:28.907609+0000","last_fullsized":"2024-02-26T06:21:28.907609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:45.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413574+0000","last_change":"2024-02-26T06:20:31.585190+0000","last_active":"2024-02-26T06:21:57.413574+0000","last_peered":"2024-02-26T06:21:57.413574+0000","last_clean":"2024-02-26T06:21:57.413574+0000","last_became_active":"2024-02-26T06:20:31.584883+0000","last_became_peered":"2024-02-26T06:20:31.584883+0000","last_unstale":"2024-02-26T06:21:57.413574+0000","last_undegraded":"2024-02-26T06:21:57.413574+0000","last_fullsized":"2024-02-26T06:21:57.413574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:47:51.843824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904061+0000","last_change":"2024-02-26T06:20:31.075573+0000","last_active":"2024-02-26T06:21:28.904061+0000","last_peered":"2024-02-26T06:21:28.904061+0000","last_clean":"2024-02-26T06:21:28.904061+0000","last_became_active":"2024-02-26T06:20:31.075153+0000","last_became_peered":"2024-02-26T06:20:31.075153+0000","last_unstale":"2024-02-26T06:21:28.904061+0000","last_undegraded":"2024-02-26T06:21:28.904061+0000","last_fullsized":"2024-02-26T06:21:28.904061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:43.886858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432306+0000","last_change":"2024-02-26T06:20:31.604400+0000","last_active":"2024-02-26T06:21:57.432306+0000","last_peered":"2024-02-26T06:21:57.432306+0000","last_clean":"2024-02-26T06:21:57.432306+0000","last_became_active":"2024-02-26T06:20:31.604068+0000","last_became_peered":"2024-02-26T06:20:31.604068+0000","last_unstale":"2024-02-26T06:21:57.432306+0000","last_undegraded":"2024-02-26T06:21:57.432306+0000","last_fullsized":"2024-02-26T06:21:57.432306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:28.792297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902928+0000","last_change":"2024-02-26T06:20:31.072896+0000","last_active":"2024-02-26T06:21:28.902928+0000","last_peered":"2024-02-26T06:21:28.902928+0000","last_clean":"2024-02-26T06:21:28.902928+0000","last_became_active":"2024-02-26T06:20:31.072571+0000","last_became_peered":"2024-02-26T06:20:31.072571+0000","last_unstale":"2024-02-26T06:21:28.902928+0000","last_undegraded":"2024-02-26T06:21:28.902928+0000","last_fullsized":"2024-02-26T06:21:28.902928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:39.054737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.415995+0000","last_change":"2024-02-26T06:21:28.906840+0000","last_active":"2024-02-26T06:21:57.415995+0000","last_peered":"2024-02-26T06:21:57.415995+0000","last_clean":"2024-02-26T06:21:57.415995+0000","last_became_active":"2024-02-26T06:21:28.905881+0000","last_became_peered":"2024-02-26T06:21:28.905881+0000","last_unstale":"2024-02-26T06:21:57.415995+0000","last_undegraded":"2024-02-26T06:21:57.415995+0000","last_fullsized":"2024-02-26T06:21:57.415995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:47.754145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.411939+0000","last_change":"2024-02-26T06:20:31.588989+0000","last_active":"2024-02-26T06:21:57.411939+0000","last_peered":"2024-02-26T06:21:57.411939+0000","last_clean":"2024-02-26T06:21:57.411939+0000","last_became_active":"2024-02-26T06:20:31.588683+0000","last_became_peered":"2024-02-26T06:20:31.588683+0000","last_unstale":"2024-02-26T06:21:57.411939+0000","last_undegraded":"2024-02-26T06:21:57.411939+0000","last_fullsized":"2024-02-26T06:21:57.411939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:29.687622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":175,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:06.418562+0000","last_change":"2024-02-26T06:20:31.081111+0000","last_active":"2024-02-26T06:23:06.418562+0000","last_peered":"2024-02-26T06:23:06.418562+0000","last_clean":"2024-02-26T06:23:06.418562+0000","last_became_active":"2024-02-26T06:20:31.080788+0000","last_became_peered":"2024-02-26T06:20:31.080788+0000","last_unstale":"2024-02-26T06:23:06.418562+0000","last_undegraded":"2024-02-26T06:23:06.418562+0000","last_fullsized":"2024-02-26T06:23:06.418562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:44:34.400256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440841+0000","last_change":"2024-02-26T06:21:29.211650+0000","last_active":"2024-02-26T06:21:57.440841+0000","last_peered":"2024-02-26T06:21:57.440841+0000","last_clean":"2024-02-26T06:21:57.440841+0000","last_became_active":"2024-02-26T06:21:29.210407+0000","last_became_peered":"2024-02-26T06:21:29.210407+0000","last_unstale":"2024-02-26T06:21:57.440841+0000","last_undegraded":"2024-02-26T06:21:57.440841+0000","last_fullsized":"2024-02-26T06:21:57.440841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:22:00.359506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898366+0000","last_change":"2024-02-26T06:20:31.061702+0000","last_active":"2024-02-26T06:21:28.898366+0000","last_peered":"2024-02-26T06:21:28.898366+0000","last_clean":"2024-02-26T06:21:28.898366+0000","last_became_active":"2024-02-26T06:20:31.061358+0000","last_became_peered":"2024-02-26T06:20:31.061358+0000","last_unstale":"2024-02-26T06:21:28.898366+0000","last_undegraded":"2024-02-26T06:21:28.898366+0000","last_fullsized":"2024-02-26T06:21:28.898366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:25:41.652032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903059+0000","last_change":"2024-02-26T06:19:10.834318+0000","last_active":"2024-02-26T06:21:28.903059+0000","last_peered":"2024-02-26T06:21:28.903059+0000","last_clean":"2024-02-26T06:21:28.903059+0000","last_became_active":"2024-02-26T06:19:10.833869+0000","last_became_peered":"2024-02-26T06:19:10.833869+0000","last_unstale":"2024-02-26T06:21:28.903059+0000","last_undegraded":"2024-02-26T06:21:28.903059+0000","last_fullsized":"2024-02-26T06:21:28.903059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_clean_scrub_stamp":"2024-02-26T06:15:16.238016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:57.078533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428655+0000","last_change":"2024-02-26T06:20:31.600738+0000","last_active":"2024-02-26T06:21:57.428655+0000","last_peered":"2024-02-26T06:21:57.428655+0000","last_clean":"2024-02-26T06:21:57.428655+0000","last_became_active":"2024-02-26T06:20:31.599144+0000","last_became_peered":"2024-02-26T06:20:31.599144+0000","last_unstale":"2024-02-26T06:21:57.428655+0000","last_undegraded":"2024-02-26T06:21:57.428655+0000","last_fullsized":"2024-02-26T06:21:57.428655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:01:16.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903180+0000","last_change":"2024-02-26T06:20:31.063563+0000","last_active":"2024-02-26T06:21:28.903180+0000","last_peered":"2024-02-26T06:21:28.903180+0000","last_clean":"2024-02-26T06:21:28.903180+0000","last_became_active":"2024-02-26T06:20:31.063212+0000","last_became_peered":"2024-02-26T06:20:31.063212+0000","last_unstale":"2024-02-26T06:21:28.903180+0000","last_undegraded":"2024-02-26T06:21:28.903180+0000","last_fullsized":"2024-02-26T06:21:28.903180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:16.480553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907970+0000","last_change":"2024-02-26T06:20:31.075104+0000","last_active":"2024-02-26T06:21:28.907970+0000","last_peered":"2024-02-26T06:21:28.907970+0000","last_clean":"2024-02-26T06:21:28.907970+0000","last_became_active":"2024-02-26T06:20:31.074777+0000","last_became_peered":"2024-02-26T06:20:31.074777+0000","last_unstale":"2024-02-26T06:21:28.907970+0000","last_undegraded":"2024-02-26T06:21:28.907970+0000","last_fullsized":"2024-02-26T06:21:28.907970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:32:20.511703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.417127+0000","last_change":"2024-02-26T06:20:31.605038+0000","last_active":"2024-02-26T06:21:57.417127+0000","last_peered":"2024-02-26T06:21:57.417127+0000","last_clean":"2024-02-26T06:21:57.417127+0000","last_became_active":"2024-02-26T06:20:31.603329+0000","last_became_peered":"2024-02-26T06:20:31.603329+0000","last_unstale":"2024-02-26T06:21:57.417127+0000","last_undegraded":"2024-02-26T06:21:57.417127+0000","last_fullsized":"2024-02-26T06:21:57.417127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:45.066836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:41.014176+0000","last_change":"2024-02-26T06:20:31.084613+0000","last_active":"2024-02-26T06:22:41.014176+0000","last_peered":"2024-02-26T06:22:41.014176+0000","last_clean":"2024-02-26T06:22:41.014176+0000","last_became_active":"2024-02-26T06:20:31.084021+0000","last_became_peered":"2024-02-26T06:20:31.084021+0000","last_unstale":"2024-02-26T06:22:41.014176+0000","last_undegraded":"2024-02-26T06:22:41.014176+0000","last_fullsized":"2024-02-26T06:22:41.014176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:52:42.096437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438954+0000","last_change":"2024-02-26T06:20:31.590367+0000","last_active":"2024-02-26T06:21:57.438954+0000","last_peered":"2024-02-26T06:21:57.438954+0000","last_clean":"2024-02-26T06:21:57.438954+0000","last_became_active":"2024-02-26T06:20:31.582914+0000","last_became_peered":"2024-02-26T06:20:31.582914+0000","last_unstale":"2024-02-26T06:21:57.438954+0000","last_undegraded":"2024-02-26T06:21:57.438954+0000","last_fullsized":"2024-02-26T06:21:57.438954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:02:47.496527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898485+0000","last_change":"2024-02-26T06:20:31.070190+0000","last_active":"2024-02-26T06:21:28.898485+0000","last_peered":"2024-02-26T06:21:28.898485+0000","last_clean":"2024-02-26T06:21:28.898485+0000","last_became_active":"2024-02-26T06:20:31.069854+0000","last_became_peered":"2024-02-26T06:20:31.069854+0000","last_unstale":"2024-02-26T06:21:28.898485+0000","last_undegraded":"2024-02-26T06:21:28.898485+0000","last_fullsized":"2024-02-26T06:21:28.898485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:57.728821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.442447+0000","last_change":"2024-02-26T06:20:31.595388+0000","last_active":"2024-02-26T06:21:57.442447+0000","last_peered":"2024-02-26T06:21:57.442447+0000","last_clean":"2024-02-26T06:21:57.442447+0000","last_became_active":"2024-02-26T06:20:31.594970+0000","last_became_peered":"2024-02-26T06:20:31.594970+0000","last_unstale":"2024-02-26T06:21:57.442447+0000","last_undegraded":"2024-02-26T06:21:57.442447+0000","last_fullsized":"2024-02-26T06:21:57.442447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:12:52.182988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902880+0000","last_change":"2024-02-26T06:20:31.066256+0000","last_active":"2024-02-26T06:21:28.902880+0000","last_peered":"2024-02-26T06:21:28.902880+0000","last_clean":"2024-02-26T06:21:28.902880+0000","last_became_active":"2024-02-26T06:20:31.065956+0000","last_became_peered":"2024-02-26T06:20:31.065956+0000","last_unstale":"2024-02-26T06:21:28.902880+0000","last_undegraded":"2024-02-26T06:21:28.902880+0000","last_fullsized":"2024-02-26T06:21:28.902880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:35.405114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.407926+0000","last_change":"2024-02-26T06:20:31.585102+0000","last_active":"2024-02-26T06:21:57.407926+0000","last_peered":"2024-02-26T06:21:57.407926+0000","last_clean":"2024-02-26T06:21:57.407926+0000","last_became_active":"2024-02-26T06:20:31.584351+0000","last_became_peered":"2024-02-26T06:20:31.584351+0000","last_unstale":"2024-02-26T06:21:57.407926+0000","last_undegraded":"2024-02-26T06:21:57.407926+0000","last_fullsized":"2024-02-26T06:21:57.407926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:09.708752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.897704+0000","last_change":"2024-02-26T06:20:31.079664+0000","last_active":"2024-02-26T06:21:28.897704+0000","last_peered":"2024-02-26T06:21:28.897704+0000","last_clean":"2024-02-26T06:21:28.897704+0000","last_became_active":"2024-02-26T06:20:31.079234+0000","last_became_peered":"2024-02-26T06:20:31.079234+0000","last_unstale":"2024-02-26T06:21:28.897704+0000","last_undegraded":"2024-02-26T06:21:28.897704+0000","last_fullsized":"2024-02-26T06:21:28.897704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:37.407215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439613+0000","last_change":"2024-02-26T06:20:31.597416+0000","last_active":"2024-02-26T06:21:57.439613+0000","last_peered":"2024-02-26T06:21:57.439613+0000","last_clean":"2024-02-26T06:21:57.439613+0000","last_became_active":"2024-02-26T06:20:31.597094+0000","last_became_peered":"2024-02-26T06:20:31.597094+0000","last_unstale":"2024-02-26T06:21:57.439613+0000","last_undegraded":"2024-02-26T06:21:57.439613+0000","last_fullsized":"2024-02-26T06:21:57.439613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:24.774298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903155+0000","last_change":"2024-02-26T06:20:31.074372+0000","last_active":"2024-02-26T06:21:28.903155+0000","last_peered":"2024-02-26T06:21:28.903155+0000","last_clean":"2024-02-26T06:21:28.903155+0000","last_became_active":"2024-02-26T06:20:31.074041+0000","last_became_peered":"2024-02-26T06:20:31.074041+0000","last_unstale":"2024-02-26T06:21:28.903155+0000","last_undegraded":"2024-02-26T06:21:28.903155+0000","last_fullsized":"2024-02-26T06:21:28.903155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:01.367668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441913+0000","last_change":"2024-02-26T06:20:31.599588+0000","last_active":"2024-02-26T06:21:57.441913+0000","last_peered":"2024-02-26T06:21:57.441913+0000","last_clean":"2024-02-26T06:21:57.441913+0000","last_became_active":"2024-02-26T06:20:31.599240+0000","last_became_peered":"2024-02-26T06:20:31.599240+0000","last_unstale":"2024-02-26T06:21:57.441913+0000","last_undegraded":"2024-02-26T06:21:57.441913+0000","last_fullsized":"2024-02-26T06:21:57.441913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:07:14.799565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914009+0000","last_change":"2024-02-26T06:21:28.914009+0000","last_active":"2024-02-26T06:21:28.914009+0000","last_peered":"2024-02-26T06:21:28.914009+0000","last_clean":"2024-02-26T06:21:28.914009+0000","last_became_active":"2024-02-26T06:21:28.906712+0000","last_became_peered":"2024-02-26T06:21:28.906712+0000","last_unstale":"2024-02-26T06:21:28.914009+0000","last_undegraded":"2024-02-26T06:21:28.914009+0000","last_fullsized":"2024-02-26T06:21:28.914009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:21:36.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432870+0000","last_change":"2024-02-26T06:20:31.603955+0000","last_active":"2024-02-26T06:21:57.432870+0000","last_peered":"2024-02-26T06:21:57.432870+0000","last_clean":"2024-02-26T06:21:57.432870+0000","last_became_active":"2024-02-26T06:20:31.603635+0000","last_became_peered":"2024-02-26T06:20:31.603635+0000","last_unstale":"2024-02-26T06:21:57.432870+0000","last_undegraded":"2024-02-26T06:21:57.432870+0000","last_fullsized":"2024-02-26T06:21:57.432870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:56:59.060204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903796+0000","last_change":"2024-02-26T06:20:31.074917+0000","last_active":"2024-02-26T06:21:28.903796+0000","last_peered":"2024-02-26T06:21:28.903796+0000","last_clean":"2024-02-26T06:21:28.903796+0000","last_became_active":"2024-02-26T06:20:31.074590+0000","last_became_peered":"2024-02-26T06:20:31.074590+0000","last_unstale":"2024-02-26T06:21:28.903796+0000","last_undegraded":"2024-02-26T06:21:28.903796+0000","last_fullsized":"2024-02-26T06:21:28.903796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:43.160268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420847+0000","last_change":"2024-02-26T06:20:31.582236+0000","last_active":"2024-02-26T06:21:57.420847+0000","last_peered":"2024-02-26T06:21:57.420847+0000","last_clean":"2024-02-26T06:21:57.420847+0000","last_became_active":"2024-02-26T06:20:31.581851+0000","last_became_peered":"2024-02-26T06:20:31.581851+0000","last_unstale":"2024-02-26T06:21:57.420847+0000","last_undegraded":"2024-02-26T06:21:57.420847+0000","last_fullsized":"2024-02-26T06:21:57.420847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:53.214338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914195+0000","last_change":"2024-02-26T06:20:31.068179+0000","last_active":"2024-02-26T06:21:28.914195+0000","last_peered":"2024-02-26T06:21:28.914195+0000","last_clean":"2024-02-26T06:21:28.914195+0000","last_became_active":"2024-02-26T06:20:31.067888+0000","last_became_peered":"2024-02-26T06:20:31.067888+0000","last_unstale":"2024-02-26T06:21:28.914195+0000","last_undegraded":"2024-02-26T06:21:28.914195+0000","last_fullsized":"2024-02-26T06:21:28.914195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:47:39.223749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441130+0000","last_change":"2024-02-26T06:20:31.598820+0000","last_active":"2024-02-26T06:21:57.441130+0000","last_peered":"2024-02-26T06:21:57.441130+0000","last_clean":"2024-02-26T06:21:57.441130+0000","last_became_active":"2024-02-26T06:20:31.598469+0000","last_became_peered":"2024-02-26T06:20:31.598469+0000","last_unstale":"2024-02-26T06:21:57.441130+0000","last_undegraded":"2024-02-26T06:21:57.441130+0000","last_fullsized":"2024-02-26T06:21:57.441130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:41:15.713748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.207193+0000","last_change":"2024-02-26T06:20:31.080988+0000","last_active":"2024-02-26T06:21:29.207193+0000","last_peered":"2024-02-26T06:21:29.207193+0000","last_clean":"2024-02-26T06:21:29.207193+0000","last_became_active":"2024-02-26T06:20:31.080654+0000","last_became_peered":"2024-02-26T06:20:31.080654+0000","last_unstale":"2024-02-26T06:21:29.207193+0000","last_undegraded":"2024-02-26T06:21:29.207193+0000","last_fullsized":"2024-02-26T06:21:29.207193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:50.315162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414613+0000","last_change":"2024-02-26T06:20:31.586057+0000","last_active":"2024-02-26T06:21:57.414613+0000","last_peered":"2024-02-26T06:21:57.414613+0000","last_clean":"2024-02-26T06:21:57.414613+0000","last_became_active":"2024-02-26T06:20:31.585756+0000","last_became_peered":"2024-02-26T06:20:31.585756+0000","last_unstale":"2024-02-26T06:21:57.414613+0000","last_undegraded":"2024-02-26T06:21:57.414613+0000","last_fullsized":"2024-02-26T06:21:57.414613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:59:07.032287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413388+0000","last_change":"2024-02-26T06:20:31.589067+0000","last_active":"2024-02-26T06:21:57.413388+0000","last_peered":"2024-02-26T06:21:57.413388+0000","last_clean":"2024-02-26T06:21:57.413388+0000","last_became_active":"2024-02-26T06:20:31.588769+0000","last_became_peered":"2024-02-26T06:20:31.588769+0000","last_unstale":"2024-02-26T06:21:57.413388+0000","last_undegraded":"2024-02-26T06:21:57.413388+0000","last_fullsized":"2024-02-26T06:21:57.413388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:43.211127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412546+0000","last_change":"2024-02-26T06:20:31.583838+0000","last_active":"2024-02-26T06:21:57.412546+0000","last_peered":"2024-02-26T06:21:57.412546+0000","last_clean":"2024-02-26T06:21:57.412546+0000","last_became_active":"2024-02-26T06:20:31.583524+0000","last_became_peered":"2024-02-26T06:20:31.583524+0000","last_unstale":"2024-02-26T06:21:57.412546+0000","last_undegraded":"2024-02-26T06:21:57.412546+0000","last_fullsized":"2024-02-26T06:21:57.412546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:21.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414078+0000","last_change":"2024-02-26T06:20:31.597901+0000","last_active":"2024-02-26T06:21:57.414078+0000","last_peered":"2024-02-26T06:21:57.414078+0000","last_clean":"2024-02-26T06:21:57.414078+0000","last_became_active":"2024-02-26T06:20:31.597633+0000","last_became_peered":"2024-02-26T06:20:31.597633+0000","last_unstale":"2024-02-26T06:21:57.414078+0000","last_undegraded":"2024-02-26T06:21:57.414078+0000","last_fullsized":"2024-02-26T06:21:57.414078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:14.084579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438958+0000","last_change":"2024-02-26T06:21:29.210694+0000","last_active":"2024-02-26T06:21:57.438958+0000","last_peered":"2024-02-26T06:21:57.438958+0000","last_clean":"2024-02-26T06:21:57.438958+0000","last_became_active":"2024-02-26T06:21:29.209421+0000","last_became_peered":"2024-02-26T06:21:29.209421+0000","last_unstale":"2024-02-26T06:21:57.438958+0000","last_undegraded":"2024-02-26T06:21:57.438958+0000","last_fullsized":"2024-02-26T06:21:57.438958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:46.511066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.430929+0000","last_change":"2024-02-26T06:20:31.605866+0000","last_active":"2024-02-26T06:21:57.430929+0000","last_peered":"2024-02-26T06:21:57.430929+0000","last_clean":"2024-02-26T06:21:57.430929+0000","last_became_active":"2024-02-26T06:20:31.605404+0000","last_became_peered":"2024-02-26T06:20:31.605404+0000","last_unstale":"2024-02-26T06:21:57.430929+0000","last_undegraded":"2024-02-26T06:21:57.430929+0000","last_fullsized":"2024-02-26T06:21:57.430929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:03:25.055462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413749+0000","last_change":"2024-02-26T06:20:31.585214+0000","last_active":"2024-02-26T06:21:57.413749+0000","last_peered":"2024-02-26T06:21:57.413749+0000","last_clean":"2024-02-26T06:21:57.413749+0000","last_became_active":"2024-02-26T06:20:31.584895+0000","last_became_peered":"2024-02-26T06:20:31.584895+0000","last_unstale":"2024-02-26T06:21:57.413749+0000","last_undegraded":"2024-02-26T06:21:57.413749+0000","last_fullsized":"2024-02-26T06:21:57.413749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:38.507499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.415101+0000","last_change":"2024-02-26T06:20:31.582189+0000","last_active":"2024-02-26T06:21:57.415101+0000","last_peered":"2024-02-26T06:21:57.415101+0000","last_clean":"2024-02-26T06:21:57.415101+0000","last_became_active":"2024-02-26T06:20:31.580729+0000","last_became_peered":"2024-02-26T06:20:31.580729+0000","last_unstale":"2024-02-26T06:21:57.415101+0000","last_undegraded":"2024-02-26T06:21:57.415101+0000","last_fullsized":"2024-02-26T06:21:57.415101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:21.337261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441676+0000","last_change":"2024-02-26T06:20:31.588894+0000","last_active":"2024-02-26T06:21:57.441676+0000","last_peered":"2024-02-26T06:21:57.441676+0000","last_clean":"2024-02-26T06:21:57.441676+0000","last_became_active":"2024-02-26T06:20:31.586362+0000","last_became_peered":"2024-02-26T06:20:31.586362+0000","last_unstale":"2024-02-26T06:21:57.441676+0000","last_undegraded":"2024-02-26T06:21:57.441676+0000","last_fullsized":"2024-02-26T06:21:57.441676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:28:30.009901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439174+0000","last_change":"2024-02-26T06:20:31.608264+0000","last_active":"2024-02-26T06:21:57.439174+0000","last_peered":"2024-02-26T06:21:57.439174+0000","last_clean":"2024-02-26T06:21:57.439174+0000","last_became_active":"2024-02-26T06:20:31.607855+0000","last_became_peered":"2024-02-26T06:20:31.607855+0000","last_unstale":"2024-02-26T06:21:57.439174+0000","last_undegraded":"2024-02-26T06:21:57.439174+0000","last_fullsized":"2024-02-26T06:21:57.439174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:20.169489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441839+0000","last_change":"2024-02-26T06:20:31.591000+0000","last_active":"2024-02-26T06:21:57.441839+0000","last_peered":"2024-02-26T06:21:57.441839+0000","last_clean":"2024-02-26T06:21:57.441839+0000","last_became_active":"2024-02-26T06:20:31.588201+0000","last_became_peered":"2024-02-26T06:20:31.588201+0000","last_unstale":"2024-02-26T06:21:57.441839+0000","last_undegraded":"2024-02-26T06:21:57.441839+0000","last_fullsized":"2024-02-26T06:21:57.441839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:47:59.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441280+0000","last_change":"2024-02-26T06:20:31.590398+0000","last_active":"2024-02-26T06:21:57.441280+0000","last_peered":"2024-02-26T06:21:57.441280+0000","last_clean":"2024-02-26T06:21:57.441280+0000","last_became_active":"2024-02-26T06:20:31.590042+0000","last_became_peered":"2024-02-26T06:20:31.590042+0000","last_unstale":"2024-02-26T06:21:57.441280+0000","last_undegraded":"2024-02-26T06:21:57.441280+0000","last_fullsized":"2024-02-26T06:21:57.441280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:40:01.233978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440874+0000","last_change":"2024-02-26T06:20:31.611698+0000","last_active":"2024-02-26T06:21:57.440874+0000","last_peered":"2024-02-26T06:21:57.440874+0000","last_clean":"2024-02-26T06:21:57.440874+0000","last_became_active":"2024-02-26T06:20:31.610979+0000","last_became_peered":"2024-02-26T06:20:31.610979+0000","last_unstale":"2024-02-26T06:21:57.440874+0000","last_undegraded":"2024-02-26T06:21:57.440874+0000","last_fullsized":"2024-02-26T06:21:57.440874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:44.604373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414255+0000","last_change":"2024-02-26T06:20:31.582016+0000","last_active":"2024-02-26T06:21:57.414255+0000","last_peered":"2024-02-26T06:21:57.414255+0000","last_clean":"2024-02-26T06:21:57.414255+0000","last_became_active":"2024-02-26T06:20:31.581689+0000","last_became_peered":"2024-02-26T06:20:31.581689+0000","last_unstale":"2024-02-26T06:21:57.414255+0000","last_undegraded":"2024-02-26T06:21:57.414255+0000","last_fullsized":"2024-02-26T06:21:57.414255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:04:11.030182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284221,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71257,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":926,"num_read_kb":536,"num_write":171,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":60985,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":213771,"data_compressed":13382,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351022,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459479,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94999999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.94599999999999995,"5min":1.018,"15min":1.018},"last":0.85699999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.61099999999999999,"15min":0.58299999999999996},"min":{"1min":0.53600000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70799999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46800000000000003,"15min":0.44700000000000001},"min":{"1min":0.40500000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.63,"15min":0.59599999999999997},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.871},{"interface":"front","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77600000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76900000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77200000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.95099999999999996,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.81100000000000005,"15min":0.83499999999999996},"min":{"1min":0.58999999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.179,"15min":1.179},"last":1.1000000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94899999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93400000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.82699999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.499,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.79500000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.91000000000000003,"15min":0.89400000000000002},"min":{"1min":0.61299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.2389999999999999,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":0.92200000000000004},"min":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.997}]},{"osd":10,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.623,"5min":0.501,"15min":0.501},"max":{"1min":0.92100000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58799999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.91500000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.84499999999999997}]}]},{"osd":10,"up_from":73,"seq":313532612659,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1118185,"data_compressed":43797,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77700000000000002,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88,"5min":1.069,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.075,"5min":6.2229999999999999,"15min":6.2229999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.45100000000000001,"15min":0.45700000000000002},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58099999999999996,"5min":0.624,"15min":0.624},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.28899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55700000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.434}]},{"osd":3,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80500000000000005,"15min":0.82999999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0129999999999999,"5min":1.403,"15min":1.403},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.57299999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66600000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.498}]},{"osd":5,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.79000000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.378},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.54000000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82199999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.85099999999999998,"15min":0.879},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0669999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91700000000000004,"15min":0.93200000000000005},"min":{"1min":0.63900000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.218,"5min":1.284,"15min":1.284},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:22:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93100000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88800000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.97199999999999998,"15min":0.98599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2190000000000001,"5min":1.266,"15min":1.266},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.63}]},{"osd":11,"last update":"Mon Feb 26 06:22:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.752,"15min":0.752},"min":{"1min":0.63600000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.94299999999999995,"5min":0.998,"15min":0.998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.58299999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.58799999999999997}]}]},{"osd":9,"up_from":68,"seq":292057776184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":459603,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.3500000000000001,"15min":1.448},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.9450000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.194,"5min":1.3100000000000001,"15min":1.3660000000000001},"min":{"1min":0.52800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.161,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2010000000000001,"15min":1.2649999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.704,"5min":1.923,"15min":1.923},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2809999999999999,"15min":1.377},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0030000000000001,"5min":2.407,"15min":2.407},"last":1.3}]},{"osd":2,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4179999999999999,"15min":1.5169999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.028,"5min":2.609,"15min":2.609},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.514,"15min":1.5920000000000001},"min":{"1min":0.72999999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.5219999999999998,"5min":3.681,"15min":3.681},"last":1.4330000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.704,"15min":1.8799999999999999},"min":{"1min":0.621,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.0659999999999998,"5min":3.016,"15min":3.016},"last":1.675},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.724,"15min":1.8080000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.3330000000000002,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.6180000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.7170000000000001,"15min":1.8100000000000001},"min":{"1min":0.78400000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.6019999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.6040000000000001,"15min":1.776},"min":{"1min":0.67500000000000004,"5min":0.624,"15min":0.624},"max":{"1min":2.1040000000000001,"5min":3.98,"15min":3.98},"last":1.798}]},{"osd":5,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":1.8049999999999999,"15min":1.9710000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.379,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":1.895},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.8360000000000001,"15min":1.968},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.3679999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.726}]},{"osd":6,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":2.085,"15min":2.2469999999999999},"min":{"1min":0.878,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6349999999999998,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.952},{"interface":"front","average":{"1min":1.8200000000000001,"5min":2.1179999999999999,"15min":2.2719999999999998},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.7480000000000002,"5min":3.835,"15min":3.835},"last":1.6000000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.8029999999999999,"5min":1.9970000000000001,"15min":2.169},"min":{"1min":0.85199999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.452,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":2.0920000000000001},{"interface":"front","average":{"1min":1.887,"5min":2.0489999999999999,"15min":2.1709999999999998},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.4929999999999999,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":1.8520000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.786,"5min":2.0920000000000001,"15min":2.2040000000000002},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":2.661,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":1.655},{"interface":"front","average":{"1min":1.8839999999999999,"5min":2.1259999999999999,"15min":2.3180000000000001},"min":{"1min":0.91100000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.5750000000000002,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":2.0049999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:14 2024","interfaces":[{"interface":"back","average":{"1min":2.1509999999999998,"5min":2.1859999999999999,"15min":2.266},"min":{"1min":1.3180000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.677,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":2.1589999999999998},{"interface":"front","average":{"1min":2.1200000000000001,"5min":2.0750000000000002,"15min":2.0950000000000002},"min":{"1min":1.091,"5min":1.087,"15min":1.087},"max":{"1min":2.7799999999999998,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":1.7709999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037823,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":459411,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.57299999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99299999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.55800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1299999999999999,"5min":1.198,"15min":1.198},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.46500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.46899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44},"min":{"1min":0.36099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65800000000000003,"5min":0.755,"15min":0.755},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.629,"5min":0.747,"15min":0.747},"last":0.70199999999999996}]},{"osd":3,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78600000000000003,"15min":0.77100000000000002},"min":{"1min":0.55900000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.22,"5min":1.379,"15min":1.379},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":0.86099999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.196,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.85999999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.60599999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2390000000000001,"5min":1.359,"15min":1.359},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.67200000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.1759999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.016}]},{"osd":7,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65100000000000002,"15min":0.64000000000000001},"min":{"1min":0.54400000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.75700000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88600000000000001,"15min":0.874},"min":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1439999999999999,"5min":1.28,"15min":1.28},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.93600000000000005},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.23,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.92500000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68500000000000005,"15min":0.66300000000000003},"min":{"1min":0.58899999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94899999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.72299999999999998,"15min":0.70099999999999996},"min":{"1min":0.61499999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.87}]},{"osd":11,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.85199999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.58499999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.89100000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051904,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40799999999999997,"15min":0.41999999999999998},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.624,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.434,"15min":0.436},"min":{"1min":0.31900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.35099999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76200000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65800000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":0.83299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1100000000000001,"5min":1.591,"15min":1.591},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.48899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93000000000000005,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.495,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.309},{"interface":"front","average":{"1min":0.499,"5min":0.47999999999999998,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67900000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42399999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.45600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.497},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60099999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":0.85599999999999998},"min":{"1min":0.58999999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.1060000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.93000000000000005,"15min":0.88400000000000001},"min":{"1min":0.5,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0429999999999999,"5min":7.4119999999999999,"15min":7.4119999999999999},"last":0.67100000000000004}]},{"osd":8,"last update":"Mon Feb 26 06:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.443},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.49099999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.83799999999999997,"5min":0.873,"15min":0.873},"last":0.59099999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.88,"15min":0.92100000000000004},"min":{"1min":0.57399999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.1599999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.748},{"interface":"front","average":{"1min":0.877,"5min":0.90800000000000003,"15min":0.94199999999999995},"min":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.89100000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63800000000000001,"15min":0.65700000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68799999999999994}]},{"osd":11,"last update":"Mon Feb 26 06:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.61799999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78500000000000003,"15min":0.78000000000000003},"min":{"1min":0.65800000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.57799999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462986,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":458992,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2689999999999999,"15min":1.2729999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.915,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.264,"5min":1.3180000000000001,"15min":1.2509999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0920000000000001,"15min":1.079},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1299999999999999,"15min":1.1060000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.855,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3520000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7829999999999999,"5min":6.4930000000000003,"15min":6.4930000000000003},"last":1}]},{"osd":3,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.645,"15min":1.5860000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.655,"15min":1.665},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4470000000000001,"15min":1.429},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.627,"15min":1.5900000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.649,"15min":1.625},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.4769999999999999,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.6950000000000001,"15min":1.7070000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.863,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":0.73199999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.7629999999999999,"15min":1.6200000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.4990000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.802,"15min":1.742},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.4350000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.76700000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:22:13 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.7969999999999999,"15min":1.7390000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.9139999999999999,"15min":1.837},"min":{"1min":0.86899999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.8050000000000002,"5min":6.625,"15min":6.625},"last":1.026}]},{"osd":9,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":2.0779999999999998,"15min":2.2400000000000002},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.536,"5min":3.258,"15min":3.258},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.907,"5min":2.133,"15min":2.2519999999999998},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8109999999999999,"5min":3.347,"15min":3.347},"last":1.1759999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:13 2024","interfaces":[{"interface":"back","average":{"1min":1.873,"5min":2.036,"15min":2.0790000000000002},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":1.073},{"interface":"front","average":{"1min":1.8089999999999999,"5min":2.1120000000000001,"15min":2.1789999999999998},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":1.202}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":525590,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.76300000000000001,"15min":0.69199999999999995},"min":{"1min":0.58499999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3069999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1,"15min":0.80500000000000005},"min":{"1min":0.58899999999999997,"5min":0.44,"15min":0.40400000000000003},"max":{"1min":1.0760000000000001,"5min":16.690999999999999,"15min":16.690999999999999},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.73199999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.72399999999999998,"5min":9.2629999999999999,"15min":9.2629999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.54800000000000004,"15min":0.55500000000000005},"min":{"1min":0.46899999999999997,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40500000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78600000000000003,"15min":0.749},"min":{"1min":0.55500000000000005,"5min":0.377,"15min":0.377},"max":{"1min":1.1279999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78300000000000003,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.073,"5min":1.391,"15min":1.391},"last":0.77700000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40500000000000003},"min":{"1min":0.374,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72199999999999998,"5min":0.748,"15min":0.748},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.443,"15min":0.40899999999999997},"min":{"1min":0.32800000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.65300000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28499999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.79600000000000004,"15min":0.66700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.032,"5min":16.672999999999998,"15min":16.672999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.625,"15min":0.60699999999999998},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.44}]},{"osd":6,"last update":"Mon Feb 26 06:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.81599999999999995,"15min":0.78800000000000003},"min":{"1min":0.55300000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0429999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.92900000000000005,"15min":0.873},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.496,"5min":8.8729999999999993,"15min":8.8729999999999993},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54500000000000004,"15min":0.51000000000000001},"min":{"1min":0.40300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47199999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63300000000000001,"15min":0.61099999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.66000000000000003},"min":{"1min":0.52900000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89800000000000002,"5min":8.9380000000000006,"15min":8.9380000000000006},"last":0.45800000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:22:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91800000000000004,"15min":0.92000000000000004},"min":{"1min":0.70199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1519999999999999,"5min":1.224,"15min":1.224},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.03,"15min":1.01},"min":{"1min":0.64900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.105,"5min":8.8979999999999997,"15min":8.8979999999999997},"last":1.0509999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.49399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0369999999999999,"5min":1.04,"15min":1.04},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79500000000000004,"15min":0.79100000000000004},"min":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.53800000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460263,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:22:03 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.004,"15min":0.84599999999999997},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.46300000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.01,"15min":0.86699999999999999},"min":{"1min":0.624,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":1.4390000000000001,"5min":1.498,"15min":1.649},"last":1.347}]},{"osd":2,"last update":"Mon Feb 26 06:22:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1160000000000001,"15min":0.93799999999999994},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"max":{"1min":1.621,"5min":6.6070000000000002,"15min":6.6070000000000002},"last":0.997},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2869999999999999,"15min":1.024},"min":{"1min":0.72299999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0600000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.081}]},{"osd":3,"last update":"Mon Feb 26 06:22:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.7370000000000001,"15min":1.7649999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2280000000000002,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.9399999999999999},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.909,"15min":1.931},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.419,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":2.2400000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:22:20 2024","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.3460000000000001,"15min":1.107},"min":{"1min":0.75700000000000001,"5min":0.436,"15min":0.436},"max":{"1min":2.0339999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.486},{"interface":"front","average":{"1min":1.357,"5min":1.288,"15min":1.171},"min":{"1min":0.69799999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.131}]},{"osd":5,"last update":"Mon Feb 26 06:22:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.631,"15min":1.4950000000000001},"min":{"1min":0.80000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2639999999999998,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":2.1549999999999998},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5549999999999999,"15min":1.4119999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.1440000000000001,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":1.6120000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:22:20 2024","interfaces":[{"interface":"back","average":{"1min":2.1000000000000001,"5min":1.8959999999999999,"15min":1.8300000000000001},"min":{"1min":0.98099999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":2.3130000000000002},{"interface":"front","average":{"1min":2.0790000000000002,"5min":1.873,"15min":1.857},"min":{"1min":0.67000000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.819}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.73,"15min":1.7190000000000001},"min":{"1min":1.2150000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.45},{"interface":"front","average":{"1min":1.73,"5min":1.599,"15min":1.577},"min":{"1min":1.123,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:22:15 2024","interfaces":[{"interface":"back","average":{"1min":2.0179999999999998,"5min":1.8360000000000001,"15min":1.7529999999999999},"min":{"1min":1.012,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6589999999999998,"5min":2.698,"15min":2.698},"last":2.375},{"interface":"front","average":{"1min":1.964,"5min":1.849,"15min":1.7709999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":1.716}]},{"osd":9,"last update":"Mon Feb 26 06:22:54 2024","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.859,"15min":1.9570000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.2709999999999999,"5min":2.754,"15min":2.754},"last":1.222},{"interface":"front","average":{"1min":1.9490000000000001,"5min":2.1110000000000002,"15min":2.1019999999999999},"min":{"1min":1.0369999999999999,"5min":0.746,"15min":0.746},"max":{"1min":2.706,"5min":3.101,"15min":3.101},"last":2.4550000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:22:15 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":2.0339999999999998,"15min":2.0539999999999998},"min":{"1min":0.91500000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7679999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":2.0310000000000001},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.8180000000000001,"15min":1.835},"min":{"1min":0.84899999999999998,"5min":0.75,"15min":0.75},"max":{"1min":2.581,"5min":2.581,"15min":2.581},"last":1.2629999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:22:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0329999999999999,"5min":1.994,"15min":1.9199999999999999},"min":{"1min":1.0569999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7989999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":2.4929999999999999},{"interface":"front","average":{"1min":1.921,"5min":2.0179999999999998,"15min":2.016},"min":{"1min":0.96199999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.714,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":2.214}]}]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":460333,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73399999999999999,"15min":0.69299999999999995},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73999999999999999,"15min":0.70699999999999996},"min":{"1min":0.51000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93200000000000005,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.35299999999999998,"15min":0.35199999999999998},"max":{"1min":0.78100000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70999999999999996}]},{"osd":3,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.77300000000000002,"15min":0.72599999999999998},"min":{"1min":0.57299999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.0649999999999999,"5min":1.359,"15min":1.359},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.73599999999999999},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.127,"15min":1.367},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.621},"min":{"1min":0.439,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73899999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:22:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47399999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.69999999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53800000000000003,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.76700000000000002},"min":{"1min":0.58499999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1180000000000001,"5min":1.206,"15min":1.206},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.81200000000000006}]},{"osd":7,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47399999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.60599999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:22:51 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.95299999999999996,"15min":0.94799999999999995},"min":{"1min":0.64400000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98399999999999999,"15min":0.98799999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.96899999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.52400000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.60099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.85799999999999998}]},{"osd":11,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.72299999999999998},"min":{"1min":0.54400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.78300000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":459303,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2150000000000001,"15min":1.0900000000000001},"min":{"1min":0.54300000000000004,"5min":0.433,"15min":0.433},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4630000000000001,"15min":1.173},"min":{"1min":0.80100000000000005,"5min":0.375,"15min":0.375},"max":{"1min":2.0840000000000001,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":0.80100000000000005}]},{"osd":1,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2090000000000001,"15min":1.056},"min":{"1min":0.68500000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.853,"5min":2.036,"15min":2.036},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.252,"5min":1.1890000000000001,"15min":1.0940000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.786,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.345,"15min":1.204},"min":{"1min":0.81799999999999995,"5min":0.70099999999999996,"15min":0.66400000000000003},"max":{"1min":1.7589999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.194,"5min":1.1220000000000001,"15min":1.0549999999999999},"min":{"1min":0.84399999999999997,"5min":0.441,"15min":0.441},"max":{"1min":1.7989999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.84399999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:22:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3740000000000001,"15min":1.2370000000000001},"min":{"1min":0.83599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.863,"5min":2.613,"15min":2.613},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.577,"5min":1.5920000000000001,"15min":1.4039999999999999},"min":{"1min":0.85599999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.0990000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.751}]},{"osd":5,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.506,"15min":1.393},"min":{"1min":0.90800000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.405,"15min":1.351},"min":{"1min":0.93799999999999994,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.266,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Feb 26 06:22:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.6439999999999999,"15min":1.5069999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.488,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.8620000000000001,"5min":1.853,"15min":1.673},"min":{"1min":0.82099999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.806,"5min":4.681,"15min":4.681},"last":0.86499999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.915,"5min":1.8089999999999999,"15min":1.75},"min":{"1min":1.2869999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.6560000000000001,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.8700000000000001,"5min":1.865,"15min":1.7250000000000001},"min":{"1min":1.381,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.468,"5min":3.544,"15min":3.544},"last":0.83399999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.806},"min":{"1min":0.97399999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.587,"15min":1.5649999999999999},"min":{"1min":0.84999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.23,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.143}]},{"osd":9,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.9590000000000001,"5min":1.9670000000000001,"15min":2.0379999999999998},"min":{"1min":1.4610000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.7610000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":2.1429999999999998,"5min":2.2290000000000001,"15min":2.403},"min":{"1min":1.619,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.734,"5min":4.266,"15min":4.266},"last":0.98099999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8280000000000001,"5min":1.9450000000000001,"15min":2.0099999999999998},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.998},{"interface":"front","average":{"1min":1.9530000000000001,"5min":2.1699999999999999,"15min":2.2589999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.6509999999999998,"5min":4.0609999999999999,"15min":4.0609999999999999},"last":0.89500000000000002}]},{"osd":11,"last update":"Mon Feb 26 06:22:34 2024","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.9399999999999999,"15min":1.946},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.6150000000000002,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":2.0800000000000001,"5min":2.0739999999999998,"15min":2.073},"min":{"1min":1.0609999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.5739999999999998,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.92900000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888150,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":526878,"data_compressed":15495,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.50600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.079,"5min":1.093,"15min":1.093},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39700000000000002,"15min":0.36099999999999999},"min":{"1min":0.34100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.38},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.34999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.41499999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.371,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69299999999999995,"15min":0.57499999999999996},"min":{"1min":0.48299999999999998,"5min":0.379,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":12.468,"15min":12.468},"last":0.51900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76200000000000001,"15min":0.71199999999999997},"min":{"1min":0.59999999999999998,"5min":0.48299999999999998,"15min":0.39800000000000002},"max":{"1min":0.94399999999999995,"5min":1.167,"15min":1.167},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.746},"min":{"1min":0.49099999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1579999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.53900000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:22:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.54800000000000004},"min":{"1min":0.52300000000000002,"5min":0.42599999999999999,"15min":0.41099999999999998},"max":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.626,"15min":0.56999999999999995},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48099999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79000000000000004,"15min":0.73799999999999999},"min":{"1min":0.628,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97499999999999998,"5min":1.327,"15min":1.327},"last":0.629},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80200000000000005,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.1180000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Feb 26 06:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.42399999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.50700000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.70199999999999996,"15min":0.66600000000000004},"min":{"1min":0.59499999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59399999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1699999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.044,"15min":1},"min":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2190000000000001,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.76700000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.61799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.5,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.49399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.67800000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724625,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1051056,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72299999999999998,"15min":0.67700000000000005},"min":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.159,"15min":1.159},"last":0.749},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.0649999999999999,"5min":1.204,"15min":1.204},"last":0.91900000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79500000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70499999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.315,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61099999999999999,"5min":0.624,"15min":0.624},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42399999999999999,"15min":0.40600000000000003},"min":{"1min":0.34799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.75,"15min":0.75},"last":0.34200000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.80800000000000005,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.78600000000000003,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78400000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Feb 26 06:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82799999999999996,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.135,"5min":1.242,"15min":1.242},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83799999999999997,"15min":0.80900000000000005},"min":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.089,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.044}]},{"osd":7,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65600000000000003,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46899999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68300000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48299999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.64500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.381,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.369},{"interface":"front","average":{"1min":1.01,"5min":0.999,"15min":1.04},"min":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.2789999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.69999999999999996,"15min":0.68400000000000005},"min":{"1min":0.56100000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.90100000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66805,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":68093,"data_compressed":9688,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68915,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:23:09.978 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-26T06:23:09.979 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:23:10.125 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:09 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2673847670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:10.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 -- 172.21.15.49:0/2929154465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 msgr2=0x7fc368107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:10.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 --2- 172.21.15.49:0/2929154465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 0x7fc368107be0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc358009860 tx=0x7fc35802bb70 comp rx=0 tx=0).stop 2024-02-26T06:23:10.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 -- 172.21.15.49:0/2929154465 shutdown_connections 2024-02-26T06:23:10.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 --2- 172.21.15.49:0/2929154465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc368109370 0x7fc36810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 --2- 172.21.15.49:0/2929154465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681089b0 0x7fc368108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 --2- 172.21.15.49:0/2929154465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 0x7fc368107be0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 -- 172.21.15.49:0/2929154465 >> 172.21.15.49:0/2929154465 conn(0x7fc368076790 msgr2=0x7fc368078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 -- 172.21.15.49:0/2929154465 shutdown_connections 2024-02-26T06:23:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.245+0000 7fc36ee00640 1 -- 172.21.15.49:0/2929154465 wait complete. 2024-02-26T06:23:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.247+0000 7fc36ee00640 1 Processor -- start 2024-02-26T06:23:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.247+0000 7fc36ee00640 1 -- start start 2024-02-26T06:23:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.247+0000 7fc36ee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 0x7fc36819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.247+0000 7fc36ee00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681089b0 0x7fc36819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.247+0000 7fc36ee00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc368109370 0x7fc3681a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.247+0000 7fc36ee00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681a5350 con 0x7fc368109370 2024-02-26T06:23:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.247+0000 7fc36ee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681a54c0 con 0x7fc3681077c0 2024-02-26T06:23:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.247+0000 7fc36ee00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681a5680 con 0x7fc3681089b0 2024-02-26T06:23:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc367e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 0x7fc36819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681089b0 0x7fc36819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc36ce00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc368109370 0x7fc3681a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681089b0 0x7fc36819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:51428/0 (socket says 172.21.15.49:51428) 2024-02-26T06:23:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc367e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 0x7fc36819d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47752/0 (socket says 172.21.15.49:47752) 2024-02-26T06:23:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 -- 172.21.15.49:0/260667273 learned_addr learned my addr 172.21.15.49:0/260667273 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 -- 172.21.15.49:0/260667273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 msgr2=0x7fc36819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 0x7fc36819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 -- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc368109370 msgr2=0x7fc3681a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc368109370 0x7fc3681a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 -- 172.21.15.49:0/260667273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc358009490 con 0x7fc3681089b0 2024-02-26T06:23:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.248+0000 7fc35f400640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681089b0 0x7fc36819dda0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc35000e9c0 tx=0x7fc35000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.249+0000 7fc365600640 1 -- 172.21.15.49:0/260667273 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35000ccc0 con 0x7fc3681089b0 2024-02-26T06:23:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.249+0000 7fc365600640 1 -- 172.21.15.49:0/260667273 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc35000ce20 con 0x7fc3681089b0 2024-02-26T06:23:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.249+0000 7fc365600640 1 -- 172.21.15.49:0/260667273 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc350010430 con 0x7fc3681089b0 2024-02-26T06:23:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.249+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3681a5960 con 0x7fc3681089b0 2024-02-26T06:23:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.249+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681a5ec0 con 0x7fc3681089b0 2024-02-26T06:23:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.250+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36810eae0 con 0x7fc3681089b0 2024-02-26T06:23:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.252+0000 7fc365600640 1 -- 172.21.15.49:0/260667273 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc350004f80 con 0x7fc3681089b0 2024-02-26T06:23:10.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.253+0000 7fc365600640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc34c0a2140 0x7fc34c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:10.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.253+0000 7fc365600640 1 -- 172.21.15.49:0/260667273 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fc350014070 con 0x7fc3681089b0 2024-02-26T06:23:10.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.253+0000 7fc367e00640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc34c0a2140 0x7fc34c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:10.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.254+0000 7fc367e00640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc34c0a2140 0x7fc34c0a4600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc358009830 tx=0x7fc358034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:10.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.257+0000 7fc365600640 1 -- 172.21.15.49:0/260667273 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc350096ea0 con 0x7fc3681089b0 2024-02-26T06:23:10.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:09 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2673847670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:10.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:09 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2673847670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.456+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc36802ceb0 con 0x7fc34c0a2140 2024-02-26T06:23:10.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.489+0000 7fc365600640 1 -- 172.21.15.49:0/260667273 <== mgr.14152 v2:172.21.15.49:6800/1431851577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+342859 (secure 0 0 0) 0x7fc36802ceb0 con 0x7fc34c0a2140 2024-02-26T06:23:10.494 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:10.496 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-02-26T06:23:10.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc34c0a2140 msgr2=0x7fc34c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:10.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc34c0a2140 0x7fc34c0a4600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc358009830 tx=0x7fc358034c00 comp rx=0 tx=0).stop 2024-02-26T06:23:10.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681089b0 msgr2=0x7fc36819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681089b0 0x7fc36819dda0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc35000e9c0 tx=0x7fc35000ee80 comp rx=0 tx=0).stop 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 shutdown_connections 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fc34c0a2140 0x7fc34c0a4600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc368109370 0x7fc3681a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681089b0 0x7fc36819dda0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 --2- 172.21.15.49:0/260667273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3681077c0 0x7fc36819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 >> 172.21.15.49:0/260667273 conn(0x7fc368076790 msgr2=0x7fc36810bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.497+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 shutdown_connections 2024-02-26T06:23:10.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.498+0000 7fc36ee00640 1 -- 172.21.15.49:0/260667273 wait complete. 2024-02-26T06:23:10.520 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-02-26T06:23:09.486557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661859,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1062,"num_read_kb":655,"num_write":354,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1241721,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360788,"kb_used_data":11636,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581036,"statfs":{"total":1151940427776,"available":1151570980864,"internally_reserved":0,"allocated":11915264,"data_stored":7490997,"data_compressed":191495,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5993,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":43,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003824"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440192+0000","last_change":"2024-02-26T06:20:31.603060+0000","last_active":"2024-02-26T06:21:57.440192+0000","last_peered":"2024-02-26T06:21:57.440192+0000","last_clean":"2024-02-26T06:21:57.440192+0000","last_became_active":"2024-02-26T06:20:31.602762+0000","last_became_peered":"2024-02-26T06:20:31.602762+0000","last_unstale":"2024-02-26T06:21:57.440192+0000","last_undegraded":"2024-02-26T06:21:57.440192+0000","last_fullsized":"2024-02-26T06:21:57.440192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:19:42.465510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.437717+0000","last_change":"2024-02-26T06:20:31.595138+0000","last_active":"2024-02-26T06:21:57.437717+0000","last_peered":"2024-02-26T06:21:57.437717+0000","last_clean":"2024-02-26T06:21:57.437717+0000","last_became_active":"2024-02-26T06:20:31.594826+0000","last_became_peered":"2024-02-26T06:20:31.594826+0000","last_unstale":"2024-02-26T06:21:57.437717+0000","last_undegraded":"2024-02-26T06:21:57.437717+0000","last_fullsized":"2024-02-26T06:21:57.437717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:49.900464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.431160+0000","last_change":"2024-02-26T06:20:31.600415+0000","last_active":"2024-02-26T06:21:57.431160+0000","last_peered":"2024-02-26T06:21:57.431160+0000","last_clean":"2024-02-26T06:21:57.431160+0000","last_became_active":"2024-02-26T06:20:31.596273+0000","last_became_peered":"2024-02-26T06:20:31.596273+0000","last_unstale":"2024-02-26T06:21:57.431160+0000","last_undegraded":"2024-02-26T06:21:57.431160+0000","last_fullsized":"2024-02-26T06:21:57.431160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:15:20.626515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410929+0000","last_change":"2024-02-26T06:20:31.581149+0000","last_active":"2024-02-26T06:21:57.410929+0000","last_peered":"2024-02-26T06:21:57.410929+0000","last_clean":"2024-02-26T06:21:57.410929+0000","last_became_active":"2024-02-26T06:20:31.580831+0000","last_became_peered":"2024-02-26T06:20:31.580831+0000","last_unstale":"2024-02-26T06:21:57.410929+0000","last_undegraded":"2024-02-26T06:21:57.410929+0000","last_fullsized":"2024-02-26T06:21:57.410929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:47:12.466502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412577+0000","last_change":"2024-02-26T06:20:31.608618+0000","last_active":"2024-02-26T06:21:57.412577+0000","last_peered":"2024-02-26T06:21:57.412577+0000","last_clean":"2024-02-26T06:21:57.412577+0000","last_became_active":"2024-02-26T06:20:31.608285+0000","last_became_peered":"2024-02-26T06:20:31.608285+0000","last_unstale":"2024-02-26T06:21:57.412577+0000","last_undegraded":"2024-02-26T06:21:57.412577+0000","last_fullsized":"2024-02-26T06:21:57.412577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:34:48.400836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.418353+0000","last_change":"2024-02-26T06:21:28.905073+0000","last_active":"2024-02-26T06:21:57.418353+0000","last_peered":"2024-02-26T06:21:57.418353+0000","last_clean":"2024-02-26T06:21:57.418353+0000","last_became_active":"2024-02-26T06:21:28.904630+0000","last_became_peered":"2024-02-26T06:21:28.904630+0000","last_unstale":"2024-02-26T06:21:57.418353+0000","last_undegraded":"2024-02-26T06:21:57.418353+0000","last_fullsized":"2024-02-26T06:21:57.418353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:04.531384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.407619+0000","last_change":"2024-02-26T06:20:31.587285+0000","last_active":"2024-02-26T06:21:57.407619+0000","last_peered":"2024-02-26T06:21:57.407619+0000","last_clean":"2024-02-26T06:21:57.407619+0000","last_became_active":"2024-02-26T06:20:31.586985+0000","last_became_peered":"2024-02-26T06:20:31.586985+0000","last_unstale":"2024-02-26T06:21:57.407619+0000","last_undegraded":"2024-02-26T06:21:57.407619+0000","last_fullsized":"2024-02-26T06:21:57.407619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:55:56.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438247+0000","last_change":"2024-02-26T06:20:31.591136+0000","last_active":"2024-02-26T06:21:57.438247+0000","last_peered":"2024-02-26T06:21:57.438247+0000","last_clean":"2024-02-26T06:21:57.438247+0000","last_became_active":"2024-02-26T06:20:31.590180+0000","last_became_peered":"2024-02-26T06:20:31.590180+0000","last_unstale":"2024-02-26T06:21:57.438247+0000","last_undegraded":"2024-02-26T06:21:57.438247+0000","last_fullsized":"2024-02-26T06:21:57.438247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:02.649407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420898+0000","last_change":"2024-02-26T06:20:31.583890+0000","last_active":"2024-02-26T06:21:57.420898+0000","last_peered":"2024-02-26T06:21:57.420898+0000","last_clean":"2024-02-26T06:21:57.420898+0000","last_became_active":"2024-02-26T06:20:31.583459+0000","last_became_peered":"2024-02-26T06:20:31.583459+0000","last_unstale":"2024-02-26T06:21:57.420898+0000","last_undegraded":"2024-02-26T06:21:57.420898+0000","last_fullsized":"2024-02-26T06:21:57.420898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:14.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412793+0000","last_change":"2024-02-26T06:20:31.579041+0000","last_active":"2024-02-26T06:21:57.412793+0000","last_peered":"2024-02-26T06:21:57.412793+0000","last_clean":"2024-02-26T06:21:57.412793+0000","last_became_active":"2024-02-26T06:20:31.578725+0000","last_became_peered":"2024-02-26T06:20:31.578725+0000","last_unstale":"2024-02-26T06:21:57.412793+0000","last_undegraded":"2024-02-26T06:21:57.412793+0000","last_fullsized":"2024-02-26T06:21:57.412793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:21.083206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420613+0000","last_change":"2024-02-26T06:20:31.581146+0000","last_active":"2024-02-26T06:21:57.420613+0000","last_peered":"2024-02-26T06:21:57.420613+0000","last_clean":"2024-02-26T06:21:57.420613+0000","last_became_active":"2024-02-26T06:20:31.580774+0000","last_became_peered":"2024-02-26T06:20:31.580774+0000","last_unstale":"2024-02-26T06:21:57.420613+0000","last_undegraded":"2024-02-26T06:21:57.420613+0000","last_fullsized":"2024-02-26T06:21:57.420613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:58:36.851504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410598+0000","last_change":"2024-02-26T06:20:31.582232+0000","last_active":"2024-02-26T06:21:57.410598+0000","last_peered":"2024-02-26T06:21:57.410598+0000","last_clean":"2024-02-26T06:21:57.410598+0000","last_became_active":"2024-02-26T06:20:31.581805+0000","last_became_peered":"2024-02-26T06:20:31.581805+0000","last_unstale":"2024-02-26T06:21:57.410598+0000","last_undegraded":"2024-02-26T06:21:57.410598+0000","last_fullsized":"2024-02-26T06:21:57.410598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:04.513814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428668+0000","last_change":"2024-02-26T06:20:31.606373+0000","last_active":"2024-02-26T06:21:57.428668+0000","last_peered":"2024-02-26T06:21:57.428668+0000","last_clean":"2024-02-26T06:21:57.428668+0000","last_became_active":"2024-02-26T06:20:31.603226+0000","last_became_peered":"2024-02-26T06:20:31.603226+0000","last_unstale":"2024-02-26T06:21:57.428668+0000","last_undegraded":"2024-02-26T06:21:57.428668+0000","last_fullsized":"2024-02-26T06:21:57.428668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:29:49.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.426327+0000","last_change":"2024-02-26T06:20:31.594587+0000","last_active":"2024-02-26T06:21:57.426327+0000","last_peered":"2024-02-26T06:21:57.426327+0000","last_clean":"2024-02-26T06:21:57.426327+0000","last_became_active":"2024-02-26T06:20:31.594032+0000","last_became_peered":"2024-02-26T06:20:31.594032+0000","last_unstale":"2024-02-26T06:21:57.426327+0000","last_undegraded":"2024-02-26T06:21:57.426327+0000","last_fullsized":"2024-02-26T06:21:57.426327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:49:18.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438329+0000","last_change":"2024-02-26T06:20:31.593115+0000","last_active":"2024-02-26T06:21:57.438329+0000","last_peered":"2024-02-26T06:21:57.438329+0000","last_clean":"2024-02-26T06:21:57.438329+0000","last_became_active":"2024-02-26T06:20:31.592782+0000","last_became_peered":"2024-02-26T06:20:31.592782+0000","last_unstale":"2024-02-26T06:21:57.438329+0000","last_undegraded":"2024-02-26T06:21:57.438329+0000","last_fullsized":"2024-02-26T06:21:57.438329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:10.709124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.427221+0000","last_change":"2024-02-26T06:20:31.597938+0000","last_active":"2024-02-26T06:21:57.427221+0000","last_peered":"2024-02-26T06:21:57.427221+0000","last_clean":"2024-02-26T06:21:57.427221+0000","last_became_active":"2024-02-26T06:20:31.597600+0000","last_became_peered":"2024-02-26T06:20:31.597600+0000","last_unstale":"2024-02-26T06:21:57.427221+0000","last_undegraded":"2024-02-26T06:21:57.427221+0000","last_fullsized":"2024-02-26T06:21:57.427221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:13.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440351+0000","last_change":"2024-02-26T06:20:31.603283+0000","last_active":"2024-02-26T06:21:57.440351+0000","last_peered":"2024-02-26T06:21:57.440351+0000","last_clean":"2024-02-26T06:21:57.440351+0000","last_became_active":"2024-02-26T06:20:31.602912+0000","last_became_peered":"2024-02-26T06:20:31.602912+0000","last_unstale":"2024-02-26T06:21:57.440351+0000","last_undegraded":"2024-02-26T06:21:57.440351+0000","last_fullsized":"2024-02-26T06:21:57.440351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:32.250914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.422411+0000","last_change":"2024-02-26T06:20:31.588670+0000","last_active":"2024-02-26T06:21:57.422411+0000","last_peered":"2024-02-26T06:21:57.422411+0000","last_clean":"2024-02-26T06:21:57.422411+0000","last_became_active":"2024-02-26T06:20:31.587971+0000","last_became_peered":"2024-02-26T06:20:31.587971+0000","last_unstale":"2024-02-26T06:21:57.422411+0000","last_undegraded":"2024-02-26T06:21:57.422411+0000","last_fullsized":"2024-02-26T06:21:57.422411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:13.053456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.425204+0000","last_change":"2024-02-26T06:20:31.604266+0000","last_active":"2024-02-26T06:21:57.425204+0000","last_peered":"2024-02-26T06:21:57.425204+0000","last_clean":"2024-02-26T06:21:57.425204+0000","last_became_active":"2024-02-26T06:20:31.604027+0000","last_became_peered":"2024-02-26T06:20:31.604027+0000","last_unstale":"2024-02-26T06:21:57.425204+0000","last_undegraded":"2024-02-26T06:21:57.425204+0000","last_fullsized":"2024-02-26T06:21:57.425204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:13.345138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904510+0000","last_change":"2024-02-26T06:20:31.068090+0000","last_active":"2024-02-26T06:21:28.904510+0000","last_peered":"2024-02-26T06:21:28.904510+0000","last_clean":"2024-02-26T06:21:28.904510+0000","last_became_active":"2024-02-26T06:20:31.067731+0000","last_became_peered":"2024-02-26T06:20:31.067731+0000","last_unstale":"2024-02-26T06:21:28.904510+0000","last_undegraded":"2024-02-26T06:21:28.904510+0000","last_fullsized":"2024-02-26T06:21:28.904510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:24.734040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413823+0000","last_change":"2024-02-26T06:21:28.915741+0000","last_active":"2024-02-26T06:21:57.413823+0000","last_peered":"2024-02-26T06:21:57.413823+0000","last_clean":"2024-02-26T06:21:57.413823+0000","last_became_active":"2024-02-26T06:21:28.915443+0000","last_became_peered":"2024-02-26T06:21:28.915443+0000","last_unstale":"2024-02-26T06:21:57.413823+0000","last_undegraded":"2024-02-26T06:21:57.413823+0000","last_fullsized":"2024-02-26T06:21:57.413823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:51.216291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898901+0000","last_change":"2024-02-26T06:20:31.083236+0000","last_active":"2024-02-26T06:21:28.898901+0000","last_peered":"2024-02-26T06:21:28.898901+0000","last_clean":"2024-02-26T06:21:28.898901+0000","last_became_active":"2024-02-26T06:20:31.081640+0000","last_became_peered":"2024-02-26T06:20:31.081640+0000","last_unstale":"2024-02-26T06:21:28.898901+0000","last_undegraded":"2024-02-26T06:21:28.898901+0000","last_fullsized":"2024-02-26T06:21:28.898901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:44.607108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439959+0000","last_change":"2024-02-26T06:20:31.593807+0000","last_active":"2024-02-26T06:21:57.439959+0000","last_peered":"2024-02-26T06:21:57.439959+0000","last_clean":"2024-02-26T06:21:57.439959+0000","last_became_active":"2024-02-26T06:20:31.593435+0000","last_became_peered":"2024-02-26T06:20:31.593435+0000","last_unstale":"2024-02-26T06:21:57.439959+0000","last_undegraded":"2024-02-26T06:21:57.439959+0000","last_fullsized":"2024-02-26T06:21:57.439959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:43.193196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.908174+0000","last_change":"2024-02-26T06:20:31.061339+0000","last_active":"2024-02-26T06:21:28.908174+0000","last_peered":"2024-02-26T06:21:28.908174+0000","last_clean":"2024-02-26T06:21:28.908174+0000","last_became_active":"2024-02-26T06:20:31.060769+0000","last_became_peered":"2024-02-26T06:20:31.060769+0000","last_unstale":"2024-02-26T06:21:28.908174+0000","last_undegraded":"2024-02-26T06:21:28.908174+0000","last_fullsized":"2024-02-26T06:21:28.908174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:28:30.501375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432762+0000","last_change":"2024-02-26T06:20:31.606936+0000","last_active":"2024-02-26T06:21:57.432762+0000","last_peered":"2024-02-26T06:21:57.432762+0000","last_clean":"2024-02-26T06:21:57.432762+0000","last_became_active":"2024-02-26T06:20:31.606582+0000","last_became_peered":"2024-02-26T06:20:31.606582+0000","last_unstale":"2024-02-26T06:21:57.432762+0000","last_undegraded":"2024-02-26T06:21:57.432762+0000","last_fullsized":"2024-02-26T06:21:57.432762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:09.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907169+0000","last_change":"2024-02-26T06:20:31.063472+0000","last_active":"2024-02-26T06:21:28.907169+0000","last_peered":"2024-02-26T06:21:28.907169+0000","last_clean":"2024-02-26T06:21:28.907169+0000","last_became_active":"2024-02-26T06:20:31.063132+0000","last_became_peered":"2024-02-26T06:20:31.063132+0000","last_unstale":"2024-02-26T06:21:28.907169+0000","last_undegraded":"2024-02-26T06:21:28.907169+0000","last_fullsized":"2024-02-26T06:21:28.907169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:58.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.418097+0000","last_change":"2024-02-26T06:20:31.589238+0000","last_active":"2024-02-26T06:21:57.418097+0000","last_peered":"2024-02-26T06:21:57.418097+0000","last_clean":"2024-02-26T06:21:57.418097+0000","last_became_active":"2024-02-26T06:20:31.588929+0000","last_became_peered":"2024-02-26T06:20:31.588929+0000","last_unstale":"2024-02-26T06:21:57.418097+0000","last_undegraded":"2024-02-26T06:21:57.418097+0000","last_fullsized":"2024-02-26T06:21:57.418097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:19.282943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":179,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:06.333402+0000","last_change":"2024-02-26T06:20:31.061714+0000","last_active":"2024-02-26T06:23:06.333402+0000","last_peered":"2024-02-26T06:23:06.333402+0000","last_clean":"2024-02-26T06:23:06.333402+0000","last_became_active":"2024-02-26T06:20:31.061234+0000","last_became_peered":"2024-02-26T06:20:31.061234+0000","last_unstale":"2024-02-26T06:23:06.333402+0000","last_undegraded":"2024-02-26T06:23:06.333402+0000","last_fullsized":"2024-02-26T06:23:06.333402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:29:17.668327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.400272+0000","last_change":"2024-02-26T06:20:31.588861+0000","last_active":"2024-02-26T06:21:57.400272+0000","last_peered":"2024-02-26T06:21:57.400272+0000","last_clean":"2024-02-26T06:21:57.400272+0000","last_became_active":"2024-02-26T06:20:31.588530+0000","last_became_peered":"2024-02-26T06:20:31.588530+0000","last_unstale":"2024-02-26T06:21:57.400272+0000","last_undegraded":"2024-02-26T06:21:57.400272+0000","last_fullsized":"2024-02-26T06:21:57.400272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:12.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.900377+0000","last_change":"2024-02-26T06:20:31.077656+0000","last_active":"2024-02-26T06:21:28.900377+0000","last_peered":"2024-02-26T06:21:28.900377+0000","last_clean":"2024-02-26T06:21:28.900377+0000","last_became_active":"2024-02-26T06:20:31.077288+0000","last_became_peered":"2024-02-26T06:20:31.077288+0000","last_unstale":"2024-02-26T06:21:28.900377+0000","last_undegraded":"2024-02-26T06:21:28.900377+0000","last_fullsized":"2024-02-26T06:21:28.900377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:56:05.173751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440086+0000","last_change":"2024-02-26T06:20:31.608089+0000","last_active":"2024-02-26T06:21:57.440086+0000","last_peered":"2024-02-26T06:21:57.440086+0000","last_clean":"2024-02-26T06:21:57.440086+0000","last_became_active":"2024-02-26T06:20:31.607786+0000","last_became_peered":"2024-02-26T06:20:31.607786+0000","last_unstale":"2024-02-26T06:21:57.440086+0000","last_undegraded":"2024-02-26T06:21:57.440086+0000","last_fullsized":"2024-02-26T06:21:57.440086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:24.113286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.207050+0000","last_change":"2024-02-26T06:20:31.077063+0000","last_active":"2024-02-26T06:21:29.207050+0000","last_peered":"2024-02-26T06:21:29.207050+0000","last_clean":"2024-02-26T06:21:29.207050+0000","last_became_active":"2024-02-26T06:20:31.075147+0000","last_became_peered":"2024-02-26T06:20:31.075147+0000","last_unstale":"2024-02-26T06:21:29.207050+0000","last_undegraded":"2024-02-26T06:21:29.207050+0000","last_fullsized":"2024-02-26T06:21:29.207050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:07:13.434635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439933+0000","last_change":"2024-02-26T06:20:31.600135+0000","last_active":"2024-02-26T06:21:57.439933+0000","last_peered":"2024-02-26T06:21:57.439933+0000","last_clean":"2024-02-26T06:21:57.439933+0000","last_became_active":"2024-02-26T06:20:31.595622+0000","last_became_peered":"2024-02-26T06:20:31.595622+0000","last_unstale":"2024-02-26T06:21:57.439933+0000","last_undegraded":"2024-02-26T06:21:57.439933+0000","last_fullsized":"2024-02-26T06:21:57.439933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:55.835700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914398+0000","last_change":"2024-02-26T06:20:31.060299+0000","last_active":"2024-02-26T06:21:28.914398+0000","last_peered":"2024-02-26T06:21:28.914398+0000","last_clean":"2024-02-26T06:21:28.914398+0000","last_became_active":"2024-02-26T06:20:31.059938+0000","last_became_peered":"2024-02-26T06:20:31.059938+0000","last_unstale":"2024-02-26T06:21:28.914398+0000","last_undegraded":"2024-02-26T06:21:28.914398+0000","last_fullsized":"2024-02-26T06:21:28.914398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:24.144958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.444272+0000","last_change":"2024-02-26T06:20:31.601945+0000","last_active":"2024-02-26T06:21:57.444272+0000","last_peered":"2024-02-26T06:21:57.444272+0000","last_clean":"2024-02-26T06:21:57.444272+0000","last_became_active":"2024-02-26T06:20:31.601560+0000","last_became_peered":"2024-02-26T06:20:31.601560+0000","last_unstale":"2024-02-26T06:21:57.444272+0000","last_undegraded":"2024-02-26T06:21:57.444272+0000","last_fullsized":"2024-02-26T06:21:57.444272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:13.932623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":173,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:05.331162+0000","last_change":"2024-02-26T06:20:31.068334+0000","last_active":"2024-02-26T06:23:05.331162+0000","last_peered":"2024-02-26T06:23:05.331162+0000","last_clean":"2024-02-26T06:23:05.331162+0000","last_became_active":"2024-02-26T06:20:31.068020+0000","last_became_peered":"2024-02-26T06:20:31.068020+0000","last_unstale":"2024-02-26T06:23:05.331162+0000","last_undegraded":"2024-02-26T06:23:05.331162+0000","last_fullsized":"2024-02-26T06:23:05.331162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:23:01.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":149,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421658+0000","last_change":"2024-02-26T06:20:31.587862+0000","last_active":"2024-02-26T06:21:57.421658+0000","last_peered":"2024-02-26T06:21:57.421658+0000","last_clean":"2024-02-26T06:21:57.421658+0000","last_became_active":"2024-02-26T06:20:31.587551+0000","last_became_peered":"2024-02-26T06:20:31.587551+0000","last_unstale":"2024-02-26T06:21:57.421658+0000","last_undegraded":"2024-02-26T06:21:57.421658+0000","last_fullsized":"2024-02-26T06:21:57.421658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:07:15.933125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902570+0000","last_change":"2024-02-26T06:20:31.058378+0000","last_active":"2024-02-26T06:21:28.902570+0000","last_peered":"2024-02-26T06:21:28.902570+0000","last_clean":"2024-02-26T06:21:28.902570+0000","last_became_active":"2024-02-26T06:20:31.058040+0000","last_became_peered":"2024-02-26T06:20:31.058040+0000","last_unstale":"2024-02-26T06:21:28.902570+0000","last_undegraded":"2024-02-26T06:21:28.902570+0000","last_fullsized":"2024-02-26T06:21:28.902570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:33:40.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440318+0000","last_change":"2024-02-26T06:20:31.601837+0000","last_active":"2024-02-26T06:21:57.440318+0000","last_peered":"2024-02-26T06:21:57.440318+0000","last_clean":"2024-02-26T06:21:57.440318+0000","last_became_active":"2024-02-26T06:20:31.597310+0000","last_became_peered":"2024-02-26T06:20:31.597310+0000","last_unstale":"2024-02-26T06:21:57.440318+0000","last_undegraded":"2024-02-26T06:21:57.440318+0000","last_fullsized":"2024-02-26T06:21:57.440318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.383510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.915022+0000","last_change":"2024-02-26T06:20:31.076158+0000","last_active":"2024-02-26T06:21:28.915022+0000","last_peered":"2024-02-26T06:21:28.915022+0000","last_clean":"2024-02-26T06:21:28.915022+0000","last_became_active":"2024-02-26T06:20:31.075795+0000","last_became_peered":"2024-02-26T06:20:31.075795+0000","last_unstale":"2024-02-26T06:21:28.915022+0000","last_undegraded":"2024-02-26T06:21:28.915022+0000","last_fullsized":"2024-02-26T06:21:28.915022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:38.491674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.421150+0000","last_change":"2024-02-26T06:20:31.586122+0000","last_active":"2024-02-26T06:21:57.421150+0000","last_peered":"2024-02-26T06:21:57.421150+0000","last_clean":"2024-02-26T06:21:57.421150+0000","last_became_active":"2024-02-26T06:20:31.585070+0000","last_became_peered":"2024-02-26T06:20:31.585070+0000","last_unstale":"2024-02-26T06:21:57.421150+0000","last_undegraded":"2024-02-26T06:21:57.421150+0000","last_fullsized":"2024-02-26T06:21:57.421150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:11.034252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":383,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:07.072497+0000","last_change":"2024-02-26T06:21:28.913681+0000","last_active":"2024-02-26T06:23:07.072497+0000","last_peered":"2024-02-26T06:23:07.072497+0000","last_clean":"2024-02-26T06:23:07.072497+0000","last_became_active":"2024-02-26T06:21:28.905006+0000","last_became_peered":"2024-02-26T06:21:28.905006+0000","last_unstale":"2024-02-26T06:23:07.072497+0000","last_undegraded":"2024-02-26T06:23:07.072497+0000","last_fullsized":"2024-02-26T06:23:07.072497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:41.432983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66805,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":308,"num_read_kb":219,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60873,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.410911+0000","last_change":"2024-02-26T06:20:31.584757+0000","last_active":"2024-02-26T06:21:57.410911+0000","last_peered":"2024-02-26T06:21:57.410911+0000","last_clean":"2024-02-26T06:21:57.410911+0000","last_became_active":"2024-02-26T06:20:31.583890+0000","last_became_peered":"2024-02-26T06:20:31.583890+0000","last_unstale":"2024-02-26T06:21:57.410911+0000","last_undegraded":"2024-02-26T06:21:57.410911+0000","last_fullsized":"2024-02-26T06:21:57.410911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:10:58.280082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903296+0000","last_change":"2024-02-26T06:20:31.076044+0000","last_active":"2024-02-26T06:21:28.903296+0000","last_peered":"2024-02-26T06:21:28.903296+0000","last_clean":"2024-02-26T06:21:28.903296+0000","last_became_active":"2024-02-26T06:20:31.075744+0000","last_became_peered":"2024-02-26T06:20:31.075744+0000","last_unstale":"2024-02-26T06:21:28.903296+0000","last_undegraded":"2024-02-26T06:21:28.903296+0000","last_fullsized":"2024-02-26T06:21:28.903296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:35.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.443819+0000","last_change":"2024-02-26T06:20:31.602169+0000","last_active":"2024-02-26T06:21:57.443819+0000","last_peered":"2024-02-26T06:21:57.443819+0000","last_clean":"2024-02-26T06:21:57.443819+0000","last_became_active":"2024-02-26T06:20:31.601280+0000","last_became_peered":"2024-02-26T06:20:31.601280+0000","last_unstale":"2024-02-26T06:21:57.443819+0000","last_undegraded":"2024-02-26T06:21:57.443819+0000","last_fullsized":"2024-02-26T06:21:57.443819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:08:07.652549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914144+0000","last_change":"2024-02-26T06:20:31.080482+0000","last_active":"2024-02-26T06:21:28.914144+0000","last_peered":"2024-02-26T06:21:28.914144+0000","last_clean":"2024-02-26T06:21:28.914144+0000","last_became_active":"2024-02-26T06:20:31.080188+0000","last_became_peered":"2024-02-26T06:20:31.080188+0000","last_unstale":"2024-02-26T06:21:28.914144+0000","last_undegraded":"2024-02-26T06:21:28.914144+0000","last_fullsized":"2024-02-26T06:21:28.914144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:32.175251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.433538+0000","last_change":"2024-02-26T06:20:31.599753+0000","last_active":"2024-02-26T06:21:57.433538+0000","last_peered":"2024-02-26T06:21:57.433538+0000","last_clean":"2024-02-26T06:21:57.433538+0000","last_became_active":"2024-02-26T06:20:31.599409+0000","last_became_peered":"2024-02-26T06:20:31.599409+0000","last_unstale":"2024-02-26T06:21:57.433538+0000","last_undegraded":"2024-02-26T06:21:57.433538+0000","last_fullsized":"2024-02-26T06:21:57.433538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:43.887431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:07.420991+0000","last_change":"2024-02-26T06:20:31.059656+0000","last_active":"2024-02-26T06:23:07.420991+0000","last_peered":"2024-02-26T06:23:07.420991+0000","last_clean":"2024-02-26T06:23:07.420991+0000","last_became_active":"2024-02-26T06:20:31.059289+0000","last_became_peered":"2024-02-26T06:20:31.059289+0000","last_unstale":"2024-02-26T06:23:07.420991+0000","last_undegraded":"2024-02-26T06:23:07.420991+0000","last_fullsized":"2024-02-26T06:23:07.420991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:08:57.650876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.404146+0000","last_change":"2024-02-26T06:21:28.907778+0000","last_active":"2024-02-26T06:21:57.404146+0000","last_peered":"2024-02-26T06:21:57.404146+0000","last_clean":"2024-02-26T06:21:57.404146+0000","last_became_active":"2024-02-26T06:21:28.907185+0000","last_became_peered":"2024-02-26T06:21:28.907185+0000","last_unstale":"2024-02-26T06:21:57.404146+0000","last_undegraded":"2024-02-26T06:21:57.404146+0000","last_fullsized":"2024-02-26T06:21:57.404146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:39.572867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.943152+0000","last_change":"2024-02-26T06:21:28.943152+0000","last_active":"2024-02-26T06:21:28.943152+0000","last_peered":"2024-02-26T06:21:28.943152+0000","last_clean":"2024-02-26T06:21:28.943152+0000","last_became_active":"2024-02-26T06:21:28.908090+0000","last_became_peered":"2024-02-26T06:21:28.908090+0000","last_unstale":"2024-02-26T06:21:28.943152+0000","last_undegraded":"2024-02-26T06:21:28.943152+0000","last_fullsized":"2024-02-26T06:21:28.943152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:56.254015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.417526+0000","last_change":"2024-02-26T06:20:31.586151+0000","last_active":"2024-02-26T06:21:57.417526+0000","last_peered":"2024-02-26T06:21:57.417526+0000","last_clean":"2024-02-26T06:21:57.417526+0000","last_became_active":"2024-02-26T06:20:31.585615+0000","last_became_peered":"2024-02-26T06:20:31.585615+0000","last_unstale":"2024-02-26T06:21:57.417526+0000","last_undegraded":"2024-02-26T06:21:57.417526+0000","last_fullsized":"2024-02-26T06:21:57.417526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907609+0000","last_change":"2024-02-26T06:20:31.062340+0000","last_active":"2024-02-26T06:21:28.907609+0000","last_peered":"2024-02-26T06:21:28.907609+0000","last_clean":"2024-02-26T06:21:28.907609+0000","last_became_active":"2024-02-26T06:20:31.062023+0000","last_became_peered":"2024-02-26T06:20:31.062023+0000","last_unstale":"2024-02-26T06:21:28.907609+0000","last_undegraded":"2024-02-26T06:21:28.907609+0000","last_fullsized":"2024-02-26T06:21:28.907609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:45.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413574+0000","last_change":"2024-02-26T06:20:31.585190+0000","last_active":"2024-02-26T06:21:57.413574+0000","last_peered":"2024-02-26T06:21:57.413574+0000","last_clean":"2024-02-26T06:21:57.413574+0000","last_became_active":"2024-02-26T06:20:31.584883+0000","last_became_peered":"2024-02-26T06:20:31.584883+0000","last_unstale":"2024-02-26T06:21:57.413574+0000","last_undegraded":"2024-02-26T06:21:57.413574+0000","last_fullsized":"2024-02-26T06:21:57.413574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:47:51.843824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.904061+0000","last_change":"2024-02-26T06:20:31.075573+0000","last_active":"2024-02-26T06:21:28.904061+0000","last_peered":"2024-02-26T06:21:28.904061+0000","last_clean":"2024-02-26T06:21:28.904061+0000","last_became_active":"2024-02-26T06:20:31.075153+0000","last_became_peered":"2024-02-26T06:20:31.075153+0000","last_unstale":"2024-02-26T06:21:28.904061+0000","last_undegraded":"2024-02-26T06:21:28.904061+0000","last_fullsized":"2024-02-26T06:21:28.904061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:43.886858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432306+0000","last_change":"2024-02-26T06:20:31.604400+0000","last_active":"2024-02-26T06:21:57.432306+0000","last_peered":"2024-02-26T06:21:57.432306+0000","last_clean":"2024-02-26T06:21:57.432306+0000","last_became_active":"2024-02-26T06:20:31.604068+0000","last_became_peered":"2024-02-26T06:20:31.604068+0000","last_unstale":"2024-02-26T06:21:57.432306+0000","last_undegraded":"2024-02-26T06:21:57.432306+0000","last_fullsized":"2024-02-26T06:21:57.432306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:28.792297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902928+0000","last_change":"2024-02-26T06:20:31.072896+0000","last_active":"2024-02-26T06:21:28.902928+0000","last_peered":"2024-02-26T06:21:28.902928+0000","last_clean":"2024-02-26T06:21:28.902928+0000","last_became_active":"2024-02-26T06:20:31.072571+0000","last_became_peered":"2024-02-26T06:20:31.072571+0000","last_unstale":"2024-02-26T06:21:28.902928+0000","last_undegraded":"2024-02-26T06:21:28.902928+0000","last_fullsized":"2024-02-26T06:21:28.902928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:39.054737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.415995+0000","last_change":"2024-02-26T06:21:28.906840+0000","last_active":"2024-02-26T06:21:57.415995+0000","last_peered":"2024-02-26T06:21:57.415995+0000","last_clean":"2024-02-26T06:21:57.415995+0000","last_became_active":"2024-02-26T06:21:28.905881+0000","last_became_peered":"2024-02-26T06:21:28.905881+0000","last_unstale":"2024-02-26T06:21:57.415995+0000","last_undegraded":"2024-02-26T06:21:57.415995+0000","last_fullsized":"2024-02-26T06:21:57.415995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:47.754145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.411939+0000","last_change":"2024-02-26T06:20:31.588989+0000","last_active":"2024-02-26T06:21:57.411939+0000","last_peered":"2024-02-26T06:21:57.411939+0000","last_clean":"2024-02-26T06:21:57.411939+0000","last_became_active":"2024-02-26T06:20:31.588683+0000","last_became_peered":"2024-02-26T06:20:31.588683+0000","last_unstale":"2024-02-26T06:21:57.411939+0000","last_undegraded":"2024-02-26T06:21:57.411939+0000","last_fullsized":"2024-02-26T06:21:57.411939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:29.687622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":175,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:23:06.418562+0000","last_change":"2024-02-26T06:20:31.081111+0000","last_active":"2024-02-26T06:23:06.418562+0000","last_peered":"2024-02-26T06:23:06.418562+0000","last_clean":"2024-02-26T06:23:06.418562+0000","last_became_active":"2024-02-26T06:20:31.080788+0000","last_became_peered":"2024-02-26T06:20:31.080788+0000","last_unstale":"2024-02-26T06:23:06.418562+0000","last_undegraded":"2024-02-26T06:23:06.418562+0000","last_fullsized":"2024-02-26T06:23:06.418562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:44:34.400256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440841+0000","last_change":"2024-02-26T06:21:29.211650+0000","last_active":"2024-02-26T06:21:57.440841+0000","last_peered":"2024-02-26T06:21:57.440841+0000","last_clean":"2024-02-26T06:21:57.440841+0000","last_became_active":"2024-02-26T06:21:29.210407+0000","last_became_peered":"2024-02-26T06:21:29.210407+0000","last_unstale":"2024-02-26T06:21:57.440841+0000","last_undegraded":"2024-02-26T06:21:57.440841+0000","last_fullsized":"2024-02-26T06:21:57.440841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:22:00.359506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898366+0000","last_change":"2024-02-26T06:20:31.061702+0000","last_active":"2024-02-26T06:21:28.898366+0000","last_peered":"2024-02-26T06:21:28.898366+0000","last_clean":"2024-02-26T06:21:28.898366+0000","last_became_active":"2024-02-26T06:20:31.061358+0000","last_became_peered":"2024-02-26T06:20:31.061358+0000","last_unstale":"2024-02-26T06:21:28.898366+0000","last_undegraded":"2024-02-26T06:21:28.898366+0000","last_fullsized":"2024-02-26T06:21:28.898366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:25:41.652032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903059+0000","last_change":"2024-02-26T06:19:10.834318+0000","last_active":"2024-02-26T06:21:28.903059+0000","last_peered":"2024-02-26T06:21:28.903059+0000","last_clean":"2024-02-26T06:21:28.903059+0000","last_became_active":"2024-02-26T06:19:10.833869+0000","last_became_peered":"2024-02-26T06:19:10.833869+0000","last_unstale":"2024-02-26T06:21:28.903059+0000","last_undegraded":"2024-02-26T06:21:28.903059+0000","last_fullsized":"2024-02-26T06:21:28.903059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:15:16.238016+0000","last_clean_scrub_stamp":"2024-02-26T06:15:16.238016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:57.078533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.428655+0000","last_change":"2024-02-26T06:20:31.600738+0000","last_active":"2024-02-26T06:21:57.428655+0000","last_peered":"2024-02-26T06:21:57.428655+0000","last_clean":"2024-02-26T06:21:57.428655+0000","last_became_active":"2024-02-26T06:20:31.599144+0000","last_became_peered":"2024-02-26T06:20:31.599144+0000","last_unstale":"2024-02-26T06:21:57.428655+0000","last_undegraded":"2024-02-26T06:21:57.428655+0000","last_fullsized":"2024-02-26T06:21:57.428655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:01:16.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903180+0000","last_change":"2024-02-26T06:20:31.063563+0000","last_active":"2024-02-26T06:21:28.903180+0000","last_peered":"2024-02-26T06:21:28.903180+0000","last_clean":"2024-02-26T06:21:28.903180+0000","last_became_active":"2024-02-26T06:20:31.063212+0000","last_became_peered":"2024-02-26T06:20:31.063212+0000","last_unstale":"2024-02-26T06:21:28.903180+0000","last_undegraded":"2024-02-26T06:21:28.903180+0000","last_fullsized":"2024-02-26T06:21:28.903180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:16.480553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.907970+0000","last_change":"2024-02-26T06:20:31.075104+0000","last_active":"2024-02-26T06:21:28.907970+0000","last_peered":"2024-02-26T06:21:28.907970+0000","last_clean":"2024-02-26T06:21:28.907970+0000","last_became_active":"2024-02-26T06:20:31.074777+0000","last_became_peered":"2024-02-26T06:20:31.074777+0000","last_unstale":"2024-02-26T06:21:28.907970+0000","last_undegraded":"2024-02-26T06:21:28.907970+0000","last_fullsized":"2024-02-26T06:21:28.907970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:32:20.511703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.417127+0000","last_change":"2024-02-26T06:20:31.605038+0000","last_active":"2024-02-26T06:21:57.417127+0000","last_peered":"2024-02-26T06:21:57.417127+0000","last_clean":"2024-02-26T06:21:57.417127+0000","last_became_active":"2024-02-26T06:20:31.603329+0000","last_became_peered":"2024-02-26T06:20:31.603329+0000","last_unstale":"2024-02-26T06:21:57.417127+0000","last_undegraded":"2024-02-26T06:21:57.417127+0000","last_fullsized":"2024-02-26T06:21:57.417127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:45.066836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:22:41.014176+0000","last_change":"2024-02-26T06:20:31.084613+0000","last_active":"2024-02-26T06:22:41.014176+0000","last_peered":"2024-02-26T06:22:41.014176+0000","last_clean":"2024-02-26T06:22:41.014176+0000","last_became_active":"2024-02-26T06:20:31.084021+0000","last_became_peered":"2024-02-26T06:20:31.084021+0000","last_unstale":"2024-02-26T06:22:41.014176+0000","last_undegraded":"2024-02-26T06:22:41.014176+0000","last_fullsized":"2024-02-26T06:22:41.014176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:52:42.096437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438954+0000","last_change":"2024-02-26T06:20:31.590367+0000","last_active":"2024-02-26T06:21:57.438954+0000","last_peered":"2024-02-26T06:21:57.438954+0000","last_clean":"2024-02-26T06:21:57.438954+0000","last_became_active":"2024-02-26T06:20:31.582914+0000","last_became_peered":"2024-02-26T06:20:31.582914+0000","last_unstale":"2024-02-26T06:21:57.438954+0000","last_undegraded":"2024-02-26T06:21:57.438954+0000","last_fullsized":"2024-02-26T06:21:57.438954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:02:47.496527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.898485+0000","last_change":"2024-02-26T06:20:31.070190+0000","last_active":"2024-02-26T06:21:28.898485+0000","last_peered":"2024-02-26T06:21:28.898485+0000","last_clean":"2024-02-26T06:21:28.898485+0000","last_became_active":"2024-02-26T06:20:31.069854+0000","last_became_peered":"2024-02-26T06:20:31.069854+0000","last_unstale":"2024-02-26T06:21:28.898485+0000","last_undegraded":"2024-02-26T06:21:28.898485+0000","last_fullsized":"2024-02-26T06:21:28.898485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:57.728821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.442447+0000","last_change":"2024-02-26T06:20:31.595388+0000","last_active":"2024-02-26T06:21:57.442447+0000","last_peered":"2024-02-26T06:21:57.442447+0000","last_clean":"2024-02-26T06:21:57.442447+0000","last_became_active":"2024-02-26T06:20:31.594970+0000","last_became_peered":"2024-02-26T06:20:31.594970+0000","last_unstale":"2024-02-26T06:21:57.442447+0000","last_undegraded":"2024-02-26T06:21:57.442447+0000","last_fullsized":"2024-02-26T06:21:57.442447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:12:52.182988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.902880+0000","last_change":"2024-02-26T06:20:31.066256+0000","last_active":"2024-02-26T06:21:28.902880+0000","last_peered":"2024-02-26T06:21:28.902880+0000","last_clean":"2024-02-26T06:21:28.902880+0000","last_became_active":"2024-02-26T06:20:31.065956+0000","last_became_peered":"2024-02-26T06:20:31.065956+0000","last_unstale":"2024-02-26T06:21:28.902880+0000","last_undegraded":"2024-02-26T06:21:28.902880+0000","last_fullsized":"2024-02-26T06:21:28.902880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:35.405114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.407926+0000","last_change":"2024-02-26T06:20:31.585102+0000","last_active":"2024-02-26T06:21:57.407926+0000","last_peered":"2024-02-26T06:21:57.407926+0000","last_clean":"2024-02-26T06:21:57.407926+0000","last_became_active":"2024-02-26T06:20:31.584351+0000","last_became_peered":"2024-02-26T06:20:31.584351+0000","last_unstale":"2024-02-26T06:21:57.407926+0000","last_undegraded":"2024-02-26T06:21:57.407926+0000","last_fullsized":"2024-02-26T06:21:57.407926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:09.708752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.897704+0000","last_change":"2024-02-26T06:20:31.079664+0000","last_active":"2024-02-26T06:21:28.897704+0000","last_peered":"2024-02-26T06:21:28.897704+0000","last_clean":"2024-02-26T06:21:28.897704+0000","last_became_active":"2024-02-26T06:20:31.079234+0000","last_became_peered":"2024-02-26T06:20:31.079234+0000","last_unstale":"2024-02-26T06:21:28.897704+0000","last_undegraded":"2024-02-26T06:21:28.897704+0000","last_fullsized":"2024-02-26T06:21:28.897704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:33:37.407215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439613+0000","last_change":"2024-02-26T06:20:31.597416+0000","last_active":"2024-02-26T06:21:57.439613+0000","last_peered":"2024-02-26T06:21:57.439613+0000","last_clean":"2024-02-26T06:21:57.439613+0000","last_became_active":"2024-02-26T06:20:31.597094+0000","last_became_peered":"2024-02-26T06:20:31.597094+0000","last_unstale":"2024-02-26T06:21:57.439613+0000","last_undegraded":"2024-02-26T06:21:57.439613+0000","last_fullsized":"2024-02-26T06:21:57.439613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:24.774298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903155+0000","last_change":"2024-02-26T06:20:31.074372+0000","last_active":"2024-02-26T06:21:28.903155+0000","last_peered":"2024-02-26T06:21:28.903155+0000","last_clean":"2024-02-26T06:21:28.903155+0000","last_became_active":"2024-02-26T06:20:31.074041+0000","last_became_peered":"2024-02-26T06:20:31.074041+0000","last_unstale":"2024-02-26T06:21:28.903155+0000","last_undegraded":"2024-02-26T06:21:28.903155+0000","last_fullsized":"2024-02-26T06:21:28.903155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:01.367668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441913+0000","last_change":"2024-02-26T06:20:31.599588+0000","last_active":"2024-02-26T06:21:57.441913+0000","last_peered":"2024-02-26T06:21:57.441913+0000","last_clean":"2024-02-26T06:21:57.441913+0000","last_became_active":"2024-02-26T06:20:31.599240+0000","last_became_peered":"2024-02-26T06:20:31.599240+0000","last_unstale":"2024-02-26T06:21:57.441913+0000","last_undegraded":"2024-02-26T06:21:57.441913+0000","last_fullsized":"2024-02-26T06:21:57.441913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:07:14.799565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914009+0000","last_change":"2024-02-26T06:21:28.914009+0000","last_active":"2024-02-26T06:21:28.914009+0000","last_peered":"2024-02-26T06:21:28.914009+0000","last_clean":"2024-02-26T06:21:28.914009+0000","last_became_active":"2024-02-26T06:21:28.906712+0000","last_became_peered":"2024-02-26T06:21:28.906712+0000","last_unstale":"2024-02-26T06:21:28.914009+0000","last_undegraded":"2024-02-26T06:21:28.914009+0000","last_fullsized":"2024-02-26T06:21:28.914009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:21:36.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.432870+0000","last_change":"2024-02-26T06:20:31.603955+0000","last_active":"2024-02-26T06:21:57.432870+0000","last_peered":"2024-02-26T06:21:57.432870+0000","last_clean":"2024-02-26T06:21:57.432870+0000","last_became_active":"2024-02-26T06:20:31.603635+0000","last_became_peered":"2024-02-26T06:20:31.603635+0000","last_unstale":"2024-02-26T06:21:57.432870+0000","last_undegraded":"2024-02-26T06:21:57.432870+0000","last_fullsized":"2024-02-26T06:21:57.432870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:56:59.060204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.903796+0000","last_change":"2024-02-26T06:20:31.074917+0000","last_active":"2024-02-26T06:21:28.903796+0000","last_peered":"2024-02-26T06:21:28.903796+0000","last_clean":"2024-02-26T06:21:28.903796+0000","last_became_active":"2024-02-26T06:20:31.074590+0000","last_became_peered":"2024-02-26T06:20:31.074590+0000","last_unstale":"2024-02-26T06:21:28.903796+0000","last_undegraded":"2024-02-26T06:21:28.903796+0000","last_fullsized":"2024-02-26T06:21:28.903796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:43.160268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.420847+0000","last_change":"2024-02-26T06:20:31.582236+0000","last_active":"2024-02-26T06:21:57.420847+0000","last_peered":"2024-02-26T06:21:57.420847+0000","last_clean":"2024-02-26T06:21:57.420847+0000","last_became_active":"2024-02-26T06:20:31.581851+0000","last_became_peered":"2024-02-26T06:20:31.581851+0000","last_unstale":"2024-02-26T06:21:57.420847+0000","last_undegraded":"2024-02-26T06:21:57.420847+0000","last_fullsized":"2024-02-26T06:21:57.420847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:53.214338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:28.914195+0000","last_change":"2024-02-26T06:20:31.068179+0000","last_active":"2024-02-26T06:21:28.914195+0000","last_peered":"2024-02-26T06:21:28.914195+0000","last_clean":"2024-02-26T06:21:28.914195+0000","last_became_active":"2024-02-26T06:20:31.067888+0000","last_became_peered":"2024-02-26T06:20:31.067888+0000","last_unstale":"2024-02-26T06:21:28.914195+0000","last_undegraded":"2024-02-26T06:21:28.914195+0000","last_fullsized":"2024-02-26T06:21:28.914195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:47:39.223749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441130+0000","last_change":"2024-02-26T06:20:31.598820+0000","last_active":"2024-02-26T06:21:57.441130+0000","last_peered":"2024-02-26T06:21:57.441130+0000","last_clean":"2024-02-26T06:21:57.441130+0000","last_became_active":"2024-02-26T06:20:31.598469+0000","last_became_peered":"2024-02-26T06:20:31.598469+0000","last_unstale":"2024-02-26T06:21:57.441130+0000","last_undegraded":"2024-02-26T06:21:57.441130+0000","last_fullsized":"2024-02-26T06:21:57.441130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:41:15.713748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:29.207193+0000","last_change":"2024-02-26T06:20:31.080988+0000","last_active":"2024-02-26T06:21:29.207193+0000","last_peered":"2024-02-26T06:21:29.207193+0000","last_clean":"2024-02-26T06:21:29.207193+0000","last_became_active":"2024-02-26T06:20:31.080654+0000","last_became_peered":"2024-02-26T06:20:31.080654+0000","last_unstale":"2024-02-26T06:21:29.207193+0000","last_undegraded":"2024-02-26T06:21:29.207193+0000","last_fullsized":"2024-02-26T06:21:29.207193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:29.832511+0000","last_clean_scrub_stamp":"2024-02-26T06:20:29.832511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:16:50.315162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414613+0000","last_change":"2024-02-26T06:20:31.586057+0000","last_active":"2024-02-26T06:21:57.414613+0000","last_peered":"2024-02-26T06:21:57.414613+0000","last_clean":"2024-02-26T06:21:57.414613+0000","last_became_active":"2024-02-26T06:20:31.585756+0000","last_became_peered":"2024-02-26T06:20:31.585756+0000","last_unstale":"2024-02-26T06:21:57.414613+0000","last_undegraded":"2024-02-26T06:21:57.414613+0000","last_fullsized":"2024-02-26T06:21:57.414613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:59:07.032287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413388+0000","last_change":"2024-02-26T06:20:31.589067+0000","last_active":"2024-02-26T06:21:57.413388+0000","last_peered":"2024-02-26T06:21:57.413388+0000","last_clean":"2024-02-26T06:21:57.413388+0000","last_became_active":"2024-02-26T06:20:31.588769+0000","last_became_peered":"2024-02-26T06:20:31.588769+0000","last_unstale":"2024-02-26T06:21:57.413388+0000","last_undegraded":"2024-02-26T06:21:57.413388+0000","last_fullsized":"2024-02-26T06:21:57.413388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:43.211127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.412546+0000","last_change":"2024-02-26T06:20:31.583838+0000","last_active":"2024-02-26T06:21:57.412546+0000","last_peered":"2024-02-26T06:21:57.412546+0000","last_clean":"2024-02-26T06:21:57.412546+0000","last_became_active":"2024-02-26T06:20:31.583524+0000","last_became_peered":"2024-02-26T06:20:31.583524+0000","last_unstale":"2024-02-26T06:21:57.412546+0000","last_undegraded":"2024-02-26T06:21:57.412546+0000","last_fullsized":"2024-02-26T06:21:57.412546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:21.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414078+0000","last_change":"2024-02-26T06:20:31.597901+0000","last_active":"2024-02-26T06:21:57.414078+0000","last_peered":"2024-02-26T06:21:57.414078+0000","last_clean":"2024-02-26T06:21:57.414078+0000","last_became_active":"2024-02-26T06:20:31.597633+0000","last_became_peered":"2024-02-26T06:20:31.597633+0000","last_unstale":"2024-02-26T06:21:57.414078+0000","last_undegraded":"2024-02-26T06:21:57.414078+0000","last_fullsized":"2024-02-26T06:21:57.414078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:14.084579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.438958+0000","last_change":"2024-02-26T06:21:29.210694+0000","last_active":"2024-02-26T06:21:57.438958+0000","last_peered":"2024-02-26T06:21:57.438958+0000","last_clean":"2024-02-26T06:21:57.438958+0000","last_became_active":"2024-02-26T06:21:29.209421+0000","last_became_peered":"2024-02-26T06:21:29.209421+0000","last_unstale":"2024-02-26T06:21:57.438958+0000","last_undegraded":"2024-02-26T06:21:57.438958+0000","last_fullsized":"2024-02-26T06:21:57.438958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:46.511066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.430929+0000","last_change":"2024-02-26T06:20:31.605866+0000","last_active":"2024-02-26T06:21:57.430929+0000","last_peered":"2024-02-26T06:21:57.430929+0000","last_clean":"2024-02-26T06:21:57.430929+0000","last_became_active":"2024-02-26T06:20:31.605404+0000","last_became_peered":"2024-02-26T06:20:31.605404+0000","last_unstale":"2024-02-26T06:21:57.430929+0000","last_undegraded":"2024-02-26T06:21:57.430929+0000","last_fullsized":"2024-02-26T06:21:57.430929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:03:25.055462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.413749+0000","last_change":"2024-02-26T06:20:31.585214+0000","last_active":"2024-02-26T06:21:57.413749+0000","last_peered":"2024-02-26T06:21:57.413749+0000","last_clean":"2024-02-26T06:21:57.413749+0000","last_became_active":"2024-02-26T06:20:31.584895+0000","last_became_peered":"2024-02-26T06:20:31.584895+0000","last_unstale":"2024-02-26T06:21:57.413749+0000","last_undegraded":"2024-02-26T06:21:57.413749+0000","last_fullsized":"2024-02-26T06:21:57.413749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:38.507499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.415101+0000","last_change":"2024-02-26T06:20:31.582189+0000","last_active":"2024-02-26T06:21:57.415101+0000","last_peered":"2024-02-26T06:21:57.415101+0000","last_clean":"2024-02-26T06:21:57.415101+0000","last_became_active":"2024-02-26T06:20:31.580729+0000","last_became_peered":"2024-02-26T06:20:31.580729+0000","last_unstale":"2024-02-26T06:21:57.415101+0000","last_undegraded":"2024-02-26T06:21:57.415101+0000","last_fullsized":"2024-02-26T06:21:57.415101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:21.337261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441676+0000","last_change":"2024-02-26T06:20:31.588894+0000","last_active":"2024-02-26T06:21:57.441676+0000","last_peered":"2024-02-26T06:21:57.441676+0000","last_clean":"2024-02-26T06:21:57.441676+0000","last_became_active":"2024-02-26T06:20:31.586362+0000","last_became_peered":"2024-02-26T06:20:31.586362+0000","last_unstale":"2024-02-26T06:21:57.441676+0000","last_undegraded":"2024-02-26T06:21:57.441676+0000","last_fullsized":"2024-02-26T06:21:57.441676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:28:30.009901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.439174+0000","last_change":"2024-02-26T06:20:31.608264+0000","last_active":"2024-02-26T06:21:57.439174+0000","last_peered":"2024-02-26T06:21:57.439174+0000","last_clean":"2024-02-26T06:21:57.439174+0000","last_became_active":"2024-02-26T06:20:31.607855+0000","last_became_peered":"2024-02-26T06:20:31.607855+0000","last_unstale":"2024-02-26T06:21:57.439174+0000","last_undegraded":"2024-02-26T06:21:57.439174+0000","last_fullsized":"2024-02-26T06:21:57.439174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:20.169489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441839+0000","last_change":"2024-02-26T06:20:31.591000+0000","last_active":"2024-02-26T06:21:57.441839+0000","last_peered":"2024-02-26T06:21:57.441839+0000","last_clean":"2024-02-26T06:21:57.441839+0000","last_became_active":"2024-02-26T06:20:31.588201+0000","last_became_peered":"2024-02-26T06:20:31.588201+0000","last_unstale":"2024-02-26T06:21:57.441839+0000","last_undegraded":"2024-02-26T06:21:57.441839+0000","last_fullsized":"2024-02-26T06:21:57.441839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:47:59.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.441280+0000","last_change":"2024-02-26T06:20:31.590398+0000","last_active":"2024-02-26T06:21:57.441280+0000","last_peered":"2024-02-26T06:21:57.441280+0000","last_clean":"2024-02-26T06:21:57.441280+0000","last_became_active":"2024-02-26T06:20:31.590042+0000","last_became_peered":"2024-02-26T06:20:31.590042+0000","last_unstale":"2024-02-26T06:21:57.441280+0000","last_undegraded":"2024-02-26T06:21:57.441280+0000","last_fullsized":"2024-02-26T06:21:57.441280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:40:01.233978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.440874+0000","last_change":"2024-02-26T06:20:31.611698+0000","last_active":"2024-02-26T06:21:57.440874+0000","last_peered":"2024-02-26T06:21:57.440874+0000","last_clean":"2024-02-26T06:21:57.440874+0000","last_became_active":"2024-02-26T06:20:31.610979+0000","last_became_peered":"2024-02-26T06:20:31.610979+0000","last_unstale":"2024-02-26T06:21:57.440874+0000","last_undegraded":"2024-02-26T06:21:57.440874+0000","last_fullsized":"2024-02-26T06:21:57.440874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:44.604373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-26T06:21:57.414255+0000","last_change":"2024-02-26T06:20:31.582016+0000","last_active":"2024-02-26T06:21:57.414255+0000","last_peered":"2024-02-26T06:21:57.414255+0000","last_clean":"2024-02-26T06:21:57.414255+0000","last_became_active":"2024-02-26T06:20:31.581689+0000","last_became_peered":"2024-02-26T06:20:31.581689+0000","last_unstale":"2024-02-26T06:21:57.414255+0000","last_undegraded":"2024-02-26T06:21:57.414255+0000","last_fullsized":"2024-02-26T06:21:57.414255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:31.035666+0000","last_clean_scrub_stamp":"2024-02-26T06:20:31.035666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:04:11.030182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284221,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71257,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":926,"num_read_kb":536,"num_write":171,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":60985,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":213771,"data_compressed":13382,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351022,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459479,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94999999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.94599999999999995,"5min":1.018,"15min":1.018},"last":0.85699999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.61099999999999999,"15min":0.58299999999999996},"min":{"1min":0.53600000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70799999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46800000000000003,"15min":0.44700000000000001},"min":{"1min":0.40500000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.63,"15min":0.59599999999999997},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.871},{"interface":"front","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77600000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76900000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77200000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.95099999999999996,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.81100000000000005,"15min":0.83499999999999996},"min":{"1min":0.58999999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.179,"15min":1.179},"last":1.1000000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94899999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93400000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.82699999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:22:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.499,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.79500000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.91000000000000003,"15min":0.89400000000000002},"min":{"1min":0.61299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.2389999999999999,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":0.92200000000000004},"min":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.997}]},{"osd":10,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.623,"5min":0.501,"15min":0.501},"max":{"1min":0.92100000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58799999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.91500000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.84499999999999997}]}]},{"osd":10,"up_from":73,"seq":313532612659,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1118185,"data_compressed":43797,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77700000000000002,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88,"5min":1.069,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.075,"5min":6.2229999999999999,"15min":6.2229999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.45100000000000001,"15min":0.45700000000000002},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58099999999999996,"5min":0.624,"15min":0.624},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.28899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55700000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.434}]},{"osd":3,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80500000000000005,"15min":0.82999999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0129999999999999,"5min":1.403,"15min":1.403},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.57299999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66600000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.498}]},{"osd":5,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.79000000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.378},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.54000000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82199999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.85099999999999998,"15min":0.879},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0669999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91700000000000004,"15min":0.93200000000000005},"min":{"1min":0.63900000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.218,"5min":1.284,"15min":1.284},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:22:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93100000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88800000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.97199999999999998,"15min":0.98599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2190000000000001,"5min":1.266,"15min":1.266},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.63}]},{"osd":11,"last update":"Mon Feb 26 06:22:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.752,"15min":0.752},"min":{"1min":0.63600000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.94299999999999995,"5min":0.998,"15min":0.998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.58299999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.58799999999999997}]}]},{"osd":9,"up_from":68,"seq":292057776184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":459603,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.3500000000000001,"15min":1.448},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.9450000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.194,"5min":1.3100000000000001,"15min":1.3660000000000001},"min":{"1min":0.52800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.161,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2010000000000001,"15min":1.2649999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.704,"5min":1.923,"15min":1.923},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2809999999999999,"15min":1.377},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0030000000000001,"5min":2.407,"15min":2.407},"last":1.3}]},{"osd":2,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4179999999999999,"15min":1.5169999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.028,"5min":2.609,"15min":2.609},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.514,"15min":1.5920000000000001},"min":{"1min":0.72999999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.5219999999999998,"5min":3.681,"15min":3.681},"last":1.4330000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.704,"15min":1.8799999999999999},"min":{"1min":0.621,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.0659999999999998,"5min":3.016,"15min":3.016},"last":1.675},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.724,"15min":1.8080000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.3330000000000002,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.6180000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.7170000000000001,"15min":1.8100000000000001},"min":{"1min":0.78400000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.6019999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.6040000000000001,"15min":1.776},"min":{"1min":0.67500000000000004,"5min":0.624,"15min":0.624},"max":{"1min":2.1040000000000001,"5min":3.98,"15min":3.98},"last":1.798}]},{"osd":5,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":1.8049999999999999,"15min":1.9710000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.379,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":1.895},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.8360000000000001,"15min":1.968},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.3679999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.726}]},{"osd":6,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":2.085,"15min":2.2469999999999999},"min":{"1min":0.878,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6349999999999998,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.952},{"interface":"front","average":{"1min":1.8200000000000001,"5min":2.1179999999999999,"15min":2.2719999999999998},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.7480000000000002,"5min":3.835,"15min":3.835},"last":1.6000000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.8029999999999999,"5min":1.9970000000000001,"15min":2.169},"min":{"1min":0.85199999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.452,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":2.0920000000000001},{"interface":"front","average":{"1min":1.887,"5min":2.0489999999999999,"15min":2.1709999999999998},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.4929999999999999,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":1.8520000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.786,"5min":2.0920000000000001,"15min":2.2040000000000002},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":2.661,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":1.655},{"interface":"front","average":{"1min":1.8839999999999999,"5min":2.1259999999999999,"15min":2.3180000000000001},"min":{"1min":0.91100000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.5750000000000002,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":2.0049999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:14 2024","interfaces":[{"interface":"back","average":{"1min":2.1509999999999998,"5min":2.1859999999999999,"15min":2.266},"min":{"1min":1.3180000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.677,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":2.1589999999999998},{"interface":"front","average":{"1min":2.1200000000000001,"5min":2.0750000000000002,"15min":2.0950000000000002},"min":{"1min":1.091,"5min":1.087,"15min":1.087},"max":{"1min":2.7799999999999998,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":1.7709999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037823,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":459411,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.57299999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99299999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.55800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1299999999999999,"5min":1.198,"15min":1.198},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.46500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.46899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44},"min":{"1min":0.36099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65800000000000003,"5min":0.755,"15min":0.755},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.629,"5min":0.747,"15min":0.747},"last":0.70199999999999996}]},{"osd":3,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78600000000000003,"15min":0.77100000000000002},"min":{"1min":0.55900000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.22,"5min":1.379,"15min":1.379},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":0.86099999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.196,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.85999999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.60599999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2390000000000001,"5min":1.359,"15min":1.359},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.67200000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.1759999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.016}]},{"osd":7,"last update":"Mon Feb 26 06:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65100000000000002,"15min":0.64000000000000001},"min":{"1min":0.54400000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.75700000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88600000000000001,"15min":0.874},"min":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1439999999999999,"5min":1.28,"15min":1.28},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.93600000000000005},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.23,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.92500000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68500000000000005,"15min":0.66300000000000003},"min":{"1min":0.58899999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94899999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.72299999999999998,"15min":0.70099999999999996},"min":{"1min":0.61499999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.87}]},{"osd":11,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.85199999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.58499999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.89100000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051904,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:19:40 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40799999999999997,"15min":0.41999999999999998},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.624,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.434,"15min":0.436},"min":{"1min":0.31900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.35099999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76200000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65800000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":0.83299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1100000000000001,"5min":1.591,"15min":1.591},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.48899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93000000000000005,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.495,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.309},{"interface":"front","average":{"1min":0.499,"5min":0.47999999999999998,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67900000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42399999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.45600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.497},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60099999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":0.85599999999999998},"min":{"1min":0.58999999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.1060000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.93000000000000005,"15min":0.88400000000000001},"min":{"1min":0.5,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0429999999999999,"5min":7.4119999999999999,"15min":7.4119999999999999},"last":0.67100000000000004}]},{"osd":8,"last update":"Mon Feb 26 06:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.443},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.49099999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.83799999999999997,"5min":0.873,"15min":0.873},"last":0.59099999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:22:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.88,"15min":0.92100000000000004},"min":{"1min":0.57399999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.1599999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.748},{"interface":"front","average":{"1min":0.877,"5min":0.90800000000000003,"15min":0.94199999999999995},"min":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.89100000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63800000000000001,"15min":0.65700000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68799999999999994}]},{"osd":11,"last update":"Mon Feb 26 06:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.61799999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78500000000000003,"15min":0.78000000000000003},"min":{"1min":0.65800000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.57799999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462986,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":458992,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2689999999999999,"15min":1.2729999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.915,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.264,"5min":1.3180000000000001,"15min":1.2509999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0920000000000001,"15min":1.079},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1299999999999999,"15min":1.1060000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.855,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3520000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7829999999999999,"5min":6.4930000000000003,"15min":6.4930000000000003},"last":1}]},{"osd":3,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.645,"15min":1.5860000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.655,"15min":1.665},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4470000000000001,"15min":1.429},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.627,"15min":1.5900000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Feb 26 06:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.649,"15min":1.625},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.4769999999999999,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.6950000000000001,"15min":1.7070000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.863,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":0.73199999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.7629999999999999,"15min":1.6200000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.4990000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.802,"15min":1.742},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.4350000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.76700000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:22:13 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.7969999999999999,"15min":1.7390000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.9139999999999999,"15min":1.837},"min":{"1min":0.86899999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.8050000000000002,"5min":6.625,"15min":6.625},"last":1.026}]},{"osd":9,"last update":"Mon Feb 26 06:22:48 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":2.0779999999999998,"15min":2.2400000000000002},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.536,"5min":3.258,"15min":3.258},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.907,"5min":2.133,"15min":2.2519999999999998},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8109999999999999,"5min":3.347,"15min":3.347},"last":1.1759999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:13 2024","interfaces":[{"interface":"back","average":{"1min":1.873,"5min":2.036,"15min":2.0790000000000002},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":1.073},{"interface":"front","average":{"1min":1.8089999999999999,"5min":2.1120000000000001,"15min":2.1789999999999998},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":1.202}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":525590,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.76300000000000001,"15min":0.69199999999999995},"min":{"1min":0.58499999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3069999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1,"15min":0.80500000000000005},"min":{"1min":0.58899999999999997,"5min":0.44,"15min":0.40400000000000003},"max":{"1min":1.0760000000000001,"5min":16.690999999999999,"15min":16.690999999999999},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.73199999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.72399999999999998,"5min":9.2629999999999999,"15min":9.2629999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.54800000000000004,"15min":0.55500000000000005},"min":{"1min":0.46899999999999997,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40500000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78600000000000003,"15min":0.749},"min":{"1min":0.55500000000000005,"5min":0.377,"15min":0.377},"max":{"1min":1.1279999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78300000000000003,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.073,"5min":1.391,"15min":1.391},"last":0.77700000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40500000000000003},"min":{"1min":0.374,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72199999999999998,"5min":0.748,"15min":0.748},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.443,"15min":0.40899999999999997},"min":{"1min":0.32800000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.65300000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28499999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.79600000000000004,"15min":0.66700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.032,"5min":16.672999999999998,"15min":16.672999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.625,"15min":0.60699999999999998},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.44}]},{"osd":6,"last update":"Mon Feb 26 06:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.81599999999999995,"15min":0.78800000000000003},"min":{"1min":0.55300000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0429999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.92900000000000005,"15min":0.873},"min":{"1min":0.64600000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.496,"5min":8.8729999999999993,"15min":8.8729999999999993},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:22:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54500000000000004,"15min":0.51000000000000001},"min":{"1min":0.40300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47199999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63300000000000001,"15min":0.61099999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.66000000000000003},"min":{"1min":0.52900000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89800000000000002,"5min":8.9380000000000006,"15min":8.9380000000000006},"last":0.45800000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:22:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91800000000000004,"15min":0.92000000000000004},"min":{"1min":0.70199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1519999999999999,"5min":1.224,"15min":1.224},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.03,"15min":1.01},"min":{"1min":0.64900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.105,"5min":8.8979999999999997,"15min":8.8979999999999997},"last":1.0509999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.49399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0369999999999999,"5min":1.04,"15min":1.04},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79500000000000004,"15min":0.79100000000000004},"min":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.53800000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460263,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:22:03 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.004,"15min":0.84599999999999997},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.46300000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.01,"15min":0.86699999999999999},"min":{"1min":0.624,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":1.4390000000000001,"5min":1.498,"15min":1.649},"last":1.347}]},{"osd":2,"last update":"Mon Feb 26 06:22:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1160000000000001,"15min":0.93799999999999994},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"max":{"1min":1.621,"5min":6.6070000000000002,"15min":6.6070000000000002},"last":0.997},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2869999999999999,"15min":1.024},"min":{"1min":0.72299999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0600000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.081}]},{"osd":3,"last update":"Mon Feb 26 06:22:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.7370000000000001,"15min":1.7649999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2280000000000002,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.9399999999999999},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.909,"15min":1.931},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.419,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":2.2400000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:22:20 2024","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.3460000000000001,"15min":1.107},"min":{"1min":0.75700000000000001,"5min":0.436,"15min":0.436},"max":{"1min":2.0339999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.486},{"interface":"front","average":{"1min":1.357,"5min":1.288,"15min":1.171},"min":{"1min":0.69799999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.131}]},{"osd":5,"last update":"Mon Feb 26 06:22:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.631,"15min":1.4950000000000001},"min":{"1min":0.80000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2639999999999998,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":2.1549999999999998},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5549999999999999,"15min":1.4119999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.1440000000000001,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":1.6120000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:22:20 2024","interfaces":[{"interface":"back","average":{"1min":2.1000000000000001,"5min":1.8959999999999999,"15min":1.8300000000000001},"min":{"1min":0.98099999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":2.3130000000000002},{"interface":"front","average":{"1min":2.0790000000000002,"5min":1.873,"15min":1.857},"min":{"1min":0.67000000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.819}]},{"osd":7,"last update":"Mon Feb 26 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.73,"15min":1.7190000000000001},"min":{"1min":1.2150000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.45},{"interface":"front","average":{"1min":1.73,"5min":1.599,"15min":1.577},"min":{"1min":1.123,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:22:15 2024","interfaces":[{"interface":"back","average":{"1min":2.0179999999999998,"5min":1.8360000000000001,"15min":1.7529999999999999},"min":{"1min":1.012,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6589999999999998,"5min":2.698,"15min":2.698},"last":2.375},{"interface":"front","average":{"1min":1.964,"5min":1.849,"15min":1.7709999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":1.716}]},{"osd":9,"last update":"Mon Feb 26 06:22:54 2024","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.859,"15min":1.9570000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.2709999999999999,"5min":2.754,"15min":2.754},"last":1.222},{"interface":"front","average":{"1min":1.9490000000000001,"5min":2.1110000000000002,"15min":2.1019999999999999},"min":{"1min":1.0369999999999999,"5min":0.746,"15min":0.746},"max":{"1min":2.706,"5min":3.101,"15min":3.101},"last":2.4550000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:22:15 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":2.0339999999999998,"15min":2.0539999999999998},"min":{"1min":0.91500000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7679999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":2.0310000000000001},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.8180000000000001,"15min":1.835},"min":{"1min":0.84899999999999998,"5min":0.75,"15min":0.75},"max":{"1min":2.581,"5min":2.581,"15min":2.581},"last":1.2629999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:22:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0329999999999999,"5min":1.994,"15min":1.9199999999999999},"min":{"1min":1.0569999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7989999999999999,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":2.4929999999999999},{"interface":"front","average":{"1min":1.921,"5min":2.0179999999999998,"15min":2.016},"min":{"1min":0.96199999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.714,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":2.214}]}]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":460333,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73399999999999999,"15min":0.69299999999999995},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73999999999999999,"15min":0.70699999999999996},"min":{"1min":0.51000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93200000000000005,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.35299999999999998,"15min":0.35199999999999998},"max":{"1min":0.78100000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70999999999999996}]},{"osd":3,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.77300000000000002,"15min":0.72599999999999998},"min":{"1min":0.57299999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.0649999999999999,"5min":1.359,"15min":1.359},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.73599999999999999},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.127,"15min":1.367},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.621},"min":{"1min":0.439,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73899999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:22:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47399999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.69999999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53800000000000003,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.76700000000000002},"min":{"1min":0.58499999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1180000000000001,"5min":1.206,"15min":1.206},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.81200000000000006}]},{"osd":7,"last update":"Mon Feb 26 06:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47399999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.60599999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:22:51 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.95299999999999996,"15min":0.94799999999999995},"min":{"1min":0.64400000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98399999999999999,"15min":0.98799999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.96899999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.52400000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.60099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.85799999999999998}]},{"osd":11,"last update":"Mon Feb 26 06:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.72299999999999998},"min":{"1min":0.54400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.78300000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":459303,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2150000000000001,"15min":1.0900000000000001},"min":{"1min":0.54300000000000004,"5min":0.433,"15min":0.433},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4630000000000001,"15min":1.173},"min":{"1min":0.80100000000000005,"5min":0.375,"15min":0.375},"max":{"1min":2.0840000000000001,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":0.80100000000000005}]},{"osd":1,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2090000000000001,"15min":1.056},"min":{"1min":0.68500000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.853,"5min":2.036,"15min":2.036},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.252,"5min":1.1890000000000001,"15min":1.0940000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.786,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Feb 26 06:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.345,"15min":1.204},"min":{"1min":0.81799999999999995,"5min":0.70099999999999996,"15min":0.66400000000000003},"max":{"1min":1.7589999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.194,"5min":1.1220000000000001,"15min":1.0549999999999999},"min":{"1min":0.84399999999999997,"5min":0.441,"15min":0.441},"max":{"1min":1.7989999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.84399999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:22:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3740000000000001,"15min":1.2370000000000001},"min":{"1min":0.83599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.863,"5min":2.613,"15min":2.613},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.577,"5min":1.5920000000000001,"15min":1.4039999999999999},"min":{"1min":0.85599999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.0990000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.751}]},{"osd":5,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.506,"15min":1.393},"min":{"1min":0.90800000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.405,"15min":1.351},"min":{"1min":0.93799999999999994,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.266,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Feb 26 06:22:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.6439999999999999,"15min":1.5069999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.488,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.8620000000000001,"5min":1.853,"15min":1.673},"min":{"1min":0.82099999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.806,"5min":4.681,"15min":4.681},"last":0.86499999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.915,"5min":1.8089999999999999,"15min":1.75},"min":{"1min":1.2869999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.6560000000000001,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.8700000000000001,"5min":1.865,"15min":1.7250000000000001},"min":{"1min":1.381,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.468,"5min":3.544,"15min":3.544},"last":0.83399999999999996}]},{"osd":8,"last update":"Mon Feb 26 06:20:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.806},"min":{"1min":0.97399999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.587,"15min":1.5649999999999999},"min":{"1min":0.84999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.23,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.143}]},{"osd":9,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.9590000000000001,"5min":1.9670000000000001,"15min":2.0379999999999998},"min":{"1min":1.4610000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.7610000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":2.1429999999999998,"5min":2.2290000000000001,"15min":2.403},"min":{"1min":1.619,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.734,"5min":4.266,"15min":4.266},"last":0.98099999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:22:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8280000000000001,"5min":1.9450000000000001,"15min":2.0099999999999998},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.998},{"interface":"front","average":{"1min":1.9530000000000001,"5min":2.1699999999999999,"15min":2.2589999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.6509999999999998,"5min":4.0609999999999999,"15min":4.0609999999999999},"last":0.89500000000000002}]},{"osd":11,"last update":"Mon Feb 26 06:22:34 2024","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.9399999999999999,"15min":1.946},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.6150000000000002,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":2.0800000000000001,"5min":2.0739999999999998,"15min":2.073},"min":{"1min":1.0609999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.5739999999999998,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.92900000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888150,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":526878,"data_compressed":15495,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.50600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.079,"5min":1.093,"15min":1.093},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39700000000000002,"15min":0.36099999999999999},"min":{"1min":0.34100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.38},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.34999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.41499999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.45200000000000001,"5min":0.371,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69299999999999995,"15min":0.57499999999999996},"min":{"1min":0.48299999999999998,"5min":0.379,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":12.468,"15min":12.468},"last":0.51900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76200000000000001,"15min":0.71199999999999997},"min":{"1min":0.59999999999999998,"5min":0.48299999999999998,"15min":0.39800000000000002},"max":{"1min":0.94399999999999995,"5min":1.167,"15min":1.167},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.746},"min":{"1min":0.49099999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1579999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.53900000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:22:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.54800000000000004},"min":{"1min":0.52300000000000002,"5min":0.42599999999999999,"15min":0.41099999999999998},"max":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.626,"15min":0.56999999999999995},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48099999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79000000000000004,"15min":0.73799999999999999},"min":{"1min":0.628,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97499999999999998,"5min":1.327,"15min":1.327},"last":0.629},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80200000000000005,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.1180000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Feb 26 06:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.42399999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.50700000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.70199999999999996,"15min":0.66600000000000004},"min":{"1min":0.59499999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59399999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1699999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.044,"15min":1},"min":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2190000000000001,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.76700000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.61799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.5,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.49399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.67800000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724625,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1051056,"data_compressed":41950,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72299999999999998,"15min":0.67700000000000005},"min":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.159,"15min":1.159},"last":0.749},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.0649999999999999,"5min":1.204,"15min":1.204},"last":0.91900000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79500000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70499999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.315,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61099999999999999,"5min":0.624,"15min":0.624},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42399999999999999,"15min":0.40600000000000003},"min":{"1min":0.34799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.75,"15min":0.75},"last":0.34200000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.80800000000000005,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.78600000000000003,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78400000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Feb 26 06:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82799999999999996,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.135,"5min":1.242,"15min":1.242},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83799999999999997,"15min":0.80900000000000005},"min":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.089,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.044}]},{"osd":7,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65600000000000003,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46899999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68300000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48299999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.64500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.381,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.369},{"interface":"front","average":{"1min":1.01,"5min":0.999,"15min":1.04},"min":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.2789999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.69999999999999996,"15min":0.68400000000000005},"min":{"1min":0.56100000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.90100000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34502,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36143,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66805,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":68093,"data_compressed":9688,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68915,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:23:10.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:23:10.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 -- 172.21.15.49:0/1379679765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff918102b70 msgr2=0x7ff918109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 --2- 172.21.15.49:0/1379679765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff918102b70 0x7ff918109400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff910009ca0 tx=0x7ff91002c100 comp rx=0 tx=0).stop 2024-02-26T06:23:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 -- 172.21.15.49:0/1379679765 shutdown_connections 2024-02-26T06:23:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 --2- 172.21.15.49:0/1379679765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9181021b0 0x7ff918102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 --2- 172.21.15.49:0/1379679765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff918102b70 0x7ff918109400 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 --2- 172.21.15.49:0/1379679765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff918100fc0 0x7ff9181013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 -- 172.21.15.49:0/1379679765 >> 172.21.15.49:0/1379679765 conn(0x7ff9180fc510 msgr2=0x7ff9180fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 -- 172.21.15.49:0/1379679765 shutdown_connections 2024-02-26T06:23:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.795+0000 7ff917200640 1 -- 172.21.15.49:0/1379679765 wait complete. 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.796+0000 7ff917200640 1 Processor -- start 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.796+0000 7ff917200640 1 -- start start 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.796+0000 7ff917200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff918100fc0 0x7ff918074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.796+0000 7ff917200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9181021b0 0x7ff918073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.796+0000 7ff917200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff918102b70 0x7ff918073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.796+0000 7ff917200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff918074210 con 0x7ff9181021b0 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.796+0000 7ff917200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff918074380 con 0x7ff918100fc0 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.796+0000 7ff917200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9181a10a0 con 0x7ff918102b70 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9181021b0 0x7ff918073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9181021b0 0x7ff918073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34884/0 (socket says 172.21.15.49:34884) 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 -- 172.21.15.49:0/3403436162 learned_addr learned my addr 172.21.15.49:0/3403436162 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff916800640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff918102b70 0x7ff918073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915e00640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff918100fc0 0x7ff918074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 -- 172.21.15.49:0/3403436162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff918102b70 msgr2=0x7ff918073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff918102b70 0x7ff918073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 -- 172.21.15.49:0/3403436162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff918100fc0 msgr2=0x7ff918074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff918100fc0 0x7ff918074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 -- 172.21.15.49:0/3403436162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff910009900 con 0x7ff9181021b0 2024-02-26T06:23:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.797+0000 7ff915400640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9181021b0 0x7ff918073570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff90400ea20 tx=0x7ff90400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.798+0000 7ff90ea00640 1 -- 172.21.15.49:0/3403436162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904009700 con 0x7ff9181021b0 2024-02-26T06:23:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.798+0000 7ff90ea00640 1 -- 172.21.15.49:0/3403436162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff90400cd80 con 0x7ff9181021b0 2024-02-26T06:23:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.798+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9181a1350 con 0x7ff9181021b0 2024-02-26T06:23:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.798+0000 7ff90ea00640 1 -- 172.21.15.49:0/3403436162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904018840 con 0x7ff9181021b0 2024-02-26T06:23:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.798+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9181a1860 con 0x7ff9181021b0 2024-02-26T06:23:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.799+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9181082e0 con 0x7ff9181021b0 2024-02-26T06:23:10.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.802+0000 7ff90ea00640 1 -- 172.21.15.49:0/3403436162 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff904010a80 con 0x7ff9181021b0 2024-02-26T06:23:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.803+0000 7ff90ea00640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff8fc0a20c0 0x7ff8fc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.803+0000 7ff90ea00640 1 -- 172.21.15.49:0/3403436162 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff904014070 con 0x7ff9181021b0 2024-02-26T06:23:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.804+0000 7ff915e00640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff8fc0a20c0 0x7ff8fc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.804+0000 7ff915e00640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff8fc0a20c0 0x7ff8fc0a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff918102010 tx=0x7ff908005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:10.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:10.808+0000 7ff90ea00640 1 -- 172.21.15.49:0/3403436162 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff904052d20 con 0x7ff9181021b0 2024-02-26T06:23:11.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.008+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff91802ce40 con 0x7ff9181021b0 2024-02-26T06:23:11.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.010+0000 7ff90ea00640 1 -- 172.21.15.49:0/3403436162 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7ff9040d1030 con 0x7ff9181021b0 2024-02-26T06:23:11.012 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:11.012 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:23:11.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff8fc0a20c0 msgr2=0x7ff8fc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff8fc0a20c0 0x7ff8fc0a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff918102010 tx=0x7ff908005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:11.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9181021b0 msgr2=0x7ff918073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9181021b0 0x7ff918073570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff90400ea20 tx=0x7ff90400eee0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 shutdown_connections 2024-02-26T06:23:11.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7ff8fc0a20c0 0x7ff8fc0a4580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff918102b70 0x7ff918073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9181021b0 0x7ff918073570 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 --2- 172.21.15.49:0/3403436162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff918100fc0 0x7ff918074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.013+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 >> 172.21.15.49:0/3403436162 conn(0x7ff9180fc510 msgr2=0x7ff9181054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:11.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.014+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 shutdown_connections 2024-02-26T06:23:11.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.014+0000 7ff917200640 1 -- 172.21.15.49:0/3403436162 wait complete. 2024-02-26T06:23:11.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:23:11.033 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:23:11.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:10 smithi049 ceph-mon[31696]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:11.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:10 smithi049 ceph-mon[31696]: pgmap v399: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-26T06:23:11.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:10 smithi049 ceph-mon[31696]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:11.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.260+0000 7fbd92c00640 1 -- 172.21.15.49:0/1457081021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 msgr2=0x7fbd8c104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.260+0000 7fbd92c00640 1 --2- 172.21.15.49:0/1457081021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 0x7fbd8c104910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009900 tx=0x7fbd7c02bc40 comp rx=0 tx=0).stop 2024-02-26T06:23:11.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.261+0000 7fbd92c00640 1 -- 172.21.15.49:0/1457081021 shutdown_connections 2024-02-26T06:23:11.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.261+0000 7fbd92c00640 1 --2- 172.21.15.49:0/1457081021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd8c104e50 0x7fbd8c10b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.261+0000 7fbd92c00640 1 --2- 172.21.15.49:0/1457081021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 0x7fbd8c104910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.261+0000 7fbd92c00640 1 --2- 172.21.15.49:0/1457081021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd8c1032a0 0x7fbd8c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.261+0000 7fbd92c00640 1 -- 172.21.15.49:0/1457081021 >> 172.21.15.49:0/1457081021 conn(0x7fbd8c0fe830 msgr2=0x7fbd8c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:11.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.261+0000 7fbd92c00640 1 -- 172.21.15.49:0/1457081021 shutdown_connections 2024-02-26T06:23:11.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.261+0000 7fbd92c00640 1 -- 172.21.15.49:0/1457081021 wait complete. 2024-02-26T06:23:11.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92c00640 1 Processor -- start 2024-02-26T06:23:11.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92c00640 1 -- start start 2024-02-26T06:23:11.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd8c1032a0 0x7fbd8c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:11.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 0x7fbd8c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd8c104e50 0x7fbd8c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c1a0dc0 con 0x7fbd8c104490 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c1a0f30 con 0x7fbd8c1032a0 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c1a10f0 con 0x7fbd8c104e50 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd90e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 0x7fbd8c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd90e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 0x7fbd8c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34904/0 (socket says 172.21.15.49:34904) 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd90e00640 1 -- 172.21.15.49:0/834501713 learned_addr learned my addr 172.21.15.49:0/834501713 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.262+0000 7fbd92200640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd8c104e50 0x7fbd8c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.263+0000 7fbd91800640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd8c1032a0 0x7fbd8c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.263+0000 7fbd90e00640 1 -- 172.21.15.49:0/834501713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd8c104e50 msgr2=0x7fbd8c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.263+0000 7fbd90e00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd8c104e50 0x7fbd8c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.263+0000 7fbd90e00640 1 -- 172.21.15.49:0/834501713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd8c1032a0 msgr2=0x7fbd8c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.263+0000 7fbd90e00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd8c1032a0 0x7fbd8c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.263+0000 7fbd90e00640 1 -- 172.21.15.49:0/834501713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd80009490 con 0x7fbd8c104490 2024-02-26T06:23:11.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.263+0000 7fbd92200640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd8c104e50 0x7fbd8c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:11.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.263+0000 7fbd90e00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 0x7fbd8c199810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c02c6b0 tx=0x7fbd7c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:11.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.264+0000 7fbd8a000640 1 -- 172.21.15.49:0/834501713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0044c0 con 0x7fbd8c104490 2024-02-26T06:23:11.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.264+0000 7fbd8a000640 1 -- 172.21.15.49:0/834501713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd7c0052d0 con 0x7fbd8c104490 2024-02-26T06:23:11.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.264+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c009560 con 0x7fbd8c104490 2024-02-26T06:23:11.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.264+0000 7fbd8a000640 1 -- 172.21.15.49:0/834501713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c003e80 con 0x7fbd8c104490 2024-02-26T06:23:11.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.264+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c1a16f0 con 0x7fbd8c104490 2024-02-26T06:23:11.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.265+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8c10a5c0 con 0x7fbd8c104490 2024-02-26T06:23:11.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.267+0000 7fbd8a000640 1 -- 172.21.15.49:0/834501713 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbd7c005440 con 0x7fbd8c104490 2024-02-26T06:23:11.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.268+0000 7fbd8a000640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbd740a20c0 0x7fbd740a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:11.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.268+0000 7fbd8a000640 1 -- 172.21.15.49:0/834501713 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fbd7c0f0680 con 0x7fbd8c104490 2024-02-26T06:23:11.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.269+0000 7fbd91800640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbd740a20c0 0x7fbd740a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:11.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.269+0000 7fbd91800640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbd740a20c0 0x7fbd740a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c1042f0 tx=0x7fbd80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:11.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.272+0000 7fbd8a000640 1 -- 172.21.15.49:0/834501713 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd7c0b9700 con 0x7fbd8c104490 2024-02-26T06:23:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:10 smithi080 ceph-mon[36206]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:10 smithi080 ceph-mon[36206]: pgmap v399: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-26T06:23:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:10 smithi080 ceph-mon[36206]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:10 smithi115 ceph-mon[36426]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:10 smithi115 ceph-mon[36426]: pgmap v399: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-26T06:23:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:10 smithi115 ceph-mon[36426]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:11.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.475+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd8c02cda0 con 0x7fbd8c104490 2024-02-26T06:23:11.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.476+0000 7fbd8a000640 1 -- 172.21.15.49:0/834501713 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd7c0430a0 con 0x7fbd8c104490 2024-02-26T06:23:11.476 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:23:11.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbd740a20c0 msgr2=0x7fbd740a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbd740a20c0 0x7fbd740a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c1042f0 tx=0x7fbd80005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:11.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 msgr2=0x7fbd8c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 0x7fbd8c199810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c02c6b0 tx=0x7fbd7c0040c0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 shutdown_connections 2024-02-26T06:23:11.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fbd740a20c0 0x7fbd740a4580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd8c104e50 0x7fbd8c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd8c104490 0x7fbd8c199810 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 --2- 172.21.15.49:0/834501713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd8c1032a0 0x7fbd8c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 >> 172.21.15.49:0/834501713 conn(0x7fbd8c0fe830 msgr2=0x7fbd8c107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:11.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.479+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 shutdown_connections 2024-02-26T06:23:11.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.480+0000 7fbd92c00640 1 -- 172.21.15.49:0/834501713 wait complete. 2024-02-26T06:23:11.498 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:23:11.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.733+0000 7f3de8a00640 1 -- 172.21.15.49:0/4145145607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de4103280 msgr2=0x7f3de41036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.733+0000 7f3de8a00640 1 --2- 172.21.15.49:0/4145145607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de4103280 0x7f3de41036a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009860 tx=0x7f3dd402bb70 comp rx=0 tx=0).stop 2024-02-26T06:23:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.733+0000 7f3de8a00640 1 -- 172.21.15.49:0/4145145607 shutdown_connections 2024-02-26T06:23:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.733+0000 7f3de8a00640 1 --2- 172.21.15.49:0/4145145607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4104e30 0x7f3de410b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.733+0000 7f3de8a00640 1 --2- 172.21.15.49:0/4145145607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3de4104470 0x7f3de41048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.733+0000 7f3de8a00640 1 --2- 172.21.15.49:0/4145145607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de4103280 0x7f3de41036a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.733+0000 7f3de8a00640 1 -- 172.21.15.49:0/4145145607 >> 172.21.15.49:0/4145145607 conn(0x7f3de40fe7f0 msgr2=0x7f3de4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.734+0000 7f3de8a00640 1 -- 172.21.15.49:0/4145145607 shutdown_connections 2024-02-26T06:23:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.734+0000 7f3de8a00640 1 -- 172.21.15.49:0/4145145607 wait complete. 2024-02-26T06:23:11.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.735+0000 7f3de8a00640 1 Processor -- start 2024-02-26T06:23:11.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.735+0000 7f3de8a00640 1 -- start start 2024-02-26T06:23:11.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.735+0000 7f3de8a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4103280 0x7f3de4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:11.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.735+0000 7f3de8a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3de4104470 0x7f3de4199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:11.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de4104e30 0x7f3de41a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:11.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de8a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de41a0e40 con 0x7f3de4103280 2024-02-26T06:23:11.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de8a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de41a0fb0 con 0x7f3de4104e30 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de8a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de41a1170 con 0x7f3de4104470 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4103280 0x7f3de4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4103280 0x7f3de4199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34924/0 (socket says 172.21.15.49:34924) 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3400640 1 -- 172.21.15.49:0/1836737866 learned_addr learned my addr 172.21.15.49:0/1836737866 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de2a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3de4104470 0x7f3de4199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3e00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de4104e30 0x7f3de41a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3400640 1 -- 172.21.15.49:0/1836737866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3de4104470 msgr2=0x7f3de4199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3400640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3de4104470 0x7f3de4199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3400640 1 -- 172.21.15.49:0/1836737866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de4104e30 msgr2=0x7f3de41a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3400640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de4104e30 0x7f3de41a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.736+0000 7f3de3400640 1 -- 172.21.15.49:0/1836737866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc009900 con 0x7f3de4103280 2024-02-26T06:23:11.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.737+0000 7f3de3400640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4103280 0x7f3de4199350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402c0b0 tx=0x7f3dd40045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:11.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.737+0000 7f3ddbe00640 1 -- 172.21.15.49:0/1836737866 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd402c510 con 0x7f3de4103280 2024-02-26T06:23:11.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.737+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd4009490 con 0x7f3de4103280 2024-02-26T06:23:11.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.737+0000 7f3ddbe00640 1 -- 172.21.15.49:0/1836737866 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dd402cb50 con 0x7f3de4103280 2024-02-26T06:23:11.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.737+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de41a1620 con 0x7f3de4103280 2024-02-26T06:23:11.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.737+0000 7f3ddbe00640 1 -- 172.21.15.49:0/1836737866 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4032600 con 0x7f3de4103280 2024-02-26T06:23:11.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.739+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de410a5c0 con 0x7f3de4103280 2024-02-26T06:23:11.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.741+0000 7f3ddbe00640 1 -- 172.21.15.49:0/1836737866 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3dd403b040 con 0x7f3de4103280 2024-02-26T06:23:11.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.742+0000 7f3ddbe00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3dc80a20c0 0x7f3dc80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:11.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.742+0000 7f3ddbe00640 1 -- 172.21.15.49:0/1836737866 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3dd40efd40 con 0x7f3de4103280 2024-02-26T06:23:11.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.743+0000 7f3de2a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3dc80a20c0 0x7f3dc80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:11.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.743+0000 7f3de2a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3dc80a20c0 0x7f3dc80a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3de419a860 tx=0x7f3dd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:11.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.744+0000 7f3ddbe00640 1 -- 172.21.15.49:0/1836737866 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3dd40b8d50 con 0x7f3de4103280 2024-02-26T06:23:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.952+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3de410a030 con 0x7f3de4103280 2024-02-26T06:23:11.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.953+0000 7f3ddbe00640 1 -- 172.21.15.49:0/1836737866 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3dd4041020 con 0x7f3de4103280 2024-02-26T06:23:11.952 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:23:11.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.955+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3dc80a20c0 msgr2=0x7f3dc80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.955+0000 7f3de8a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3dc80a20c0 0x7f3dc80a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3de419a860 tx=0x7f3dd0005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.955+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4103280 msgr2=0x7f3de4199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.955+0000 7f3de8a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4103280 0x7f3de4199350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402c0b0 tx=0x7f3dd40045a0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.956+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 shutdown_connections 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.956+0000 7f3de8a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3dc80a20c0 0x7f3dc80a4580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.956+0000 7f3de8a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de4104e30 0x7f3de41a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.956+0000 7f3de8a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3de4104470 0x7f3de4199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.956+0000 7f3de8a00640 1 --2- 172.21.15.49:0/1836737866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4103280 0x7f3de4199350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.956+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 >> 172.21.15.49:0/1836737866 conn(0x7f3de40fe7f0 msgr2=0x7f3de4107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.956+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 shutdown_connections 2024-02-26T06:23:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:11.956+0000 7f3de8a00640 1 -- 172.21.15.49:0/1836737866 wait complete. 2024-02-26T06:23:11.978 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:23:12.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:11 smithi049 ceph-mon[31696]: from='client.25294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:12.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:11 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3403436162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:12.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:11 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/834501713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:12.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 -- 172.21.15.49:0/3509750344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c104470 msgr2=0x7efd7c1048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 --2- 172.21.15.49:0/3509750344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c104470 0x7efd7c1048f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd68009900 tx=0x7efd6802bc40 comp rx=0 tx=0).stop 2024-02-26T06:23:12.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 -- 172.21.15.49:0/3509750344 shutdown_connections 2024-02-26T06:23:12.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 --2- 172.21.15.49:0/3509750344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd7c104e30 0x7efd7c10b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 --2- 172.21.15.49:0/3509750344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c104470 0x7efd7c1048f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 --2- 172.21.15.49:0/3509750344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c103280 0x7efd7c1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 -- 172.21.15.49:0/3509750344 >> 172.21.15.49:0/3509750344 conn(0x7efd7c0fe7f0 msgr2=0x7efd7c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:12.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 -- 172.21.15.49:0/3509750344 shutdown_connections 2024-02-26T06:23:12.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.212+0000 7efd81c00640 1 -- 172.21.15.49:0/3509750344 wait complete. 2024-02-26T06:23:12.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.214+0000 7efd81c00640 1 Processor -- start 2024-02-26T06:23:12.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.214+0000 7efd81c00640 1 -- start start 2024-02-26T06:23:12.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.214+0000 7efd81c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c103280 0x7efd7c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:12.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.214+0000 7efd81c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd7c104470 0x7efd7c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:12.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.214+0000 7efd81c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c104e30 0x7efd7c1a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.214+0000 7efd81c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c1a0e00 con 0x7efd7c103280 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.214+0000 7efd81c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c1a0f70 con 0x7efd7c104470 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.214+0000 7efd81c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c1a1130 con 0x7efd7c104e30 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c103280 0x7efd7c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c103280 0x7efd7c199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34934/0 (socket says 172.21.15.49:34934) 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7be00640 1 -- 172.21.15.49:0/2556772906 learned_addr learned my addr 172.21.15.49:0/2556772906 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7b400640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd7c104470 0x7efd7c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd81200640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c104e30 0x7efd7c1a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7be00640 1 -- 172.21.15.49:0/2556772906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c104e30 msgr2=0x7efd7c1a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7be00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c104e30 0x7efd7c1a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7be00640 1 -- 172.21.15.49:0/2556772906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd7c104470 msgr2=0x7efd7c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7be00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd7c104470 0x7efd7c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.215+0000 7efd7be00640 1 -- 172.21.15.49:0/2556772906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68009560 con 0x7efd7c103280 2024-02-26T06:23:12.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.216+0000 7efd7be00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c103280 0x7efd7c199310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00e9c0 tx=0x7efd6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:12.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.216+0000 7efd78c00640 1 -- 172.21.15.49:0/2556772906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c009700 con 0x7efd7c103280 2024-02-26T06:23:12.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.216+0000 7efd78c00640 1 -- 172.21.15.49:0/2556772906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd6c00cd80 con 0x7efd7c103280 2024-02-26T06:23:12.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.216+0000 7efd78c00640 1 -- 172.21.15.49:0/2556772906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c010430 con 0x7efd7c103280 2024-02-26T06:23:12.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.216+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c1a1410 con 0x7efd7c103280 2024-02-26T06:23:12.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.216+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c1a1940 con 0x7efd7c103280 2024-02-26T06:23:12.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.217+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c10a540 con 0x7efd7c103280 2024-02-26T06:23:12.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.220+0000 7efd78c00640 1 -- 172.21.15.49:0/2556772906 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efd6c003680 con 0x7efd7c103280 2024-02-26T06:23:12.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.222+0000 7efd78c00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efd600a2140 0x7efd600a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:12.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.222+0000 7efd78c00640 1 -- 172.21.15.49:0/2556772906 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7efd6c014070 con 0x7efd7c103280 2024-02-26T06:23:12.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.223+0000 7efd7b400640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efd600a2140 0x7efd600a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:12.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.223+0000 7efd7b400640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efd600a2140 0x7efd600a4600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd6802c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.227+0000 7efd78c00640 1 -- 172.21.15.49:0/2556772906 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd6c096fa0 con 0x7efd7c103280 2024-02-26T06:23:12.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:11 smithi080 ceph-mon[36206]: from='client.25294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:12.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:11 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3403436162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:12.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:11 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/834501713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:12.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:11 smithi115 ceph-mon[36426]: from='client.25294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:23:12.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:11 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3403436162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:12.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:11 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/834501713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:23:12.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.430+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd7c02cdd0 con 0x7efd7c103280 2024-02-26T06:23:12.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.431+0000 7efd78c00640 1 -- 172.21.15.49:0/2556772906 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd6c09a660 con 0x7efd7c103280 2024-02-26T06:23:12.431 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:23:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efd600a2140 msgr2=0x7efd600a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efd600a2140 0x7efd600a4600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd6802c7f0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c103280 msgr2=0x7efd7c199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c103280 0x7efd7c199310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00e9c0 tx=0x7efd6c00ee80 comp rx=0 tx=0).stop 2024-02-26T06:23:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 shutdown_connections 2024-02-26T06:23:12.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7efd600a2140 0x7efd600a4600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd7c104470 0x7efd7c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd7c104e30 0x7efd7c1a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 --2- 172.21.15.49:0/2556772906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd7c103280 0x7efd7c199310 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 >> 172.21.15.49:0/2556772906 conn(0x7efd7c0fe7f0 msgr2=0x7efd7c107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 shutdown_connections 2024-02-26T06:23:12.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.436+0000 7efd81c00640 1 -- 172.21.15.49:0/2556772906 wait complete. 2024-02-26T06:23:12.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:23:12.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 -- 172.21.15.49:0/177151864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 msgr2=0x7f3e901037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 --2- 172.21.15.49:0/177151864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 0x7f3e901037e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009900 tx=0x7f3e8402bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:12.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 -- 172.21.15.49:0/177151864 shutdown_connections 2024-02-26T06:23:12.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 --2- 172.21.15.49:0/177151864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e90104f70 0x7f3e9010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 --2- 172.21.15.49:0/177151864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e901045b0 0x7f3e90104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 --2- 172.21.15.49:0/177151864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 0x7f3e901037e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 -- 172.21.15.49:0/177151864 >> 172.21.15.49:0/177151864 conn(0x7f3e900fe970 msgr2=0x7f3e90100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:12.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 -- 172.21.15.49:0/177151864 shutdown_connections 2024-02-26T06:23:12.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.685+0000 7f3e98200640 1 -- 172.21.15.49:0/177151864 wait complete. 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.686+0000 7f3e98200640 1 Processor -- start 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.686+0000 7f3e98200640 1 -- start start 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.686+0000 7f3e98200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 0x7f3e90199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e98200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e901045b0 0x7f3e901999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e98200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e90104f70 0x7f3e901a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e98200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901a0f60 con 0x7f3e901045b0 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e98200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901a10d0 con 0x7f3e901033c0 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e98200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901a1290 con 0x7f3e90104f70 2024-02-26T06:23:12.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e94e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e901045b0 0x7f3e901999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e94e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e901045b0 0x7f3e901999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34964/0 (socket says 172.21.15.49:34964) 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e94e00640 1 -- 172.21.15.49:0/3739926045 learned_addr learned my addr 172.21.15.49:0/3739926045 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e95800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 0x7f3e90199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e96200640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e90104f70 0x7f3e901a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e94e00640 1 -- 172.21.15.49:0/3739926045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e90104f70 msgr2=0x7f3e901a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e94e00640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e90104f70 0x7f3e901a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e94e00640 1 -- 172.21.15.49:0/3739926045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 msgr2=0x7f3e90199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e94e00640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 0x7f3e90199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e94e00640 1 -- 172.21.15.49:0/3739926045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80009840 con 0x7f3e901045b0 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.687+0000 7f3e95800640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 0x7f3e90199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.688+0000 7f3e94e00640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e901045b0 0x7f3e901999b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009d50 tx=0x7f3e8000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.688+0000 7f3e8e000640 1 -- 172.21.15.49:0/3739926045 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80003b70 con 0x7f3e901045b0 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.688+0000 7f3e8e000640 1 -- 172.21.15.49:0/3739926045 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e80003cd0 con 0x7f3e901045b0 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.688+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e84009560 con 0x7f3e901045b0 2024-02-26T06:23:12.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.688+0000 7f3e8e000640 1 -- 172.21.15.49:0/3739926045 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80010990 con 0x7f3e901045b0 2024-02-26T06:23:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.688+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e901a18f0 con 0x7f3e901045b0 2024-02-26T06:23:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.689+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9010a6e0 con 0x7f3e901045b0 2024-02-26T06:23:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.691+0000 7f3e8e000640 1 -- 172.21.15.49:0/3739926045 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3e80004000 con 0x7f3e901045b0 2024-02-26T06:23:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.693+0000 7f3e8e000640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e780a20c0 0x7f3e780a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.693+0000 7f3e8e000640 1 -- 172.21.15.49:0/3739926045 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3e80014070 con 0x7f3e901045b0 2024-02-26T06:23:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.693+0000 7f3e95800640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e780a20c0 0x7f3e780a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.694+0000 7f3e95800640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e780a20c0 0x7f3e780a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3e8402c240 tx=0x7f3e8402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.697+0000 7f3e8e000640 1 -- 172.21.15.49:0/3739926045 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e80097450 con 0x7f3e901045b0 2024-02-26T06:23:12.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.940+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e9010ee20 con 0x7f3e901045b0 2024-02-26T06:23:12.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.941+0000 7f3e8e000640 1 -- 172.21.15.49:0/3739926045 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11331 (secure 0 0 0) 0x7f3e8009ab10 con 0x7f3e901045b0 2024-02-26T06:23:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:12.941 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:23:03.712829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:23:12.942 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-02-26T06:23:12.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e780a20c0 msgr2=0x7f3e780a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e780a20c0 0x7f3e780a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3e8402c240 tx=0x7f3e8402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e901045b0 msgr2=0x7f3e901999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e901045b0 0x7f3e901999b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009d50 tx=0x7f3e8000c6a0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 shutdown_connections 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f3e780a20c0 0x7f3e780a4580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e90104f70 0x7f3e901a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e901045b0 0x7f3e901999b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 --2- 172.21.15.49:0/3739926045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e901033c0 0x7f3e90199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:12.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 >> 172.21.15.49:0/3739926045 conn(0x7f3e900fe970 msgr2=0x7f3e901078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:12.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.944+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 shutdown_connections 2024-02-26T06:23:12.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:12.945+0000 7f3e98200640 1 -- 172.21.15.49:0/3739926045 wait complete. 2024-02-26T06:23:12.966 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:23:13.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 -- 172.21.15.49:0/3233227258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 msgr2=0x7f32541035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 --2- 172.21.15.49:0/3233227258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 0x7f32541035f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3244009860 tx=0x7f324402bb70 comp rx=0 tx=0).stop 2024-02-26T06:23:13.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 -- 172.21.15.49:0/3233227258 shutdown_connections 2024-02-26T06:23:13.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 --2- 172.21.15.49:0/3233227258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3254104d80 0x7f325410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 --2- 172.21.15.49:0/3233227258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32541043c0 0x7f3254104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 --2- 172.21.15.49:0/3233227258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 0x7f32541035f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 -- 172.21.15.49:0/3233227258 >> 172.21.15.49:0/3233227258 conn(0x7f32540fe760 msgr2=0x7f3254100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 -- 172.21.15.49:0/3233227258 shutdown_connections 2024-02-26T06:23:13.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.192+0000 7f3253e00640 1 -- 172.21.15.49:0/3233227258 wait complete. 2024-02-26T06:23:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.193+0000 7f3253e00640 1 Processor -- start 2024-02-26T06:23:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.193+0000 7f3253e00640 1 -- start start 2024-02-26T06:23:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3253e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 0x7f3254199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3253e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32541043c0 0x7f32541997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3253e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3254104d80 0x7f32541a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3253e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32541a0d60 con 0x7f32541031d0 2024-02-26T06:23:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3253e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32541a0ed0 con 0x7f32541043c0 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3253e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32541a1090 con 0x7f3254104d80 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3252a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 0x7f3254199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3252a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 0x7f3254199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34990/0 (socket says 172.21.15.49:34990) 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3252a00640 1 -- 172.21.15.49:0/3648994085 learned_addr learned my addr 172.21.15.49:0/3648994085 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3252000640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32541043c0 0x7f32541997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.194+0000 7f3253400640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3254104d80 0x7f32541a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f3252a00640 1 -- 172.21.15.49:0/3648994085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3254104d80 msgr2=0x7f32541a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f3252a00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3254104d80 0x7f32541a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f3252a00640 1 -- 172.21.15.49:0/3648994085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32541043c0 msgr2=0x7f32541997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f3252a00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32541043c0 0x7f32541997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f3252a00640 1 -- 172.21.15.49:0/3648994085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c009840 con 0x7f32541031d0 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f3252000640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32541043c0 0x7f32541997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f3252a00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 0x7f3254199270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f324402c0b0 tx=0x7f3244005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f324b400640 1 -- 172.21.15.49:0/3648994085 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324402c510 con 0x7f32541031d0 2024-02-26T06:23:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f324b400640 1 -- 172.21.15.49:0/3648994085 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f324402cb50 con 0x7f32541031d0 2024-02-26T06:23:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3244009490 con 0x7f32541031d0 2024-02-26T06:23:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.195+0000 7f324b400640 1 -- 172.21.15.49:0/3648994085 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440035f0 con 0x7f32541031d0 2024-02-26T06:23:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.196+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32541a1540 con 0x7f32541031d0 2024-02-26T06:23:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.197+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325410a570 con 0x7f32541031d0 2024-02-26T06:23:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.199+0000 7f324b400640 1 -- 172.21.15.49:0/3648994085 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f324403b040 con 0x7f32541031d0 2024-02-26T06:23:13.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.200+0000 7f324b400640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32380a2140 0x7f32380a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.200+0000 7f324b400640 1 -- 172.21.15.49:0/3648994085 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f32440efbc0 con 0x7f32541031d0 2024-02-26T06:23:13.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.200+0000 7f3252000640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32380a2140 0x7f32380a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.201+0000 7f3252000640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32380a2140 0x7f32380a4600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f325419a780 tx=0x7f3240009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.202+0000 7f324b400640 1 -- 172.21.15.49:0/3648994085 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32440b8ba0 con 0x7f32541031d0 2024-02-26T06:23:13.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:12 smithi080 ceph-mon[36206]: pgmap v400: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-26T06:23:13.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:12 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1836737866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:13.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:12 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2556772906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:13.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:12 smithi115 ceph-mon[36426]: pgmap v400: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-26T06:23:13.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:12 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1836737866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:13.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:12 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2556772906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:13.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.404+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f325402cdd0 con 0x7f32541031d0 2024-02-26T06:23:13.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.408+0000 7f324b400640 1 -- 172.21.15.49:0/3648994085 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22704 (secure 0 0 0) 0x7f3244039090 con 0x7f32541031d0 2024-02-26T06:23:13.408 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:13.408 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:21:28.893557+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:23:13.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32380a2140 msgr2=0x7f32380a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32380a2140 0x7f32380a4600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f325419a780 tx=0x7f3240009260 comp rx=0 tx=0).stop 2024-02-26T06:23:13.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 msgr2=0x7f3254199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 0x7f3254199270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f324402c0b0 tx=0x7f3244005d50 comp rx=0 tx=0).stop 2024-02-26T06:23:13.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 shutdown_connections 2024-02-26T06:23:13.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f32380a2140 0x7f32380a4600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32541043c0 0x7f32541997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3254104d80 0x7f32541a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 --2- 172.21.15.49:0/3648994085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32541031d0 0x7f3254199270 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 >> 172.21.15.49:0/3648994085 conn(0x7f32540fe760 msgr2=0x7f3254107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 shutdown_connections 2024-02-26T06:23:13.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.411+0000 7f3253e00640 1 -- 172.21.15.49:0/3648994085 wait complete. 2024-02-26T06:23:13.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:12 smithi049 ceph-mon[31696]: pgmap v400: 97 pgs: 97 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-26T06:23:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:12 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1836737866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:23:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:12 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2556772906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:23:13.429 DEBUG:tasks.fwd_scrub:Yielding 2024-02-26T06:23:13.430 INFO:teuthology.run_tasks:Running task exec... 2024-02-26T06:23:13.439 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T06:23:13.439 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:23:13.439 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-02-26T06:23:13.439 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-26T06:23:13.439 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:23:13.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:13.440 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:23:13.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.633+0000 7f10f6e00640 1 -- 172.21.15.49:0/960612270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f0100ff0 msgr2=0x7f10f0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.633+0000 7f10f6e00640 1 --2- 172.21.15.49:0/960612270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f0100ff0 0x7f10f0101410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009900 tx=0x7f10e402bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:13.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.634+0000 7f10f6e00640 1 -- 172.21.15.49:0/960612270 shutdown_connections 2024-02-26T06:23:13.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.634+0000 7f10f6e00640 1 --2- 172.21.15.49:0/960612270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10f0102ba0 0x7f10f0109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.634+0000 7f10f6e00640 1 --2- 172.21.15.49:0/960612270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f01021e0 0x7f10f0102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.634+0000 7f10f6e00640 1 --2- 172.21.15.49:0/960612270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f0100ff0 0x7f10f0101410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.634+0000 7f10f6e00640 1 -- 172.21.15.49:0/960612270 >> 172.21.15.49:0/960612270 conn(0x7f10f00fc560 msgr2=0x7f10f00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.634+0000 7f10f6e00640 1 -- 172.21.15.49:0/960612270 shutdown_connections 2024-02-26T06:23:13.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.634+0000 7f10f6e00640 1 -- 172.21.15.49:0/960612270 wait complete. 2024-02-26T06:23:13.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.635+0000 7f10f6e00640 1 Processor -- start 2024-02-26T06:23:13.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.635+0000 7f10f6e00640 1 -- start start 2024-02-26T06:23:13.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.635+0000 7f10f6e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10f0100ff0 0x7f10f0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.635+0000 7f10f6e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f01021e0 0x7f10f01975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.635+0000 7f10f6e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f0102ba0 0x7f10f019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.635+0000 7f10f6e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f019eb70 con 0x7f10f01021e0 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.635+0000 7f10f6e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f019ece0 con 0x7f10f0100ff0 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.635+0000 7f10f6e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f019eea0 con 0x7f10f0102ba0 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f5000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f01021e0 0x7f10f01975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f5a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10f0100ff0 0x7f10f0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f5000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f01021e0 0x7f10f01975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35016/0 (socket says 172.21.15.49:35016) 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f6400640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f0102ba0 0x7f10f019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f5000640 1 -- 172.21.15.49:0/2468907800 learned_addr learned my addr 172.21.15.49:0/2468907800 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f6400640 1 -- 172.21.15.49:0/2468907800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10f0100ff0 msgr2=0x7f10f0197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f6400640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10f0100ff0 0x7f10f0197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f6400640 1 -- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f01021e0 msgr2=0x7f10f01975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f6400640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f01021e0 0x7f10f01975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f6400640 1 -- 172.21.15.49:0/2468907800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009560 con 0x7f10f0102ba0 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f5a00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10f0100ff0 0x7f10f0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:13.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.636+0000 7f10f6400640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f0102ba0 0x7f10f019e630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009fd0 tx=0x7f10e800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.637+0000 7f10ee000640 1 -- 172.21.15.49:0/2468907800 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800ce20 con 0x7f10f0102ba0 2024-02-26T06:23:13.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.637+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f019f180 con 0x7f10f0102ba0 2024-02-26T06:23:13.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.637+0000 7f10ee000640 1 -- 172.21.15.49:0/2468907800 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10e8005020 con 0x7f10f0102ba0 2024-02-26T06:23:13.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.637+0000 7f10ee000640 1 -- 172.21.15.49:0/2468907800 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8010690 con 0x7f10f0102ba0 2024-02-26T06:23:13.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.637+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f019f690 con 0x7f10f0102ba0 2024-02-26T06:23:13.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.638+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f0108390 con 0x7f10f0102ba0 2024-02-26T06:23:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.641+0000 7f10ee000640 1 -- 172.21.15.49:0/2468907800 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f10e8005500 con 0x7f10f0102ba0 2024-02-26T06:23:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.643+0000 7f10ee000640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f10d80a20b0 0x7f10d80a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.643+0000 7f10ee000640 1 -- 172.21.15.49:0/2468907800 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f10e8014070 con 0x7f10f0102ba0 2024-02-26T06:23:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.643+0000 7f10f5a00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f10d80a20b0 0x7f10d80a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.644+0000 7f10ee000640 1 -- 172.21.15.49:0/2468907800 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f10e80973b0 con 0x7f10f0102ba0 2024-02-26T06:23:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.644+0000 7f10f5a00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f10d80a20b0 0x7f10d80a4570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10e402c240 tx=0x7f10e402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.646+0000 7fa501000640 1 -- 172.21.15.49:0/2978791511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc101280 msgr2=0x7fa4fc101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.646+0000 7fa501000640 1 --2- 172.21.15.49:0/2978791511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc101280 0x7fa4fc101700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009900 tx=0x7fa4e402bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:13.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.646+0000 7fa501000640 1 -- 172.21.15.49:0/2978791511 shutdown_connections 2024-02-26T06:23:13.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.646+0000 7fa501000640 1 --2- 172.21.15.49:0/2978791511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4fc101c40 0x7fa4fc1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.646+0000 7fa501000640 1 --2- 172.21.15.49:0/2978791511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc101280 0x7fa4fc101700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.646+0000 7fa501000640 1 --2- 172.21.15.49:0/2978791511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4fc100090 0x7fa4fc1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.646+0000 7fa501000640 1 -- 172.21.15.49:0/2978791511 >> 172.21.15.49:0/2978791511 conn(0x7fa4fc0fb620 msgr2=0x7fa4fc0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.646+0000 7fa501000640 1 -- 172.21.15.49:0/2978791511 shutdown_connections 2024-02-26T06:23:13.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.647+0000 7fa501000640 1 -- 172.21.15.49:0/2978791511 wait complete. 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.647+0000 7fa501000640 1 Processor -- start 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.647+0000 7fa501000640 1 -- start start 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa501000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc100090 0x7fa4fc197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa501000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4fc101280 0x7fa4fc1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa501000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4fc101c40 0x7fa4fc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa501000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc19ed60 con 0x7fa4fc100090 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa501000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc19eed0 con 0x7fa4fc101280 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa501000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc19f090 con 0x7fa4fc101c40 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa4f9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc100090 0x7fa4fc197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa4f9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc100090 0x7fa4fc197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35040/0 (socket says 172.21.15.49:35040) 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa4f9e00640 1 -- 172.21.15.49:0/282925847 learned_addr learned my addr 172.21.15.49:0/282925847 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa4f1400640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4fc101280 0x7fa4fc1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.648+0000 7fa4fa800640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4fc101c40 0x7fa4fc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4fa800640 1 -- 172.21.15.49:0/282925847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4fc101280 msgr2=0x7fa4fc1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4fa800640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4fc101280 0x7fa4fc1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4fa800640 1 -- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc100090 msgr2=0x7fa4fc197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4fa800640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc100090 0x7fa4fc197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4fa800640 1 -- 172.21.15.49:0/282925847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4009560 con 0x7fa4fc101c40 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc100090 0x7fa4fc197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4f1400640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4fc101280 0x7fa4fc1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4fa800640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4fc101c40 0x7fa4fc19e820 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400e940 tx=0x7fa4f400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4f3400640 1 -- 172.21.15.49:0/282925847 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4004d20 con 0x7fa4fc101c40 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4f3400640 1 -- 172.21.15.49:0/282925847 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4004e80 con 0x7fa4fc101c40 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa501000640 1 -- 172.21.15.49:0/282925847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc19f370 con 0x7fa4fc101c40 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa4f3400640 1 -- 172.21.15.49:0/282925847 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40116d0 con 0x7fa4fc101c40 2024-02-26T06:23:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.649+0000 7fa501000640 1 -- 172.21.15.49:0/282925847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc19f800 con 0x7fa4fc101c40 2024-02-26T06:23:13.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.653+0000 7fa4f3400640 1 -- 172.21.15.49:0/282925847 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa4f400cb20 con 0x7fa4fc101c40 2024-02-26T06:23:13.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.654+0000 7fa4f3400640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa4e00a20c0 0x7fa4e00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.654+0000 7fa503f72740 1 Processor -- start 2024-02-26T06:23:13.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.654+0000 7fa4f3400640 1 -- 172.21.15.49:0/282925847 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fa4f40cdd50 con 0x7fa4fc101c40 2024-02-26T06:23:13.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.654+0000 7fa503f72740 1 -- start start 2024-02-26T06:23:13.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.654+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa4e00a20c0 0x7fa4e00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa503f72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d6660760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa503f72740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6665830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa503f72740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6668330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa503f72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d66182d0 con 0x5652d6660340 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa503f72740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d65cae20 con 0x5652d6660d30 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa503f72740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d65c8350 con 0x5652d6665ea0 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4f1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6665830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4fa800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6668330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4f1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6665830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47864/0 (socket says 172.21.15.49:47864) 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4f1400640 1 -- 172.21.15.49:0/1081317538 learned_addr learned my addr 172.21.15.49:0/1081317538 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa4e00a20c0 0x7fa4e00a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0058c0 tx=0x7fa4ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/1081317538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d6660760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4fa800640 1 -- 172.21.15.49:0/1081317538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 msgr2=0x5652d6665830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4fa800640 1 --2- 172.21.15.49:0/1081317538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6665830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4fa800640 1 -- 172.21.15.49:0/1081317538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 msgr2=0x5652d6660760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4fa800640 1 --2- 172.21.15.49:0/1081317538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d6660760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.655+0000 7fa4fa800640 1 -- 172.21.15.49:0/1081317538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652d658cc50 con 0x5652d6665ea0 2024-02-26T06:23:13.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.656+0000 7fa4fa800640 1 --2- 172.21.15.49:0/1081317538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6668330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400c640 tx=0x7fa4f409a9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae05d1d14816040 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.656+0000 7fa4eb400640 1 -- 172.21.15.49:0/1081317538 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40a31b0 con 0x5652d6665ea0 2024-02-26T06:23:13.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.656+0000 7fa4eb400640 1 -- 172.21.15.49:0/1081317538 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4098b60 con 0x5652d6665ea0 2024-02-26T06:23:13.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.656+0000 7fa4eb400640 1 -- 172.21.15.49:0/1081317538 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4097130 con 0x5652d6665ea0 2024-02-26T06:23:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 -- 172.21.15.49:0/1081317538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 msgr2=0x5652d6668330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 --2- 172.21.15.49:0/1081317538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6668330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400c640 tx=0x7fa4f409a9a0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 -- 172.21.15.49:0/1081317538 shutdown_connections 2024-02-26T06:23:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 --2- 172.21.15.49:0/1081317538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6665830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 --2- 172.21.15.49:0/1081317538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6668330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 --2- 172.21.15.49:0/1081317538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d6660760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 -- 172.21.15.49:0/1081317538 >> 172.21.15.49:0/1081317538 conn(0x5652d66574a0 msgr2=0x5652d66578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 -- 172.21.15.49:0/1081317538 shutdown_connections 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.657+0000 7fa503f72740 1 -- 172.21.15.49:0/1081317538 wait complete. 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.658+0000 7fa503f72740 1 -- start start 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.658+0000 7fa503f72740 1 Processor -- start 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa503f72740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d666a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa503f72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660d30 0x5652d666aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa503f72740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6665ea0 0x5652d6671a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa503f72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d65c3b30 con 0x5652d6660d30 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa503f72740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6568fa0 con 0x5652d6665ea0 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa503f72740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6568c40 con 0x5652d6660340 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa503f72740 10 client.0 _ll_register_callbacks cb 0x5652d62d4c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa4f1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660d30 0x5652d666aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa4f1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660d30 0x5652d666aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35056/0 (socket says 172.21.15.49:35056) 2024-02-26T06:23:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa4f1400640 1 -- 172.21.15.49:0/1922389105 learned_addr learned my addr 172.21.15.49:0/1922389105 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d666a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.659+0000 7fa4fa800640 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6665ea0 0x5652d6671a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4f1400640 1 -- 172.21.15.49:0/1922389105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 msgr2=0x5652d666a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4f1400640 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d666a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4f1400640 1 -- 172.21.15.49:0/1922389105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6665ea0 msgr2=0x5652d6671a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4f1400640 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6665ea0 0x5652d6671a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4f1400640 1 -- 172.21.15.49:0/1922389105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc19f800 con 0x5652d6660d30 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d666a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4f1400640 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660d30 0x5652d666aa30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4000c00 tx=0x7fa4e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b060ddce8d838b4d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4dfe00640 1 -- 172.21.15.49:0/1922389105 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4004df0 con 0x5652d6660d30 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa4dfe00640 1 -- 172.21.15.49:0/1922389105 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fa4e4046070 con 0x5652d6660d30 2024-02-26T06:23:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.660+0000 7fa503f72740 1 -- 172.21.15.49:0/1922389105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652d62c3860 con 0x5652d6660d30 2024-02-26T06:23:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa4dfe00640 1 -- 172.21.15.49:0/1922389105 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4e4004f50 con 0x5652d6660d30 2024-02-26T06:23:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa503f72740 10 client.15216 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa503f72740 1 -- 172.21.15.49:0/1922389105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5652d65c21b0 con 0x5652d6660d30 2024-02-26T06:23:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa4dfe00640 1 -- 172.21.15.49:0/1922389105 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5652d62c3860 con 0x5652d6660d30 2024-02-26T06:23:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa4dfe00640 1 -- 172.21.15.49:0/1922389105 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa4e403c040 con 0x5652d6660d30 2024-02-26T06:23:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa503f72740 10 client.15216 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa503f72740 10 client.15216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa503f72740 20 client.15216 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa503f72740 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652d6676790 0x5652d6678bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa503f72740 4 client.15216 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.661+0000 7fa503f72740 1 -- 172.21.15.49:0/1922389105 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652d6314410 con 0x5652d6676790 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.662+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652d6676790 0x5652d6678bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.662+0000 7fa4dfe00640 10 client.15216 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.662+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652d6676790 0x5652d6678bb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d09cab8e7c77e2 server_cookie=fa2a52087839e2b3 in_seq=0 out_seq=0 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa4dfe00640 1 -- 172.21.15.49:0/1922389105 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652d6314410 con 0x5652d6676790 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa4dfe00640 10 client.15216 handle_command_reply: tid=0 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 client.15216 shutdown 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 -- 172.21.15.49:0/1922389105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660d30 msgr2=0x5652d666aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660d30 0x5652d666aa30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4000c00 tx=0x7fa4e4004270 comp rx=0 tx=0).stop 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 -- 172.21.15.49:0/1922389105 shutdown_connections 2024-02-26T06:23:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652d6676790 0x5652d6678bb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6665ea0 0x5652d6671a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660d30 0x5652d666aa30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 --2- 172.21.15.49:0/1922389105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d666a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa4dfe00640 0 client.15216 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.663+0000 7fa503f72740 1 -- 172.21.15.49:0/1922389105 >> 172.21.15.49:0/1922389105 conn(0x5652d66574a0 msgr2=0x5652d66566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.664+0000 7fa503f72740 1 -- 172.21.15.49:0/1922389105 shutdown_connections 2024-02-26T06:23:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.664+0000 7fa503f72740 1 -- 172.21.15.49:0/1922389105 wait complete. 2024-02-26T06:23:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.664+0000 7fa503f72740 20 client.15216 trim_cache size 0 max 16384 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.680+0000 7fa503f72740 1 Processor -- start 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.680+0000 7fa503f72740 1 -- start start 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.680+0000 7fa503f72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d66621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.680+0000 7fa503f72740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6662730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.680+0000 7fa503f72740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6671a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.680+0000 7fa503f72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d630c0c0 con 0x5652d6660340 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.680+0000 7fa503f72740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6657ef0 con 0x5652d6660d30 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.680+0000 7fa503f72740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6314410 con 0x5652d6665ea0 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d66621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d66621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35068/0 (socket says 172.21.15.49:35068) 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 -- 172.21.15.49:0/670318211 learned_addr learned my addr 172.21.15.49:0/670318211 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4fa800640 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6671a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f1400640 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6662730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 -- 172.21.15.49:0/670318211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 msgr2=0x5652d6671a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6671a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 -- 172.21.15.49:0/670318211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 msgr2=0x5652d6662730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6662730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 -- 172.21.15.49:0/670318211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652d655ea60 con 0x5652d6660340 2024-02-26T06:23:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f1400640 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6662730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:23:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.681+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d66621f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00fca0 tx=0x7fa4ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195ccc66d16c2671 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.682+0000 7fa4eaa00640 1 -- 172.21.15.49:0/670318211 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec021040 con 0x5652d6660340 2024-02-26T06:23:13.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.682+0000 7fa4eaa00640 1 -- 172.21.15.49:0/670318211 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec022040 con 0x5652d6660340 2024-02-26T06:23:13.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.682+0000 7fa4eaa00640 1 -- 172.21.15.49:0/670318211 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec005380 con 0x5652d6660340 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 -- 172.21.15.49:0/670318211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 msgr2=0x5652d66621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d66621f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00fca0 tx=0x7fa4ec014000 comp rx=0 tx=0).stop 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 -- 172.21.15.49:0/670318211 shutdown_connections 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d6662730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6665ea0 0x5652d6671a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 --2- 172.21.15.49:0/670318211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6660340 0x5652d66621f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 -- 172.21.15.49:0/670318211 >> 172.21.15.49:0/670318211 conn(0x5652d66574a0 msgr2=0x5652d6661730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 -- 172.21.15.49:0/670318211 shutdown_connections 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 -- 172.21.15.49:0/670318211 wait complete. 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.683+0000 7fa503f72740 1 -- start start 2024-02-26T06:23:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.684+0000 7fa503f72740 1 Processor -- start 2024-02-26T06:23:13.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.684+0000 7fa503f72740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d66716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.684+0000 7fa503f72740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d666fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.684+0000 7fa503f72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6665ea0 0x5652d6670230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.684+0000 7fa503f72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6671ed0 con 0x5652d6665ea0 2024-02-26T06:23:13.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.684+0000 7fa503f72740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6670770 con 0x5652d6660d30 2024-02-26T06:23:13.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.684+0000 7fa503f72740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6670930 con 0x5652d6660340 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa503f72740 10 client.0 _ll_register_callbacks cb 0x7fa4f4096640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6665ea0 0x5652d6670230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6665ea0 0x5652d6670230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35080/0 (socket says 172.21.15.49:35080) 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 -- 172.21.15.49:0/2185766099 learned_addr learned my addr 172.21.15.49:0/2185766099 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d66716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4f1400640 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d666fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 -- 172.21.15.49:0/2185766099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 msgr2=0x5652d66716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d66716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 -- 172.21.15.49:0/2185766099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 msgr2=0x5652d666fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d666fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 -- 172.21.15.49:0/2185766099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ec017050 con 0x5652d6665ea0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4fa800640 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6665ea0 0x5652d6670230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40cb7e0 tx=0x7fa4f4096e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b50bcdb55fca046 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.685+0000 7fa4dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa4dfe00640 1 -- 172.21.15.49:0/2185766099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40a31b0 con 0x5652d6665ea0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa4dfe00640 1 -- 172.21.15.49:0/2185766099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fa4f40052b0 con 0x5652d6665ea0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 1 -- 172.21.15.49:0/2185766099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4f4098b60 con 0x5652d6665ea0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa4dfe00640 1 -- 172.21.15.49:0/2185766099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4f40059c0 con 0x5652d6665ea0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 1 -- 172.21.15.49:0/2185766099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5652d6584670 con 0x5652d6665ea0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa4dfe00640 1 -- 172.21.15.49:0/2185766099 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4098b60 con 0x5652d6665ea0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa4dfe00640 1 -- 172.21.15.49:0/2185766099 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa4f4018030 con 0x5652d6665ea0 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:13.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 10 client.15222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 20 client.15222 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652d66cdf20 0x5652d66d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 4 client.15222 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.686+0000 7fa503f72740 1 -- 172.21.15.49:0/2185766099 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5652d63007b0 con 0x5652d66cdf20 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.687+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652d66cdf20 0x5652d66d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.687+0000 7fa4f9e00640 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652d66cdf20 0x5652d66d0340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88a7c40359fb6a1 server_cookie=e209adb54ab38f50 in_seq=0 out_seq=0 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.687+0000 7fa4dfe00640 10 client.15222 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.688+0000 7fa4dfe00640 1 -- 172.21.15.49:0/2185766099 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5652d63007b0 con 0x5652d66cdf20 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.688+0000 7fa4dfe00640 10 client.15222 handle_command_reply: tid=0 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.688+0000 7fa503f72740 1 client.15222 shutdown 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 -- 172.21.15.49:0/2185766099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6665ea0 msgr2=0x5652d6670230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6665ea0 0x5652d6670230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40cb7e0 tx=0x7fa4f4096e90 comp rx=0 tx=0).stop 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 -- 172.21.15.49:0/2185766099 shutdown_connections 2024-02-26T06:23:13.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652d66cdf20 0x5652d66d0340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652d6660d30 0x5652d666fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652d6665ea0 0x5652d6670230 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 --2- 172.21.15.49:0/2185766099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652d6660340 0x5652d66716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 -- 172.21.15.49:0/2185766099 >> 172.21.15.49:0/2185766099 conn(0x5652d66574a0 msgr2=0x5652d66621f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa4dfe00640 0 client.15222 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 -- 172.21.15.49:0/2185766099 shutdown_connections 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 1 -- 172.21.15.49:0/2185766099 wait complete. 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.689+0000 7fa503f72740 20 client.15222 trim_cache size 0 max 16384 2024-02-26T06:23:13.693 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:23:13.694 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:23:13.694 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "4e4363ff-afb5-4667-a9a8-f5c2ced3a294", 2024-02-26T06:23:13.694 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:23:13.694 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 -- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa4e00a20c0 msgr2=0x7fa4e00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa4e00a20c0 0x7fa4e00a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0058c0 tx=0x7fa4ec005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 -- 172.21.15.49:0/282925847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4fc101c40 msgr2=0x7fa4fc19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4fc101c40 0x7fa4fc19e820 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400e940 tx=0x7fa4f400ee00 comp rx=0 tx=0).stop 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 -- 172.21.15.49:0/282925847 shutdown_connections 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fa4e00a20c0 0x7fa4e00a4580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4fc101280 0x7fa4fc1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4fc101c40 0x7fa4fc19e820 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 --2- 172.21.15.49:0/282925847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4fc100090 0x7fa4fc197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 -- 172.21.15.49:0/282925847 >> 172.21.15.49:0/282925847 conn(0x7fa4fc0fb620 msgr2=0x7fa4fc0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 -- 172.21.15.49:0/282925847 shutdown_connections 2024-02-26T06:23:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.690+0000 7fa501000640 1 -- 172.21.15.49:0/282925847 wait complete. 2024-02-26T06:23:13.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:23:13.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:23:13.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.913+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f10f002cdd0 con 0x7f10f0102ba0 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 -- 172.21.15.49:0/1315468988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20104530 msgr2=0x7f1f20106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 --2- 172.21.15.49:0/1315468988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20104530 0x7f1f20106920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009860 tx=0x7f1f0c02bb70 comp rx=0 tx=0).stop 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 -- 172.21.15.49:0/1315468988 shutdown_connections 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 --2- 172.21.15.49:0/1315468988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f20106e60 0x7f1f201092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 --2- 172.21.15.49:0/1315468988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20104530 0x7f1f20106920 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 --2- 172.21.15.49:0/1315468988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f20101c00 0x7f1f20103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 -- 172.21.15.49:0/1315468988 >> 172.21.15.49:0/1315468988 conn(0x7f1f200fb610 msgr2=0x7f1f200fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 -- 172.21.15.49:0/1315468988 shutdown_connections 2024-02-26T06:23:13.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.919+0000 7f1f25c00640 1 -- 172.21.15.49:0/1315468988 wait complete. 2024-02-26T06:23:13.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.920+0000 7f1f25c00640 1 Processor -- start 2024-02-26T06:23:13.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.920+0000 7f1f25c00640 1 -- start start 2024-02-26T06:23:13.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f25c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20101c00 0x7f1f20197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f25c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f20104530 0x7f1f201977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f25c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f20106e60 0x7f1f2019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f25c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2019ed60 con 0x7f1f20101c00 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f25c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2019eed0 con 0x7f1f20106e60 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f25c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2019f090 con 0x7f1f20104530 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f1ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20101c00 0x7f1f20197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f1e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f20104530 0x7f1f201977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f1ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20101c00 0x7f1f20197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35094/0 (socket says 172.21.15.49:35094) 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f1ea00640 1 -- 172.21.15.49:0/3730796914 learned_addr learned my addr 172.21.15.49:0/3730796914 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f1f400640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f20106e60 0x7f1f2019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f1e000640 1 -- 172.21.15.49:0/3730796914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f20106e60 msgr2=0x7f1f2019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.921+0000 7f1f1e000640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f20106e60 0x7f1f2019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f1e000640 1 -- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20101c00 msgr2=0x7f1f20197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f1e000640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20101c00 0x7f1f20197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f1e000640 1 -- 172.21.15.49:0/3730796914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c009490 con 0x7f1f20104530 2024-02-26T06:23:13.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20101c00 0x7f1f20197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:13.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f1e000640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f20104530 0x7f1f201977b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c02c0b0 tx=0x7f1f0c0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f17400640 1 -- 172.21.15.49:0/3730796914 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c02c510 con 0x7f1f20104530 2024-02-26T06:23:13.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f17400640 1 -- 172.21.15.49:0/3730796914 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f0c02cb50 con 0x7f1f20104530 2024-02-26T06:23:13.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f25c00640 1 -- 172.21.15.49:0/3730796914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f200ff9f0 con 0x7f1f20104530 2024-02-26T06:23:13.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f17400640 1 -- 172.21.15.49:0/3730796914 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0035f0 con 0x7f1f20104530 2024-02-26T06:23:13.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.922+0000 7f1f25c00640 1 -- 172.21.15.49:0/3730796914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f200ffea0 con 0x7f1f20104530 2024-02-26T06:23:13.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.925+0000 7f1f28a09740 1 Processor -- start 2024-02-26T06:23:13.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f28a09740 1 -- start start 2024-02-26T06:23:13.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645756111f0 0x564575611610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x5645756166e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 0x5645756191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f28a09740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564575581120 con 0x5645756111f0 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f28a09740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457557e650 con 0x564575611be0 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f28a09740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564575579e30 con 0x564575616d50 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f1f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645756111f0 0x564575611610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f1f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645756111f0 0x564575611610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35108/0 (socket says 172.21.15.49:35108) 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f1f400640 1 -- 172.21.15.49:0/3593541557 learned_addr learned my addr 172.21.15.49:0/3593541557 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f17400640 1 -- 172.21.15.49:0/3730796914 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1f0c03d740 con 0x7f1f20104530 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.926+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/3593541557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x5645756166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f1f400640 1 -- 172.21.15.49:0/3593541557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 msgr2=0x5645756191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f1f400640 1 --2- 172.21.15.49:0/3593541557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 0x5645756191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f1f400640 1 -- 172.21.15.49:0/3593541557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 msgr2=0x5645756166e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f1f400640 1 --2- 172.21.15.49:0/3593541557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x5645756166e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f1f400640 1 -- 172.21.15.49:0/3593541557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564575533030 con 0x5645756111f0 2024-02-26T06:23:13.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f1f400640 1 --2- 172.21.15.49:0/3593541557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645756111f0 0x564575611610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009fd0 tx=0x7f1f180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5830c83e7616ba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f0be00640 1 -- 172.21.15.49:0/3593541557 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18004d20 con 0x5645756111f0 2024-02-26T06:23:13.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f0be00640 1 -- 172.21.15.49:0/3593541557 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f18004e80 con 0x5645756111f0 2024-02-26T06:23:13.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.927+0000 7f1f0be00640 1 -- 172.21.15.49:0/3593541557 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18005280 con 0x5645756111f0 2024-02-26T06:23:13.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f28a09740 1 -- 172.21.15.49:0/3593541557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645756111f0 msgr2=0x564575611610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f28a09740 1 --2- 172.21.15.49:0/3593541557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645756111f0 0x564575611610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009fd0 tx=0x7f1f180118d0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f17400640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1f040a20c0 0x7f1f040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f28a09740 1 -- 172.21.15.49:0/3593541557 shutdown_connections 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f28a09740 1 --2- 172.21.15.49:0/3593541557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x5645756166e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f28a09740 1 --2- 172.21.15.49:0/3593541557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 0x5645756191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f28a09740 1 --2- 172.21.15.49:0/3593541557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645756111f0 0x564575611610 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f17400640 1 -- 172.21.15.49:0/3730796914 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1f0c0f39a0 con 0x7f1f20104530 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f28a09740 1 -- 172.21.15.49:0/3593541557 >> 172.21.15.49:0/3593541557 conn(0x5645755d9570 msgr2=0x56457560cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1f040a20c0 0x7f1f040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.928+0000 7f1f28a09740 1 -- 172.21.15.49:0/3593541557 shutdown_connections 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.929+0000 7f1f28a09740 1 -- 172.21.15.49:0/3593541557 wait complete. 2024-02-26T06:23:13.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.929+0000 7f1f28a09740 1 -- start start 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.929+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1f040a20c0 0x7f1f040a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009fd0 tx=0x7f1f10014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.929+0000 7f1f28a09740 1 Processor -- start 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x56457561b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575611be0 0x56457561b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 0x564575622980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f28a09740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457551ee90 con 0x564575616d50 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f28a09740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457551eb30 con 0x5645756111f0 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f28a09740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645752e81c0 con 0x564575611be0 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f28a09740 10 client.0 _ll_register_callbacks cb 0x564575532930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x56457561b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x56457561b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47936/0 (socket says 172.21.15.49:47936) 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1ea00640 1 -- 172.21.15.49:0/949559588 learned_addr learned my addr 172.21.15.49:0/949559588 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1e000640 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575611be0 0x56457561b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1f400640 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 0x564575622980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1ea00640 1 -- 172.21.15.49:0/949559588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575611be0 msgr2=0x56457561b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575611be0 0x56457561b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1ea00640 1 -- 172.21.15.49:0/949559588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 msgr2=0x564575622980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.930+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 0x564575622980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.931+0000 7f1f1ea00640 1 -- 172.21.15.49:0/949559588 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f18009c80 con 0x5645756111f0 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.931+0000 7f1f1e000640 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575611be0 0x56457561b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.931+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x56457561b400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f100038a0 tx=0x7f1f10004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141873765227dd6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.931+0000 7f1f08c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.933+0000 7f1f08c00640 1 -- 172.21.15.49:0/949559588 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000fd50 con 0x5645756111f0 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.933+0000 7f1f08c00640 1 -- 172.21.15.49:0/949559588 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1f10005020 con 0x5645756111f0 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.933+0000 7f1f28a09740 1 -- 172.21.15.49:0/949559588 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645752b6720 con 0x5645756111f0 2024-02-26T06:23:13.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.933+0000 7f1f08c00640 1 -- 172.21.15.49:0/949559588 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f100051e0 con 0x5645756111f0 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.933+0000 7f1f28a09740 10 client.25343 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.933+0000 7f1f28a09740 1 -- 172.21.15.49:0/949559588 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5645755784b0 con 0x5645756111f0 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f08c00640 1 -- 172.21.15.49:0/949559588 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5645752b6720 con 0x5645756111f0 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f08c00640 1 -- 172.21.15.49:0/949559588 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1f1001e420 con 0x5645756111f0 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f28a09740 10 client.25343 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f28a09740 10 client.25343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f28a09740 20 client.25343 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f28a09740 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564575627280 0x564575629670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f28a09740 4 client.25343 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f28a09740 1 -- 172.21.15.49:0/949559588 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564575305890 con 0x564575627280 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.934+0000 7f1f1e000640 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564575627280 0x564575629670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.935+0000 7f1f1e000640 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564575627280 0x564575629670 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0df7c13dd1b90ef server_cookie=3739fa43a1d39df5 in_seq=0 out_seq=0 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.935+0000 7f1f08c00640 10 client.25343 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f08c00640 1 -- 172.21.15.49:0/949559588 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564575305890 con 0x564575627280 2024-02-26T06:23:13.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f08c00640 10 client.25343 handle_command_reply: tid=0 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 client.25343 shutdown 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 -- 172.21.15.49:0/949559588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 msgr2=0x56457561b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x56457561b400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f100038a0 tx=0x7f1f10004730 comp rx=0 tx=0).stop 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 -- 172.21.15.49:0/949559588 shutdown_connections 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564575627280 0x564575629670 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 0x564575622980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575611be0 0x56457561b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 --2- 172.21.15.49:0/949559588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x56457561b400 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f08c00640 0 client.25343 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 -- 172.21.15.49:0/949559588 >> 172.21.15.49:0/949559588 conn(0x5645755d9570 msgr2=0x56457560be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 -- 172.21.15.49:0/949559588 shutdown_connections 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 1 -- 172.21.15.49:0/949559588 wait complete. 2024-02-26T06:23:13.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.936+0000 7f1f28a09740 20 client.25343 trim_cache size 0 max 16384 2024-02-26T06:23:13.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.948+0000 7f1f17400640 1 -- 172.21.15.49:0/3730796914 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mgrmap(e 19) v1 ==== 136785+0+0 (secure 0 0 0) 0x7f1f0c0e00b0 con 0x7f1f20104530 2024-02-26T06:23:13.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.948+0000 7f10ee000640 1 -- 172.21.15.49:0/2468907800 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 19) v1 ==== 136785+0+0 (secure 0 0 0) 0x7f10e8097780 con 0x7f10f0102ba0 2024-02-26T06:23:13.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.948+0000 7f10ee000640 1 -- 172.21.15.49:0/2468907800 <== mon.1 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f10e80cf990 con 0x7f10f0102ba0 2024-02-26T06:23:13.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f10d80a20b0 msgr2=0x7f10d80a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f10d80a20b0 0x7f10d80a4570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10e402c240 tx=0x7f10e402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f0102ba0 msgr2=0x7f10f019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f0102ba0 0x7f10f019e630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009fd0 tx=0x7f10e800ef10 comp rx=0 tx=0).stop 2024-02-26T06:23:13.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 shutdown_connections 2024-02-26T06:23:13.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f10d80a20b0 0x7f10d80a4570 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f0102ba0 0x7f10f019e630 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f01021e0 0x7f10f01975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 --2- 172.21.15.49:0/2468907800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10f0100ff0 0x7f10f0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 >> 172.21.15.49:0/2468907800 conn(0x7f10f00fc560 msgr2=0x7f10f0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.951+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 shutdown_connections 2024-02-26T06:23:13.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.952+0000 7f10f6e00640 1 -- 172.21.15.49:0/2468907800 wait complete. 2024-02-26T06:23:13.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.963+0000 7f1f28a09740 1 Processor -- start 2024-02-26T06:23:13.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.963+0000 7f1f28a09740 1 -- start start 2024-02-26T06:23:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.963+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x564575613100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.963+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575611be0 0x564575613640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.963+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 0x564575622910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.963+0000 7f1f28a09740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457561a3b0 con 0x564575611be0 2024-02-26T06:23:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.963+0000 7f1f28a09740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564575305890 con 0x5645756111f0 2024-02-26T06:23:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.963+0000 7f1f28a09740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645752b6720 con 0x564575616d50 2024-02-26T06:23:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575611be0 0x564575613640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 0x564575622910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575611be0 0x564575613640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35124/0 (socket says 172.21.15.49:35124) 2024-02-26T06:23:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 -- 172.21.15.49:0/359289135 learned_addr learned my addr 172.21.15.49:0/359289135 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/359289135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x564575613100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 -- 172.21.15.49:0/359289135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 msgr2=0x564575622910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 --2- 172.21.15.49:0/359289135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 0x564575622910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 -- 172.21.15.49:0/359289135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 msgr2=0x564575613100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 --2- 172.21.15.49:0/359289135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x564575613100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 -- 172.21.15.49:0/359289135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564575514920 con 0x564575611be0 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.964+0000 7f1f1e000640 1 --2- 172.21.15.49:0/359289135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575611be0 0x564575613640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0b1050 tx=0x7f1f0c02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1f8a907106be30 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.965+0000 7f1f0b400640 1 -- 172.21.15.49:0/359289135 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0b48d0 con 0x564575611be0 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.965+0000 7f1f0b400640 1 -- 172.21.15.49:0/359289135 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f0c0c0d60 con 0x564575611be0 2024-02-26T06:23:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.965+0000 7f1f0b400640 1 -- 172.21.15.49:0/359289135 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0b3d10 con 0x564575611be0 2024-02-26T06:23:13.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 -- 172.21.15.49:0/359289135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575611be0 msgr2=0x564575613640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 --2- 172.21.15.49:0/359289135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575611be0 0x564575613640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0b1050 tx=0x7f1f0c02bb70 comp rx=0 tx=0).stop 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 -- 172.21.15.49:0/359289135 shutdown_connections 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 --2- 172.21.15.49:0/359289135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564575616d50 0x564575622910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 --2- 172.21.15.49:0/359289135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575611be0 0x564575613640 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 --2- 172.21.15.49:0/359289135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5645756111f0 0x564575613100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 -- 172.21.15.49:0/359289135 >> 172.21.15.49:0/359289135 conn(0x5645755d9570 msgr2=0x56457562e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 -- 172.21.15.49:0/359289135 shutdown_connections 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.966+0000 7f1f28a09740 1 -- 172.21.15.49:0/359289135 wait complete. 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 1 -- start start 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 1 Processor -- start 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645756111f0 0x564575622310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x564575622850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 0x564575620920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564575620fd0 con 0x564575616d50 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564575621140 con 0x564575611be0 2024-02-26T06:23:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564575621300 con 0x5645756111f0 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.967+0000 7f1f28a09740 10 client.0 _ll_register_callbacks cb 0x7f1f10009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645756111f0 0x564575622310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x564575622850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 0x564575620920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x564575622850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47954/0 (socket says 172.21.15.49:47954) 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645756111f0 0x564575622310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:51684/0 (socket says 172.21.15.49:51684) 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 -- 172.21.15.49:0/2365752056 learned_addr learned my addr 172.21.15.49:0/2365752056 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 -- 172.21.15.49:0/2365752056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645756111f0 msgr2=0x564575622310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645756111f0 0x564575622310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 -- 172.21.15.49:0/2365752056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 msgr2=0x564575620920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 0x564575620920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 -- 172.21.15.49:0/2365752056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f200ffea0 con 0x564575611be0 2024-02-26T06:23:13.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645756111f0 0x564575622310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:13.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f1e000640 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x564575622850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0e0400 tx=0x7f1f0c0c0ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac72bc968fd9186 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:13.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.968+0000 7f1f08c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:13.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f08c00640 1 -- 172.21.15.49:0/2365752056 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0b48d0 con 0x564575611be0 2024-02-26T06:23:13.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f08c00640 1 -- 172.21.15.49:0/2365752056 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1f0c0b3fa0 con 0x564575611be0 2024-02-26T06:23:13.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 1 -- 172.21.15.49:0/2365752056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645752797e0 con 0x564575611be0 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f08c00640 1 -- 172.21.15.49:0/2365752056 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f0c0b8110 con 0x564575611be0 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 10 client.25349 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 1 -- 172.21.15.49:0/2365752056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56457553a400 con 0x564575611be0 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f08c00640 1 -- 172.21.15.49:0/2365752056 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5645752797e0 con 0x564575611be0 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f08c00640 1 -- 172.21.15.49:0/2365752056 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1f0c04d060 con 0x564575611be0 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 10 client.25349 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 10 client.25349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 20 client.25349 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56457567e820 0x564575680c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 4 client.25349 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.969+0000 7f1f28a09740 1 -- 172.21.15.49:0/2365752056 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564575225700 con 0x56457567e820 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.970+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56457567e820 0x564575680c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.970+0000 7f1f1ea00640 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56457567e820 0x564575680c40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db827db63a102be8 server_cookie=84fd65371c0cbadb in_seq=0 out_seq=0 2024-02-26T06:23:13.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.970+0000 7f1f08c00640 10 client.25349 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:13.973 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:13 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3739926045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:13.973 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:13 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3648994085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:13.973 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:13 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2468907800' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:23:13.973 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:13 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:23:13.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.971+0000 7f1f08c00640 1 -- 172.21.15.49:0/2365752056 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564575225700 con 0x56457567e820 2024-02-26T06:23:13.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.971+0000 7f1f08c00640 10 client.25349 handle_command_reply: tid=0 2024-02-26T06:23:13.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.971+0000 7f1f28a09740 1 client.25349 shutdown 2024-02-26T06:23:13.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.971+0000 7f1f28a09740 1 -- 172.21.15.49:0/2365752056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 msgr2=0x564575622850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.971+0000 7f1f28a09740 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x564575622850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0e0400 tx=0x7f1f0c0c0ed0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 1 -- 172.21.15.49:0/2365752056 shutdown_connections 2024-02-26T06:23:13.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56457567e820 0x564575680c40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564575611be0 0x564575622850 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564575616d50 0x564575620920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 1 --2- 172.21.15.49:0/2365752056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645756111f0 0x564575622310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f08c00640 0 client.25349 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 1 -- 172.21.15.49:0/2365752056 >> 172.21.15.49:0/2365752056 conn(0x5645755d9570 msgr2=0x564575613100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 1 -- 172.21.15.49:0/2365752056 shutdown_connections 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 1 -- 172.21.15.49:0/2365752056 wait complete. 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.972+0000 7f1f28a09740 20 client.25349 trim_cache size 0 max 16384 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:23:13.975 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:23:13.976 INFO:teuthology.orchestra.run.smithi049.stdout: "4e4363ff-afb5-4667-a9a8-f5c2ced3a294": { 2024-02-26T06:23:13.976 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:23:13.976 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "4e4363ff-afb5-4667-a9a8-f5c2ced3a294", 2024-02-26T06:23:13.976 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:23:13.976 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:23:13.976 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:23:13.976 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:23:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 -- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1f040a20c0 msgr2=0x7f1f040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1f040a20c0 0x7f1f040a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009fd0 tx=0x7f1f10014040 comp rx=0 tx=0).stop 2024-02-26T06:23:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 -- 172.21.15.49:0/3730796914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f20104530 msgr2=0x7f1f201977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f20104530 0x7f1f201977b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c02c0b0 tx=0x7f1f0c0045a0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.977 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-02-26T06:23:13.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 -- 172.21.15.49:0/3730796914 shutdown_connections 2024-02-26T06:23:13.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f1f040a20c0 0x7f1f040a4580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f20106e60 0x7f1f2019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f20104530 0x7f1f201977b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 --2- 172.21.15.49:0/3730796914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f20101c00 0x7f1f20197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:13.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 -- 172.21.15.49:0/3730796914 >> 172.21.15.49:0/3730796914 conn(0x7f1f200fb610 msgr2=0x7f1f20102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:13.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 -- 172.21.15.49:0/3730796914 shutdown_connections 2024-02-26T06:23:13.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:13.973+0000 7f1f25c00640 1 -- 172.21.15.49:0/3730796914 wait complete. 2024-02-26T06:23:13.988 INFO:tasks.cephfs.filesystem:scrub status for tag:4e4363ff-afb5-4667-a9a8-f5c2ced3a294 - {'path': '/', 'tag': '4e4363ff-afb5-4667-a9a8-f5c2ced3a294', 'options': 'recursive,force'} 2024-02-26T06:23:14.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.164+0000 7f7122e00640 1 -- 172.21.15.49:0/3270305962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 msgr2=0x7f711c1037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:14.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.164+0000 7f7122e00640 1 --2- 172.21.15.49:0/3270305962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 0x7f711c1037f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f710c009900 tx=0x7f710c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:14.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.164+0000 7f7122e00640 1 -- 172.21.15.49:0/3270305962 shutdown_connections 2024-02-26T06:23:14.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.164+0000 7f7122e00640 1 --2- 172.21.15.49:0/3270305962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f711c104f80 0x7f711c10b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.164+0000 7f7122e00640 1 --2- 172.21.15.49:0/3270305962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f711c1045c0 0x7f711c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.164+0000 7f7122e00640 1 --2- 172.21.15.49:0/3270305962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 0x7f711c1037f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.164+0000 7f7122e00640 1 -- 172.21.15.49:0/3270305962 >> 172.21.15.49:0/3270305962 conn(0x7f711c0fe940 msgr2=0x7f711c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:14.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.165+0000 7f7122e00640 1 -- 172.21.15.49:0/3270305962 shutdown_connections 2024-02-26T06:23:14.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.165+0000 7f7122e00640 1 -- 172.21.15.49:0/3270305962 wait complete. 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7122e00640 1 Processor -- start 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7122e00640 1 -- start start 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7122e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 0x7f711c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7122e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f711c1045c0 0x7f711c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7122e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f711c104f80 0x7f711c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7122e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c1a0ed0 con 0x7f711c1033d0 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7122e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c1a1040 con 0x7f711c104f80 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7122e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c1a1200 con 0x7f711c1045c0 2024-02-26T06:23:14.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f711be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 0x7f711c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f711b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f711c1045c0 0x7f711c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f711be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 0x7f711c1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35166/0 (socket says 172.21.15.49:35166) 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f711be00640 1 -- 172.21.15.49:0/223127642 learned_addr learned my addr 172.21.15.49:0/223127642 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.166+0000 7f7120e00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f711c104f80 0x7f711c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.167+0000 7f711b400640 1 -- 172.21.15.49:0/223127642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f711c104f80 msgr2=0x7f711c1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.167+0000 7f711b400640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f711c104f80 0x7f711c1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.167+0000 7f711b400640 1 -- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 msgr2=0x7f711c1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.167+0000 7f711b400640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 0x7f711c1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.167+0000 7f711b400640 1 -- 172.21.15.49:0/223127642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c009560 con 0x7f711c1045c0 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.167+0000 7f711be00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 0x7f711c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.167+0000 7f7120e00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f711c104f80 0x7f711c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.167+0000 7f711b400640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f711c1045c0 0x7f711c199920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f710800e9f0 tx=0x7f710800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.168+0000 7f7118c00640 1 -- 172.21.15.49:0/223127642 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800ccc0 con 0x7f711c1045c0 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.168+0000 7f7118c00640 1 -- 172.21.15.49:0/223127642 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f710800ce20 con 0x7f711c1045c0 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.168+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c1a14e0 con 0x7f711c1045c0 2024-02-26T06:23:14.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.168+0000 7f7118c00640 1 -- 172.21.15.49:0/223127642 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108018920 con 0x7f711c1045c0 2024-02-26T06:23:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.168+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c1a1a40 con 0x7f711c1045c0 2024-02-26T06:23:14.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.169+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c10a6f0 con 0x7f711c1045c0 2024-02-26T06:23:14.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.174+0000 7f7118c00640 1 -- 172.21.15.49:0/223127642 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 136785+0+0 (secure 0 0 0) 0x7f7108018a80 con 0x7f711c1045c0 2024-02-26T06:23:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.175+0000 7f7118c00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f71000a2230 0x7f71000a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.175+0000 7f711be00640 1 -- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f71000a2230 msgr2=0x7f71000a46f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.175+0000 7f7118c00640 1 -- 172.21.15.49:0/223127642 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7108014070 con 0x7f711c1045c0 2024-02-26T06:23:14.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.175+0000 7f711be00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f71000a2230 0x7f71000a46f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:23:14.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.179+0000 7f7118c00640 1 -- 172.21.15.49:0/223127642 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7108052d20 con 0x7f711c1045c0 2024-02-26T06:23:14.241 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:13 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3739926045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:14.241 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:13 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3648994085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:14.241 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:13 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2468907800' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:23:14.241 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:13 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:23:14.242 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: ignoring --setuser ceph since I am not root 2024-02-26T06:23:14.242 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: ignoring --setgroup ceph since I am not root 2024-02-26T06:23:14.242 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:13 smithi115 ceph-mgr[38183]: -- 172.21.15.115:0/1854191535 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563d9131c1a0 con 0x563d912f3000 2024-02-26T06:23:14.330 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:13 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: ignoring --setuser ceph since I am not root 2024-02-26T06:23:14.330 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:13 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: ignoring --setgroup ceph since I am not root 2024-02-26T06:23:14.330 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:13 smithi080 ceph-mgr[37596]: -- 172.21.15.80:0/167508408 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56203a2fe1a0 con 0x56203a2d5000 2024-02-26T06:23:14.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:13 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3739926045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:14.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:13 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3648994085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:23:14.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:13 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2468907800' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:23:14.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:13 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:23:14.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.376+0000 7f711be00640 1 -- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f71000a2230 msgr2=0x7f71000a46f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:14.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.376+0000 7f711be00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f71000a2230 0x7f71000a46f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:23:14.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.382+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f711c10edb0 con 0x7f711c1045c0 2024-02-26T06:23:14.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.389+0000 7f7118c00640 1 -- 172.21.15.49:0/223127642 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f710809aaf0 con 0x7f711c1045c0 2024-02-26T06:23:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f71000a2230 msgr2=0x7f71000a46f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:23:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f71000a2230 0x7f71000a46f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f711c1045c0 msgr2=0x7f711c199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f711c1045c0 0x7f711c199920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f710800e9f0 tx=0x7f710800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 shutdown_connections 2024-02-26T06:23:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f71000a2230 0x7f71000a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f711c104f80 0x7f711c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f711c1045c0 0x7f711c199920 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 --2- 172.21.15.49:0/223127642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f711c1033d0 0x7f711c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 >> 172.21.15.49:0/223127642 conn(0x7f711c0fe940 msgr2=0x7f711c1078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:14.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 shutdown_connections 2024-02-26T06:23:14.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.392+0000 7f7122e00640 1 -- 172.21.15.49:0/223127642 wait complete. 2024-02-26T06:23:14.408 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: ignoring --setuser ceph since I am not root 2024-02-26T06:23:14.408 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: ignoring --setgroup ceph since I am not root 2024-02-26T06:23:14.413 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-02-26T06:23:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2681843147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc1045d0 msgr2=0x7fe8dc104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2681843147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc1045d0 0x7fe8dc104a50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009900 tx=0x7fe8c802bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2681843147 shutdown_connections 2024-02-26T06:23:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2681843147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8dc104f90 0x7fe8dc10b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2681843147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc1045d0 0x7fe8dc104a50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2681843147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8dc1033e0 0x7fe8dc103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2681843147 >> 172.21.15.49:0/2681843147 conn(0x7fe8dc0fe970 msgr2=0x7fe8dc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:14.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2681843147 shutdown_connections 2024-02-26T06:23:14.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.618+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2681843147 wait complete. 2024-02-26T06:23:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.619+0000 7fe8e2e00640 1 Processor -- start 2024-02-26T06:23:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.619+0000 7fe8e2e00640 1 -- start start 2024-02-26T06:23:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e2e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8dc1033e0 0x7fe8dc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e2e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8dc1045d0 0x7fe8dc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e2e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc104f90 0x7fe8dc1a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e2e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc1a0e40 con 0x7fe8dc104f90 2024-02-26T06:23:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e2e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc1a0fb0 con 0x7fe8dc1033e0 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e2e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc1a1170 con 0x7fe8dc1045d0 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e0e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc104f90 0x7fe8dc1a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8db400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8dc1045d0 0x7fe8dc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8dbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8dc1033e0 0x7fe8dc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e0e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc104f90 0x7fe8dc1a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35190/0 (socket says 172.21.15.49:35190) 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.620+0000 7fe8e0e00640 1 -- 172.21.15.49:0/2450722772 learned_addr learned my addr 172.21.15.49:0/2450722772 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8db400640 1 -- 172.21.15.49:0/2450722772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8dc1033e0 msgr2=0x7fe8dc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8db400640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8dc1033e0 0x7fe8dc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8db400640 1 -- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc104f90 msgr2=0x7fe8dc1a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8db400640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc104f90 0x7fe8dc1a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8db400640 1 -- 172.21.15.49:0/2450722772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc009490 con 0x7fe8dc1045d0 2024-02-26T06:23:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8db400640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8dc1045d0 0x7fe8dc199890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8c802c170 tx=0x7fe8c8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8d8c00640 1 -- 172.21.15.49:0/2450722772 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80043d0 con 0x7fe8dc1045d0 2024-02-26T06:23:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8d8c00640 1 -- 172.21.15.49:0/2450722772 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8c80052d0 con 0x7fe8dc1045d0 2024-02-26T06:23:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c8009560 con 0x7fe8dc1045d0 2024-02-26T06:23:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.621+0000 7fe8d8c00640 1 -- 172.21.15.49:0/2450722772 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8003cd0 con 0x7fe8dc1045d0 2024-02-26T06:23:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.622+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc1a1740 con 0x7fe8dc1045d0 2024-02-26T06:23:14.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.623+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc10a630 con 0x7fe8dc1045d0 2024-02-26T06:23:14.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.625+0000 7fe8d8c00640 1 -- 172.21.15.49:0/2450722772 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 136785+0+0 (secure 0 0 0) 0x7fe8c8005440 con 0x7fe8dc1045d0 2024-02-26T06:23:14.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.626+0000 7fe8d8c00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe8c00a21b0 0x7fe8c00a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:14.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.626+0000 7fe8dbe00640 1 -- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe8c00a21b0 msgr2=0x7fe8c00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:14.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.626+0000 7fe8d8c00640 1 -- 172.21.15.49:0/2450722772 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe8c80f0b20 con 0x7fe8dc1045d0 2024-02-26T06:23:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.626+0000 7fe8dbe00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe8c00a21b0 0x7fe8c00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:23:14.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.631+0000 7fe8d8c00640 1 -- 172.21.15.49:0/2450722772 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8c80b9a80 con 0x7fe8dc1045d0 2024-02-26T06:23:14.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.825+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe8dc02ceb0 con 0x7fe8dc1045d0 2024-02-26T06:23:14.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.827+0000 7fe8dbe00640 1 -- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe8c00a21b0 msgr2=0x7fe8c00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:14.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.827+0000 7fe8dbe00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe8c00a21b0 0x7fe8c00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:23:14.830 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:14.505+0000 7f432d92b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:23:14.830 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:14.597+0000 7f432d92b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:23:14.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.834+0000 7fe8d8c00640 1 -- 172.21.15.49:0/2450722772 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe8c80bd140 con 0x7fe8dc1045d0 2024-02-26T06:23:14.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe8c00a21b0 msgr2=0x7fe8c00a4670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:23:14.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe8c00a21b0 0x7fe8c00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8dc1045d0 msgr2=0x7fe8dc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:14.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8dc1045d0 0x7fe8dc199890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8c802c170 tx=0x7fe8c8005e80 comp rx=0 tx=0).stop 2024-02-26T06:23:14.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 shutdown_connections 2024-02-26T06:23:14.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7fe8c00a21b0 0x7fe8c00a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8dc104f90 0x7fe8dc1a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8dc1045d0 0x7fe8dc199890 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 --2- 172.21.15.49:0/2450722772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8dc1033e0 0x7fe8dc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:14.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 >> 172.21.15.49:0/2450722772 conn(0x7fe8dc0fe970 msgr2=0x7fe8dc107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:14.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 shutdown_connections 2024-02-26T06:23:14.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:14.836+0000 7fe8e2e00640 1 -- 172.21.15.49:0/2450722772 wait complete. 2024-02-26T06:23:14.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:14.520+0000 7f0e008df200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:23:14.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:14.621+0000 7f0e008df200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:23:14.853 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 -- 172.21.15.49:0/2172344557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc1011b0 msgr2=0x7f7cdc1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 --2- 172.21.15.49:0/2172344557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc1011b0 0x7f7cdc1015d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009900 tx=0x7f7ccc02bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 -- 172.21.15.49:0/2172344557 shutdown_connections 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 --2- 172.21.15.49:0/2172344557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7cdc1023a0 0x7f7cdc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 --2- 172.21.15.49:0/2172344557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc102d60 0x7f7cdc1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 --2- 172.21.15.49:0/2172344557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc1011b0 0x7f7cdc1015d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 -- 172.21.15.49:0/2172344557 >> 172.21.15.49:0/2172344557 conn(0x7f7cdc0fc740 msgr2=0x7f7cdc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 -- 172.21.15.49:0/2172344557 shutdown_connections 2024-02-26T06:23:15.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.068+0000 7f7ce2600640 1 -- 172.21.15.49:0/2172344557 wait complete. 2024-02-26T06:23:15.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.070+0000 7f7ce2600640 1 Processor -- start 2024-02-26T06:23:15.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.070+0000 7f7ce2600640 1 -- start start 2024-02-26T06:23:15.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.070+0000 7f7ce2600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc1011b0 0x7f7cdc074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:15.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.070+0000 7f7ce2600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7cdc1023a0 0x7f7cdc073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.070+0000 7f7ce2600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc102d60 0x7f7cdc073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.070+0000 7f7ce2600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc074210 con 0x7f7cdc102d60 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.070+0000 7f7ce2600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc074380 con 0x7f7cdc1023a0 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.070+0000 7f7ce2600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1a1260 con 0x7f7cdc1011b0 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc1011b0 0x7f7cdc074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc102d60 0x7f7cdc073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7cdc1023a0 0x7f7cdc073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc102d60 0x7f7cdc073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35204/0 (socket says 172.21.15.49:35204) 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 -- 172.21.15.49:0/3005190937 learned_addr learned my addr 172.21.15.49:0/3005190937 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc1011b0 0x7f7cdc074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:51748/0 (socket says 172.21.15.49:51748) 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc1011b0 msgr2=0x7f7cdc074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc1011b0 0x7f7cdc074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7cdc1023a0 msgr2=0x7f7cdc073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7cdc1023a0 0x7f7cdc073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 -- 172.21.15.49:0/3005190937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc8009490 con 0x7f7cdc102d60 2024-02-26T06:23:15.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdaa00640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7cdc1023a0 0x7f7cdc073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.071+0000 7f7cdbe00640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc102d60 0x7f7cdc073ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400ead0 tx=0x7f7cd400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.072+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400cde0 con 0x7f7cdc102d60 2024-02-26T06:23:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.072+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cd4005020 con 0x7f7cdc102d60 2024-02-26T06:23:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.072+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc009560 con 0x7f7cdc102d60 2024-02-26T06:23:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.072+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40116b0 con 0x7f7cdc102d60 2024-02-26T06:23:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.072+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc1a1840 con 0x7f7cdc102d60 2024-02-26T06:23:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.073+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc108470 con 0x7f7cdc102d60 2024-02-26T06:23:15.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.075+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 136785+0+0 (secure 0 0 0) 0x7f7cd4005520 con 0x7f7cdc102d60 2024-02-26T06:23:15.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.077+0000 7f7cd3e00640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 0x7f7cc00a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:15.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.077+0000 7f7cdb400640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 msgr2=0x7f7cc00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:15.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.077+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7cd40cdee0 con 0x7f7cdc102d60 2024-02-26T06:23:15.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.077+0000 7f7cdb400640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 0x7f7cc00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:23:15.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.082+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cd4096ef0 con 0x7f7cdc102d60 2024-02-26T06:23:15.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:14.792+0000 7f6f57491200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:23:15.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:14.925+0000 7f6f57491200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:23:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:23:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:23:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-26T06:23:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-mon[31696]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-02-26T06:23:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' 2024-02-26T06:23:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:14 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' 2024-02-26T06:23:15.225 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:23:15.225 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:15.225 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:23:15.225 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:15.225 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-26T06:23:15.225 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-mon[36206]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-02-26T06:23:15.226 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' 2024-02-26T06:23:15.226 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:14 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' 2024-02-26T06:23:15.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.278+0000 7f7cdb400640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 msgr2=0x7f7cc00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:15.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.278+0000 7f7cdb400640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 0x7f7cc00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:23:15.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:23:15.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:15.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:23:15.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:15.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-26T06:23:15.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-mon[36426]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-02-26T06:23:15.282 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' 2024-02-26T06:23:15.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:14 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' 2024-02-26T06:23:15.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.305+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 --> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cdc02ceb0 con 0x7f7cc00a21b0 2024-02-26T06:23:15.547 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:15.225+0000 7f432d92b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:23:15.548 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:15.316+0000 7f432d92b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:23:15.548 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:15.455+0000 7f432d92b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:23:15.548 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:15.547+0000 7f432d92b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:23:15.590 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:15.282+0000 7f0e008df200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:23:15.590 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:15.377+0000 7f0e008df200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:23:15.590 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:15.521+0000 7f0e008df200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:23:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.679+0000 7f7cdb400640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 msgr2=0x7f7cc00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:15.679+0000 7f7cdb400640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 0x7f7cc00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T06:23:15.830 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:15.630+0000 7f432d92b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:23:15.964 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:15.618+0000 7f0e008df200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:23:15.964 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:15.702+0000 7f0e008df200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:23:16.080 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:15 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:15.765+0000 7f6f57491200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:23:16.080 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:15 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:15.893+0000 7f6f57491200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:23:16.169 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:15.888+0000 7f432d92b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:23:16.169 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:15.994+0000 7f432d92b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:23:16.169 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:16 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:16.077+0000 7f432d92b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:23:16.169 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:16 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:16.169+0000 7f432d92b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:23:16.256 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:15.964+0000 7f0e008df200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:23:16.256 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:16.074+0000 7f0e008df200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:23:16.256 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:16.161+0000 7f0e008df200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:23:16.256 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:16.256+0000 7f0e008df200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:23:16.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:16.079+0000 7f6f57491200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:23:16.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:16.211+0000 7f6f57491200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:23:16.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:16.325+0000 7f6f57491200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:23:16.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:16.480+0000 7f7cdb400640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 msgr2=0x7f7cc00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:16.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:16.480+0000 7f7cdb400640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 0x7f7cc00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T06:23:16.580 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:16 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:16.342+0000 7f432d92b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:23:16.590 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:16.431+0000 7f0e008df200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:23:16.939 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:16.675+0000 7f6f57491200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:23:16.939 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:16.821+0000 7f6f57491200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:23:17.303 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:16.938+0000 7f6f57491200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:23:17.303 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:17 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:17.071+0000 7f6f57491200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:23:17.303 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:17 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:17.302+0000 7f6f57491200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:23:17.580 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:17 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:17.285+0000 7f432d92b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:23:17.580 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:17 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:17.411+0000 7f432d92b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:23:17.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:17.391+0000 7f0e008df200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:23:17.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:17.522+0000 7f0e008df200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:23:18.015 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:17 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:17.738+0000 7f432d92b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:23:18.015 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:17 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:17.825+0000 7f432d92b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:23:18.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:18.082+0000 7f7cdb400640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 msgr2=0x7f7cc00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:18.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:18.082+0000 7f7cdb400640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 0x7f7cc00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T06:23:18.136 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:17.853+0000 7f0e008df200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:23:18.136 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:17.943+0000 7f0e008df200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:23:18.330 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:18.015+0000 7f432d92b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:23:18.330 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:18.195+0000 7f432d92b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:23:18.590 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:18.136+0000 7f0e008df200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:23:18.590 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:18.321+0000 7f0e008df200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:23:18.726 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:18.472+0000 7f432d92b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:23:18.860 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:18.602+0000 7f0e008df200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:23:18.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:18.657+0000 7f6f57491200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:23:18.919 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:18.838+0000 7f6f57491200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:23:19.080 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:18.726+0000 7f432d92b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:23:19.080 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:18.956+0000 7f432d92b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:23:19.340 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:18.860+0000 7f0e008df200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:23:19.340 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:19.096+0000 7f0e008df200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:23:19.670 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:19.282+0000 7f6f57491200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:23:20.815 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:19.409+0000 7f6f57491200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:23:20.817 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:19.897+0000 7f0e008df200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:23:20.817 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:20.465+0000 7f0e008df200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:23:20.817 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:20.564+0000 7f0e008df200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:23:20.817 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:19 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:19.749+0000 7f432d92b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:23:20.817 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:20.310+0000 7f432d92b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:23:20.817 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:20.405+0000 7f432d92b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:23:20.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-mon[36426]: Standby manager daemon y restarted 2024-02-26T06:23:20.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-mon[36426]: Standby manager daemon y started 2024-02-26T06:23:20.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:23:20.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:23:20.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:23:20.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:23:20.818 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:19.683+0000 7f6f57491200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:23:20.819 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:19.942+0000 7f6f57491200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:23:20.819 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:20.339+0000 7f6f57491200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:23:20.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:20.706+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 20) v1 ==== 136785+0+0 (secure 0 0 0) 0x7f7cd40972c0 con 0x7f7cdc102d60 2024-02-26T06:23:21.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:20 smithi049 ceph-mon[31696]: Standby manager daemon y restarted 2024-02-26T06:23:21.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:20 smithi049 ceph-mon[31696]: Standby manager daemon y started 2024-02-26T06:23:21.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:20 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:23:21.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:20 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:23:21.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:20 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:23:21.013 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:20 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:23:21.013 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:20.677+0000 7f6f57491200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:23:21.080 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y[37592]: 2024-02-26T06:23:20.617+0000 7f432d92b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:23:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-mon[36206]: Standby manager daemon y restarted 2024-02-26T06:23:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-mon[36206]: Standby manager daemon y started 2024-02-26T06:23:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:23:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:23:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:23:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:20 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.80:0/1711048798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:23:21.090 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:23:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z[38140]: 2024-02-26T06:23:20.782+0000 7f0e008df200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:23:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:21.285+0000 7f7cdb400640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 msgr2=0x7f7cc00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1431851577 2024-02-26T06:23:21.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:21.285+0000 7f7cdb400640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 0x7f7cc00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-26T06:23:21.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:21.007+0000 7f6f57491200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:23:21.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:21.730+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 21) v1 ==== 136785+0+0 (secure 0 0 0) 0x7f7cd40cb9f0 con 0x7f7cdc102d60 2024-02-26T06:23:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:21 smithi080 ceph-mon[36206]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-26T06:23:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:21 smithi080 ceph-mon[36206]: Standby manager daemon z restarted 2024-02-26T06:23:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:21 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:23:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:21 smithi080 ceph-mon[36206]: Standby manager daemon z started 2024-02-26T06:23:22.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:21 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:23:22.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:21 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:23:22.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:21 smithi080 ceph-mon[36206]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:23:22.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:21 smithi115 ceph-mon[36426]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-26T06:23:22.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:21 smithi115 ceph-mon[36426]: Standby manager daemon z restarted 2024-02-26T06:23:22.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:21 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:23:22.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:21 smithi115 ceph-mon[36426]: Standby manager daemon z started 2024-02-26T06:23:22.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:21 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:23:22.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:21 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:23:22.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:21 smithi115 ceph-mon[36426]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:23:22.109 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:21 smithi049 ceph-mon[31696]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-26T06:23:22.109 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:21 smithi049 ceph-mon[31696]: Standby manager daemon z restarted 2024-02-26T06:23:22.109 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:21 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:23:22.109 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:21 smithi049 ceph-mon[31696]: Standby manager daemon z started 2024-02-26T06:23:22.109 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:21 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:23:22.109 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:21 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:23:22.109 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:21 smithi049 ceph-mon[31696]: from='mgr.? 172.21.15.115:0/4039198563' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:23:22.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:22.109+0000 7f6f57491200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:23:22.987 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:22 smithi049 ceph-mon[31696]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-02-26T06:23:22.987 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:22.847+0000 7f6f57491200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:23:22.988 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:22.987+0000 7f6f57491200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:23:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:22 smithi080 ceph-mon[36206]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-02-26T06:23:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:22 smithi115 ceph-mon[36426]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-02-26T06:23:23.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:23.310+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 9 ==== mgrmap(e 22) v1 ==== 136453+0+0 (secure 0 0 0) 0x7f7cd4005190 con 0x7f7cdc102d60 2024-02-26T06:23:23.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:23.312+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 msgr2=0x7f7cc00a4670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:23:23.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:23.312+0000 7f7cd3e00640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/1431851577,v1:172.21.15.49:6801/1431851577] conn(0x7f7cc00a21b0 0x7f7cc00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:23.614 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.285+0000 7f6f57491200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:23:23.789 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: Active manager daemon x restarted 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: Activating manager daemon x 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: mgrmap e22: x(active, starting, since 0.0243135s), standbys: z, y 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:23:23.790 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:23:23.791 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: Manager daemon x is now available 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:23:23.792 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:23 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: Active manager daemon x restarted 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: Activating manager daemon x 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: mgrmap e22: x(active, starting, since 0.0243135s), standbys: z, y 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:23:23.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:23:23.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:23:23.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:23:23.844 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: Manager daemon x is now available 2024-02-26T06:23:23.844 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:23.844 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:23.844 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:23:23.844 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:23 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:23:23.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: Active manager daemon x restarted 2024-02-26T06:23:23.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: Activating manager daemon x 2024-02-26T06:23:23.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T06:23:23.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: mgrmap e22: x(active, starting, since 0.0243135s), standbys: z, y 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:23:23.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.613+0000 7f6eac600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.890 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.613+0000 7f6eac600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.613+0000 7f6eac600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.613+0000 7f6eac600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.613+0000 7f6eac600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.614+0000 7f6ea8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.614+0000 7f6ea8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.614+0000 7f6ea8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.614+0000 7f6ea8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:23:23.614+0000 7f6ea8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:23:23.891 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: Manager daemon x is now available 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:23:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:23 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:23:24.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.324+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mon.0 v2:172.21.15.49:3300/0 10 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f7cd40cb6e0 con 0x7f7cdc102d60 2024-02-26T06:23:24.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.324+0000 7f7cd3e00640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7cc0067880 0x7f7cc0067c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:24.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.324+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 --> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cdc02ceb0 con 0x7f7cc0067880 2024-02-26T06:23:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.331+0000 7f7cdb400640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7cc0067880 0x7f7cc0067c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:24.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.338+0000 7f7cdb400640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7cc0067880 0x7f7cc0067c60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009230 tx=0x7f7ccc02c740 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:24.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.429+0000 7f7cd3e00640 1 -- 172.21.15.49:0/3005190937 <== mgr.25384 v2:172.21.15.49:6800/2212175168 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7cdc02ceb0 con 0x7f7cc0067880 2024-02-26T06:23:24.429 INFO:teuthology.orchestra.run.smithi049.stdout:Schedule set for path / 2024-02-26T06:23:24.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.432+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7cc0067880 msgr2=0x7f7cc0067c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:24.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.432+0000 7f7ce2600640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7cc0067880 0x7f7cc0067c60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009230 tx=0x7f7ccc02c740 comp rx=0 tx=0).stop 2024-02-26T06:23:24.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.433+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc102d60 msgr2=0x7f7cdc073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:24.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.433+0000 7f7ce2600640 1 --2- 172.21.15.49:0/3005190937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cdc102d60 0x7f7cdc073ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400ead0 tx=0x7f7cd400ef90 comp rx=0 tx=0).stop 2024-02-26T06:23:24.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.433+0000 7f7cdbe00640 1 -- 172.21.15.49:0/3005190937 reap_dead start 2024-02-26T06:23:24.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.433+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 shutdown_connections 2024-02-26T06:23:24.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.433+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 >> 172.21.15.49:0/3005190937 conn(0x7f7cdc0fc740 msgr2=0x7f7cdc105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:24.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.433+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 shutdown_connections 2024-02-26T06:23:24.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.433+0000 7f7ce2600640 1 -- 172.21.15.49:0/3005190937 wait complete. 2024-02-26T06:23:24.458 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 -- 172.21.15.49:0/2954893232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0073be0 msgr2=0x7f9ee0074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2954893232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0073be0 0x7f9ee0074060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009900 tx=0x7f9ecc02bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 -- 172.21.15.49:0/2954893232 shutdown_connections 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2954893232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0073be0 0x7f9ee0074060 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2954893232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee0109070 0x7f9ee010b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2954893232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ee00751c0 0x7f9ee0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 -- 172.21.15.49:0/2954893232 >> 172.21.15.49:0/2954893232 conn(0x7f9ee00fc740 msgr2=0x7f9ee00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 -- 172.21.15.49:0/2954893232 shutdown_connections 2024-02-26T06:23:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.658+0000 7f9ee7000640 1 -- 172.21.15.49:0/2954893232 wait complete. 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.659+0000 7f9ee7000640 1 Processor -- start 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.659+0000 7f9ee7000640 1 -- start start 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.659+0000 7f9ee7000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee0073be0 0x7f9ee0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.659+0000 7f9ee7000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ee00751c0 0x7f9ee01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.659+0000 7f9ee7000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0109070 0x7f9ee01a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.659+0000 7f9ee7000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01a0e80 con 0x7f9ee0073be0 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.659+0000 7f9ee7000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01a0ff0 con 0x7f9ee0109070 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.659+0000 7f9ee7000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01a11b0 con 0x7f9ee00751c0 2024-02-26T06:23:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0109070 0x7f9ee01a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9edf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ee00751c0 0x7f9ee01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0109070 0x7f9ee01a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33510/0 (socket says 172.21.15.49:33510) 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 -- 172.21.15.49:0/2851956665 learned_addr learned my addr 172.21.15.49:0/2851956665 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9edfe00640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee0073be0 0x7f9ee0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 -- 172.21.15.49:0/2851956665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ee00751c0 msgr2=0x7f9ee01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ee00751c0 0x7f9ee01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 -- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee0073be0 msgr2=0x7f9ee0199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee0073be0 0x7f9ee0199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 -- 172.21.15.49:0/2851956665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009490 con 0x7f9ee0109070 2024-02-26T06:23:24.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.660+0000 7f9ee5000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0109070 0x7f9ee01a0940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800eaa0 tx=0x7f9ed800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:24.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.661+0000 7f9edcc00640 1 -- 172.21.15.49:0/2851956665 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800cdb0 con 0x7f9ee0109070 2024-02-26T06:23:24.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.661+0000 7f9edcc00640 1 -- 172.21.15.49:0/2851956665 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8005020 con 0x7f9ee0109070 2024-02-26T06:23:24.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.661+0000 7f9edcc00640 1 -- 172.21.15.49:0/2851956665 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80116b0 con 0x7f9ee0109070 2024-02-26T06:23:24.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.661+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc009560 con 0x7f9ee0109070 2024-02-26T06:23:24.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.661+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee0100d40 con 0x7f9ee0109070 2024-02-26T06:23:24.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.662+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0107da0 con 0x7f9ee0109070 2024-02-26T06:23:24.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.666+0000 7f9edcc00640 1 -- 172.21.15.49:0/2851956665 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f9ed8005520 con 0x7f9ee0109070 2024-02-26T06:23:24.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.667+0000 7f9edcc00640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9ec40a1f70 0x7f9ec40a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.667+0000 7f9edcc00640 1 -- 172.21.15.49:0/2851956665 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9ed80cd9d0 con 0x7f9ee0109070 2024-02-26T06:23:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.667+0000 7f9edfe00640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9ec40a1f70 0x7f9ec40a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:24.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.669+0000 7f9edfe00640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9ec40a1f70 0x7f9ec40a4430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009d60 tx=0x7f9ed0005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:24.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.672+0000 7f9edcc00640 1 -- 172.21.15.49:0/2851956665 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ed8096c30 con 0x7f9ee0109070 2024-02-26T06:23:24.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:24.939+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 --> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ee002cf20 con 0x7f9ec40a1f70 2024-02-26T06:23:25.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.016+0000 7f9edcc00640 1 -- 172.21.15.49:0/2851956665 <== mgr.25384 v2:172.21.15.49:6800/2212175168 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f9ee002cf20 con 0x7f9ec40a1f70 2024-02-26T06:23:25.017 INFO:teuthology.orchestra.run.smithi049.stdout:Retention added to path / 2024-02-26T06:23:25.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.018+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9ec40a1f70 msgr2=0x7f9ec40a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.018+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9ec40a1f70 0x7f9ec40a4430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009d60 tx=0x7f9ed0005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:25.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.018+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0109070 msgr2=0x7f9ee01a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.018+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0109070 0x7f9ee01a0940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800eaa0 tx=0x7f9ed800ef60 comp rx=0 tx=0).stop 2024-02-26T06:23:25.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.019+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 shutdown_connections 2024-02-26T06:23:25.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.019+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9ec40a1f70 0x7f9ec40a4430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.019+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ee0109070 0x7f9ee01a0940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.019+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ee00751c0 0x7f9ee01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.019+0000 7f9ee7000640 1 --2- 172.21.15.49:0/2851956665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee0073be0 0x7f9ee0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.019+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 >> 172.21.15.49:0/2851956665 conn(0x7f9ee00fc740 msgr2=0x7f9ee0104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:25.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.019+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 shutdown_connections 2024-02-26T06:23:25.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.019+0000 7f9ee7000640 1 -- 172.21.15.49:0/2851956665 wait complete. 2024-02-26T06:23:25.036 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-26T06:23:25.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.221+0000 7f7696800640 1 -- 172.21.15.49:0/4269576833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 msgr2=0x7f76900771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.221+0000 7f7696800640 1 --2- 172.21.15.49:0/4269576833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 0x7f76900771e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f767c009900 tx=0x7f767c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:25.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.222+0000 7f7696800640 1 -- 172.21.15.49:0/4269576833 shutdown_connections 2024-02-26T06:23:25.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.222+0000 7f7696800640 1 --2- 172.21.15.49:0/4269576833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7690077720 0x7f7690077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.222+0000 7f7696800640 1 --2- 172.21.15.49:0/4269576833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 0x7f76900771e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.222+0000 7f7696800640 1 --2- 172.21.15.49:0/4269576833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7690075b40 0x7f7690075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.222+0000 7f7696800640 1 -- 172.21.15.49:0/4269576833 >> 172.21.15.49:0/4269576833 conn(0x7f76900fe790 msgr2=0x7f7690100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:25.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.222+0000 7f7696800640 1 -- 172.21.15.49:0/4269576833 shutdown_connections 2024-02-26T06:23:25.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.222+0000 7f7696800640 1 -- 172.21.15.49:0/4269576833 wait complete. 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.223+0000 7f7696800640 1 Processor -- start 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.223+0000 7f7696800640 1 -- start start 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7696800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7690075b40 0x7f769019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7696800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 0x7f769019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7696800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7690077720 0x7f76901a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7696800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76901a5120 con 0x7f7690077720 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7696800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76901a5290 con 0x7f7690075b40 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7696800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76901a5450 con 0x7f7690076d80 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7694a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 0x7f769019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7695400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7690075b40 0x7f769019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7694a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 0x7f769019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:54008/0 (socket says 172.21.15.49:54008) 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7695e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7690077720 0x7f76901a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7694a00640 1 -- 172.21.15.49:0/2627996073 learned_addr learned my addr 172.21.15.49:0/2627996073 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7694a00640 1 -- 172.21.15.49:0/2627996073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7690075b40 msgr2=0x7f769019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7694a00640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7690075b40 0x7f769019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7694a00640 1 -- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7690077720 msgr2=0x7f76901a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7694a00640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7690077720 0x7f76901a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.224+0000 7f7694a00640 1 -- 172.21.15.49:0/2627996073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c009560 con 0x7f7690076d80 2024-02-26T06:23:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.225+0000 7f7694a00640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 0x7f769019db70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f767c006010 tx=0x7f767c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.225+0000 7f768e000640 1 -- 172.21.15.49:0/2627996073 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0043d0 con 0x7f7690076d80 2024-02-26T06:23:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.225+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76901a56d0 con 0x7f7690076d80 2024-02-26T06:23:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.225+0000 7f768e000640 1 -- 172.21.15.49:0/2627996073 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f767c0052d0 con 0x7f7690076d80 2024-02-26T06:23:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.225+0000 7f768e000640 1 -- 172.21.15.49:0/2627996073 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c003cc0 con 0x7f7690076d80 2024-02-26T06:23:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.225+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76901a5be0 con 0x7f7690076d80 2024-02-26T06:23:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.226+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769010fb70 con 0x7f7690076d80 2024-02-26T06:23:25.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.228+0000 7f768e000640 1 -- 172.21.15.49:0/2627996073 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f767c005440 con 0x7f7690076d80 2024-02-26T06:23:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.230+0000 7f768e000640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76780a1ea0 0x7f76780a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.230+0000 7f768e000640 1 -- 172.21.15.49:0/2627996073 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f767c0efed0 con 0x7f7690076d80 2024-02-26T06:23:25.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.230+0000 7f7695400640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76780a1ea0 0x7f76780a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.231+0000 7f7695400640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76780a1ea0 0x7f76780a4360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7684009bd0 tx=0x7f7684005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:25.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.236+0000 7f768e000640 1 -- 172.21.15.49:0/2627996073 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f767c0b9040 con 0x7f7690076d80 2024-02-26T06:23:25.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.386+0000 7f768e000640 1 -- 172.21.15.49:0/2627996073 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 24) v1 ==== 136888+0+0 (secure 0 0 0) 0x7f767c0b9410 con 0x7f7690076d80 2024-02-26T06:23:25.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.482+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 --> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f769002ce40 con 0x7f76780a1ea0 2024-02-26T06:23:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.491+0000 7f768e000640 1 -- 172.21.15.49:0/2627996073 <== mgr.25384 v2:172.21.15.49:6800/2212175168 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f769002ce40 con 0x7f76780a1ea0 2024-02-26T06:23:25.492 INFO:teuthology.orchestra.run.smithi049.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-26T00:00:00", "created": "2024-02-26T06:23:24", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76780a1ea0 msgr2=0x7f76780a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76780a1ea0 0x7f76780a4360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7684009bd0 tx=0x7f7684005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 msgr2=0x7f769019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 0x7f769019db70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f767c006010 tx=0x7f767c005e80 comp rx=0 tx=0).stop 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 shutdown_connections 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76780a1ea0 0x7f76780a4360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7690077720 0x7f76901a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7690076d80 0x7f769019db70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 --2- 172.21.15.49:0/2627996073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7690075b40 0x7f769019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 >> 172.21.15.49:0/2627996073 conn(0x7f76900fe790 msgr2=0x7f7690100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:25.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 shutdown_connections 2024-02-26T06:23:25.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.493+0000 7f7696800640 1 -- 172.21.15.49:0/2627996073 wait complete. 2024-02-26T06:23:25.511 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-26T06:23:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:25 smithi080 ceph-mon[36206]: mgrmap e23: x(active, since 1.03143s), standbys: z, y 2024-02-26T06:23:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:25 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:25 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:25.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:25 smithi080 ceph-mon[36206]: [26/Feb/2024:06:23:24] ENGINE Bus STARTING 2024-02-26T06:23:25.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:25 smithi080 ceph-mon[36206]: [26/Feb/2024:06:23:24] ENGINE Serving on https://172.21.15.49:7150 2024-02-26T06:23:25.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:25 smithi080 ceph-mon[36206]: [26/Feb/2024:06:23:24] ENGINE Serving on http://172.21.15.49:8765 2024-02-26T06:23:25.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:25 smithi080 ceph-mon[36206]: [26/Feb/2024:06:23:24] ENGINE Bus STARTED 2024-02-26T06:23:25.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:25 smithi080 ceph-mon[36206]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:25.583 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:25 smithi049 ceph-mon[31696]: mgrmap e23: x(active, since 1.03143s), standbys: z, y 2024-02-26T06:23:25.583 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:25 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:25.583 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:25 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:25.583 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:25 smithi049 ceph-mon[31696]: [26/Feb/2024:06:23:24] ENGINE Bus STARTING 2024-02-26T06:23:25.583 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:25 smithi049 ceph-mon[31696]: [26/Feb/2024:06:23:24] ENGINE Serving on https://172.21.15.49:7150 2024-02-26T06:23:25.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:25 smithi049 ceph-mon[31696]: [26/Feb/2024:06:23:24] ENGINE Serving on http://172.21.15.49:8765 2024-02-26T06:23:25.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:25 smithi049 ceph-mon[31696]: [26/Feb/2024:06:23:24] ENGINE Bus STARTED 2024-02-26T06:23:25.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:25 smithi049 ceph-mon[31696]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:25 smithi115 ceph-mon[36426]: mgrmap e23: x(active, since 1.03143s), standbys: z, y 2024-02-26T06:23:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:25 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:25 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:25 smithi115 ceph-mon[36426]: [26/Feb/2024:06:23:24] ENGINE Bus STARTING 2024-02-26T06:23:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:25 smithi115 ceph-mon[36426]: [26/Feb/2024:06:23:24] ENGINE Serving on https://172.21.15.49:7150 2024-02-26T06:23:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:25 smithi115 ceph-mon[36426]: [26/Feb/2024:06:23:24] ENGINE Serving on http://172.21.15.49:8765 2024-02-26T06:23:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:25 smithi115 ceph-mon[36426]: [26/Feb/2024:06:23:24] ENGINE Bus STARTED 2024-02-26T06:23:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:25 smithi115 ceph-mon[36426]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3859814783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ccc077720 msgr2=0x7f3ccc077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3859814783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ccc077720 0x7f3ccc077fc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009ca0 tx=0x7f3cc402c100 comp rx=0 tx=0).stop 2024-02-26T06:23:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3859814783 shutdown_connections 2024-02-26T06:23:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3859814783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc076d80 0x7f3ccc0771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3859814783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ccc077720 0x7f3ccc077fc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3859814783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ccc075b40 0x7f3ccc075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3859814783 >> 172.21.15.49:0/3859814783 conn(0x7f3ccc0fe970 msgr2=0x7f3ccc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:25.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3859814783 shutdown_connections 2024-02-26T06:23:25.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.711+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3859814783 wait complete. 2024-02-26T06:23:25.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3cd2e00640 1 Processor -- start 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3cd2e00640 1 -- start start 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3cd2e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc075b40 0x7f3ccc19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3cd2e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ccc076d80 0x7f3ccc19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3cd2e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ccc077720 0x7f3ccc1a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3cd2e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a5200 con 0x7f3ccc076d80 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3cd2e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a5370 con 0x7f3ccc075b40 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3cd2e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a5530 con 0x7f3ccc077720 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.713+0000 7f3ccbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc075b40 0x7f3ccc19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc075b40 0x7f3ccc19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33568/0 (socket says 172.21.15.49:33568) 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccbe00640 1 -- 172.21.15.49:0/3028400167 learned_addr learned my addr 172.21.15.49:0/3028400167 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccb400640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ccc076d80 0x7f3ccc19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3cd0e00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ccc077720 0x7f3ccc1a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccbe00640 1 -- 172.21.15.49:0/3028400167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ccc077720 msgr2=0x7f3ccc1a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccbe00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ccc077720 0x7f3ccc1a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccbe00640 1 -- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ccc076d80 msgr2=0x7f3ccc19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccbe00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ccc076d80 0x7f3ccc19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccbe00640 1 -- 172.21.15.49:0/3028400167 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009900 con 0x7f3ccc075b40 2024-02-26T06:23:25.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.714+0000 7f3ccbe00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc075b40 0x7f3ccc19d7b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00e9f0 tx=0x7f3cbc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:25.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.715+0000 7f3cc8c00640 1 -- 172.21.15.49:0/3028400167 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc009720 con 0x7f3ccc075b40 2024-02-26T06:23:25.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.715+0000 7f3cc8c00640 1 -- 172.21.15.49:0/3028400167 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cbc00cd80 con 0x7f3ccc075b40 2024-02-26T06:23:25.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.715+0000 7f3cc8c00640 1 -- 172.21.15.49:0/3028400167 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc010690 con 0x7f3ccc075b40 2024-02-26T06:23:25.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.715+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc1a5810 con 0x7f3ccc075b40 2024-02-26T06:23:25.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.715+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc1a5cd0 con 0x7f3ccc075b40 2024-02-26T06:23:25.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.716+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc10fd50 con 0x7f3ccc075b40 2024-02-26T06:23:25.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.718+0000 7f3cc8c00640 1 -- 172.21.15.49:0/3028400167 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 24) v1 ==== 136888+0+0 (secure 0 0 0) 0x7f3cbc0046c0 con 0x7f3ccc075b40 2024-02-26T06:23:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.719+0000 7f3cc8c00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3cb00a2350 0x7f3cb00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:25.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.719+0000 7f3cc8c00640 1 -- 172.21.15.49:0/3028400167 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f3cbc014070 con 0x7f3ccc075b40 2024-02-26T06:23:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.719+0000 7f3ccb400640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3cb00a2350 0x7f3cb00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:25.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.720+0000 7f3ccb400640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3cb00a2350 0x7f3cb00a4810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc19ec20 tx=0x7f3cb8005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:25.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.723+0000 7f3cc8c00640 1 -- 172.21.15.49:0/3028400167 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3cbc097040 con 0x7f3ccc075b40 2024-02-26T06:23:25.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.940+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 --> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3ccc02ceb0 con 0x7f3cb00a2350 2024-02-26T06:23:25.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.948+0000 7f3cc8c00640 1 -- 172.21.15.49:0/3028400167 <== mgr.25384 v2:172.21.15.49:6800/2212175168 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f3ccc02ceb0 con 0x7f3cb00a2350 2024-02-26T06:23:25.949 INFO:teuthology.orchestra.run.smithi049.stdout:/ 1m 6m3h 2024-02-26T06:23:25.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3cb00a2350 msgr2=0x7f3cb00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3cb00a2350 0x7f3cb00a4810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc19ec20 tx=0x7f3cb8005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:25.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc075b40 msgr2=0x7f3ccc19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:25.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc075b40 0x7f3ccc19d7b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00e9f0 tx=0x7f3cbc00edf0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 shutdown_connections 2024-02-26T06:23:25.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3cb00a2350 0x7f3cb00a4810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ccc077720 0x7f3ccc1a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ccc076d80 0x7f3ccc19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 --2- 172.21.15.49:0/3028400167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc075b40 0x7f3ccc19d7b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:25.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 >> 172.21.15.49:0/3028400167 conn(0x7f3ccc0fe970 msgr2=0x7f3ccc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:25.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.951+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 shutdown_connections 2024-02-26T06:23:25.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:25.952+0000 7f3cd2e00640 1 -- 172.21.15.49:0/3028400167 wait complete. 2024-02-26T06:23:25.968 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-02-26T06:23:26.048 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-26T06:23:26.061 INFO:teuthology.run_tasks:Running task background_exec... 2024-02-26T06:23:26.076 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:23:26.076 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-02-26T06:23:26.084 INFO:teuthology.run_tasks:Running task exec... 2024-02-26T06:23:26.093 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T06:23:26.093 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:23:26.093 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-02-26T06:23:26.213 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:26.263 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem Size Used Avail Use% Mounted on 2024-02-26T06:23:26.264 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:3300,172.21.15.80:3300,172.21.15.115:3300:/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 340G 0 340G 0% /tmp/cephfs 2024-02-26T06:23:26.270 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-02-26T06:23:26.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-02-26T06:23:26.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:26.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-02-26T06:23:26.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:23:26.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:26.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: from='client.25439 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:26.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.525 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:26 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.586 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-02-26T06:23:26.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:26.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-02-26T06:23:26.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:23:26.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: from='client.25439 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:26 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: from='client.25439 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:26 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi080 to 174.1M 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: Unable to set osd_memory_target on smithi080 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.832 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:27 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi080 to 174.1M 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: Unable to set osd_memory_target on smithi080 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:27 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi080 to 174.1M 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: Unable to set osd_memory_target on smithi080 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:27.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:27 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:28 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-02-26T06:23:28.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:28 smithi080 ceph-mon[36206]: Unable to set osd_memory_target on smithi115 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-02-26T06:23:28.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:28 smithi080 ceph-mon[36206]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:28.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:28 smithi080 ceph-mon[36206]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-02-26T06:23:28.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:28 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-02-26T06:23:28.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:28 smithi115 ceph-mon[36426]: Unable to set osd_memory_target on smithi115 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-02-26T06:23:28.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:28 smithi115 ceph-mon[36426]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:28.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:28 smithi115 ceph-mon[36426]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-02-26T06:23:28.847 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:28 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-02-26T06:23:28.847 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:28 smithi049 ceph-mon[31696]: Unable to set osd_memory_target on smithi115 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-02-26T06:23:28.847 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:28 smithi049 ceph-mon[31696]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:28.847 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:28 smithi049 ceph-mon[31696]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-02-26T06:23:29.172 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-02-26T06:23:29.287 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-02-26T06:23:29.293 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-02-26T06:23:29.418 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-02-26T06:23:29.534 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2024-02-26T06:23:29.534 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:29.534 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:23:29.534 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 .. 2024-02-26T06:23:29.534 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 data 2024-02-26T06:23:29.534 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:29.534 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2024-02-26T06:23:29.535 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:29.535 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:29.535 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 .. 2024-02-26T06:23:29.540 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-02-26T06:23:29.699 INFO:teuthology.orchestra.run.smithi049.stdout:[Service] 2024-02-26T06:23:29.699 INFO:teuthology.orchestra.run.smithi049.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-02-26T06:23:29.699 INFO:teuthology.orchestra.run.smithi049.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-02-26T06:23:29.705 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-02-26T06:23:29.931 INFO:teuthology.orchestra.run.smithi049.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi049 to 176.7M 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: Unable to set osd_memory_target on smithi049 to 185363763: error parsing value: Value '185363763' is below minimum 939524096 2024-02-26T06:23:30.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:30.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:30.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: Updating smithi049:/etc/ceph/ceph.conf 2024-02-26T06:23:30.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: Updating smithi080:/etc/ceph/ceph.conf 2024-02-26T06:23:30.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:29 smithi049 ceph-mon[31696]: Updating smithi115:/etc/ceph/ceph.conf 2024-02-26T06:23:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi049 to 176.7M 2024-02-26T06:23:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: Unable to set osd_memory_target on smithi049 to 185363763: error parsing value: Value '185363763' is below minimum 939524096 2024-02-26T06:23:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: Updating smithi049:/etc/ceph/ceph.conf 2024-02-26T06:23:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: Updating smithi080:/etc/ceph/ceph.conf 2024-02-26T06:23:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:29 smithi080 ceph-mon[36206]: Updating smithi115:/etc/ceph/ceph.conf 2024-02-26T06:23:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi049 to 176.7M 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: Unable to set osd_memory_target on smithi049 to 185363763: error parsing value: Value '185363763' is below minimum 939524096 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: Updating smithi049:/etc/ceph/ceph.conf 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: Updating smithi080:/etc/ceph/ceph.conf 2024-02-26T06:23:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:29 smithi115 ceph-mon[36426]: Updating smithi115:/etc/ceph/ceph.conf 2024-02-26T06:23:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:31.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:31.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:30 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:31.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:31.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:31.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:31.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:31.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:30 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:23:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:23:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: Updating smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:31.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: Updating smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:31.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:31.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:30 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:31 smithi080 ceph-mon[36206]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:31 smithi080 ceph-mon[36206]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:31 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:31 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:31 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:31 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:31 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:31 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:31 smithi115 ceph-mon[36426]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:31 smithi115 ceph-mon[36426]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:31 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:31 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:31 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:31 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:31 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:31 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:31 smithi049 ceph-mon[31696]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:23:32.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:31 smithi049 ceph-mon[31696]: Updating smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:23:32.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:31 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:31 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:31 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:32.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:31 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:32.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:31 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:32.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:31 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:32 smithi080 ceph-mon[36206]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-02-26T06:23:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:32 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:32 smithi115 ceph-mon[36426]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-02-26T06:23:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:32 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:33.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:32 smithi049 ceph-mon[31696]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-02-26T06:23:33.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:32 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:23:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:34 smithi080 ceph-mon[36206]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-02-26T06:23:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:34 smithi115 ceph-mon[36426]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-02-26T06:23:35.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:34 smithi049 ceph-mon[31696]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-02-26T06:23:36.873 INFO:teuthology.orchestra.run.smithi049.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-02-26T06:23:36.883 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-02-26T06:23:37.058 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 .. 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 data 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:total 35 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 . 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 .. 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 base 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Feb 26 06:23 global 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 log 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_commit_ts 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_dynshmem 2024-02-26T06:23:37.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Feb 26 06:23 pg_hba.conf 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Feb 26 06:23 pg_ident.conf 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 pg_logical 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 pg_multixact 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_notify 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_replslot 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_serial 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_snapshots 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_stat 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_stat_tmp 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 pg_subtrans 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_tblspc 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_twophase 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 PG_VERSION 2024-02-26T06:23:37.060 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 pg_wal 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 pg_xact 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Feb 26 06:23 postgresql.auto.conf 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Feb 26 06:23 postgresql.conf 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base: 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 . 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 1 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 13493 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 13494 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/1: 2024-02-26T06:23:37.061 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 . 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 .. 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 112 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 113 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 1247 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1247_fsm 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1247_vm 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 26 06:23 1249 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1249_fsm 2024-02-26T06:23:37.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1249_vm 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 26 06:23 1255 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1255_fsm 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1255_vm 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 26 06:23 1259 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1259_fsm 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1259_vm 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 26 06:23 13344 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13344_fsm 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13344_vm 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13346 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13348 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13349_fsm 2024-02-26T06:23:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349_vm 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13351 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13353 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13354_fsm 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354_vm 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13356 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13358 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13359_fsm 2024-02-26T06:23:37.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359_vm 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13361 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13363 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1417 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1418 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 174 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 175 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2187 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2224 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2328 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2336 2024-02-26T06:23:37.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2337 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2579 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2600 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2600_fsm 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2600_vm 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2601_fsm 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601_vm 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2602 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2602_fsm 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2602_vm 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2603 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2603_fsm 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2603_vm 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2604 2024-02-26T06:23:37.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2605 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2605_fsm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2605_vm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2606_fsm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606_vm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2607 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2607_fsm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2607_vm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 26 06:23 2608 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2608_fsm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2608_vm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 26 06:23 2609 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2609_fsm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2609_vm 2024-02-26T06:23:37.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2610 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2610_fsm 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2610_vm 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2611 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2612_fsm 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612_vm 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2613 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2615_fsm 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615_vm 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2616 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2616_fsm 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2616_vm 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 26 06:23 2617 2024-02-26T06:23:37.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2617_fsm 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2617_vm 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 26 06:23 2618 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2618_fsm 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2618_vm 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 26 06:23 2619 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2619_fsm 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2619_vm 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2620 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2650 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2651 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2652 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2653 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2654 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2655 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2656 2024-02-26T06:23:37.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2657 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 26 06:23 2658 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 2659 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2660 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2661 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2662 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2663 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2664 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2665 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2666 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2667 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2668 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2669 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2670 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 26 06:23 2673 2024-02-26T06:23:37.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 26 06:23 2674 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 26 06:23 2675 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2678 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2679 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2680 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2681 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2682 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2683 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2684 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2685 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2686 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2687 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2688 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2689 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 26 06:23 2690 2024-02-26T06:23:37.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 26 06:23 2691 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2692 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2693 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2696 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2699 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2701 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2702 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2703 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2704 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2753 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2753_fsm 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2753_vm 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2754 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2755 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2756 2024-02-26T06:23:37.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2757 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2830 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2831 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2832 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2833 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2834 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2835 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2836 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2837 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 26 06:23 2838 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2838_fsm 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2838_vm 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2839 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840_fsm 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2840_vm 2024-02-26T06:23:37.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2841 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2995 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2996 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3079_fsm 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079_vm 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3080 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3081 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3085 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3118 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3119 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 26 06:23 3164 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3256 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3257 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3258 2024-02-26T06:23:37.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3350 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3351 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3379 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3380 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3381 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3394 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3394_fsm 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3394_vm 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3395 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3429 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3430 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3431 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3433 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3439 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3440 2024-02-26T06:23:37.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3455 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 26 06:23 3456 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3456_fsm 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3456_vm 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3466 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3467 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3468 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3501 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3502 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3503 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3534 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3541_fsm 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541_vm 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3542 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3574 2024-02-26T06:23:37.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3575 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3576 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3596 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3597 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3598 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3599 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3600_fsm 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600_vm 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3601_fsm 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601_vm 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3602_fsm 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602_vm 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603_fsm 2024-02-26T06:23:37.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3603_vm 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3604 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3605 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3606 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3607 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3608 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 3609 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3712 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3764_fsm 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764_vm 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3766 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3767 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3997 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4143 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4144 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4145 2024-02-26T06:23:37.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4146 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4147 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4148 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4149 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4150 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4151 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4152 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4153 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4154 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4155 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4156 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4157 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4158 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4159 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4160 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4161 2024-02-26T06:23:37.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4162 2024-02-26T06:23:37.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4163 2024-02-26T06:23:37.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4164 2024-02-26T06:23:37.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4165 2024-02-26T06:23:37.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4166 2024-02-26T06:23:37.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4167 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4168 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4169 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4170 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4171 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4172 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4173 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4174 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 5002 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 548 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 549 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6102 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6104 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6106 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6110 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6111 2024-02-26T06:23:37.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6112 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6113 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6117 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 826 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 827 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 828 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 26 06:23 pg_filenode.map 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 PG_VERSION 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 . 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 .. 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 112 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 113 2024-02-26T06:23:37.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 1247 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1247_fsm 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1247_vm 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 26 06:23 1249 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1249_fsm 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1249_vm 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 26 06:23 1255 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1255_fsm 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1255_vm 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 26 06:23 1259 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1259_fsm 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1259_vm 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 26 06:23 13344 2024-02-26T06:23:37.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13344_fsm 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13344_vm 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13346 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13348 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13349_fsm 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349_vm 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13351 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13353 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13354_fsm 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354_vm 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13356 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13358 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13359_fsm 2024-02-26T06:23:37.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359_vm 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13361 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13363 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1417 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1418 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 174 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 175 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2187 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2224 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2328 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2336 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2337 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2579 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2600 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2600_fsm 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2600_vm 2024-02-26T06:23:37.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2601_fsm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601_vm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2602 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2602_fsm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2602_vm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2603 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2603_fsm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2603_vm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2604 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2605 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2605_fsm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2605_vm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2606_fsm 2024-02-26T06:23:37.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606_vm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2607 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2607_fsm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2607_vm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 26 06:23 2608 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2608_fsm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2608_vm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 26 06:23 2609 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2609_fsm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2609_vm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2610 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2610_fsm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2610_vm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2611 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2612_fsm 2024-02-26T06:23:37.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612_vm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2613 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2615_fsm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615_vm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2616 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2616_fsm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2616_vm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 26 06:23 2617 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2617_fsm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2617_vm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 26 06:23 2618 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2618_fsm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2618_vm 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 26 06:23 2619 2024-02-26T06:23:37.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2619_fsm 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2619_vm 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2620 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2650 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2651 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2652 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2653 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2654 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2655 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2656 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2657 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 26 06:23 2658 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 2659 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2660 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2661 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2662 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2663 2024-02-26T06:23:37.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2664 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2665 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2666 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2667 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2668 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2669 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2670 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 26 06:23 2673 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 26 06:23 2674 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 26 06:23 2675 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2678 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2679 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2680 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2681 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2682 2024-02-26T06:23:37.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2683 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2684 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2685 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2686 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2687 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2688 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2689 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 26 06:23 2690 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 26 06:23 2691 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2692 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2693 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2696 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2699 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2701 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2702 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2703 2024-02-26T06:23:37.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2704 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2753 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2753_fsm 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2753_vm 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2754 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2755 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2756 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2757 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2830 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2831 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2832 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2833 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2834 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2835 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2836 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2837 2024-02-26T06:23:37.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 26 06:23 2838 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2838_fsm 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2838_vm 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2839 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840_fsm 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2840_vm 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2841 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2995 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2996 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3079_fsm 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079_vm 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3080 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3081 2024-02-26T06:23:37.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3085 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3118 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3119 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 26 06:23 3164 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3256 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3257 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3258 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3350 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3351 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3379 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3380 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3381 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3394 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3394_fsm 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3394_vm 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3395 2024-02-26T06:23:37.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3429 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3430 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3431 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3433 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3439 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3440 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3455 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 26 06:23 3456 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3456_fsm 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3456_vm 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3466 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3467 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3468 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3501 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3502 2024-02-26T06:23:37.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3503 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3534 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3541_fsm 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541_vm 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3542 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3574 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3575 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3576 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3596 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3597 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3598 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3599 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3600_fsm 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600_vm 2024-02-26T06:23:37.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3601_fsm 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601_vm 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3602_fsm 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602_vm 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603_fsm 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3603_vm 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3604 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3605 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3606 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3607 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3608 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 3609 2024-02-26T06:23:37.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3712 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3764_fsm 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764_vm 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3766 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3767 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3997 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4143 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4144 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4145 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4146 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4147 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4148 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4149 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4150 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4151 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4152 2024-02-26T06:23:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4153 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4154 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4155 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4156 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4157 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4158 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4159 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4160 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4161 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4162 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4163 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4164 2024-02-26T06:23:37.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4165 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4166 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4167 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4168 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4169 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4170 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4171 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4172 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4173 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4174 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 5002 2024-02-26T06:23:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 548 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 549 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6102 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6104 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6106 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6110 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6111 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6112 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6113 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6117 2024-02-26T06:23:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 826 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 827 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 828 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 26 06:23 pg_filenode.map 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 PG_VERSION 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 . 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 .. 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 112 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 113 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 1247 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1247_fsm 2024-02-26T06:23:37.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1247_vm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 26 06:23 1249 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1249_fsm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1249_vm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 26 06:23 1255 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1255_fsm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1255_vm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 26 06:23 1259 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1259_fsm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1259_vm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 26 06:23 13344 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13344_fsm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13344_vm 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13346 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13348 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349 2024-02-26T06:23:37.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13349_fsm 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349_vm 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13351 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13353 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13354_fsm 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354_vm 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13356 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13358 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13359_fsm 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359_vm 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13361 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13363 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1417 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1418 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 174 2024-02-26T06:23:37.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 175 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2187 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2224 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2328 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2336 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2337 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2579 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2600 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2600_fsm 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2600_vm 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2601_fsm 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601_vm 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2602 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2602_fsm 2024-02-26T06:23:37.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2602_vm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2603 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2603_fsm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2603_vm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2604 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2605 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2605_fsm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2605_vm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2606_fsm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606_vm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2607 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2607_fsm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2607_vm 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 26 06:23 2608 2024-02-26T06:23:37.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2608_fsm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2608_vm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 26 06:23 2609 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2609_fsm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2609_vm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2610 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2610_fsm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2610_vm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2611 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2612_fsm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612_vm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2613 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2615_fsm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615_vm 2024-02-26T06:23:37.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2616 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2616_fsm 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2616_vm 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 26 06:23 2617 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2617_fsm 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2617_vm 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 26 06:23 2618 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2618_fsm 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2618_vm 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 26 06:23 2619 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2619_fsm 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2619_vm 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2620 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2650 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2651 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2652 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2653 2024-02-26T06:23:37.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2654 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2655 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2656 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2657 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 26 06:23 2658 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 2659 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2660 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2661 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2662 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2663 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2664 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2665 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2666 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2667 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2668 2024-02-26T06:23:37.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2669 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2670 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 26 06:23 2673 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 26 06:23 2674 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 26 06:23 2675 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2678 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2679 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2680 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2681 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2682 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2683 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2684 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2685 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2686 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2687 2024-02-26T06:23:37.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2688 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2689 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 26 06:23 2690 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 26 06:23 2691 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2692 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2693 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2696 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2699 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2701 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2702 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2703 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2704 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2753 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2753_fsm 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2753_vm 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2754 2024-02-26T06:23:37.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2755 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2756 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2757 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2830 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2831 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2832 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2833 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2834 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2835 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2836 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2837 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 26 06:23 2838 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2838_fsm 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2838_vm 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2839 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840 2024-02-26T06:23:37.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840_fsm 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2840_vm 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2841 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2995 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2996 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3079_fsm 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079_vm 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3080 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3081 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3085 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3118 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3119 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 26 06:23 3164 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3256 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3257 2024-02-26T06:23:37.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3258 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3350 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3351 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3379 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3380 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3381 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3394 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3394_fsm 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3394_vm 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3395 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3429 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3430 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3431 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3433 2024-02-26T06:23:37.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3439 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3440 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3455 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 26 06:23 3456 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3456_fsm 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3456_vm 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3466 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3467 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3468 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3501 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3502 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3503 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3534 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3541_fsm 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541_vm 2024-02-26T06:23:37.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3542 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3574 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3575 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3576 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3596 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3597 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3598 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3599 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3600_fsm 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600_vm 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3601_fsm 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601_vm 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3602_fsm 2024-02-26T06:23:37.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602_vm 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603_fsm 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3603_vm 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3604 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3605 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3606 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3607 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3608 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 3609 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3712 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3764_fsm 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764_vm 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3766 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3767 2024-02-26T06:23:37.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3997 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4143 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4144 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4145 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4146 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4147 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4148 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4149 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4150 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4151 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4152 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4153 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4154 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4155 2024-02-26T06:23:37.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4156 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4157 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4158 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4159 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4160 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4161 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4162 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4163 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4164 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4165 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4166 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4167 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4168 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4169 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4170 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4171 2024-02-26T06:23:37.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4172 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4173 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4174 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 5002 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 548 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 549 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6102 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6104 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6106 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6110 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6111 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6112 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6113 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6117 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 826 2024-02-26T06:23:37.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 827 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 828 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 26 06:23 pg_filenode.map 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 PG_VERSION 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/global: 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:total 537 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Feb 26 06:23 . 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1213 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1213_fsm 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1213_vm 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1214 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1214_fsm 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1214_vm 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 1232 2024-02-26T06:23:37.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 1233 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1260 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1260_fsm 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1260_vm 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1261 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1261_fsm 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1261_vm 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1262 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1262_fsm 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1262_vm 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2396 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2396_fsm 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2396_vm 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2397 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2671 2024-02-26T06:23:37.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2672 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2676 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2677 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2694 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2695 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2697 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2698 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2846 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2847 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2964 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2965 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2966 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2967 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3592 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3593 2024-02-26T06:23:37.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4060 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4061 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4175 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4176 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4177 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4178 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4181 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4182 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4183 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4184 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4185 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4186 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6000 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6001 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6002 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6100 2024-02-26T06:23:37.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6114 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6115 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 pg_control 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 26 06:23 pg_filenode.map 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/log: 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.127 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:total 1 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 . 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 mappings 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Feb 26 06:23 replorigin_checkpoint 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 snapshots 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.128 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 .. 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 .. 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 . 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 members 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 offsets 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-02-26T06:23:37.129 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 .. 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 0000 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 .. 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 0000 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.130 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-02-26T06:23:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 0000 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.132 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:total 16384 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 . 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:23 000000010000000000000001 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 archive_status 2024-02-26T06:23:37.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 .. 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 26 06:23 .. 2024-02-26T06:23:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 0000 2024-02-26T06:23:37.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-02-26T06:23:37.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:36 smithi049 ceph-mon[31696]: pgmap v9: 97 pgs: 97 active+clean; 21 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.9 MiB/s wr, 62 op/s 2024-02-26T06:23:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:36 smithi080 ceph-mon[36206]: pgmap v9: 97 pgs: 97 active+clean; 21 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.9 MiB/s wr, 62 op/s 2024-02-26T06:23:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:36 smithi115 ceph-mon[36426]: pgmap v9: 97 pgs: 97 active+clean; 21 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.9 MiB/s wr, 62 op/s 2024-02-26T06:23:37.475 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-02-26T06:23:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:dropping old tables... 2024-02-26T06:23:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-02-26T06:23:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-02-26T06:23:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-02-26T06:23:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-02-26T06:23:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:creating tables... 2024-02-26T06:23:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:generating data (client-side)... 2024-02-26T06:23:37.957 INFO:teuthology.orchestra.run.smithi049.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.19 s, remaining 5.97 s) 2024-02-26T06:23:38.342 INFO:teuthology.orchestra.run.smithi049.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.58 s, remaining 8.67 s) 2024-02-26T06:23:38.668 INFO:teuthology.orchestra.run.smithi049.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.90 s, remaining 8.74 s) 2024-02-26T06:23:38.760 INFO:teuthology.orchestra.run.smithi049.stderr:400000 of 3200000 tuples (12%) done (elapsed 1.00 s, remaining 6.97 s) 2024-02-26T06:23:39.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:38 smithi049 ceph-mon[31696]: pgmap v10: 97 pgs: 97 active+clean; 21 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.7 MiB/s wr, 56 op/s 2024-02-26T06:23:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.41 s, remaining 7.59 s) 2024-02-26T06:23:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:39 smithi080 ceph-mon[36206]: pgmap v10: 97 pgs: 97 active+clean; 21 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.7 MiB/s wr, 56 op/s 2024-02-26T06:23:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:39 smithi115 ceph-mon[36426]: pgmap v10: 97 pgs: 97 active+clean; 21 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.7 MiB/s wr, 56 op/s 2024-02-26T06:23:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 -- 172.21.15.49:0/3713530070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f102810d8e0 msgr2=0x7f102810fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 --2- 172.21.15.49:0/3713530070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f102810d8e0 0x7f102810fcd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1020009be0 tx=0x7f102002bef0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 -- 172.21.15.49:0/3713530070 shutdown_connections 2024-02-26T06:23:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 --2- 172.21.15.49:0/3713530070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f102810d8e0 0x7f102810fcd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 --2- 172.21.15.49:0/3713530070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10281006e0 0x7f102810d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 --2- 172.21.15.49:0/3713530070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280ffd30 0x7f1028100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 -- 172.21.15.49:0/3713530070 >> 172.21.15.49:0/3713530070 conn(0x7f10280fb990 msgr2=0x7f10280fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 -- 172.21.15.49:0/3713530070 shutdown_connections 2024-02-26T06:23:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.354+0000 7f1027200640 1 -- 172.21.15.49:0/3713530070 wait complete. 2024-02-26T06:23:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1027200640 1 Processor -- start 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1027200640 1 -- start start 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1027200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280ffd30 0x7f102819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1027200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10281006e0 0x7f102819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1027200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f102810d8e0 0x7f10281996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1027200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028199e40 con 0x7f102810d8e0 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1027200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028199fb0 con 0x7f10281006e0 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1027200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102819a170 con 0x7f10280ffd30 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.356+0000 7f1025400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10281006e0 0x7f102819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10281006e0 0x7f102819faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50544/0 (socket says 172.21.15.49:50544) 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025400640 1 -- 172.21.15.49:0/2004910361 learned_addr learned my addr 172.21.15.49:0/2004910361 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025e00640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280ffd30 0x7f102819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1026800640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f102810d8e0 0x7f10281996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025400640 1 -- 172.21.15.49:0/2004910361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280ffd30 msgr2=0x7f102819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025400640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280ffd30 0x7f102819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025400640 1 -- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f102810d8e0 msgr2=0x7f10281996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025400640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f102810d8e0 0x7f10281996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025400640 1 -- 172.21.15.49:0/2004910361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1020009840 con 0x7f10281006e0 2024-02-26T06:23:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.357+0000 7f1025400640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10281006e0 0x7f102819faa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9c0 tx=0x7f101400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.358+0000 7f101ea00640 1 -- 172.21.15.49:0/2004910361 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400ccc0 con 0x7f10281006e0 2024-02-26T06:23:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.358+0000 7f101ea00640 1 -- 172.21.15.49:0/2004910361 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f101400ce20 con 0x7f10281006e0 2024-02-26T06:23:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.358+0000 7f101ea00640 1 -- 172.21.15.49:0/2004910361 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014010430 con 0x7f10281006e0 2024-02-26T06:23:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.358+0000 7f1027200640 1 -- 172.21.15.49:0/2004910361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102819a450 con 0x7f10281006e0 2024-02-26T06:23:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.358+0000 7f1027200640 1 -- 172.21.15.49:0/2004910361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281a6550 con 0x7f10281006e0 2024-02-26T06:23:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.361+0000 7f102ff97740 1 Processor -- start 2024-02-26T06:23:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.361+0000 7f102ff97740 1 -- start start 2024-02-26T06:23:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f101ea00640 1 -- 172.21.15.49:0/2004910361 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f1014004f80 con 0x7f10281006e0 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f102ff97740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99875d30 0x564c99874a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f102ff97740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c99874fc0 0x564c99875440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f102ff97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c9987e540 0x564c99880930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f102ff97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c998305f0 con 0x564c9987e540 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f102ff97740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c997e2890 con 0x564c99875d30 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f102ff97740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c997dfdc0 con 0x564c99874fc0 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f1026800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99875d30 0x564c99874a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f1025400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c9987e540 0x564c99880930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f1026800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99875d30 0x564c99874a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50546/0 (socket says 172.21.15.49:50546) 2024-02-26T06:23:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.362+0000 7f1026800640 1 -- 172.21.15.49:0/3136380387 learned_addr learned my addr 172.21.15.49:0/3136380387 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.363+0000 7f1025e00640 1 --2- 172.21.15.49:0/3136380387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c99874fc0 0x564c99875440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.363+0000 7f1026800640 1 -- 172.21.15.49:0/3136380387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c99874fc0 msgr2=0x564c99875440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.363+0000 7f1026800640 1 --2- 172.21.15.49:0/3136380387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c99874fc0 0x564c99875440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.363+0000 7f1026800640 1 -- 172.21.15.49:0/3136380387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c9987e540 msgr2=0x564c99880930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.363+0000 7f1026800640 1 --2- 172.21.15.49:0/3136380387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c9987e540 0x564c99880930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.363+0000 7f1026800640 1 -- 172.21.15.49:0/3136380387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c9975cad0 con 0x564c99875d30 2024-02-26T06:23:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.363+0000 7f1026800640 1 --2- 172.21.15.49:0/3136380387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99875d30 0x564c99874a80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1028100d10 tx=0x7f102002cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad045fcb404b5057 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.364+0000 7f1013400640 1 -- 172.21.15.49:0/3136380387 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102002ca90 con 0x564c99875d30 2024-02-26T06:23:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.364+0000 7f1013400640 1 -- 172.21.15.49:0/3136380387 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10200333e0 con 0x564c99875d30 2024-02-26T06:23:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.364+0000 7f1013400640 1 -- 172.21.15.49:0/3136380387 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200042a0 con 0x564c99875d30 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.364+0000 7f102ff97740 1 -- 172.21.15.49:0/3136380387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99875d30 msgr2=0x564c99874a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.364+0000 7f102ff97740 1 --2- 172.21.15.49:0/3136380387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99875d30 0x564c99874a80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1028100d10 tx=0x7f102002cd60 comp rx=0 tx=0).stop 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f102ff97740 1 -- 172.21.15.49:0/3136380387 shutdown_connections 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f102ff97740 1 --2- 172.21.15.49:0/3136380387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c9987e540 0x564c99880930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f102ff97740 1 --2- 172.21.15.49:0/3136380387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c99874fc0 0x564c99875440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f102ff97740 1 --2- 172.21.15.49:0/3136380387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99875d30 0x564c99874a80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f102ff97740 1 -- 172.21.15.49:0/3136380387 >> 172.21.15.49:0/3136380387 conn(0x564c9983c480 msgr2=0x564c9986fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f101ea00640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f100c0a2330 0x7f100c0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f102ff97740 1 -- 172.21.15.49:0/3136380387 shutdown_connections 2024-02-26T06:23:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f102ff97740 1 -- 172.21.15.49:0/3136380387 wait complete. 2024-02-26T06:23:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f1026800640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f100c0a2330 0x7f100c0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.365+0000 7f101ea00640 1 -- 172.21.15.49:0/2004910361 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1014014070 con 0x7f10281006e0 2024-02-26T06:23:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.366+0000 7f102ff97740 1 -- start start 2024-02-26T06:23:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.366+0000 7f102ff97740 1 Processor -- start 2024-02-26T06:23:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.366+0000 7f1026800640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f100c0a2330 0x7f100c0a47f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f102002c430 tx=0x7f102002bef0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f102ff97740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99874fc0 0x564c9987b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f102ff97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f102ff97740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c9987bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f102ff97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c997db5a0 con 0x564c99875d30 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f102ff97740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c99780b20 con 0x564c99874fc0 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f102ff97740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c997807c0 con 0x564c9987e540 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f102ff97740 10 client.0 _ll_register_callbacks cb 0x564c99794520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f1025e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99874fc0 0x564c9987b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f1025e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99874fc0 0x564c9987b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50558/0 (socket says 172.21.15.49:50558) 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.367+0000 7f1025e00640 1 -- 172.21.15.49:0/3818342185 learned_addr learned my addr 172.21.15.49:0/3818342185 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1026800640 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c9987bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1025400640 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1025e00640 1 -- 172.21.15.49:0/3818342185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 msgr2=0x564c9987bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1025e00640 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c9987bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1025e00640 1 -- 172.21.15.49:0/3818342185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 msgr2=0x564c9987bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1025e00640 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1025e00640 1 -- 172.21.15.49:0/3818342185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1028110900 con 0x564c99874fc0 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1025400640 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:23:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1025e00640 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99874fc0 0x564c9987b570 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1018013710 tx=0x7f1018013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7addf44e982ed21d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.368+0000 7f1007e00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.369+0000 7f1007e00640 1 -- 172.21.15.49:0/3818342185 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018011e90 con 0x564c99874fc0 2024-02-26T06:23:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.369+0000 7f1007e00640 1 -- 172.21.15.49:0/3818342185 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1018004020 con 0x564c99874fc0 2024-02-26T06:23:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.369+0000 7f102ff97740 1 -- 172.21.15.49:0/3818342185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c997adca0 con 0x564c99874fc0 2024-02-26T06:23:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.369+0000 7f1007e00640 1 -- 172.21.15.49:0/3818342185 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10180041e0 con 0x564c99874fc0 2024-02-26T06:23:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.369+0000 7f102ff97740 10 client.25451 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.369+0000 7f102ff97740 1 -- 172.21.15.49:0/3818342185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564c997d9c20 con 0x564c99874fc0 2024-02-26T06:23:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.370+0000 7f1007e00640 1 -- 172.21.15.49:0/3818342185 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1018028020 con 0x564c99874fc0 2024-02-26T06:23:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.371+0000 7f1007e00640 1 -- 172.21.15.49:0/3818342185 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1018024450 con 0x564c99874fc0 2024-02-26T06:23:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.371+0000 7f102ff97740 10 client.25451 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.371+0000 7f102ff97740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.371+0000 7f102ff97740 20 client.25451 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.371+0000 7f102ff97740 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564c9988e590 0x564c99890980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.371+0000 7f102ff97740 4 client.25451 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.371+0000 7f102ff97740 1 -- 172.21.15.49:0/3818342185 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c997d1cb0 con 0x564c9988e590 2024-02-26T06:23:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.372+0000 7f1025400640 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564c9988e590 0x564c99890980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.372+0000 7f1025400640 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564c9988e590 0x564c99890980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26f3dfa80e1de21 server_cookie=af4abfa5ebb018fb in_seq=0 out_seq=0 2024-02-26T06:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.372+0000 7f1007e00640 10 client.25451 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.373+0000 7f1007e00640 1 -- 172.21.15.49:0/3818342185 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c997d1cb0 con 0x564c9988e590 2024-02-26T06:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.373+0000 7f1007e00640 10 client.25451 handle_command_reply: tid=0 2024-02-26T06:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.373+0000 7f102ff97740 1 client.25451 shutdown 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 -- 172.21.15.49:0/3818342185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99874fc0 msgr2=0x564c9987b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99874fc0 0x564c9987b570 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1018013710 tx=0x7f1018013bd0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 -- 172.21.15.49:0/3818342185 shutdown_connections 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564c9988e590 0x564c99890980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c9987bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f1007e00640 0 client.25451 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 --2- 172.21.15.49:0/3818342185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c99874fc0 0x564c9987b570 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 -- 172.21.15.49:0/3818342185 >> 172.21.15.49:0/3818342185 conn(0x564c9983c480 msgr2=0x564c99871a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 -- 172.21.15.49:0/3818342185 shutdown_connections 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.374+0000 7f102ff97740 1 -- 172.21.15.49:0/3818342185 wait complete. 2024-02-26T06:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.375+0000 7f102ff97740 20 client.25451 trim_cache size 0 max 16384 2024-02-26T06:23:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.407+0000 7f102ff97740 1 Processor -- start 2024-02-26T06:23:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.407+0000 7f102ff97740 1 -- start start 2024-02-26T06:23:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f102ff97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f102ff97740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9987e540 0x564c9987be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f102ff97740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9986fef0 0x564c99879e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f102ff97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c99523e60 con 0x564c99875d30 2024-02-26T06:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f102ff97740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c9988afe0 con 0x564c9987e540 2024-02-26T06:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f102ff97740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c9988e390 con 0x564c9986fef0 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f1025400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9987e540 0x564c9987be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f1025e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f1025400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9987e540 0x564c9987be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50568/0 (socket says 172.21.15.49:50568) 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f1025400640 1 -- 172.21.15.49:0/1987261785 learned_addr learned my addr 172.21.15.49:0/1987261785 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.408+0000 7f1026800640 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9986fef0 0x564c99879e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1025400640 1 -- 172.21.15.49:0/1987261785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9986fef0 msgr2=0x564c99879e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1025400640 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9986fef0 0x564c99879e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1025400640 1 -- 172.21.15.49:0/1987261785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 msgr2=0x564c9987dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1025400640 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1025400640 1 -- 172.21.15.49:0/1987261785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c99776500 con 0x564c9987e540 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1025e00640 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1025400640 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9987e540 0x564c9987be40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1014097640 tx=0x7f10140d1c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3bfafafcc534291 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1012a00640 1 -- 172.21.15.49:0/1987261785 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101409a7b0 con 0x564c9987e540 2024-02-26T06:23:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.409+0000 7f1012a00640 1 -- 172.21.15.49:0/1987261785 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f101409a5d0 con 0x564c9987e540 2024-02-26T06:23:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f1012a00640 1 -- 172.21.15.49:0/1987261785 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014098f70 con 0x564c9987e540 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f102ff97740 1 -- 172.21.15.49:0/1987261785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9987e540 msgr2=0x564c9987be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f102ff97740 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9987e540 0x564c9987be40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1014097640 tx=0x7f10140d1c60 comp rx=0 tx=0).stop 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f102ff97740 1 -- 172.21.15.49:0/1987261785 shutdown_connections 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f102ff97740 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9987e540 0x564c9987be40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f102ff97740 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9986fef0 0x564c99879e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f102ff97740 1 --2- 172.21.15.49:0/1987261785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c9987dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f102ff97740 1 -- 172.21.15.49:0/1987261785 >> 172.21.15.49:0/1987261785 conn(0x564c9983c480 msgr2=0x564c9986f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.410+0000 7f102ff97740 1 -- 172.21.15.49:0/1987261785 shutdown_connections 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.411+0000 7f102ff97740 1 -- 172.21.15.49:0/1987261785 wait complete. 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.411+0000 7f102ff97740 1 -- start start 2024-02-26T06:23:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.411+0000 7f102ff97740 1 Processor -- start 2024-02-26T06:23:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.416+0000 7f102ff97740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9986fef0 0x564c99884360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f102ff97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c998848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f102ff97740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c998e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f102ff97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c99884fa0 con 0x564c99875d30 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f102ff97740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c998e4ff0 con 0x564c9986fef0 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f102ff97740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c998e5130 con 0x564c9987e540 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f102ff97740 10 client.0 _ll_register_callbacks cb 0x7f1018009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f1026800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c998e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f1026800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c998e4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:54696/0 (socket says 172.21.15.49:54696) 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f1026800640 1 -- 172.21.15.49:0/2306367130 learned_addr learned my addr 172.21.15.49:0/2306367130 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f1025e00640 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9986fef0 0x564c99884360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f1025400640 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c998848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.417+0000 7f1025e00640 1 -- 172.21.15.49:0/2306367130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 msgr2=0x564c998e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.418+0000 7f1025e00640 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c998e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.418+0000 7f1025e00640 1 -- 172.21.15.49:0/2306367130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 msgr2=0x564c998848a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.418+0000 7f1025e00640 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c998848a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.418+0000 7f1025e00640 1 -- 172.21.15.49:0/2306367130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c997d9c20 con 0x564c9986fef0 2024-02-26T06:23:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.418+0000 7f1026800640 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c998e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.418+0000 7f1025e00640 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9986fef0 0x564c99884360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f10180247b0 tx=0x7f1018005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f343fcf01c19b90a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.418+0000 7f1007e00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.419+0000 7f1007e00640 1 -- 172.21.15.49:0/2306367130 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018011c90 con 0x564c9986fef0 2024-02-26T06:23:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.419+0000 7f1007e00640 1 -- 172.21.15.49:0/2306367130 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1018000e40 con 0x564c9986fef0 2024-02-26T06:23:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.419+0000 7f102ff97740 1 -- 172.21.15.49:0/2306367130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c99800d00 con 0x564c9986fef0 2024-02-26T06:23:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.419+0000 7f1007e00640 1 -- 172.21.15.49:0/2306367130 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1018011df0 con 0x564c9986fef0 2024-02-26T06:23:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.419+0000 7f102ff97740 10 client.25457 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.419+0000 7f102ff97740 1 -- 172.21.15.49:0/2306367130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f10281a6550 con 0x564c9986fef0 2024-02-26T06:23:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.420+0000 7f1007e00640 1 -- 172.21.15.49:0/2306367130 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1018039080 con 0x564c9986fef0 2024-02-26T06:23:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.420+0000 7f1007e00640 1 -- 172.21.15.49:0/2306367130 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f101802b070 con 0x564c9986fef0 2024-02-26T06:23:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.420+0000 7f102ff97740 10 client.25457 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.420+0000 7f102ff97740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.420+0000 7f102ff97740 20 client.25457 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.420+0000 7f102ff97740 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564c998e6870 0x564c998e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.420+0000 7f102ff97740 4 client.25457 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:23:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.420+0000 7f102ff97740 1 -- 172.21.15.49:0/2306367130 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564c994db600 con 0x564c998e6870 2024-02-26T06:23:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.421+0000 7f1025400640 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564c998e6870 0x564c998e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.421+0000 7f1025400640 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564c998e6870 0x564c998e8c90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2846378212b43e30 server_cookie=961c48695d3d5084 in_seq=0 out_seq=0 2024-02-26T06:23:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.421+0000 7f1007e00640 10 client.25457 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:39.631 INFO:teuthology.orchestra.run.smithi049.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.87 s, remaining 8.09 s) 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f1007e00640 1 -- 172.21.15.49:0/2306367130 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564c994db600 con 0x564c998e6870 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f1007e00640 10 client.25457 handle_command_reply: tid=0 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 client.25457 shutdown 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 -- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9986fef0 msgr2=0x564c99884360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9986fef0 0x564c99884360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f10180247b0 tx=0x7f1018005cc0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 -- 172.21.15.49:0/2306367130 shutdown_connections 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564c998e6870 0x564c998e8c90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9987e540 0x564c998e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f1007e00640 0 client.25457 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c99875d30 0x564c998848a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 --2- 172.21.15.49:0/2306367130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c9986fef0 0x564c99884360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 -- 172.21.15.49:0/2306367130 >> 172.21.15.49:0/2306367130 conn(0x564c9983c480 msgr2=0x564c99879e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 -- 172.21.15.49:0/2306367130 shutdown_connections 2024-02-26T06:23:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.640+0000 7f102ff97740 1 -- 172.21.15.49:0/2306367130 wait complete. 2024-02-26T06:23:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f102ff97740 20 client.25457 trim_cache size 0 max 16384 2024-02-26T06:23:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:23:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:23:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:23:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:23:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 -- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f100c0a2330 msgr2=0x7f100c0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f100c0a2330 0x7f100c0a47f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f102002c430 tx=0x7f102002bef0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 -- 172.21.15.49:0/2004910361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10281006e0 msgr2=0x7f102819faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10281006e0 0x7f102819faa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9c0 tx=0x7f101400ee80 comp rx=0 tx=0).stop 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 -- 172.21.15.49:0/2004910361 shutdown_connections 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f100c0a2330 0x7f100c0a47f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10281006e0 0x7f102819faa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f102810d8e0 0x7f10281996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.641+0000 7f1027200640 1 --2- 172.21.15.49:0/2004910361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280ffd30 0x7f102819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.642+0000 7f1027200640 1 -- 172.21.15.49:0/2004910361 >> 172.21.15.49:0/2004910361 conn(0x7f10280fb990 msgr2=0x7f10280fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.642+0000 7f1027200640 1 -- 172.21.15.49:0/2004910361 shutdown_connections 2024-02-26T06:23:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:39.642+0000 7f1027200640 1 -- 172.21.15.49:0/2004910361 wait complete. 2024-02-26T06:23:39.724 INFO:teuthology.orchestra.run.smithi049.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.96 s, remaining 7.00 s) 2024-02-26T06:23:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:800000 of 3200000 tuples (25%) done (elapsed 2.42 s, remaining 7.25 s) 2024-02-26T06:23:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.51 s, remaining 6.43 s) 2024-02-26T06:23:40.630 INFO:teuthology.orchestra.run.smithi049.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.87 s, remaining 6.31 s) 2024-02-26T06:23:41.158 INFO:teuthology.orchestra.run.smithi049.stderr:1100000 of 3200000 tuples (34%) done (elapsed 3.39 s, remaining 6.48 s) 2024-02-26T06:23:41.276 INFO:teuthology.orchestra.run.smithi049.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.51 s, remaining 5.85 s) 2024-02-26T06:23:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:41 smithi080 ceph-mon[36206]: pgmap v11: 97 pgs: 97 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.6 MiB/s wr, 64 op/s 2024-02-26T06:23:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:41 smithi115 ceph-mon[36426]: pgmap v11: 97 pgs: 97 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.6 MiB/s wr, 64 op/s 2024-02-26T06:23:41.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:41 smithi049 ceph-mon[31696]: pgmap v11: 97 pgs: 97 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.6 MiB/s wr, 64 op/s 2024-02-26T06:23:41.682 INFO:teuthology.orchestra.run.smithi049.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.92 s, remaining 5.73 s) 2024-02-26T06:23:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:1400000 of 3200000 tuples (43%) done (elapsed 4.38 s, remaining 5.63 s) 2024-02-26T06:23:42.255 INFO:teuthology.orchestra.run.smithi049.stderr:1500000 of 3200000 tuples (46%) done (elapsed 4.49 s, remaining 5.09 s) 2024-02-26T06:23:42.703 INFO:teuthology.orchestra.run.smithi049.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.94 s, remaining 4.94 s) 2024-02-26T06:23:42.914 INFO:teuthology.orchestra.run.smithi049.stderr:1700000 of 3200000 tuples (53%) done (elapsed 5.15 s, remaining 4.54 s) 2024-02-26T06:23:43.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:43 smithi080 ceph-mon[36206]: pgmap v12: 97 pgs: 97 active+clean; 93 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 13 MiB/s wr, 195 op/s 2024-02-26T06:23:43.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:43 smithi115 ceph-mon[36426]: pgmap v12: 97 pgs: 97 active+clean; 93 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 13 MiB/s wr, 195 op/s 2024-02-26T06:23:43.348 INFO:teuthology.orchestra.run.smithi049.stderr:1800000 of 3200000 tuples (56%) done (elapsed 5.58 s, remaining 4.34 s) 2024-02-26T06:23:43.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:43 smithi049 ceph-mon[31696]: pgmap v12: 97 pgs: 97 active+clean; 93 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 13 MiB/s wr, 195 op/s 2024-02-26T06:23:43.744 INFO:teuthology.orchestra.run.smithi049.stderr:1900000 of 3200000 tuples (59%) done (elapsed 5.98 s, remaining 4.09 s) 2024-02-26T06:23:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2000000 of 3200000 tuples (62%) done (elapsed 6.15 s, remaining 3.69 s) 2024-02-26T06:23:43.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:23:43.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:23:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 -- 172.21.15.49:0/4035087418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 msgr2=0x7f91200689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 --2- 172.21.15.49:0/4035087418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 0x7f91200689d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9110009900 tx=0x7f911002bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 -- 172.21.15.49:0/4035087418 shutdown_connections 2024-02-26T06:23:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 --2- 172.21.15.49:0/4035087418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91201064c0 0x7f912010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 --2- 172.21.15.49:0/4035087418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9120068fa0 0x7f9120105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 --2- 172.21.15.49:0/4035087418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 0x7f91200689d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 -- 172.21.15.49:0/4035087418 >> 172.21.15.49:0/4035087418 conn(0x7f9120077180 msgr2=0x7f9120077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:44.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 -- 172.21.15.49:0/4035087418 shutdown_connections 2024-02-26T06:23:44.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.229+0000 7f9127000640 1 -- 172.21.15.49:0/4035087418 wait complete. 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.230+0000 7f9127000640 1 Processor -- start 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9127000640 1 -- start start 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9127000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 0x7f912019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9127000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91201064c0 0x7f912019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9127000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91201975b0 0x7f9120197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9127000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9120197fa0 con 0x7f91201975b0 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9127000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9120198110 con 0x7f91201064c0 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9127000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201982d0 con 0x7f91200685f0 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9117e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91201064c0 0x7f912019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9117e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91201064c0 0x7f912019d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49732/0 (socket says 172.21.15.49:49732) 2024-02-26T06:23:44.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9117e00640 1 -- 172.21.15.49:0/2687266924 learned_addr learned my addr 172.21.15.49:0/2687266924 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:44.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f911fe00640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 0x7f912019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.231+0000 7f9125000640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91201975b0 0x7f9120197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f9117e00640 1 -- 172.21.15.49:0/2687266924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 msgr2=0x7f912019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f9117e00640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 0x7f912019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f9117e00640 1 -- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91201975b0 msgr2=0x7f9120197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f9117e00640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91201975b0 0x7f9120197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f9117e00640 1 -- 172.21.15.49:0/2687266924 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110009560 con 0x7f91201064c0 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f911fe00640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 0x7f912019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f9117e00640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91201064c0 0x7f912019d940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f910800d950 tx=0x7f910800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f911d600640 1 -- 172.21.15.49:0/2687266924 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108009c30 con 0x7f91201064c0 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f911d600640 1 -- 172.21.15.49:0/2687266924 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f910800b600 con 0x7f91201064c0 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.232+0000 7f9127000640 1 -- 172.21.15.49:0/2687266924 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91201a42c0 con 0x7f91201064c0 2024-02-26T06:23:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.233+0000 7f911d600640 1 -- 172.21.15.49:0/2687266924 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108004a00 con 0x7f91201064c0 2024-02-26T06:23:44.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.233+0000 7f9127000640 1 -- 172.21.15.49:0/2687266924 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91201a4820 con 0x7f91201064c0 2024-02-26T06:23:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.237+0000 7f9129f64740 1 Processor -- start 2024-02-26T06:23:44.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.237+0000 7f9129f64740 1 -- start start 2024-02-26T06:23:44.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f911d600640 1 -- 172.21.15.49:0/2687266924 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f9108004020 con 0x7f91201064c0 2024-02-26T06:23:44.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9129f64740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9129f64740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7290 0x5652928c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9129f64740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 0x5652928d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9129f64740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x565292836120 con 0x5652928d0600 2024-02-26T06:23:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9129f64740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x565292833650 con 0x5652928c7f70 2024-02-26T06:23:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9129f64740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529282ee30 con 0x5652928c7290 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9117e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 0x5652928d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9125000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f911fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7290 0x5652928c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9125000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49738/0 (socket says 172.21.15.49:49738) 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.238+0000 7f9125000640 1 -- 172.21.15.49:0/1385438438 learned_addr learned my addr 172.21.15.49:0/1385438438 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.239+0000 7f9125000640 1 -- 172.21.15.49:0/1385438438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7290 msgr2=0x5652928c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.239+0000 7f9125000640 1 --2- 172.21.15.49:0/1385438438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7290 0x5652928c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.239+0000 7f9125000640 1 -- 172.21.15.49:0/1385438438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 msgr2=0x5652928d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.239+0000 7f9125000640 1 --2- 172.21.15.49:0/1385438438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 0x5652928d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.239+0000 7f9125000640 1 -- 172.21.15.49:0/1385438438 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652927e8030 con 0x5652928c7f70 2024-02-26T06:23:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.239+0000 7f9125000640 1 --2- 172.21.15.49:0/1385438438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928c6cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9118009f50 tx=0x7f91180118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2d270ffc0ab2327 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.239+0000 7f9115600640 1 -- 172.21.15.49:0/1385438438 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118004d20 con 0x5652928c7f70 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.240+0000 7f9115600640 1 -- 172.21.15.49:0/1385438438 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9118004e80 con 0x5652928c7f70 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.240+0000 7f9115600640 1 -- 172.21.15.49:0/1385438438 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118005280 con 0x5652928c7f70 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.240+0000 7f911d600640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91040a2340 0x7f91040a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.240+0000 7f911d600640 1 -- 172.21.15.49:0/2687266924 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f91080cde60 con 0x7f91201064c0 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.240+0000 7f911fe00640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91040a2340 0x7f91040a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 -- 172.21.15.49:0/1385438438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 msgr2=0x5652928c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 --2- 172.21.15.49:0/1385438438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928c6cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9118009f50 tx=0x7f91180118d0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 -- 172.21.15.49:0/1385438438 shutdown_connections 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 --2- 172.21.15.49:0/1385438438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 0x5652928d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 --2- 172.21.15.49:0/1385438438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7290 0x5652928c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 --2- 172.21.15.49:0/1385438438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928c6cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 -- 172.21.15.49:0/1385438438 >> 172.21.15.49:0/1385438438 conn(0x56529288e570 msgr2=0x5652928c1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 -- 172.21.15.49:0/1385438438 shutdown_connections 2024-02-26T06:23:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f9129f64740 1 -- 172.21.15.49:0/1385438438 wait complete. 2024-02-26T06:23:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.241+0000 7f911fe00640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91040a2340 0x7f91040a4800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9110005ab0 tx=0x7f9110005a20 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 1 -- start start 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 1 Processor -- start 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928da610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928dab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928d0600 0x5652928d4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652927d3e90 con 0x5652928c7290 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652927d3b30 con 0x5652928c7f70 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529259d1c0 con 0x5652928d0600 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.242+0000 7f9129f64740 10 client.0 _ll_register_callbacks cb 0x5652927e7930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f911fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928da610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f911fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928da610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40584/0 (socket says 172.21.15.49:40584) 2024-02-26T06:23:44.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f911fe00640 1 -- 172.21.15.49:0/4059213358 learned_addr learned my addr 172.21.15.49:0/4059213358 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f9125000640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928d0600 0x5652928d4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f9117e00640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928dab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f911fe00640 1 -- 172.21.15.49:0/4059213358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928d0600 msgr2=0x5652928d4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f911fe00640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928d0600 0x5652928d4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f911fe00640 1 -- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 msgr2=0x5652928dab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f911fe00640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928dab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f911fe00640 1 -- 172.21.15.49:0/4059213358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9118009c00 con 0x5652928c7290 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f9125000640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928d0600 0x5652928d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.243+0000 7f9117e00640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928dab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.244+0000 7f911fe00640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928da610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f9110009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11102878b361af7c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.244+0000 7f910e000640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.244+0000 7f910e000640 1 -- 172.21.15.49:0/4059213358 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911002dea0 con 0x5652928c7290 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.244+0000 7f9129f64740 1 -- 172.21.15.49:0/4059213358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56529256b720 con 0x5652928c7290 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.244+0000 7f910e000640 1 -- 172.21.15.49:0/4059213358 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9110005400 con 0x5652928c7290 2024-02-26T06:23:44.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.244+0000 7f910e000640 1 -- 172.21.15.49:0/4059213358 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91100055c0 con 0x5652928c7290 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.244+0000 7f9129f64740 10 client.15354 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.244+0000 7f9129f64740 1 -- 172.21.15.49:0/4059213358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56529282d4b0 con 0x5652928c7290 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.245+0000 7f910e000640 1 -- 172.21.15.49:0/4059213358 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9110061080 con 0x5652928c7290 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.245+0000 7f910e000640 1 -- 172.21.15.49:0/4059213358 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9110048040 con 0x5652928c7290 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.245+0000 7f9129f64740 10 client.15354 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.245+0000 7f9129f64740 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.245+0000 7f9129f64740 20 client.15354 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.245+0000 7f9129f64740 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652928e4be0 0x5652928e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.245+0000 7f9129f64740 4 client.15354 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.245+0000 7f9129f64740 1 -- 172.21.15.49:0/4059213358 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652925ba890 con 0x5652928e4be0 2024-02-26T06:23:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.246+0000 7f9117e00640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652928e4be0 0x5652928e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.246+0000 7f9117e00640 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652928e4be0 0x5652928e6fd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e8a2d65892db90f server_cookie=d5fa3dd4e7fd4a8c in_seq=0 out_seq=0 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.246+0000 7f910e000640 10 client.15354 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f910e000640 1 -- 172.21.15.49:0/4059213358 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652925ba890 con 0x5652928e4be0 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f910e000640 10 client.15354 handle_command_reply: tid=0 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 client.15354 shutdown 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 -- 172.21.15.49:0/4059213358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 msgr2=0x5652928da610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928da610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f9110009260 comp rx=0 tx=0).stop 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 -- 172.21.15.49:0/4059213358 shutdown_connections 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5652928e4be0 0x5652928e6fd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7f70 0x5652928dab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928d0600 0x5652928d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 --2- 172.21.15.49:0/4059213358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928da610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f910e000640 0 client.15354 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 -- 172.21.15.49:0/4059213358 >> 172.21.15.49:0/4059213358 conn(0x56529288e570 msgr2=0x5652928c3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 -- 172.21.15.49:0/4059213358 shutdown_connections 2024-02-26T06:23:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 1 -- 172.21.15.49:0/4059213358 wait complete. 2024-02-26T06:23:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.247+0000 7f9129f64740 20 client.15354 trim_cache size 0 max 16384 2024-02-26T06:23:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2100000 of 3200000 tuples (65%) done (elapsed 6.48 s, remaining 3.40 s) 2024-02-26T06:23:44.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.264+0000 7f9129f64740 1 Processor -- start 2024-02-26T06:23:44.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.264+0000 7f9129f64740 1 -- start start 2024-02-26T06:23:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.264+0000 7f9129f64740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7290 0x5652928d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9129f64740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9129f64740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 0x5652928d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9129f64740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652928c2f10 con 0x5652928d0600 2024-02-26T06:23:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9129f64740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652928c6510 con 0x5652928c7290 2024-02-26T06:23:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9129f64740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652928c2ce0 con 0x5652928c7f70 2024-02-26T06:23:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9117e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9117e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57280/0 (socket says 172.21.15.49:57280) 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9117e00640 1 -- 172.21.15.49:0/349414683 learned_addr learned my addr 172.21.15.49:0/349414683 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9125000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 0x5652928d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f911fe00640 1 --2- 172.21.15.49:0/349414683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7290 0x5652928d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9117e00640 1 -- 172.21.15.49:0/349414683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7290 msgr2=0x5652928d49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9117e00640 1 --2- 172.21.15.49:0/349414683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7290 0x5652928d49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9117e00640 1 -- 172.21.15.49:0/349414683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 msgr2=0x5652928d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9117e00640 1 --2- 172.21.15.49:0/349414683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 0x5652928d3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.265+0000 7f9117e00640 1 -- 172.21.15.49:0/349414683 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652927c9920 con 0x5652928c7f70 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.266+0000 7f9117e00640 1 --2- 172.21.15.49:0/349414683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d3540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9108097060 tx=0x7f91080ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1533f537bdc91 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.266+0000 7f9114c00640 1 -- 172.21.15.49:0/349414683 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910809a460 con 0x5652928c7f70 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.266+0000 7f9114c00640 1 -- 172.21.15.49:0/349414683 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f910809a1d0 con 0x5652928c7f70 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.266+0000 7f9114c00640 1 -- 172.21.15.49:0/349414683 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080a2e30 con 0x5652928c7f70 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 -- 172.21.15.49:0/349414683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 msgr2=0x5652928d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 --2- 172.21.15.49:0/349414683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d3540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9108097060 tx=0x7f91080ba040 comp rx=0 tx=0).stop 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 -- 172.21.15.49:0/349414683 shutdown_connections 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 --2- 172.21.15.49:0/349414683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928d0600 0x5652928d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 --2- 172.21.15.49:0/349414683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d3540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 --2- 172.21.15.49:0/349414683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928c7290 0x5652928d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 -- 172.21.15.49:0/349414683 >> 172.21.15.49:0/349414683 conn(0x56529288e570 msgr2=0x5652928d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 -- 172.21.15.49:0/349414683 shutdown_connections 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 -- 172.21.15.49:0/349414683 wait complete. 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.267+0000 7f9129f64740 1 -- start start 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.268+0000 7f9129f64740 1 Processor -- start 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.268+0000 7f9129f64740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928d9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.268+0000 7f9129f64740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.268+0000 7f9129f64740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928d0600 0x56529293b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.268+0000 7f9129f64740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652928da010 con 0x5652928c7290 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.268+0000 7f9129f64740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652928da150 con 0x5652928d0600 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.268+0000 7f9129f64740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529293b640 con 0x5652928c7f70 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.268+0000 7f9129f64740 10 client.0 _ll_register_callbacks cb 0x7f9110005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f911fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928d9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57296/0 (socket says 172.21.15.49:57296) 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 -- 172.21.15.49:0/799476117 learned_addr learned my addr 172.21.15.49:0/799476117 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9125000640 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928d0600 0x56529293b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 -- 172.21.15.49:0/799476117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928d0600 msgr2=0x56529293b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928d0600 0x56529293b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 -- 172.21.15.49:0/799476117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 msgr2=0x5652928d9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928d9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 -- 172.21.15.49:0/799476117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91201a4820 con 0x5652928c7f70 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f9117e00640 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d9880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9108005fd0 tx=0x7f91080d1400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f8c63767586d34 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.269+0000 7f910e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f910e000640 1 -- 172.21.15.49:0/799476117 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108099ff0 con 0x5652928c7f70 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f910e000640 1 -- 172.21.15.49:0/799476117 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f910809a690 con 0x5652928c7f70 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 1 -- 172.21.15.49:0/799476117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56529252e7e0 con 0x5652928c7f70 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f910e000640 1 -- 172.21.15.49:0/799476117 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f910809a460 con 0x5652928c7f70 2024-02-26T06:23:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 10 client.25489 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 1 -- 172.21.15.49:0/799476117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5652927ef400 con 0x5652928c7f70 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f910e000640 1 -- 172.21.15.49:0/799476117 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56529252e7e0 con 0x5652928c7f70 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f910e000640 1 -- 172.21.15.49:0/799476117 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91080042d0 con 0x5652928c7f70 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 10 client.25489 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 10 client.25489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 20 client.25489 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56529293c980 0x56529293ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 4 client.25489 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.270+0000 7f9129f64740 1 -- 172.21.15.49:0/799476117 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652924da700 con 0x56529293c980 2024-02-26T06:23:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.271+0000 7f911fe00640 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56529293c980 0x56529293ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.271+0000 7f911fe00640 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56529293c980 0x56529293ed70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b1b14d1917d934a server_cookie=5b31a0e341ae5067 in_seq=0 out_seq=0 2024-02-26T06:23:44.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.271+0000 7f910e000640 10 client.25489 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:44.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.271+0000 7f910e000640 1 -- 172.21.15.49:0/799476117 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5652924da700 con 0x56529293c980 2024-02-26T06:23:44.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.271+0000 7f910e000640 10 client.25489 handle_command_reply: tid=0 2024-02-26T06:23:44.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.271+0000 7f9129f64740 1 client.25489 shutdown 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 -- 172.21.15.49:0/799476117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 msgr2=0x5652928d9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d9880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9108005fd0 tx=0x7f91080d1400 comp rx=0 tx=0).stop 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 -- 172.21.15.49:0/799476117 shutdown_connections 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56529293c980 0x56529293ed70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652928d0600 0x56529293b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652928c7f70 0x5652928d9880 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 --2- 172.21.15.49:0/799476117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5652928c7290 0x5652928d9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 -- 172.21.15.49:0/799476117 >> 172.21.15.49:0/799476117 conn(0x56529288e570 msgr2=0x5652928c38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f910e000640 0 client.25489 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 -- 172.21.15.49:0/799476117 shutdown_connections 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 1 -- 172.21.15.49:0/799476117 wait complete. 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.272+0000 7f9129f64740 20 client.25489 trim_cache size 0 max 16384 2024-02-26T06:23:44.275 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:23:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:23:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:23:44.276 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:23:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 -- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91040a2340 msgr2=0x7f91040a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91040a2340 0x7f91040a4800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9110005ab0 tx=0x7f9110005a20 comp rx=0 tx=0).stop 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 -- 172.21.15.49:0/2687266924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91201064c0 msgr2=0x7f912019d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91201064c0 0x7f912019d940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f910800d950 tx=0x7f910800de10 comp rx=0 tx=0).stop 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 -- 172.21.15.49:0/2687266924 shutdown_connections 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91040a2340 0x7f91040a4800 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91201064c0 0x7f912019d940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91201975b0 0x7f9120197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 --2- 172.21.15.49:0/2687266924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91200685f0 0x7f912019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 -- 172.21.15.49:0/2687266924 >> 172.21.15.49:0/2687266924 conn(0x7f9120077180 msgr2=0x7f91200fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 -- 172.21.15.49:0/2687266924 shutdown_connections 2024-02-26T06:23:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.273+0000 7f9127000640 1 -- 172.21.15.49:0/2687266924 wait complete. 2024-02-26T06:23:44.289 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:23:44.290 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 -- 172.21.15.49:0/3929409484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c101b90 msgr2=0x7fee4c105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 --2- 172.21.15.49:0/3929409484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c101b90 0x7fee4c105bd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee38009930 tx=0x7fee3802bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 -- 172.21.15.49:0/3929409484 shutdown_connections 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 --2- 172.21.15.49:0/3929409484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c101b90 0x7fee4c105bd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 --2- 172.21.15.49:0/3929409484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee4c1062b0 0x7fee4c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 --2- 172.21.15.49:0/3929409484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee4c1011e0 0x7fee4c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 -- 172.21.15.49:0/3929409484 >> 172.21.15.49:0/3929409484 conn(0x7fee4c0fca60 msgr2=0x7fee4c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 -- 172.21.15.49:0/3929409484 shutdown_connections 2024-02-26T06:23:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.531+0000 7fee4be00640 1 -- 172.21.15.49:0/3929409484 wait complete. 2024-02-26T06:23:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.532+0000 7fee4be00640 1 Processor -- start 2024-02-26T06:23:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.532+0000 7fee4be00640 1 -- start start 2024-02-26T06:23:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee4c1011e0 0x7fee4c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c1062b0 0x7fee4c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee4c19be90 0x7fee4c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c19c880 con 0x7fee4c1011e0 2024-02-26T06:23:44.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c19c9f0 con 0x7fee4c1062b0 2024-02-26T06:23:44.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4be00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c19cb60 con 0x7fee4c19be90 2024-02-26T06:23:44.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c1062b0 0x7fee4c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee4c1011e0 0x7fee4c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee4c19be90 0x7fee4c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c1062b0 0x7fee4c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49774/0 (socket says 172.21.15.49:49774) 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.533+0000 7fee4a000640 1 -- 172.21.15.49:0/2467881087 learned_addr learned my addr 172.21.15.49:0/2467881087 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee4a000640 1 -- 172.21.15.49:0/2467881087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee4c19be90 msgr2=0x7fee4c19c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee4a000640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee4c19be90 0x7fee4c19c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee4a000640 1 -- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee4c1011e0 msgr2=0x7fee4c197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee4a000640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee4c1011e0 0x7fee4c197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee4a000640 1 -- 172.21.15.49:0/2467881087 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee38009560 con 0x7fee4c1062b0 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee4b400640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee4c19be90 0x7fee4c19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:44.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee4a000640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c1062b0 0x7fee4c197b60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee38005ab0 tx=0x7fee3802cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:44.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee43400640 1 -- 172.21.15.49:0/2467881087 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38004350 con 0x7fee4c1062b0 2024-02-26T06:23:44.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.534+0000 7fee43400640 1 -- 172.21.15.49:0/2467881087 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee380044b0 con 0x7fee4c1062b0 2024-02-26T06:23:44.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.535+0000 7fee43400640 1 -- 172.21.15.49:0/2467881087 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38003cc0 con 0x7fee4c1062b0 2024-02-26T06:23:44.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.535+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c1a4140 con 0x7fee4c1062b0 2024-02-26T06:23:44.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.535+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c1a4510 con 0x7fee4c1062b0 2024-02-26T06:23:44.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.536+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c1026b0 con 0x7fee4c1062b0 2024-02-26T06:23:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.538+0000 7fee43400640 1 -- 172.21.15.49:0/2467881087 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7fee38004df0 con 0x7fee4c1062b0 2024-02-26T06:23:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.539+0000 7fee43400640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee300a2340 0x7fee300a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.539+0000 7fee43400640 1 -- 172.21.15.49:0/2467881087 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fee380f0760 con 0x7fee4c1062b0 2024-02-26T06:23:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.540+0000 7fee4aa00640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee300a2340 0x7fee300a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:44.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.540+0000 7fee4aa00640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee300a2340 0x7fee300a4800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009bd0 tx=0x7fee3c005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:44.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.543+0000 7fee43400640 1 -- 172.21.15.49:0/2467881087 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee380b9430 con 0x7fee4c1062b0 2024-02-26T06:23:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2200000 of 3200000 tuples (68%) done (elapsed 6.83 s, remaining 3.10 s) 2024-02-26T06:23:44.772 INFO:teuthology.orchestra.run.smithi049.stderr:2300000 of 3200000 tuples (71%) done (elapsed 7.01 s, remaining 2.74 s) 2024-02-26T06:23:44.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.801+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee4c198920 con 0x7fee4c1062b0 2024-02-26T06:23:44.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.802+0000 7fee43400640 1 -- 172.21.15.49:0/2467881087 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11331 (secure 0 0 0) 0x7fee380bcaf0 con 0x7fee4c1062b0 2024-02-26T06:23:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:23:44.802 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:23:03.712829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:23:44.803 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee300a2340 msgr2=0x7fee300a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee300a2340 0x7fee300a4800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009bd0 tx=0x7fee3c005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c1062b0 msgr2=0x7fee4c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c1062b0 0x7fee4c197b60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee38005ab0 tx=0x7fee3802cd80 comp rx=0 tx=0).stop 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 shutdown_connections 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee300a2340 0x7fee300a4800 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee4c1062b0 0x7fee4c197b60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee4c19be90 0x7fee4c19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 --2- 172.21.15.49:0/2467881087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee4c1011e0 0x7fee4c197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:44.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 >> 172.21.15.49:0/2467881087 conn(0x7fee4c0fca60 msgr2=0x7fee4c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:44.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 shutdown_connections 2024-02-26T06:23:44.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:44.805+0000 7fee4be00640 1 -- 172.21.15.49:0/2467881087 wait complete. 2024-02-26T06:23:44.824 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:23:44.824 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:23:45.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.050+0000 7f06f7e00640 1 -- 172.21.15.49:0/925485969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06f0101390 msgr2=0x7f06f0101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.050+0000 7f06f7e00640 1 --2- 172.21.15.49:0/925485969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06f0101390 0x7f06f0101770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009930 tx=0x7f06e002bc20 comp rx=0 tx=0).stop 2024-02-26T06:23:45.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.051+0000 7f06f7e00640 1 -- 172.21.15.49:0/925485969 shutdown_connections 2024-02-26T06:23:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.051+0000 7f06f7e00640 1 --2- 172.21.15.49:0/925485969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06f0101d40 0x7f06f0105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.051+0000 7f06f7e00640 1 --2- 172.21.15.49:0/925485969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f0106460 0x7f06f010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.051+0000 7f06f7e00640 1 --2- 172.21.15.49:0/925485969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06f0101390 0x7f06f0101770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.051+0000 7f06f7e00640 1 -- 172.21.15.49:0/925485969 >> 172.21.15.49:0/925485969 conn(0x7f06f00fcbb0 msgr2=0x7f06f00fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.051+0000 7f06f7e00640 1 -- 172.21.15.49:0/925485969 shutdown_connections 2024-02-26T06:23:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.051+0000 7f06f7e00640 1 -- 172.21.15.49:0/925485969 wait complete. 2024-02-26T06:23:45.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.052+0000 7f06f7e00640 1 Processor -- start 2024-02-26T06:23:45.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.052+0000 7f06f7e00640 1 -- start start 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.052+0000 7f06f7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f0101390 0x7f06f019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.052+0000 7f06f7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06f0106460 0x7f06f019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.052+0000 7f06f7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06f0199860 0x7f06f0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.052+0000 7f06f7e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f019a250 con 0x7f06f0106460 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.052+0000 7f06f7e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f019a3c0 con 0x7f06f0199860 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.052+0000 7f06f7e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f019a580 con 0x7f06f0101390 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06f0199860 0x7f06f0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06f0106460 0x7f06f019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06f0199860 0x7f06f0199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49784/0 (socket says 172.21.15.49:49784) 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5e00640 1 -- 172.21.15.49:0/2141969450 learned_addr learned my addr 172.21.15.49:0/2141969450 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5400640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f0101390 0x7f06f019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5e00640 1 -- 172.21.15.49:0/2141969450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f0101390 msgr2=0x7f06f019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f0101390 0x7f06f019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5e00640 1 -- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06f0106460 msgr2=0x7f06f019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06f0106460 0x7f06f019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.053+0000 7f06f5e00640 1 -- 172.21.15.49:0/2141969450 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e4009490 con 0x7f06f0199860 2024-02-26T06:23:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.054+0000 7f06f5e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06f0199860 0x7f06f0199d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06e800eb10 tx=0x7f06e800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.054+0000 7f06ee000640 1 -- 172.21.15.49:0/2141969450 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8004b20 con 0x7f06f0199860 2024-02-26T06:23:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.054+0000 7f06ee000640 1 -- 172.21.15.49:0/2141969450 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e8004c80 con 0x7f06f0199860 2024-02-26T06:23:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.054+0000 7f06ee000640 1 -- 172.21.15.49:0/2141969450 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8010650 con 0x7f06f0199860 2024-02-26T06:23:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.054+0000 7f06f7e00640 1 -- 172.21.15.49:0/2141969450 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e0009560 con 0x7f06f0199860 2024-02-26T06:23:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.054+0000 7f06f7e00640 1 -- 172.21.15.49:0/2141969450 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f01a6870 con 0x7f06f0199860 2024-02-26T06:23:45.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.057+0000 7f06ee000640 1 -- 172.21.15.49:0/2141969450 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f06e8005020 con 0x7f06f0199860 2024-02-26T06:23:45.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.058+0000 7f06ee000640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f06d80a2340 0x7f06d80a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.059+0000 7f06ee000640 1 -- 172.21.15.49:0/2141969450 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f06e8014070 con 0x7f06f0199860 2024-02-26T06:23:45.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.059+0000 7f06f5400640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f06d80a2340 0x7f06d80a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.059+0000 7f06fac7e740 1 Processor -- start 2024-02-26T06:23:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.059+0000 7f06fac7e740 1 -- start start 2024-02-26T06:23:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06f5400640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f06d80a2340 0x7f06d80a4800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005ab0 tx=0x7f06e0005fd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dbc10 0x56540c1dbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1e5110 0x56540c1e7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06fac7e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c14a1e0 con 0x56540c1dbc10 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06fac7e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c147710 con 0x56540c1dc530 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06fac7e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c142ef0 con 0x56540c1e5110 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49794/0 (socket says 172.21.15.49:49794) 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06f4a00640 1 -- 172.21.15.49:0/794280354 learned_addr learned my addr 172.21.15.49:0/794280354 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06f5400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dbc10 0x56540c1dbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.060+0000 7f06f5e00640 1 --2- 172.21.15.49:0/794280354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1e5110 0x56540c1e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06f4a00640 1 -- 172.21.15.49:0/794280354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1e5110 msgr2=0x56540c1e7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06f4a00640 1 --2- 172.21.15.49:0/794280354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1e5110 0x56540c1e7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06f4a00640 1 -- 172.21.15.49:0/794280354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dbc10 msgr2=0x56540c1dbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06f4a00640 1 --2- 172.21.15.49:0/794280354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dbc10 0x56540c1dbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06f4a00640 1 -- 172.21.15.49:0/794280354 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56540c104dc0 con 0x56540c1dc530 2024-02-26T06:23:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06f4a00640 1 --2- 172.21.15.49:0/794280354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e4bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009db0 tx=0x7f06e4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748d17b363eb742f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06de600640 1 -- 172.21.15.49:0/794280354 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4004230 con 0x56540c1dc530 2024-02-26T06:23:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06de600640 1 -- 172.21.15.49:0/794280354 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e4004390 con 0x56540c1dc530 2024-02-26T06:23:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.061+0000 7f06de600640 1 -- 172.21.15.49:0/794280354 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e40035f0 con 0x56540c1dc530 2024-02-26T06:23:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.062+0000 7f06fac7e740 1 -- 172.21.15.49:0/794280354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 msgr2=0x56540c1e4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.062+0000 7f06fac7e740 1 --2- 172.21.15.49:0/794280354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e4bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009db0 tx=0x7f06e4011810 comp rx=0 tx=0).stop 2024-02-26T06:23:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.062+0000 7f06fac7e740 1 -- 172.21.15.49:0/794280354 shutdown_connections 2024-02-26T06:23:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.062+0000 7f06fac7e740 1 --2- 172.21.15.49:0/794280354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e4bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.062+0000 7f06fac7e740 1 --2- 172.21.15.49:0/794280354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1e5110 0x56540c1e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.062+0000 7f06fac7e740 1 --2- 172.21.15.49:0/794280354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dbc10 0x56540c1dbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.062+0000 7f06fac7e740 1 -- 172.21.15.49:0/794280354 >> 172.21.15.49:0/794280354 conn(0x56540c1d6340 msgr2=0x56540c1d6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.063+0000 7f06fac7e740 1 -- 172.21.15.49:0/794280354 shutdown_connections 2024-02-26T06:23:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.063+0000 7f06fac7e740 1 -- 172.21.15.49:0/794280354 wait complete. 2024-02-26T06:23:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.063+0000 7f06fac7e740 1 -- start start 2024-02-26T06:23:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.064+0000 7f06fac7e740 1 Processor -- start 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.064+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.064+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.064+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c1e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.064+0000 7f06fac7e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c0e8120 con 0x56540c1e5110 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.064+0000 7f06fac7e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c0e7dc0 con 0x56540c1dc530 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.064+0000 7f06fac7e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540beb11c0 con 0x56540c1dbc10 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.064+0000 7f06fac7e740 10 client.0 _ll_register_callbacks cb 0x56540c137280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c1e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c1e2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40666/0 (socket says 172.21.15.49:40666) 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49800/0 (socket says 172.21.15.49:49800) 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f5e00640 1 -- 172.21.15.49:0/1139324013 learned_addr learned my addr 172.21.15.49:0/1139324013 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f5400640 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f4a00640 1 -- 172.21.15.49:0/1139324013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 msgr2=0x56540c1e2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f4a00640 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f4a00640 1 -- 172.21.15.49:0/1139324013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 msgr2=0x56540c1e2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f4a00640 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c1e2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f4a00640 1 -- 172.21.15.49:0/1139324013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e40099e0 con 0x56540c1dc530 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.065+0000 7f06f4a00640 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e26d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f06e4000c00 tx=0x7f06e4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4349d2d23a6cfc1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.066+0000 7f06d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.066+0000 7f06d2a00640 1 -- 172.21.15.49:0/1139324013 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e40055e0 con 0x56540c1dc530 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.066+0000 7f06fac7e740 1 -- 172.21.15.49:0/1139324013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56540be427e0 con 0x56540c1dc530 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.066+0000 7f06d2a00640 1 -- 172.21.15.49:0/1139324013 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f06e4024420 con 0x56540c1dc530 2024-02-26T06:23:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.066+0000 7f06d2a00640 1 -- 172.21.15.49:0/1139324013 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06e4004500 con 0x56540c1dc530 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.066+0000 7f06fac7e740 10 client.25493 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.066+0000 7f06fac7e740 1 -- 172.21.15.49:0/1139324013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56540c141570 con 0x56540c1dc530 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06d2a00640 1 -- 172.21.15.49:0/1139324013 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56540be427e0 con 0x56540c1dc530 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06d2a00640 1 -- 172.21.15.49:0/1139324013 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f06e4019720 con 0x56540c1dc530 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06fac7e740 10 client.25493 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06fac7e740 10 client.25493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06fac7e740 20 client.25493 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06fac7e740 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56540c1f50c0 0x56540c1f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06fac7e740 4 client.25493 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06fac7e740 1 -- 172.21.15.49:0/1139324013 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56540be7f720 con 0x56540c1f50c0 2024-02-26T06:23:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.067+0000 7f06f5400640 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56540c1f50c0 0x56540c1f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.068+0000 7f06f5400640 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56540c1f50c0 0x56540c1f74e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3f7313063a27d40 server_cookie=104553f42e0072df in_seq=0 out_seq=0 2024-02-26T06:23:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.068+0000 7f06d2a00640 10 client.25493 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06d2a00640 1 -- 172.21.15.49:0/1139324013 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56540be7f720 con 0x56540c1f50c0 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06d2a00640 10 client.25493 handle_command_reply: tid=0 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 client.25493 shutdown 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 -- 172.21.15.49:0/1139324013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 msgr2=0x56540c1e26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e26d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f06e4000c00 tx=0x7f06e4005270 comp rx=0 tx=0).stop 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 -- 172.21.15.49:0/1139324013 shutdown_connections 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56540c1f50c0 0x56540c1f74e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dc530 0x56540c1e26d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c1e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 --2- 172.21.15.49:0/1139324013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06d2a00640 0 client.25493 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 -- 172.21.15.49:0/1139324013 >> 172.21.15.49:0/1139324013 conn(0x56540c1d6340 msgr2=0x56540c1d8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 -- 172.21.15.49:0/1139324013 shutdown_connections 2024-02-26T06:23:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 1 -- 172.21.15.49:0/1139324013 wait complete. 2024-02-26T06:23:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.069+0000 7f06fac7e740 20 client.25493 trim_cache size 0 max 16384 2024-02-26T06:23:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.083+0000 7f06fac7e740 1 Processor -- start 2024-02-26T06:23:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.083+0000 7f06fac7e740 1 -- start start 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.083+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.083+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dc530 0x56540c1e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.083+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1e5110 0x56540c1df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.083+0000 7f06fac7e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c1d1ab0 con 0x56540c1dc530 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.083+0000 7f06fac7e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c1db3b0 con 0x56540c1e5110 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.083+0000 7f06fac7e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c1d7340 con 0x56540c1dbc10 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1e5110 0x56540c1df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dc530 0x56540c1e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1e5110 0x56540c1df3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49804/0 (socket says 172.21.15.49:49804) 2024-02-26T06:23:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5e00640 1 -- 172.21.15.49:0/797468068 learned_addr learned my addr 172.21.15.49:0/797468068 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5400640 1 -- 172.21.15.49:0/797468068 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1e5110 msgr2=0x56540c1df3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5400640 1 --2- 172.21.15.49:0/797468068 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1e5110 0x56540c1df3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5400640 1 -- 172.21.15.49:0/797468068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dc530 msgr2=0x56540c1e2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5400640 1 --2- 172.21.15.49:0/797468068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dc530 0x56540c1e2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5400640 1 -- 172.21.15.49:0/797468068 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56540c0dda80 con 0x56540c1dbc10 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5e00640 1 --2- 172.21.15.49:0/797468068 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1e5110 0x56540c1df3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.084+0000 7f06f5400640 1 --2- 172.21.15.49:0/797468068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e48c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009230 tx=0x7f06e003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1423933a869fd4b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.085+0000 7f06ddc00640 1 -- 172.21.15.49:0/797468068 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e002cb20 con 0x56540c1dbc10 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.085+0000 7f06ddc00640 1 -- 172.21.15.49:0/797468068 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e00035f0 con 0x56540c1dbc10 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.085+0000 7f06ddc00640 1 -- 172.21.15.49:0/797468068 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0004520 con 0x56540c1dbc10 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.085+0000 7f06fac7e740 1 -- 172.21.15.49:0/797468068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 msgr2=0x56540c1e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.085+0000 7f06fac7e740 1 --2- 172.21.15.49:0/797468068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e48c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009230 tx=0x7f06e003f000 comp rx=0 tx=0).stop 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.085+0000 7f06fac7e740 1 -- 172.21.15.49:0/797468068 shutdown_connections 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.085+0000 7f06fac7e740 1 --2- 172.21.15.49:0/797468068 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1e5110 0x56540c1df3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.086+0000 7f06fac7e740 1 --2- 172.21.15.49:0/797468068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1dc530 0x56540c1e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.086+0000 7f06fac7e740 1 --2- 172.21.15.49:0/797468068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dbc10 0x56540c1e48c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.086+0000 7f06fac7e740 1 -- 172.21.15.49:0/797468068 >> 172.21.15.49:0/797468068 conn(0x56540c1d6340 msgr2=0x56540c1dd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.086+0000 7f06fac7e740 1 -- 172.21.15.49:0/797468068 shutdown_connections 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.086+0000 7f06fac7e740 1 -- 172.21.15.49:0/797468068 wait complete. 2024-02-26T06:23:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.086+0000 7f06fac7e740 1 -- start start 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.086+0000 7f06fac7e740 1 Processor -- start 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dbc10 0x56540c1eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dc530 0x56540c1eb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06fac7e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c24b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06fac7e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c1eb950 con 0x56540c1e5110 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06fac7e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c1ebac0 con 0x56540c1dbc10 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06fac7e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540c24bb30 con 0x56540c1dc530 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06fac7e740 10 client.0 _ll_register_callbacks cb 0x7f06e4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06f5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dbc10 0x56540c1eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dc530 0x56540c1eb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06f5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dbc10 0x56540c1eacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49808/0 (socket says 172.21.15.49:49808) 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06f5400640 1 -- 172.21.15.49:0/3566847437 learned_addr learned my addr 172.21.15.49:0/3566847437 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.087+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c24b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06f4a00640 1 -- 172.21.15.49:0/3566847437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dbc10 msgr2=0x56540c1eacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06f4a00640 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dbc10 0x56540c1eacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06f4a00640 1 -- 172.21.15.49:0/3566847437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 msgr2=0x56540c24b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06f4a00640 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c24b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06f4a00640 1 -- 172.21.15.49:0/3566847437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f01a6870 con 0x56540c1dc530 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06f5e00640 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c24b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06f4a00640 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dc530 0x56540c1eb1f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f06e4031040 tx=0x7f06e4005510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f207b827697b3c51 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06d2a00640 1 -- 172.21.15.49:0/3566847437 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4019800 con 0x56540c1dc530 2024-02-26T06:23:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06fac7e740 1 -- 172.21.15.49:0/3566847437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06e4004390 con 0x56540c1dc530 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.088+0000 7f06d2a00640 1 -- 172.21.15.49:0/3566847437 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f06e4009c00 con 0x56540c1dc530 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06d2a00640 1 -- 172.21.15.49:0/3566847437 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06e4019ca0 con 0x56540c1dc530 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06fac7e740 10 client.25510 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06fac7e740 1 -- 172.21.15.49:0/3566847437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f06f010e840 con 0x56540c1dc530 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06d2a00640 1 -- 172.21.15.49:0/3566847437 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e4004390 con 0x56540c1dc530 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06d2a00640 1 -- 172.21.15.49:0/3566847437 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f06e4037b10 con 0x56540c1dc530 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06fac7e740 10 client.25510 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06fac7e740 10 client.25510 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06fac7e740 20 client.25510 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06fac7e740 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56540c24d040 0x56540c24f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06fac7e740 4 client.25510 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.089+0000 7f06fac7e740 1 -- 172.21.15.49:0/3566847437 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56540bece890 con 0x56540c24d040 2024-02-26T06:23:45.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.090+0000 7f06f5400640 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56540c24d040 0x56540c24f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.090+0000 7f06f5400640 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56540c24d040 0x56540c24f430 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72f62060adb87b8f server_cookie=9bb1da80843b55e4 in_seq=0 out_seq=0 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.090+0000 7f06d2a00640 10 client.25510 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.090+0000 7f06d2a00640 1 -- 172.21.15.49:0/3566847437 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56540bece890 con 0x56540c24d040 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.090+0000 7f06d2a00640 10 client.25510 handle_command_reply: tid=0 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.090+0000 7f06fac7e740 1 client.25510 shutdown 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 -- 172.21.15.49:0/3566847437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dc530 msgr2=0x56540c1eb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dc530 0x56540c1eb1f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f06e4031040 tx=0x7f06e4005510 comp rx=0 tx=0).stop 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 -- 172.21.15.49:0/3566847437 shutdown_connections 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56540c24d040 0x56540c24f430 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56540c1e5110 0x56540c24b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56540c1dc530 0x56540c1eb1f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 --2- 172.21.15.49:0/3566847437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56540c1dbc10 0x56540c1eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06d2a00640 0 client.25510 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:23:45.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 -- 172.21.15.49:0/3566847437 >> 172.21.15.49:0/3566847437 conn(0x56540c1d6340 msgr2=0x56540c1d4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 -- 172.21.15.49:0/3566847437 shutdown_connections 2024-02-26T06:23:45.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 1 -- 172.21.15.49:0/3566847437 wait complete. 2024-02-26T06:23:45.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.091+0000 7f06fac7e740 20 client.25510 trim_cache size 0 max 16384 2024-02-26T06:23:45.094 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 -- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f06d80a2340 msgr2=0x7f06d80a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f06d80a2340 0x7f06d80a4800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005ab0 tx=0x7f06e0005fd0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 -- 172.21.15.49:0/2141969450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06f0199860 msgr2=0x7f06f0199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06f0199860 0x7f06f0199d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06e800eb10 tx=0x7f06e800c2d0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 -- 172.21.15.49:0/2141969450 shutdown_connections 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f06d80a2340 0x7f06d80a4800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06f0199860 0x7f06f0199d10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06f0106460 0x7f06f019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 --2- 172.21.15.49:0/2141969450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f0101390 0x7f06f019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 -- 172.21.15.49:0/2141969450 >> 172.21.15.49:0/2141969450 conn(0x7f06f00fcbb0 msgr2=0x7f06f00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 -- 172.21.15.49:0/2141969450 shutdown_connections 2024-02-26T06:23:45.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.092+0000 7f06f7e00640 1 -- 172.21.15.49:0/2141969450 wait complete. 2024-02-26T06:23:45.106 INFO:teuthology.orchestra.run.smithi049.stderr:2400000 of 3200000 tuples (75%) done (elapsed 7.34 s, remaining 2.45 s) 2024-02-26T06:23:45.114 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:45 smithi049 ceph-mon[31696]: pgmap v13: 97 pgs: 97 active+clean; 93 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 13 MiB/s wr, 150 op/s 2024-02-26T06:23:45.114 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:45 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2467881087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:45.118 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:23:45.118 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:23:45.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:45 smithi080 ceph-mon[36206]: pgmap v13: 97 pgs: 97 active+clean; 93 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 13 MiB/s wr, 150 op/s 2024-02-26T06:23:45.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:45 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2467881087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:45.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:45 smithi115 ceph-mon[36426]: pgmap v13: 97 pgs: 97 active+clean; 93 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 13 MiB/s wr, 150 op/s 2024-02-26T06:23:45.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:45 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2467881087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:23:45.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 -- 172.21.15.49:0/2390115781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 msgr2=0x7f7108107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 --2- 172.21.15.49:0/2390115781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 0x7f7108107ba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009900 tx=0x7f70f402bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:45.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 -- 172.21.15.49:0/2390115781 shutdown_connections 2024-02-26T06:23:45.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 --2- 172.21.15.49:0/2390115781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 0x7f7108107ba0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 --2- 172.21.15.49:0/2390115781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71081080e0 0x7f71081145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 --2- 172.21.15.49:0/2390115781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f710810d730 0x7f710810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 -- 172.21.15.49:0/2390115781 >> 172.21.15.49:0/2390115781 conn(0x7f71080756c0 msgr2=0x7f7108077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 -- 172.21.15.49:0/2390115781 shutdown_connections 2024-02-26T06:23:45.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.373+0000 7f710e600640 1 -- 172.21.15.49:0/2390115781 wait complete. 2024-02-26T06:23:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.374+0000 7f710e600640 1 Processor -- start 2024-02-26T06:23:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.374+0000 7f710e600640 1 -- start start 2024-02-26T06:23:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f710e600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 0x7f710819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f710e600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71081080e0 0x7f710819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f710e600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f710810d730 0x7f71081a2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f710e600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081a2d50 con 0x7f71081080e0 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f710e600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081a2ec0 con 0x7f7108107720 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f710e600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081a3080 con 0x7f710810d730 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f7106a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71081080e0 0x7f710819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f7106a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71081080e0 0x7f710819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40708/0 (socket says 172.21.15.49:40708) 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f7106a00640 1 -- 172.21.15.49:0/2593009128 learned_addr learned my addr 172.21.15.49:0/2593009128 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f7107400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 0x7f710819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f7107e00640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f710810d730 0x7f71081a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.375+0000 7f7106a00640 1 -- 172.21.15.49:0/2593009128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f710810d730 msgr2=0x7f71081a2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f7106a00640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f710810d730 0x7f71081a2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f7106a00640 1 -- 172.21.15.49:0/2593009128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 msgr2=0x7f710819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f7106a00640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 0x7f710819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f7106a00640 1 -- 172.21.15.49:0/2593009128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f8009490 con 0x7f71081080e0 2024-02-26T06:23:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f7107e00640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f710810d730 0x7f71081a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:45.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f7107400640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 0x7f710819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:45.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f7106a00640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71081080e0 0x7f710819e3d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000c00 tx=0x7f70f402c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f70ffe00640 1 -- 172.21.15.49:0/2593009128 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f402dea0 con 0x7f71081080e0 2024-02-26T06:23:45.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f710e600640 1 -- 172.21.15.49:0/2593009128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f4009560 con 0x7f71081080e0 2024-02-26T06:23:45.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f70ffe00640 1 -- 172.21.15.49:0/2593009128 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70f4004490 con 0x7f71081080e0 2024-02-26T06:23:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f70ffe00640 1 -- 172.21.15.49:0/2593009128 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4003c90 con 0x7f71081080e0 2024-02-26T06:23:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.376+0000 7f710e600640 1 -- 172.21.15.49:0/2593009128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71081aaa00 con 0x7f71081080e0 2024-02-26T06:23:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.380+0000 7f70ffe00640 1 -- 172.21.15.49:0/2593009128 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f70f4004df0 con 0x7f71081080e0 2024-02-26T06:23:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.380+0000 7f71115a1740 1 Processor -- start 2024-02-26T06:23:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.380+0000 7f71115a1740 1 -- start start 2024-02-26T06:23:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f71115a1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c6419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f71115a1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f71115a1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f71115a1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6410d1e0 con 0x564c641a8110 2024-02-26T06:23:45.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f71115a1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6410a710 con 0x564c6419ec10 2024-02-26T06:23:45.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f71115a1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c64105ef0 con 0x564c6419f530 2024-02-26T06:23:45.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c6419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7106a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c6419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49834/0 (socket says 172.21.15.49:49834) 2024-02-26T06:23:45.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107e00640 1 -- 172.21.15.49:0/205019371 learned_addr learned my addr 172.21.15.49:0/205019371 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107400640 1 -- 172.21.15.49:0/205019371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 msgr2=0x564c6419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107400640 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c6419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107400640 1 -- 172.21.15.49:0/205019371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 msgr2=0x564c641aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107400640 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107400640 1 -- 172.21.15.49:0/205019371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c640c7dc0 con 0x564c6419f530 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.382+0000 7f7106a00640 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.381+0000 7f7107e00640 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c6419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.382+0000 7f70ffe00640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f70ec0a2340 0x7f70ec0a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.382+0000 7f7107400640 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a7bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70f80115a0 tx=0x7f70f8011a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ecf79ff33677406 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.382+0000 7f70ffe00640 1 -- 172.21.15.49:0/2593009128 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f70f40f06c0 con 0x7f71081080e0 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.382+0000 7f7107e00640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f70ec0a2340 0x7f70ec0a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.382+0000 7f70fcc00640 1 -- 172.21.15.49:0/205019371 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8004020 con 0x564c6419f530 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.382+0000 7f70fcc00640 1 -- 172.21.15.49:0/205019371 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70f8005500 con 0x564c6419f530 2024-02-26T06:23:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.382+0000 7f70fcc00640 1 -- 172.21.15.49:0/205019371 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8004d80 con 0x564c6419f530 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 -- 172.21.15.49:0/205019371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 msgr2=0x564c641a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a7bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70f80115a0 tx=0x7f70f8011a60 comp rx=0 tx=0).stop 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f7107e00640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f70ec0a2340 0x7f70ec0a4800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7100009fd0 tx=0x7f710000d040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 -- 172.21.15.49:0/205019371 shutdown_connections 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a7bd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 --2- 172.21.15.49:0/205019371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c6419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 -- 172.21.15.49:0/205019371 >> 172.21.15.49:0/205019371 conn(0x564c64199340 msgr2=0x564c64199720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 -- 172.21.15.49:0/205019371 shutdown_connections 2024-02-26T06:23:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.383+0000 7f71115a1740 1 -- 172.21.15.49:0/205019371 wait complete. 2024-02-26T06:23:45.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.384+0000 7f71115a1740 1 -- start start 2024-02-26T06:23:45.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.384+0000 7f71115a1740 1 Processor -- start 2024-02-26T06:23:45.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f71115a1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f71115a1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f71115a1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f71115a1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c640ab120 con 0x564c641a8110 2024-02-26T06:23:45.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f71115a1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c640aadc0 con 0x564c6419ec10 2024-02-26T06:23:45.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f71115a1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63e741c0 con 0x564c6419f530 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641a5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49836/0 (socket says 172.21.15.49:49836) 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f71115a1740 10 client.0 _ll_register_callbacks cb 0x564c640fa280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107400640 1 -- 172.21.15.49:0/220442296 learned_addr learned my addr 172.21.15.49:0/220442296 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107e00640 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7106a00640 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107400640 1 -- 172.21.15.49:0/220442296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 msgr2=0x564c641a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107400640 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107400640 1 -- 172.21.15.49:0/220442296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 msgr2=0x564c641a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107400640 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.385+0000 7f7107400640 1 -- 172.21.15.49:0/220442296 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f8009d90 con 0x564c6419ec10 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.386+0000 7f7107400640 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641a5160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000c00 tx=0x7f70f800fd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=263b346b58f09271 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.386+0000 7f70f1600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:45.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.386+0000 7f70f1600640 1 -- 172.21.15.49:0/220442296 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8005670 con 0x564c6419ec10 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.386+0000 7f70f1600640 1 -- 172.21.15.49:0/220442296 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f70f8004480 con 0x564c6419ec10 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.386+0000 7f71115a1740 1 -- 172.21.15.49:0/220442296 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c63e057e0 con 0x564c6419ec10 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.386+0000 7f70f1600640 1 -- 172.21.15.49:0/220442296 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f8004a80 con 0x564c6419ec10 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.386+0000 7f71115a1740 10 client.25511 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.386+0000 7f71115a1740 1 -- 172.21.15.49:0/220442296 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564c64104570 con 0x564c6419ec10 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f70f1600640 1 -- 172.21.15.49:0/220442296 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564c63e057e0 con 0x564c6419ec10 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f70f1600640 1 -- 172.21.15.49:0/220442296 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f70f8023750 con 0x564c6419ec10 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f71115a1740 10 client.25511 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f71115a1740 10 client.25511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f71115a1740 20 client.25511 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f71115a1740 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x564c641b81e0 0x564c641ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f71115a1740 4 client.25511 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f71115a1740 1 -- 172.21.15.49:0/220442296 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c63e42720 con 0x564c641b81e0 2024-02-26T06:23:45.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.387+0000 7f7106a00640 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x564c641b81e0 0x564c641ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.388+0000 7f7106a00640 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x564c641b81e0 0x564c641ba5d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f0086e85b30ac38 server_cookie=911391dc734e5f98 in_seq=0 out_seq=0 2024-02-26T06:23:45.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.388+0000 7f70f1600640 10 client.25511 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.388+0000 7f70f1600640 1 -- 172.21.15.49:0/220442296 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c63e42720 con 0x564c641b81e0 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.388+0000 7f70f1600640 10 client.25511 handle_command_reply: tid=0 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 client.25511 shutdown 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 -- 172.21.15.49:0/220442296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 msgr2=0x564c641a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641a5160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000c00 tx=0x7f70f800fd80 comp rx=0 tx=0).stop 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 -- 172.21.15.49:0/220442296 shutdown_connections 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x564c641b81e0 0x564c641ba5d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f70f1600640 0 client.25511 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:23:45.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 --2- 172.21.15.49:0/220442296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641a5160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 -- 172.21.15.49:0/220442296 >> 172.21.15.49:0/220442296 conn(0x564c64199340 msgr2=0x564c6419b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 -- 172.21.15.49:0/220442296 shutdown_connections 2024-02-26T06:23:45.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 1 -- 172.21.15.49:0/220442296 wait complete. 2024-02-26T06:23:45.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.389+0000 7f71115a1740 20 client.25511 trim_cache size 0 max 16384 2024-02-26T06:23:45.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f71115a1740 1 Processor -- start 2024-02-26T06:23:45.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f71115a1740 1 -- start start 2024-02-26T06:23:45.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f71115a1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419ec10 0x564c641a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f71115a1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c6419f530 0x564c641a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f71115a1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c641a8110 0x564c641a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f71115a1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c641bf860 con 0x564c6419f530 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f71115a1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6419ff80 con 0x564c641a8110 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f71115a1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6419df40 con 0x564c6419ec10 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f7106a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c6419f530 0x564c641a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f7106a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c6419f530 0x564c641a56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40738/0 (socket says 172.21.15.49:40738) 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.406+0000 7f7106a00640 1 -- 172.21.15.49:0/3351923378 learned_addr learned my addr 172.21.15.49:0/3351923378 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7107e00640 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c641a8110 0x564c641a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7107400640 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419ec10 0x564c641a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7107e00640 1 -- 172.21.15.49:0/3351923378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419ec10 msgr2=0x564c641a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7107e00640 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419ec10 0x564c641a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7107e00640 1 -- 172.21.15.49:0/3351923378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c6419f530 msgr2=0x564c641a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7107e00640 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c6419f530 0x564c641a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7107e00640 1 -- 172.21.15.49:0/3351923378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c640a0a80 con 0x564c641a8110 2024-02-26T06:23:45.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7106a00640 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c6419f530 0x564c641a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:45.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.407+0000 7f7107e00640 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c641a8110 0x564c641a5bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7100011bf0 tx=0x7f7100014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=516f575383e19b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f70f3e00640 1 -- 172.21.15.49:0/3351923378 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100010040 con 0x564c641a8110 2024-02-26T06:23:45.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f70f3e00640 1 -- 172.21.15.49:0/3351923378 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7100018df0 con 0x564c641a8110 2024-02-26T06:23:45.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f70f3e00640 1 -- 172.21.15.49:0/3351923378 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100024760 con 0x564c641a8110 2024-02-26T06:23:45.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f71115a1740 1 -- 172.21.15.49:0/3351923378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c641a8110 msgr2=0x564c641a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f71115a1740 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c641a8110 0x564c641a5bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7100011bf0 tx=0x7f7100014000 comp rx=0 tx=0).stop 2024-02-26T06:23:45.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f71115a1740 1 -- 172.21.15.49:0/3351923378 shutdown_connections 2024-02-26T06:23:45.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f71115a1740 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c641a8110 0x564c641a5bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f71115a1740 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c6419f530 0x564c641a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f71115a1740 1 --2- 172.21.15.49:0/3351923378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419ec10 0x564c641a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.408+0000 7f71115a1740 1 -- 172.21.15.49:0/3351923378 >> 172.21.15.49:0/3351923378 conn(0x564c64199340 msgr2=0x564c641a0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.409+0000 7f71115a1740 1 -- 172.21.15.49:0/3351923378 shutdown_connections 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.409+0000 7f71115a1740 1 -- 172.21.15.49:0/3351923378 wait complete. 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.409+0000 7f71115a1740 1 -- start start 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.409+0000 7f71115a1740 1 Processor -- start 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f71115a1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f71115a1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f71115a1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f71115a1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6420e8a0 con 0x564c641a8110 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f71115a1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6420ea10 con 0x564c6419ec10 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f71115a1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6420ebd0 con 0x564c6419f530 2024-02-26T06:23:45.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f71115a1740 10 client.0 _ll_register_callbacks cb 0x7f70f8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49848/0 (socket says 172.21.15.49:49848) 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107400640 1 -- 172.21.15.49:0/967872878 learned_addr learned my addr 172.21.15.49:0/967872878 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107e00640 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7106a00640 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107400640 1 -- 172.21.15.49:0/967872878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 msgr2=0x564c641ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107400640 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107400640 1 -- 172.21.15.49:0/967872878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 msgr2=0x564c641ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107400640 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.410+0000 7f7107400640 1 -- 172.21.15.49:0/967872878 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f4009560 con 0x564c6419ec10 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.411+0000 7f7107400640 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641adca0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005fe0 tx=0x7f70f800f4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5638a9ef6b580cc5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.411+0000 7f70f1600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:45.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.411+0000 7f70f1600640 1 -- 172.21.15.49:0/967872878 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8011230 con 0x564c6419ec10 2024-02-26T06:23:45.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.411+0000 7f70f1600640 1 -- 172.21.15.49:0/967872878 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f70f8023470 con 0x564c6419ec10 2024-02-26T06:23:45.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.411+0000 7f71115a1740 1 -- 172.21.15.49:0/967872878 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70f8005500 con 0x564c6419ec10 2024-02-26T06:23:45.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.411+0000 7f70f1600640 1 -- 172.21.15.49:0/967872878 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f8023630 con 0x564c6419ec10 2024-02-26T06:23:45.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.411+0000 7f71115a1740 10 client.25517 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:45.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.411+0000 7f71115a1740 1 -- 172.21.15.49:0/967872878 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f71000118d0 con 0x564c6419ec10 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f70f1600640 1 -- 172.21.15.49:0/967872878 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70f8044020 con 0x564c6419ec10 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f70f1600640 1 -- 172.21.15.49:0/967872878 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f70f8010430 con 0x564c6419ec10 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f71115a1740 10 client.25517 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f71115a1740 10 client.25517 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f71115a1740 20 client.25517 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f71115a1740 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x564c64210070 0x564c64212460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f71115a1740 4 client.25517 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f71115a1740 1 -- 172.21.15.49:0/967872878 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c63e91890 con 0x564c64210070 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.412+0000 7f7106a00640 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x564c64210070 0x564c64212460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f7106a00640 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x564c64210070 0x564c64212460 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c67a5495f8a89170 server_cookie=1458022aea8cf7f3 in_seq=0 out_seq=0 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f70f1600640 10 client.25517 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f70f1600640 1 -- 172.21.15.49:0/967872878 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c63e91890 con 0x564c64210070 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f70f1600640 10 client.25517 handle_command_reply: tid=0 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 client.25517 shutdown 2024-02-26T06:23:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 -- 172.21.15.49:0/967872878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 msgr2=0x564c641adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641adca0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005fe0 tx=0x7f70f800f4b0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 -- 172.21.15.49:0/967872878 shutdown_connections 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x564c64210070 0x564c64212460 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c641a8110 0x564c641ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6419f530 0x564c641ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 --2- 172.21.15.49:0/967872878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c6419ec10 0x564c641adca0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 -- 172.21.15.49:0/967872878 >> 172.21.15.49:0/967872878 conn(0x564c64199340 msgr2=0x564c64197bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f70f1600640 0 client.25517 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 -- 172.21.15.49:0/967872878 shutdown_connections 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 1 -- 172.21.15.49:0/967872878 wait complete. 2024-02-26T06:23:45.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.413+0000 7f71115a1740 20 client.25517 trim_cache size 0 max 16384 2024-02-26T06:23:45.418 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:23:45.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 -- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f70ec0a2340 msgr2=0x7f70ec0a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f70ec0a2340 0x7f70ec0a4800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7100009fd0 tx=0x7f710000d040 comp rx=0 tx=0).stop 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 -- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71081080e0 msgr2=0x7f710819e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71081080e0 0x7f710819e3d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000c00 tx=0x7f70f402c720 comp rx=0 tx=0).stop 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 -- 172.21.15.49:0/2593009128 shutdown_connections 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f70ec0a2340 0x7f70ec0a4800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f710810d730 0x7f71081a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71081080e0 0x7f710819e3d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 --2- 172.21.15.49:0/2593009128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7108107720 0x7f710819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 -- 172.21.15.49:0/2593009128 >> 172.21.15.49:0/2593009128 conn(0x7f71080756c0 msgr2=0x7f7108075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 -- 172.21.15.49:0/2593009128 shutdown_connections 2024-02-26T06:23:45.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.414+0000 7f710e600640 1 -- 172.21.15.49:0/2593009128 wait complete. 2024-02-26T06:23:45.438 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:23:45.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:23:45.445 INFO:teuthology.orchestra.run.smithi049.stderr:2500000 of 3200000 tuples (78%) done (elapsed 7.68 s, remaining 2.15 s) 2024-02-26T06:23:45.633 INFO:teuthology.orchestra.run.smithi049.stderr:2600000 of 3200000 tuples (81%) done (elapsed 7.87 s, remaining 1.82 s) 2024-02-26T06:23:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.678+0000 7f7ee6e00640 1 -- 172.21.15.49:0/1202867453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073550 msgr2=0x7f7ee0073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.678+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/1202867453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073550 0x7f7ee0073930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009ca0 tx=0x7f7ed402c190 comp rx=0 tx=0).stop 2024-02-26T06:23:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.678+0000 7f7ee6e00640 1 -- 172.21.15.49:0/1202867453 shutdown_connections 2024-02-26T06:23:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.678+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/1202867453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee0073e70 0x7f7ee010d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.678+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/1202867453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee010d890 0x7f7ee010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.678+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/1202867453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073550 0x7f7ee0073930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.678+0000 7f7ee6e00640 1 -- 172.21.15.49:0/1202867453 >> 172.21.15.49:0/1202867453 conn(0x7f7ee00fcb20 msgr2=0x7f7ee00fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.678+0000 7f7ee6e00640 1 -- 172.21.15.49:0/1202867453 shutdown_connections 2024-02-26T06:23:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.679+0000 7f7ee6e00640 1 -- 172.21.15.49:0/1202867453 wait complete. 2024-02-26T06:23:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.680+0000 7f7ee6e00640 1 Processor -- start 2024-02-26T06:23:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.680+0000 7f7ee6e00640 1 -- start start 2024-02-26T06:23:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.680+0000 7f7ee6e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee0073550 0x7f7ee019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.680+0000 7f7ee6e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073e70 0x7f7ee019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.680+0000 7f7ee6e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee010d890 0x7f7ee01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.680+0000 7f7ee6e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0199e70 con 0x7f7ee010d890 2024-02-26T06:23:45.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.680+0000 7f7ee6e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0199fe0 con 0x7f7ee0073550 2024-02-26T06:23:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.680+0000 7f7ee6e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee019a1a0 con 0x7f7ee0073e70 2024-02-26T06:23:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073e70 0x7f7ee019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee0073550 0x7f7ee019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee6400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee010d890 0x7f7ee01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073e70 0x7f7ee019fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57412/0 (socket says 172.21.15.49:57412) 2024-02-26T06:23:45.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5000640 1 -- 172.21.15.49:0/3873007124 learned_addr learned my addr 172.21.15.49:0/3873007124 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5a00640 1 -- 172.21.15.49:0/3873007124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073e70 msgr2=0x7f7ee019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5a00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073e70 0x7f7ee019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5a00640 1 -- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee010d890 msgr2=0x7f7ee01996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5a00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee010d890 0x7f7ee01996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5a00640 1 -- 172.21.15.49:0/3873007124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc009490 con 0x7f7ee0073550 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.681+0000 7f7ee5a00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee0073550 0x7f7ee019f620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4006010 tx=0x7f7ed4034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.682+0000 7f7ede000640 1 -- 172.21.15.49:0/3873007124 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4003970 con 0x7f7ee0073550 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.682+0000 7f7ede000640 1 -- 172.21.15.49:0/3873007124 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed40327b0 con 0x7f7ee0073550 2024-02-26T06:23:45.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.682+0000 7f7ede000640 1 -- 172.21.15.49:0/3873007124 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40044e0 con 0x7f7ee0073550 2024-02-26T06:23:45.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.682+0000 7f7ee6e00640 1 -- 172.21.15.49:0/3873007124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed4009900 con 0x7f7ee0073550 2024-02-26T06:23:45.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.682+0000 7f7ee6e00640 1 -- 172.21.15.49:0/3873007124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee01a63c0 con 0x7f7ee0073550 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.686+0000 7f7eeb3db740 1 Processor -- start 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.686+0000 7f7eeb3db740 1 -- start start 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.686+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x55812849d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.686+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 0x5581284a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.686+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5581284a62c0 0x5581284a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.686+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812840b1e0 con 0x55812849d6e0 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.686+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558128408710 con 0x5581284a62c0 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.686+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558128403ef0 con 0x55812849cdc0 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee6400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x55812849d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee6400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x55812849d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57428/0 (socket says 172.21.15.49:57428) 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee6400640 1 -- 172.21.15.49:0/2073929786 learned_addr learned my addr 172.21.15.49:0/2073929786 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee5000640 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5581284a62c0 0x5581284a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ede000640 1 -- 172.21.15.49:0/3873007124 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f7ed403b050 con 0x7f7ee0073550 2024-02-26T06:23:45.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee6400640 1 -- 172.21.15.49:0/2073929786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5581284a62c0 msgr2=0x5581284a86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee6400640 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5581284a62c0 0x5581284a86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee6400640 1 -- 172.21.15.49:0/2073929786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 msgr2=0x5581284a5d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:23:45.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee6400640 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 0x5581284a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee6400640 1 -- 172.21.15.49:0/2073929786 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581283c5dc0 con 0x55812849cdc0 2024-02-26T06:23:45.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.687+0000 7f7ee5000640 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5581284a62c0 0x5581284a86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:23:45.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.688+0000 7f7ee6400640 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x55812849d1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009b80 tx=0x7f7ed8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c95360ffdcd25c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.688+0000 7f7ed2a00640 1 -- 172.21.15.49:0/2073929786 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8004030 con 0x55812849cdc0 2024-02-26T06:23:45.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.688+0000 7f7ed2a00640 1 -- 172.21.15.49:0/2073929786 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed80035f0 con 0x55812849cdc0 2024-02-26T06:23:45.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.688+0000 7f7ed2a00640 1 -- 172.21.15.49:0/2073929786 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8019680 con 0x55812849cdc0 2024-02-26T06:23:45.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.689+0000 7f7ede000640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7ec80a22c0 0x7f7ec80a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.689+0000 7f7ede000640 1 -- 172.21.15.49:0/3873007124 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7ed40f00e0 con 0x7f7ee0073550 2024-02-26T06:23:45.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7ee5a00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7ec80a22c0 0x7f7ec80a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2073929786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 msgr2=0x55812849d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x55812849d1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009b80 tx=0x7f7ed8011810 comp rx=0 tx=0).stop 2024-02-26T06:23:45.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2073929786 shutdown_connections 2024-02-26T06:23:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5581284a62c0 0x5581284a86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 0x5581284a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2073929786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x55812849d1a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7ee5a00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7ec80a22c0 0x7f7ec80a4780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40ed890 tx=0x7f7ed40bc430 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.690+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2073929786 >> 172.21.15.49:0/2073929786 conn(0x5581284974f0 msgr2=0x5581284978d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.691+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2073929786 shutdown_connections 2024-02-26T06:23:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.691+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2073929786 wait complete. 2024-02-26T06:23:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.691+0000 7f7eeb3db740 1 -- start start 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.691+0000 7f7eeb3db740 1 Processor -- start 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849cdc0 0x5581284a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 0x5581284a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581283a9120 con 0x55812849d6e0 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581283a8dc0 con 0x55812849cdc0 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581281721c0 con 0x5581284a62c0 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7eeb3db740 10 client.0 _ll_register_callbacks cb 0x5581283f8280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7ee6400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849cdc0 0x5581284a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7ee6400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849cdc0 0x5581284a32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49892/0 (socket says 172.21.15.49:49892) 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.692+0000 7f7ee6400640 1 -- 172.21.15.49:0/2609815365 learned_addr learned my addr 172.21.15.49:0/2609815365 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee5000640 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 0x5581284a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee6400640 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee6400640 1 -- 172.21.15.49:0/2609815365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 msgr2=0x5581284a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee6400640 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee6400640 1 -- 172.21.15.49:0/2609815365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 msgr2=0x5581284a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee6400640 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 0x5581284a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee6400640 1 -- 172.21.15.49:0/2609815365 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed8009830 con 0x55812849cdc0 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee5000640 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 0x5581284a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ee6400640 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849cdc0 0x5581284a32a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8003f10 tx=0x7f7ed8003d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ce4fc4f8769aed server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ec7400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:45.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ec7400640 1 -- 172.21.15.49:0/2609815365 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8005050 con 0x55812849cdc0 2024-02-26T06:23:45.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7ec7400640 1 -- 172.21.15.49:0/2609815365 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7ed80051b0 con 0x55812849cdc0 2024-02-26T06:23:45.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.693+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2609815365 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581281037e0 con 0x55812849cdc0 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.694+0000 7f7ec7400640 1 -- 172.21.15.49:0/2609815365 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ed8005370 con 0x55812849cdc0 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.694+0000 7f7eeb3db740 10 client.25529 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.694+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2609815365 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558128402570 con 0x55812849cdc0 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.694+0000 7f7ec7400640 1 -- 172.21.15.49:0/2609815365 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed8036080 con 0x55812849cdc0 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.694+0000 7f7ec7400640 1 -- 172.21.15.49:0/2609815365 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ed802a4d0 con 0x55812849cdc0 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7eeb3db740 10 client.25529 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7eeb3db740 10 client.25529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7eeb3db740 20 client.25529 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5581284b62f0 0x5581284b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7eeb3db740 4 client.25529 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2609815365 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558128140720 con 0x5581284b62f0 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7ee5000640 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5581284b62f0 0x5581284b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7ee5000640 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5581284b62f0 0x5581284b8710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e392791dbceb1d server_cookie=131e51ae0a90c09 in_seq=0 out_seq=0 2024-02-26T06:23:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.695+0000 7f7ec7400640 10 client.25529 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.696+0000 7f7ec7400640 1 -- 172.21.15.49:0/2609815365 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558128140720 con 0x5581284b62f0 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.696+0000 7f7ec7400640 10 client.25529 handle_command_reply: tid=0 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.696+0000 7f7eeb3db740 1 client.25529 shutdown 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2609815365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849cdc0 msgr2=0x5581284a32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849cdc0 0x5581284a32a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8003f10 tx=0x7f7ed8003d00 comp rx=0 tx=0).stop 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2609815365 shutdown_connections 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5581284b62f0 0x5581284b8710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849d6e0 0x5581284a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/2609815365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849cdc0 0x5581284a32a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7ec7400640 0 client.25529 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2609815365 >> 172.21.15.49:0/2609815365 conn(0x5581284974f0 msgr2=0x558128499240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2609815365 shutdown_connections 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 1 -- 172.21.15.49:0/2609815365 wait complete. 2024-02-26T06:23:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.697+0000 7f7eeb3db740 20 client.25529 trim_cache size 0 max 16384 2024-02-26T06:23:45.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.711+0000 7f7eeb3db740 1 Processor -- start 2024-02-26T06:23:45.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.711+0000 7f7eeb3db740 1 -- start start 2024-02-26T06:23:45.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.711+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849cdc0 0x5581284a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.711+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284a0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.711+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.711+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581284984f0 con 0x55812849cdc0 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.711+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812849c790 con 0x55812849d6e0 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.711+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812849bfd0 con 0x5581284a62c0 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee6400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849cdc0 0x5581284a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49906/0 (socket says 172.21.15.49:49906) 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 -- 172.21.15.49:0/3164440162 learned_addr learned my addr 172.21.15.49:0/3164440162 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee6400640 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 -- 172.21.15.49:0/3164440162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 msgr2=0x5581284a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 -- 172.21.15.49:0/3164440162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849cdc0 msgr2=0x5581284a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849cdc0 0x5581284a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 -- 172.21.15.49:0/3164440162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55812839ea80 con 0x55812849d6e0 2024-02-26T06:23:45.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee6400640 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:23:45.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.712+0000 7f7ee5000640 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284a0520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009f40 tx=0x7f7ecc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb76150b7ceabb7c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.713+0000 7f7ed2000640 1 -- 172.21.15.49:0/3164440162 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc004020 con 0x55812849d6e0 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.713+0000 7f7ed2000640 1 -- 172.21.15.49:0/3164440162 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ecc018040 con 0x55812849d6e0 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.713+0000 7f7ed2000640 1 -- 172.21.15.49:0/3164440162 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc0053a0 con 0x55812849d6e0 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 -- 172.21.15.49:0/3164440162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 msgr2=0x5581284a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284a0520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009f40 tx=0x7f7ecc005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 -- 172.21.15.49:0/3164440162 shutdown_connections 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284a0520 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581284a62c0 0x5581284a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/3164440162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55812849cdc0 0x5581284a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 -- 172.21.15.49:0/3164440162 >> 172.21.15.49:0/3164440162 conn(0x5581284974f0 msgr2=0x55812849e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 -- 172.21.15.49:0/3164440162 shutdown_connections 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 -- 172.21.15.49:0/3164440162 wait complete. 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 -- start start 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.714+0000 7f7eeb3db740 1 Processor -- start 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x5581284ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7eeb3db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5581284a62c0 0x55812850dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812850e510 con 0x5581284a62c0 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812850e680 con 0x55812849d6e0 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7eeb3db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812850e840 con 0x55812849cdc0 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7eeb3db740 10 client.0 _ll_register_callbacks cb 0x7f7ed8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7ee6400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x5581284ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7ee5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7ee6400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x5581284ac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57460/0 (socket says 172.21.15.49:57460) 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7ee5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49922/0 (socket says 172.21.15.49:49922) 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7ee6400640 1 -- 172.21.15.49:0/279324410 learned_addr learned my addr 172.21.15.49:0/279324410 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:45.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.715+0000 7f7ee6400640 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5581284a62c0 0x55812850dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ee6400640 1 -- 172.21.15.49:0/279324410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 msgr2=0x5581284acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ee6400640 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ee6400640 1 -- 172.21.15.49:0/279324410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5581284a62c0 msgr2=0x55812850dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ee6400640 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5581284a62c0 0x55812850dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ee6400640 1 -- 172.21.15.49:0/279324410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc011520 con 0x55812849cdc0 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ee6400640 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x5581284ac5d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8011850 tx=0x7f7ed8034040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f215e11b2d2d98 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ec7400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ec7400640 1 -- 172.21.15.49:0/279324410 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80055e0 con 0x55812849cdc0 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7eeb3db740 1 -- 172.21.15.49:0/279324410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ed80035f0 con 0x55812849cdc0 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.716+0000 7f7ec7400640 1 -- 172.21.15.49:0/279324410 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7ed802aaa0 con 0x55812849cdc0 2024-02-26T06:23:45.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7ec7400640 1 -- 172.21.15.49:0/279324410 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ed802a670 con 0x55812849cdc0 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7eeb3db740 10 client.25540 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7eeb3db740 1 -- 172.21.15.49:0/279324410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5581283c4620 con 0x55812849cdc0 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7ec7400640 1 -- 172.21.15.49:0/279324410 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed80035f0 con 0x55812849cdc0 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7ec7400640 1 -- 172.21.15.49:0/279324410 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ed801ac20 con 0x55812849cdc0 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7eeb3db740 10 client.25540 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7eeb3db740 10 client.25540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7eeb3db740 20 client.25540 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558128516ed0 0x5581285192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7eeb3db740 4 client.25540 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7eeb3db740 1 -- 172.21.15.49:0/279324410 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55812818f890 con 0x558128516ed0 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.717+0000 7f7ee5000640 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558128516ed0 0x5581285192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7ee5000640 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558128516ed0 0x5581285192f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f429a875e5fccc server_cookie=b06d7aa4856a6336 in_seq=0 out_seq=0 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7ec7400640 10 client.25540 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7ec7400640 1 -- 172.21.15.49:0/279324410 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55812818f890 con 0x558128516ed0 2024-02-26T06:23:45.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7ec7400640 10 client.25540 handle_command_reply: tid=0 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 client.25540 shutdown 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 -- 172.21.15.49:0/279324410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 msgr2=0x5581284ac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x5581284ac5d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8011850 tx=0x7f7ed8034040 comp rx=0 tx=0).stop 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 -- 172.21.15.49:0/279324410 shutdown_connections 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558128516ed0 0x5581285192f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55812849d6e0 0x5581284acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5581284a62c0 0x55812850dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 --2- 172.21.15.49:0/279324410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55812849cdc0 0x5581284ac5d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7eeb3db740 1 -- 172.21.15.49:0/279324410 >> 172.21.15.49:0/279324410 conn(0x5581284974f0 msgr2=0x558128495e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.718+0000 7f7ec7400640 0 client.25540 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7eeb3db740 1 -- 172.21.15.49:0/279324410 shutdown_connections 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7eeb3db740 1 -- 172.21.15.49:0/279324410 wait complete. 2024-02-26T06:23:45.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7eeb3db740 20 client.25540 trim_cache size 0 max 16384 2024-02-26T06:23:45.722 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:23:45.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 -- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7ec80a22c0 msgr2=0x7f7ec80a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7ec80a22c0 0x7f7ec80a4780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40ed890 tx=0x7f7ed40bc430 comp rx=0 tx=0).stop 2024-02-26T06:23:45.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 -- 172.21.15.49:0/3873007124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee0073550 msgr2=0x7f7ee019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee0073550 0x7f7ee019f620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4006010 tx=0x7f7ed4034440 comp rx=0 tx=0).stop 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 -- 172.21.15.49:0/3873007124 shutdown_connections 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7ec80a22c0 0x7f7ec80a4780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee010d890 0x7f7ee01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee0073e70 0x7f7ee019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.719+0000 7f7ee6e00640 1 --2- 172.21.15.49:0/3873007124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee0073550 0x7f7ee019f620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.720+0000 7f7ee6e00640 1 -- 172.21.15.49:0/3873007124 >> 172.21.15.49:0/3873007124 conn(0x7f7ee00fcb20 msgr2=0x7f7ee00fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.720+0000 7f7ee6e00640 1 -- 172.21.15.49:0/3873007124 shutdown_connections 2024-02-26T06:23:45.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:45.720+0000 7f7ee6e00640 1 -- 172.21.15.49:0/3873007124 wait complete. 2024-02-26T06:23:45.948 INFO:teuthology.orchestra.run.smithi049.stderr:2700000 of 3200000 tuples (84%) done (elapsed 8.18 s, remaining 1.52 s) 2024-02-26T06:23:46.285 INFO:teuthology.orchestra.run.smithi049.stderr:2800000 of 3200000 tuples (87%) done (elapsed 8.52 s, remaining 1.22 s) 2024-02-26T06:23:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2900000 of 3200000 tuples (90%) done (elapsed 8.70 s, remaining 0.90 s) 2024-02-26T06:23:46.738 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:23:46.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:23:46.739 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:23:46.786 INFO:teuthology.orchestra.run.smithi049.stderr:3000000 of 3200000 tuples (93%) done (elapsed 9.02 s, remaining 0.60 s) 2024-02-26T06:23:46.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.978+0000 7f60cc000640 1 -- 172.21.15.49:0/491796826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4106430 msgr2=0x7f60c410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.978+0000 7f60cc000640 1 --2- 172.21.15.49:0/491796826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4106430 0x7f60c410e000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009ca0 tx=0x7f60bc02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.979+0000 7f60cc000640 1 -- 172.21.15.49:0/491796826 shutdown_connections 2024-02-26T06:23:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.979+0000 7f60cc000640 1 --2- 172.21.15.49:0/491796826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4106430 0x7f60c410e000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.979+0000 7f60cc000640 1 --2- 172.21.15.49:0/491796826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c4101d10 0x7f60c4105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.979+0000 7f60cc000640 1 --2- 172.21.15.49:0/491796826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c4101360 0x7f60c4101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.979+0000 7f60cc000640 1 -- 172.21.15.49:0/491796826 >> 172.21.15.49:0/491796826 conn(0x7f60c40fcbe0 msgr2=0x7f60c40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.979+0000 7f60cc000640 1 -- 172.21.15.49:0/491796826 shutdown_connections 2024-02-26T06:23:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.979+0000 7f60cc000640 1 -- 172.21.15.49:0/491796826 wait complete. 2024-02-26T06:23:46.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.980+0000 7f60cc000640 1 Processor -- start 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.980+0000 7f60cc000640 1 -- start start 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.980+0000 7f60cc000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c4101360 0x7f60c419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.980+0000 7f60cc000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4106430 0x7f60c419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.980+0000 7f60cc000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c4199850 0x7f60c4199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.980+0000 7f60cc000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c419a300 con 0x7f60c4199850 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.980+0000 7f60cc000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c419a470 con 0x7f60c4101360 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60cc000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c419a5e0 con 0x7f60c4106430 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c4101360 0x7f60c419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c4101360 0x7f60c419f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49928/0 (socket says 172.21.15.49:49928) 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 -- 172.21.15.49:0/3501671301 learned_addr learned my addr 172.21.15.49:0/3501671301 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c8c00640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4106430 0x7f60c419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60ca000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c4199850 0x7f60c4199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 -- 172.21.15.49:0/3501671301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4106430 msgr2=0x7f60c419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4106430 0x7f60c419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 -- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c4199850 msgr2=0x7f60c4199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c4199850 0x7f60c4199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 -- 172.21.15.49:0/3501671301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc009900 con 0x7f60c4101360 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60ca000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c4199850 0x7f60c4199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.981+0000 7f60c9600640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c4101360 0x7f60c419f690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60b400e9f0 tx=0x7f60b400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.982+0000 7f60c2000640 1 -- 172.21.15.49:0/3501671301 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4009720 con 0x7f60c4101360 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.982+0000 7f60c2000640 1 -- 172.21.15.49:0/3501671301 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b400cd80 con 0x7f60c4101360 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.982+0000 7f60cc000640 1 -- 172.21.15.49:0/3501671301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c41a6540 con 0x7f60c4101360 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.982+0000 7f60c2000640 1 -- 172.21.15.49:0/3501671301 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4010690 con 0x7f60c4101360 2024-02-26T06:23:46.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.982+0000 7f60cc000640 1 -- 172.21.15.49:0/3501671301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c41a6a20 con 0x7f60c4101360 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60cef76740 1 Processor -- start 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60c2000640 1 -- 172.21.15.49:0/3501671301 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f60b40046c0 con 0x7f60c4101360 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60cef76740 1 -- start start 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60cef76740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60cef76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60cef76740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60cef76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a4732d0 con 0x55ec4a4b8830 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60cef76740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a425e20 con 0x55ec4a4c1410 2024-02-26T06:23:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.987+0000 7f60cef76740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a423350 con 0x55ec4a4b7f10 2024-02-26T06:23:46.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60c9600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60ca000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60ca000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57496/0 (socket says 172.21.15.49:57496) 2024-02-26T06:23:46.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60ca000640 1 -- 172.21.15.49:0/213767320 learned_addr learned my addr 172.21.15.49:0/213767320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:46.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60c8c00640 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60ca000640 1 -- 172.21.15.49:0/213767320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4c1410 msgr2=0x55ec4a4c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:46.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60ca000640 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60ca000640 1 -- 172.21.15.49:0/213767320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 msgr2=0x55ec4a4c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60ca000640 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60ca000640 1 -- 172.21.15.49:0/213767320 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec4a3e7c50 con 0x55ec4a4b7f10 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.988+0000 7f60c8c00640 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.989+0000 7f60ca000640 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4b82f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009ca0 tx=0x7f60bc032d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ef2429f6a204898 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.989+0000 7f60c2000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f60a80a2340 0x7f60a80a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.989+0000 7f60c2000640 1 -- 172.21.15.49:0/3501671301 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f60b4014070 con 0x7f60c4101360 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.989+0000 7f60ba600640 1 -- 172.21.15.49:0/213767320 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc032340 con 0x55ec4a4b7f10 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.989+0000 7f60ba600640 1 -- 172.21.15.49:0/213767320 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60bc0324a0 con 0x55ec4a4b7f10 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.989+0000 7f60ba600640 1 -- 172.21.15.49:0/213767320 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc003970 con 0x55ec4a4b7f10 2024-02-26T06:23:46.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.989+0000 7f60c9600640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f60a80a2340 0x7f60a80a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 -- 172.21.15.49:0/213767320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 msgr2=0x55ec4a4b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60c9600640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f60a80a2340 0x7f60a80a4800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60b40cbcd0 tx=0x7f60b40d1e50 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4b82f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009ca0 tx=0x7f60bc032d60 comp rx=0 tx=0).stop 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 -- 172.21.15.49:0/213767320 shutdown_connections 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 --2- 172.21.15.49:0/213767320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4b82f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 -- 172.21.15.49:0/213767320 >> 172.21.15.49:0/213767320 conn(0x55ec4a4b2640 msgr2=0x55ec4a4b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:46.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 -- 172.21.15.49:0/213767320 shutdown_connections 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.990+0000 7f60cef76740 1 -- 172.21.15.49:0/213767320 wait complete. 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.991+0000 7f60cef76740 1 -- start start 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.991+0000 7f60cef76740 1 Processor -- start 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60cef76740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60cef76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4be920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60cef76740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 0x55ec4a4bee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60cef76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a41eb30 con 0x55ec4a4b8830 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60cef76740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a3c3fa0 con 0x55ec4a4b7f10 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60cef76740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a3c3c40 con 0x55ec4a4c1410 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60cef76740 10 client.0 _ll_register_callbacks cb 0x55ec4a12fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4be920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40854/0 (socket says 172.21.15.49:40854) 2024-02-26T06:23:46.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60c8c00640 1 -- 172.21.15.49:0/4038342116 learned_addr learned my addr 172.21.15.49:0/4038342116 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60ca000640 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60ca000640 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 0x55ec4a4bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60c8c00640 1 -- 172.21.15.49:0/4038342116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 msgr2=0x55ec4a4bee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60c8c00640 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 0x55ec4a4bee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60c8c00640 1 -- 172.21.15.49:0/4038342116 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 msgr2=0x55ec4a4be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60c8c00640 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.992+0000 7f60c8c00640 1 -- 172.21.15.49:0/4038342116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c410e810 con 0x55ec4a4b8830 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.993+0000 7f60c8c00640 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4be920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60b0012730 tx=0x7f60b0012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a0f3d7a551221a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.993+0000 7f60aea00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.993+0000 7f60aea00640 1 -- 172.21.15.49:0/4038342116 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0017070 con 0x55ec4a4b8830 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.993+0000 7f60cef76740 1 -- 172.21.15.49:0/4038342116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec4a11e860 con 0x55ec4a4b8830 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.993+0000 7f60aea00640 1 -- 172.21.15.49:0/4038342116 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f60b0004050 con 0x55ec4a4b8830 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.993+0000 7f60aea00640 1 -- 172.21.15.49:0/4038342116 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b0004210 con 0x55ec4a4b8830 2024-02-26T06:23:46.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.993+0000 7f60cef76740 10 client.15429 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.993+0000 7f60cef76740 1 -- 172.21.15.49:0/4038342116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ec4a41d1b0 con 0x55ec4a4b8830 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60aea00640 1 -- 172.21.15.49:0/4038342116 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ec4a11e860 con 0x55ec4a4b8830 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60aea00640 1 -- 172.21.15.49:0/4038342116 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60b0023070 con 0x55ec4a4b8830 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60cef76740 10 client.15429 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60cef76740 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60cef76740 20 client.15429 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60cef76740 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec4a4d1800 0x55ec4a4d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60cef76740 4 client.15429 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60cef76740 1 -- 172.21.15.49:0/4038342116 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec4a16f410 con 0x55ec4a4d1800 2024-02-26T06:23:46.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.994+0000 7f60ca000640 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec4a4d1800 0x55ec4a4d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:46.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.995+0000 7f60ca000640 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec4a4d1800 0x55ec4a4d3c20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a1f353d655afb1 server_cookie=7ef8f59399a38345 in_seq=0 out_seq=0 2024-02-26T06:23:46.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.995+0000 7f60aea00640 10 client.15429 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:46.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60aea00640 1 -- 172.21.15.49:0/4038342116 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec4a16f410 con 0x55ec4a4d1800 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60aea00640 10 client.15429 handle_command_reply: tid=0 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 client.15429 shutdown 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 -- 172.21.15.49:0/4038342116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 msgr2=0x55ec4a4be920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4be920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60b0012730 tx=0x7f60b0012bf0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 -- 172.21.15.49:0/4038342116 shutdown_connections 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec4a4d1800 0x55ec4a4d3c20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 0x55ec4a4bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4be920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 --2- 172.21.15.49:0/4038342116 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 -- 172.21.15.49:0/4038342116 >> 172.21.15.49:0/4038342116 conn(0x55ec4a4b2640 msgr2=0x55ec4a4b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60aea00640 0 client.15429 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 -- 172.21.15.49:0/4038342116 shutdown_connections 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 1 -- 172.21.15.49:0/4038342116 wait complete. 2024-02-26T06:23:46.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:46.996+0000 7f60cef76740 20 client.15429 trim_cache size 0 max 16384 2024-02-26T06:23:47.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.013+0000 7f60cef76740 1 Processor -- start 2024-02-26T06:23:47.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.013+0000 7f60cef76740 1 -- start start 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60cef76740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60cef76740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b8830 0x55ec4a4b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60cef76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60cef76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a1670c0 con 0x55ec4a4c1410 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60cef76740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a4a4360 con 0x55ec4a4b8830 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60cef76740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a4b3640 con 0x55ec4a4b7f10 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b8830 0x55ec4a4b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b8830 0x55ec4a4b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49952/0 (socket says 172.21.15.49:49952) 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60c8c00640 1 -- 172.21.15.49:0/3445137352 learned_addr learned my addr 172.21.15.49:0/3445137352 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:47.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60ca000640 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60ca000640 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.014+0000 7f60c8c00640 1 -- 172.21.15.49:0/3445137352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 msgr2=0x55ec4a4bec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.015+0000 7f60c8c00640 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4bec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.015+0000 7f60c8c00640 1 -- 172.21.15.49:0/3445137352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4c1410 msgr2=0x55ec4a4c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.015+0000 7f60c8c00640 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.015+0000 7f60c8c00640 1 -- 172.21.15.49:0/3445137352 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec4a3b9a60 con 0x55ec4a4b8830 2024-02-26T06:23:47.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.015+0000 7f60c8c00640 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b8830 0x55ec4a4b4110 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60b0012420 tx=0x7f60b00098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b94ac462d49073 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:47.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.018+0000 7f60b9c00640 1 -- 172.21.15.49:0/3445137352 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0017070 con 0x55ec4a4b8830 2024-02-26T06:23:47.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.018+0000 7f60b9c00640 1 -- 172.21.15.49:0/3445137352 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b0009e80 con 0x55ec4a4b8830 2024-02-26T06:23:47.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.018+0000 7f60b9c00640 1 -- 172.21.15.49:0/3445137352 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b00056a0 con 0x55ec4a4b8830 2024-02-26T06:23:47.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 -- 172.21.15.49:0/3445137352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b8830 msgr2=0x55ec4a4b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b8830 0x55ec4a4b4110 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60b0012420 tx=0x7f60b00098d0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 -- 172.21.15.49:0/3445137352 shutdown_connections 2024-02-26T06:23:47.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b8830 0x55ec4a4b4110 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4c1410 0x55ec4a4c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 --2- 172.21.15.49:0/3445137352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4bec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 -- 172.21.15.49:0/3445137352 >> 172.21.15.49:0/3445137352 conn(0x55ec4a4b2640 msgr2=0x55ec4a4b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 -- 172.21.15.49:0/3445137352 shutdown_connections 2024-02-26T06:23:47.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.019+0000 7f60cef76740 1 -- 172.21.15.49:0/3445137352 wait complete. 2024-02-26T06:23:47.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 1 -- start start 2024-02-26T06:23:47.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 1 Processor -- start 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 0x55ec4a521d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a522290 con 0x55ec4a4b8830 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a5223d0 con 0x55ec4a4b7f10 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4a522590 con 0x55ec4a4c1410 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.020+0000 7f60cef76740 10 client.0 _ll_register_callbacks cb 0x7f60bc037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40874/0 (socket says 172.21.15.49:40874) 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60c8c00640 1 -- 172.21.15.49:0/709682505 learned_addr learned my addr 172.21.15.49:0/709682505 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60ca000640 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 0x55ec4a521d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60ca000640 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60c8c00640 1 -- 172.21.15.49:0/709682505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 msgr2=0x55ec4a521d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60c8c00640 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 0x55ec4a521d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60c8c00640 1 -- 172.21.15.49:0/709682505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 msgr2=0x55ec4a4c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60c8c00640 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.021+0000 7f60c8c00640 1 -- 172.21.15.49:0/709682505 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec4a41d1b0 con 0x55ec4a4b8830 2024-02-26T06:23:47.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.022+0000 7f60c8c00640 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c7a70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f60b0012c30 tx=0x7f60b0005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab36bee681a9842 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:47.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.022+0000 7f60aea00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:23:47.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.022+0000 7f60aea00640 1 -- 172.21.15.49:0/709682505 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0033070 con 0x55ec4a4b8830 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.022+0000 7f60cef76740 1 -- 172.21.15.49:0/709682505 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60bc0324a0 con 0x55ec4a4b8830 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.022+0000 7f60aea00640 1 -- 172.21.15.49:0/709682505 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f60b000fb30 con 0x55ec4a4b8830 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.022+0000 7f60aea00640 1 -- 172.21.15.49:0/709682505 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b0017040 con 0x55ec4a4b8830 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.022+0000 7f60cef76740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.022+0000 7f60cef76740 1 -- 172.21.15.49:0/709682505 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ec4a3df670 con 0x55ec4a4b8830 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60aea00640 1 -- 172.21.15.49:0/709682505 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60bc0324a0 con 0x55ec4a4b8830 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60aea00640 1 -- 172.21.15.49:0/709682505 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60b0036020 con 0x55ec4a4b8830 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60cef76740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60cef76740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60cef76740 20 client.15435 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60cef76740 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec4a529aa0 0x55ec4a52bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60cef76740 4 client.15435 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60cef76740 1 -- 172.21.15.49:0/709682505 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ec4a15b7b0 con 0x55ec4a529aa0 2024-02-26T06:23:47.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.023+0000 7f60ca000640 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec4a529aa0 0x55ec4a52bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.024+0000 7f60aea00640 10 client.15435 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:47.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.024+0000 7f60ca000640 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec4a529aa0 0x55ec4a52bec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87c7aea66e547e5 server_cookie=133260ea542858a1 in_seq=0 out_seq=0 2024-02-26T06:23:47.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.025+0000 7f60aea00640 1 -- 172.21.15.49:0/709682505 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec4a15b7b0 con 0x55ec4a529aa0 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.025+0000 7f60aea00640 10 client.15435 handle_command_reply: tid=0 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.025+0000 7f60cef76740 1 client.15435 shutdown 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.025+0000 7f60cef76740 1 -- 172.21.15.49:0/709682505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 msgr2=0x55ec4a4c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.025+0000 7f60cef76740 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c7a70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f60b0012c30 tx=0x7f60b0005530 comp rx=0 tx=0).stop 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60cef76740 1 -- 172.21.15.49:0/709682505 shutdown_connections 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60cef76740 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec4a529aa0 0x55ec4a52bec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60cef76740 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec4a4c1410 0x55ec4a521d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60cef76740 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec4a4b8830 0x55ec4a4c7a70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60cef76740 1 --2- 172.21.15.49:0/709682505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec4a4b7f10 0x55ec4a4c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60cef76740 1 -- 172.21.15.49:0/709682505 >> 172.21.15.49:0/709682505 conn(0x55ec4a4b2640 msgr2=0x55ec4a4b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60aea00640 0 client.15435 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60cef76740 1 -- 172.21.15.49:0/709682505 shutdown_connections 2024-02-26T06:23:47.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.026+0000 7f60cef76740 1 -- 172.21.15.49:0/709682505 wait complete. 2024-02-26T06:23:47.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.027+0000 7f60cef76740 20 client.15435 trim_cache size 0 max 16384 2024-02-26T06:23:47.027 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:23:47.027 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:23:47.027 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "fd1500c3-782d-496a-a5ce-b52a59991ada", 2024-02-26T06:23:47.027 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:23:47.027 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:23:47.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 -- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f60a80a2340 msgr2=0x7f60a80a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f60a80a2340 0x7f60a80a4800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60b40cbcd0 tx=0x7f60b40d1e50 comp rx=0 tx=0).stop 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 -- 172.21.15.49:0/3501671301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c4101360 msgr2=0x7f60c419f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c4101360 0x7f60c419f690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60b400e9f0 tx=0x7f60b400edf0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 -- 172.21.15.49:0/3501671301 shutdown_connections 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f60a80a2340 0x7f60a80a4800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60c4199850 0x7f60c4199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4106430 0x7f60c419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 --2- 172.21.15.49:0/3501671301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c4101360 0x7f60c419f690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 -- 172.21.15.49:0/3501671301 >> 172.21.15.49:0/3501671301 conn(0x7f60c40fcbe0 msgr2=0x7f60c40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 -- 172.21.15.49:0/3501671301 shutdown_connections 2024-02-26T06:23:47.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.028+0000 7f60cc000640 1 -- 172.21.15.49:0/3501671301 wait complete. 2024-02-26T06:23:47.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:23:47.057 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:23:47.121 INFO:teuthology.orchestra.run.smithi049.stderr:3100000 of 3200000 tuples (96%) done (elapsed 9.36 s, remaining 0.30 s) 2024-02-26T06:23:47.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:47 smithi049 ceph-mon[31696]: pgmap v14: 97 pgs: 97 active+clean; 253 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 39 MiB/s wr, 348 op/s 2024-02-26T06:23:47.234 INFO:teuthology.orchestra.run.smithi049.stderr:3200000 of 3200000 tuples (100%) done (elapsed 9.47 s, remaining 0.00 s) 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 -- 172.21.15.49:0/1595181441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5900ff980 msgr2=0x7fd5901102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 --2- 172.21.15.49:0/1595181441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5900ff980 0x7fd5901102e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd588009be0 tx=0x7fd58802bef0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 -- 172.21.15.49:0/1595181441 shutdown_connections 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 --2- 172.21.15.49:0/1595181441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5900ff980 0x7fd5901102e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 --2- 172.21.15.49:0/1595181441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd590109e50 0x7fd5900ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 --2- 172.21.15.49:0/1595181441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5901094a0 0x7fd590109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 -- 172.21.15.49:0/1595181441 >> 172.21.15.49:0/1595181441 conn(0x7fd5900761d0 msgr2=0x7fd5900fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 -- 172.21.15.49:0/1595181441 shutdown_connections 2024-02-26T06:23:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.317+0000 7fd595e00640 1 -- 172.21.15.49:0/1595181441 wait complete. 2024-02-26T06:23:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:vacuuming... 2024-02-26T06:23:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.318+0000 7fd595e00640 1 Processor -- start 2024-02-26T06:23:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.318+0000 7fd595e00640 1 -- start start 2024-02-26T06:23:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.318+0000 7fd595e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5900ff980 0x7fd59019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.318+0000 7fd595e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5901094a0 0x7fd59019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd595e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd590199790 0x7fd590199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd595e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59019a260 con 0x7fd5900ff980 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd595e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59019a3d0 con 0x7fd590199790 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd595e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59019a590 con 0x7fd5901094a0 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5900ff980 0x7fd59019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5900ff980 0x7fd59019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40884/0 (socket says 172.21.15.49:40884) 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58ea00640 1 -- 172.21.15.49:0/327943206 learned_addr learned my addr 172.21.15.49:0/327943206 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58f400640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd590199790 0x7fd590199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58e000640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5901094a0 0x7fd59019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58ea00640 1 -- 172.21.15.49:0/327943206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5901094a0 msgr2=0x7fd59019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58ea00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5901094a0 0x7fd59019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58ea00640 1 -- 172.21.15.49:0/327943206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd590199790 msgr2=0x7fd590199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58ea00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd590199790 0x7fd590199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58ea00640 1 -- 172.21.15.49:0/327943206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd588009840 con 0x7fd5900ff980 2024-02-26T06:23:47.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.319+0000 7fd58f400640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd590199790 0x7fd590199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.320+0000 7fd58ea00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5900ff980 0x7fd59019f680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e9f0 tx=0x7fd58000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.320+0000 7fd594a00640 1 -- 172.21.15.49:0/327943206 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58000ccc0 con 0x7fd5900ff980 2024-02-26T06:23:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.320+0000 7fd595e00640 1 -- 172.21.15.49:0/327943206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901a64e0 con 0x7fd5900ff980 2024-02-26T06:23:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.320+0000 7fd594a00640 1 -- 172.21.15.49:0/327943206 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd58000ce20 con 0x7fd5900ff980 2024-02-26T06:23:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.320+0000 7fd594a00640 1 -- 172.21.15.49:0/327943206 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580010430 con 0x7fd5900ff980 2024-02-26T06:23:47.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.320+0000 7fd595e00640 1 -- 172.21.15.49:0/327943206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5901a6a10 con 0x7fd5900ff980 2024-02-26T06:23:47.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.325+0000 7fd598da4740 1 Processor -- start 2024-02-26T06:23:47.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.325+0000 7fd598da4740 1 -- start start 2024-02-26T06:23:47.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.325+0000 7fd594a00640 1 -- 172.21.15.49:0/327943206 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7fd580003fb0 con 0x7fd5900ff980 2024-02-26T06:23:47.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.325+0000 7fd598da4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdc8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.325+0000 7fd598da4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdc9290 0x5599ecdc9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.325+0000 7fd598da4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdd2600 0x5599ecdd49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.325+0000 7fd598da4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ecd38120 con 0x5599ecdd2600 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd598da4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ecd35650 con 0x5599ecdc9290 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd598da4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ecd30e30 con 0x5599ecdc9f70 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdc8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdc8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57542/0 (socket says 172.21.15.49:57542) 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdc9290 0x5599ecdc9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58f400640 1 -- 172.21.15.49:0/2042317340 learned_addr learned my addr 172.21.15.49:0/2042317340 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58e000640 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdd2600 0x5599ecdd49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58ea00640 1 -- 172.21.15.49:0/2042317340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 msgr2=0x5599ecdc8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58ea00640 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdc8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58ea00640 1 -- 172.21.15.49:0/2042317340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdd2600 msgr2=0x5599ecdd49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58ea00640 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdd2600 0x5599ecdd49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58ea00640 1 -- 172.21.15.49:0/2042317340 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599eccea030 con 0x5599ecdc9290 2024-02-26T06:23:47.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.326+0000 7fd58f400640 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdc8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:47.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.327+0000 7fd58ea00640 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdc9290 0x5599ecdc9710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd5800cbab0 tx=0x7fd5800d66a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa328dd4316f079 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:47.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.327+0000 7fd57be00640 1 -- 172.21.15.49:0/2042317340 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5800d6ea0 con 0x5599ecdc9290 2024-02-26T06:23:47.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.327+0000 7fd57be00640 1 -- 172.21.15.49:0/2042317340 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5800d04e0 con 0x5599ecdc9290 2024-02-26T06:23:47.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.327+0000 7fd57be00640 1 -- 172.21.15.49:0/2042317340 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5800dcad0 con 0x5599ecdc9290 2024-02-26T06:23:47.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.327+0000 7fd594a00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd5740a2340 0x7fd5740a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.327+0000 7fd594a00640 1 -- 172.21.15.49:0/327943206 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd580014070 con 0x7fd5900ff980 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.327+0000 7fd58f400640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd5740a2340 0x7fd5740a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 -- 172.21.15.49:0/2042317340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdc9290 msgr2=0x5599ecdc9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdc9290 0x5599ecdc9710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd5800cbab0 tx=0x7fd5800d66a0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 -- 172.21.15.49:0/2042317340 shutdown_connections 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdc9290 0x5599ecdc9710 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdd2600 0x5599ecdd49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 --2- 172.21.15.49:0/2042317340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdc8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 -- 172.21.15.49:0/2042317340 >> 172.21.15.49:0/2042317340 conn(0x5599ecd90570 msgr2=0x5599ecdc3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 -- 172.21.15.49:0/2042317340 shutdown_connections 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd598da4740 1 -- 172.21.15.49:0/2042317340 wait complete. 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.328+0000 7fd58f400640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd5740a2340 0x7fd5740a4800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd588009330 tx=0x7fd588009280 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 1 -- start start 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 1 Processor -- start 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9290 0x5599ecdcf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599eccd5e90 con 0x5599ecdc9290 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599eccd5b30 con 0x5599ecdd2600 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599eca9f1c0 con 0x5599ecdc9f70 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.329+0000 7fd598da4740 10 client.0 _ll_register_callbacks cb 0x5599ecce9930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9290 0x5599ecdcf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49990/0 (socket says 172.21.15.49:49990) 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58f400640 1 -- 172.21.15.49:0/866605610 learned_addr learned my addr 172.21.15.49:0/866605610 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58e000640 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58ea00640 1 -- 172.21.15.49:0/866605610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 msgr2=0x5599ecdcfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58ea00640 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58ea00640 1 -- 172.21.15.49:0/866605610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 msgr2=0x5599ecdd0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58ea00640 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58ea00640 1 -- 172.21.15.49:0/866605610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5901a6a10 con 0x5599ecdc9290 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.330+0000 7fd58ea00640 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9290 0x5599ecdcf5d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd58009a9a0 tx=0x7fd580009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e613c5811fcbd86d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.331+0000 7fd578c00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.331+0000 7fd578c00640 1 -- 172.21.15.49:0/866605610 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5800d1b10 con 0x5599ecdc9290 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.331+0000 7fd578c00640 1 -- 172.21.15.49:0/866605610 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd58009b3a0 con 0x5599ecdc9290 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.331+0000 7fd598da4740 1 -- 172.21.15.49:0/866605610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599eca307e0 con 0x5599ecdc9290 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.331+0000 7fd578c00640 1 -- 172.21.15.49:0/866605610 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd580096f40 con 0x5599ecdc9290 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.331+0000 7fd598da4740 10 client.15447 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.331+0000 7fd598da4740 1 -- 172.21.15.49:0/866605610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5599ecd2f4b0 con 0x5599ecdc9290 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd578c00640 1 -- 172.21.15.49:0/866605610 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5599eca307e0 con 0x5599ecdc9290 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd578c00640 1 -- 172.21.15.49:0/866605610 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd58001f060 con 0x5599ecdc9290 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd598da4740 10 client.15447 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:47.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd598da4740 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd598da4740 20 client.15447 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd598da4740 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599ecde2470 0x5599ecde4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd598da4740 4 client.15447 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd598da4740 1 -- 172.21.15.49:0/866605610 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599eca6d720 con 0x5599ecde2470 2024-02-26T06:23:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.332+0000 7fd58e000640 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599ecde2470 0x5599ecde4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.333+0000 7fd58e000640 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599ecde2470 0x5599ecde4860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70002b71f649a015 server_cookie=46126c9f2eea116f in_seq=0 out_seq=0 2024-02-26T06:23:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.333+0000 7fd578c00640 10 client.15447 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:47.337 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:47 smithi080 ceph-mon[36206]: pgmap v14: 97 pgs: 97 active+clean; 253 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 39 MiB/s wr, 348 op/s 2024-02-26T06:23:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.333+0000 7fd578c00640 1 -- 172.21.15.49:0/866605610 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599eca6d720 con 0x5599ecde2470 2024-02-26T06:23:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.333+0000 7fd578c00640 10 client.15447 handle_command_reply: tid=0 2024-02-26T06:23:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 client.15447 shutdown 2024-02-26T06:23:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 -- 172.21.15.49:0/866605610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9290 msgr2=0x5599ecdcf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9290 0x5599ecdcf5d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd58009a9a0 tx=0x7fd580009f30 comp rx=0 tx=0).stop 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 -- 172.21.15.49:0/866605610 shutdown_connections 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599ecde2470 0x5599ecde4860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 --2- 172.21.15.49:0/866605610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9290 0x5599ecdcf5d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd578c00640 0 client.15447 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 -- 172.21.15.49:0/866605610 >> 172.21.15.49:0/866605610 conn(0x5599ecd90570 msgr2=0x5599ecdc5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 -- 172.21.15.49:0/866605610 shutdown_connections 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 1 -- 172.21.15.49:0/866605610 wait complete. 2024-02-26T06:23:47.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.334+0000 7fd598da4740 20 client.15447 trim_cache size 0 max 16384 2024-02-26T06:23:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:47 smithi115 ceph-mon[36426]: pgmap v14: 97 pgs: 97 active+clean; 253 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 39 MiB/s wr, 348 op/s 2024-02-26T06:23:47.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.351+0000 7fd598da4740 1 Processor -- start 2024-02-26T06:23:47.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd598da4740 1 -- start start 2024-02-26T06:23:47.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd598da4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd598da4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdc57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd598da4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 0x5599ecdd3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd598da4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599eca307e0 con 0x5599ecdc9f70 2024-02-26T06:23:47.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd598da4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ecdbed40 con 0x5599ecdd2600 2024-02-26T06:23:47.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd598da4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ecdcb240 con 0x5599ecdd39f0 2024-02-26T06:23:47.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd58e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdc57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd58f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 0x5599ecdd3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40918/0 (socket says 172.21.15.49:40918) 2024-02-26T06:23:47.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd58ea00640 1 -- 172.21.15.49:0/1248973915 learned_addr learned my addr 172.21.15.49:0/1248973915 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.352+0000 7fd58e000640 1 --2- 172.21.15.49:0/1248973915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdc57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50006/0 (socket says 172.21.15.49:50006) 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd58e000640 1 -- 172.21.15.49:0/1248973915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 msgr2=0x5599ecdd3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd58e000640 1 --2- 172.21.15.49:0/1248973915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 0x5599ecdd3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd58e000640 1 -- 172.21.15.49:0/1248973915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 msgr2=0x5599ecdcfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd58e000640 1 --2- 172.21.15.49:0/1248973915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd58e000640 1 -- 172.21.15.49:0/1248973915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599ecccb920 con 0x5599ecdd2600 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd58e000640 1 --2- 172.21.15.49:0/1248973915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdc57f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x5599ecde5100 tx=0x7fd57c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff13c4e4d9b6273 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd57b400640 1 -- 172.21.15.49:0/1248973915 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c017070 con 0x5599ecdd2600 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd57b400640 1 -- 172.21.15.49:0/1248973915 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd57c018040 con 0x5599ecdd2600 2024-02-26T06:23:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.353+0000 7fd57b400640 1 -- 172.21.15.49:0/1248973915 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c010720 con 0x5599ecdd2600 2024-02-26T06:23:47.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 -- 172.21.15.49:0/1248973915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 msgr2=0x5599ecdc57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 --2- 172.21.15.49:0/1248973915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdc57f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x5599ecde5100 tx=0x7fd57c005f90 comp rx=0 tx=0).stop 2024-02-26T06:23:47.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 -- 172.21.15.49:0/1248973915 shutdown_connections 2024-02-26T06:23:47.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 --2- 172.21.15.49:0/1248973915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdc57f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 --2- 172.21.15.49:0/1248973915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 0x5599ecdd3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 --2- 172.21.15.49:0/1248973915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdcfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 -- 172.21.15.49:0/1248973915 >> 172.21.15.49:0/1248973915 conn(0x5599ecd90570 msgr2=0x5599ecdc50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 -- 172.21.15.49:0/1248973915 shutdown_connections 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.354+0000 7fd598da4740 1 -- 172.21.15.49:0/1248973915 wait complete. 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.355+0000 7fd598da4740 1 -- start start 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.355+0000 7fd598da4740 1 Processor -- start 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd598da4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdd9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd598da4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd598da4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 0x5599ecdd7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd598da4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ecdd8160 con 0x5599ecdc9f70 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd598da4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ecdd82d0 con 0x5599ecdd2600 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd598da4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ece39910 con 0x5599ecdd39f0 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd598da4740 10 client.0 _ll_register_callbacks cb 0x7fd5800df070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdd9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdd9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40932/0 (socket says 172.21.15.49:40932) 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58ea00640 1 -- 172.21.15.49:0/2507251137 learned_addr learned my addr 172.21.15.49:0/2507251137 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58f400640 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 0x5599ecdd7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58e000640 1 -- 172.21.15.49:0/2507251137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 msgr2=0x5599ecdd7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58e000640 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 0x5599ecdd7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58e000640 1 -- 172.21.15.49:0/2507251137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 msgr2=0x5599ecdd9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58e000640 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdd9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58e000640 1 -- 172.21.15.49:0/2507251137 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c009ad0 con 0x5599ecdd2600 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.356+0000 7fd58ea00640 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdd9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.357+0000 7fd58e000640 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd7550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009da0 tx=0x7fd57c003d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0584838267d50db server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.357+0000 7fd578c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.357+0000 7fd578c00640 1 -- 172.21.15.49:0/2507251137 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c017070 con 0x5599ecdd2600 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.357+0000 7fd598da4740 1 -- 172.21.15.49:0/2507251137 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5800d04e0 con 0x5599ecdd2600 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.357+0000 7fd578c00640 1 -- 172.21.15.49:0/2507251137 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd57c01f460 con 0x5599ecdd2600 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.357+0000 7fd578c00640 1 -- 172.21.15.49:0/2507251137 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd57c01f620 con 0x5599ecdd2600 2024-02-26T06:23:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.357+0000 7fd598da4740 10 client.25571 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.357+0000 7fd598da4740 1 -- 172.21.15.49:0/2507251137 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5599eccf1400 con 0x5599ecdd2600 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd578c00640 1 -- 172.21.15.49:0/2507251137 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5800d04e0 con 0x5599ecdd2600 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd578c00640 1 -- 172.21.15.49:0/2507251137 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd57c026070 con 0x5599ecdd2600 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd598da4740 10 client.25571 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd598da4740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd598da4740 20 client.25571 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd598da4740 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599ece3ae20 0x5599ece3d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd598da4740 4 client.25571 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd598da4740 1 -- 172.21.15.49:0/2507251137 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599ecabc890 con 0x5599ece3ae20 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.358+0000 7fd58ea00640 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599ece3ae20 0x5599ece3d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd58ea00640 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599ece3ae20 0x5599ece3d240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bfa5d3d942dd53 server_cookie=2a39ee87d9d2d11 in_seq=0 out_seq=0 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd578c00640 10 client.25571 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd578c00640 1 -- 172.21.15.49:0/2507251137 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5599ecabc890 con 0x5599ece3ae20 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd578c00640 10 client.25571 handle_command_reply: tid=0 2024-02-26T06:23:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 client.25571 shutdown 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 -- 172.21.15.49:0/2507251137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 msgr2=0x5599ecdd7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd7550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009da0 tx=0x7fd57c003d00 comp rx=0 tx=0).stop 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 -- 172.21.15.49:0/2507251137 shutdown_connections 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599ece3ae20 0x5599ece3d240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599ecdd2600 0x5599ecdd7550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599ecdd39f0 0x5599ecdd7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 --2- 172.21.15.49:0/2507251137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599ecdc9f70 0x5599ecdd9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 -- 172.21.15.49:0/2507251137 >> 172.21.15.49:0/2507251137 conn(0x5599ecd90570 msgr2=0x5599ecdc5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd578c00640 0 client.25571 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 -- 172.21.15.49:0/2507251137 shutdown_connections 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.359+0000 7fd598da4740 1 -- 172.21.15.49:0/2507251137 wait complete. 2024-02-26T06:23:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd598da4740 20 client.25571 trim_cache size 0 max 16384 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout: "fd1500c3-782d-496a-a5ce-b52a59991ada": { 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "fd1500c3-782d-496a-a5ce-b52a59991ada", 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:23:47.364 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 -- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd5740a2340 msgr2=0x7fd5740a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd5740a2340 0x7fd5740a4800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd588009330 tx=0x7fd588009280 comp rx=0 tx=0).stop 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 -- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5900ff980 msgr2=0x7fd59019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5900ff980 0x7fd59019f680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e9f0 tx=0x7fd58000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 -- 172.21.15.49:0/327943206 shutdown_connections 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd5740a2340 0x7fd5740a4800 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd590199790 0x7fd590199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5901094a0 0x7fd59019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 --2- 172.21.15.49:0/327943206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5900ff980 0x7fd59019f680 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.360+0000 7fd595e00640 1 -- 172.21.15.49:0/327943206 >> 172.21.15.49:0/327943206 conn(0x7fd5900761d0 msgr2=0x7fd590100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.361+0000 7fd595e00640 1 -- 172.21.15.49:0/327943206 shutdown_connections 2024-02-26T06:23:47.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:23:47.361+0000 7fd595e00640 1 -- 172.21.15.49:0/327943206 wait complete. 2024-02-26T06:23:47.377 INFO:tasks.cephfs.filesystem:scrub status for tag:fd1500c3-782d-496a-a5ce-b52a59991ada - {'path': '/', 'tag': 'fd1500c3-782d-496a-a5ce-b52a59991ada', 'options': 'recursive,force'} 2024-02-26T06:23:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:48 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:23:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:48 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:48 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:23:48.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:48 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:48 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:23:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:48 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:48.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:48 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:23:48.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:48 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:48.405 INFO:teuthology.orchestra.run.smithi049.stderr:creating primary keys... 2024-02-26T06:23:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:48 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:23:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:48 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:48 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:23:48.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:48 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:23:49.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:49 smithi080 ceph-mon[36206]: pgmap v15: 97 pgs: 97 active+clean; 253 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 37 MiB/s wr, 336 op/s 2024-02-26T06:23:49.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:49 smithi115 ceph-mon[36426]: pgmap v15: 97 pgs: 97 active+clean; 253 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 37 MiB/s wr, 336 op/s 2024-02-26T06:23:49.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:49 smithi049 ceph-mon[31696]: pgmap v15: 97 pgs: 97 active+clean; 253 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 37 MiB/s wr, 336 op/s 2024-02-26T06:23:51.133 INFO:teuthology.orchestra.run.smithi049.stderr:done in 13.45 s (drop tables 0.00 s, create tables 0.03 s, client-side generate 9.61 s, vacuum 1.09 s, primary keys 2.73 s). 2024-02-26T06:23:51.141 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-02-26T06:23:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:starting vacuum...end. 2024-02-26T06:23:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:51 smithi080 ceph-mon[36206]: pgmap v16: 97 pgs: 97 active+clean; 293 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 44 MiB/s wr, 377 op/s 2024-02-26T06:23:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:51 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:23:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:51 smithi115 ceph-mon[36426]: pgmap v16: 97 pgs: 97 active+clean; 293 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 44 MiB/s wr, 377 op/s 2024-02-26T06:23:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:51 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:23:51.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:51 smithi049 ceph-mon[31696]: pgmap v16: 97 pgs: 97 active+clean; 293 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 44 MiB/s wr, 377 op/s 2024-02-26T06:23:51.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:51 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:23:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 -- 172.21.15.49:0/554022744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10fd30 msgr2=0x7f448c10da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 --2- 172.21.15.49:0/554022744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10fd30 0x7f448c10da20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f447c009900 tx=0x7f447c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 -- 172.21.15.49:0/554022744 shutdown_connections 2024-02-26T06:23:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 --2- 172.21.15.49:0/554022744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f448c11a960 0x7f448c11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 --2- 172.21.15.49:0/554022744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10fd30 0x7f448c10da20 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f447c009900 tx=0x7f447c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 --2- 172.21.15.49:0/554022744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c10f8c0 0x7f448c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 -- 172.21.15.49:0/554022744 >> 172.21.15.49:0/554022744 conn(0x7f448c06c220 msgr2=0x7f448c06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 -- 172.21.15.49:0/554022744 shutdown_connections 2024-02-26T06:23:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.832+0000 7f4493e00640 1 -- 172.21.15.49:0/554022744 wait complete. 2024-02-26T06:23:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.833+0000 7f4493e00640 1 Processor -- start 2024-02-26T06:23:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4493e00640 1 -- start start 2024-02-26T06:23:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4493e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10f8c0 0x7f448c117f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4493e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c11a960 0x7f448c110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4493e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f448c111520 0x7f448c1119d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4493e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c118740 con 0x7f448c11a960 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4493e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c115a80 con 0x7f448c111520 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4493e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c115c40 con 0x7f448c10f8c0 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4491e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f448c111520 0x7f448c1119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4491e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f448c111520 0x7f448c1119d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33184/0 (socket says 172.21.15.49:33184) 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4491e00640 1 -- 172.21.15.49:0/540354590 learned_addr learned my addr 172.21.15.49:0/540354590 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.834+0000 7f4490a00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c11a960 0x7f448c110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.835+0000 7f4491400640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10f8c0 0x7f448c117f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.835+0000 7f4491e00640 1 -- 172.21.15.49:0/540354590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10f8c0 msgr2=0x7f448c117f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.835+0000 7f4491e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10f8c0 0x7f448c117f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.835+0000 7f4491e00640 1 -- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c11a960 msgr2=0x7f448c110fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.835+0000 7f4491e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c11a960 0x7f448c110fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.835+0000 7f4491e00640 1 -- 172.21.15.49:0/540354590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480009490 con 0x7f448c111520 2024-02-26T06:23:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.835+0000 7f4491400640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10f8c0 0x7f448c117f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:23:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.836+0000 7f4491e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f448c111520 0x7f448c1119d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f448400eb40 tx=0x7f448400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.836+0000 7f448a000640 1 -- 172.21.15.49:0/540354590 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484004b20 con 0x7f448c111520 2024-02-26T06:23:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.836+0000 7f448a000640 1 -- 172.21.15.49:0/540354590 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4484004c80 con 0x7f448c111520 2024-02-26T06:23:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.837+0000 7f448a000640 1 -- 172.21.15.49:0/540354590 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840107b0 con 0x7f448c111520 2024-02-26T06:23:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.837+0000 7f4493e00640 1 -- 172.21.15.49:0/540354590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c009560 con 0x7f448c111520 2024-02-26T06:23:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.837+0000 7f4493e00640 1 -- 172.21.15.49:0/540354590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c116220 con 0x7f448c111520 2024-02-26T06:23:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.842+0000 7f4496dc5740 1 Processor -- start 2024-02-26T06:23:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.843+0000 7f448a000640 1 -- 172.21.15.49:0/540354590 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f4484010910 con 0x7f448c111520 2024-02-26T06:23:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.845+0000 7f4496dc5740 1 -- start start 2024-02-26T06:23:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.845+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ad1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.845+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 0x562896ad2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.845+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896add6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.845+0000 7f4496dc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896a8d5f0 con 0x562896adb300 2024-02-26T06:23:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.845+0000 7f4496dc5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896a3f890 con 0x562896ad1d80 2024-02-26T06:23:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.845+0000 7f4496dc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896a3cdc0 con 0x562896ad2af0 2024-02-26T06:23:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.846+0000 7f4491400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 0x562896ad2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.846+0000 7f4491400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 0x562896ad2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33192/0 (socket says 172.21.15.49:33192) 2024-02-26T06:23:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.847+0000 7f4490a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896add6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.847+0000 7f4490a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896add6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32936/0 (socket says 172.21.15.49:32936) 2024-02-26T06:23:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.847+0000 7f4490a00640 1 -- 172.21.15.49:0/3592008717 learned_addr learned my addr 172.21.15.49:0/3592008717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.847+0000 7f4490a00640 1 --2- 172.21.15.49:0/3592008717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ad1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.847+0000 7f448a000640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f44740a23f0 0x7f44740a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.848+0000 7f4490a00640 1 -- 172.21.15.49:0/3592008717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 msgr2=0x562896ad2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.848+0000 7f4490a00640 1 --2- 172.21.15.49:0/3592008717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 0x562896ad2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.848+0000 7f4490a00640 1 -- 172.21.15.49:0/3592008717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 msgr2=0x562896add6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.848+0000 7f4490a00640 1 --2- 172.21.15.49:0/3592008717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896add6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.848+0000 7f4490a00640 1 -- 172.21.15.49:0/3592008717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628969b9ad0 con 0x562896ad2af0 2024-02-26T06:23:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.848+0000 7f448a000640 1 -- 172.21.15.49:0/540354590 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4484014070 con 0x7f448c111520 2024-02-26T06:23:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.849+0000 7f4490a00640 1 --2- 172.21.15.49:0/3592008717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ad1840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f447c005ec0 tx=0x7f447c02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c08be4153e2a0310 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.849+0000 7f4491e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f44740a23f0 0x7f44740a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.850+0000 7f447a600640 1 -- 172.21.15.49:0/3592008717 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c02c890 con 0x562896ad2af0 2024-02-26T06:23:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.850+0000 7f4491e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f44740a23f0 0x7f44740a48b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f44840cc000 tx=0x7f4484004a40 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.851+0000 7f447a600640 1 -- 172.21.15.49:0/3592008717 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f447c004510 con 0x562896ad2af0 2024-02-26T06:23:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.851+0000 7f447a600640 1 -- 172.21.15.49:0/3592008717 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c02c890 con 0x562896ad2af0 2024-02-26T06:23:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.851+0000 7f4496dc5740 1 -- 172.21.15.49:0/3592008717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 msgr2=0x562896ad1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.851+0000 7f4496dc5740 1 --2- 172.21.15.49:0/3592008717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ad1840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f447c005ec0 tx=0x7f447c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:23:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.852+0000 7f4496dc5740 1 -- 172.21.15.49:0/3592008717 shutdown_connections 2024-02-26T06:23:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.852+0000 7f4496dc5740 1 --2- 172.21.15.49:0/3592008717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 0x562896ad2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.852+0000 7f4496dc5740 1 --2- 172.21.15.49:0/3592008717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896add6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.852+0000 7f4496dc5740 1 --2- 172.21.15.49:0/3592008717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ad1840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.852+0000 7f4496dc5740 1 -- 172.21.15.49:0/3592008717 >> 172.21.15.49:0/3592008717 conn(0x562896a99480 msgr2=0x562896acca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.852+0000 7f4496dc5740 1 -- 172.21.15.49:0/3592008717 shutdown_connections 2024-02-26T06:23:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.852+0000 7f4496dc5740 1 -- 172.21.15.49:0/3592008717 wait complete. 2024-02-26T06:23:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.853+0000 7f4496dc5740 1 -- start start 2024-02-26T06:23:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.853+0000 7f4496dc5740 1 Processor -- start 2024-02-26T06:23:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.854+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad1d80 0x562896ad83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.854+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad2af0 0x562896ad88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.854+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ad8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.854+0000 7f4496dc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896a385a0 con 0x562896adb300 2024-02-26T06:23:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.854+0000 7f4496dc5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628969ddb20 con 0x562896ad2af0 2024-02-26T06:23:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.854+0000 7f4496dc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628969dd7c0 con 0x562896ad1d80 2024-02-26T06:23:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.854+0000 7f4496dc5740 10 client.0 _ll_register_callbacks cb 0x5628969f1520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.855+0000 7f4491400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad2af0 0x562896ad88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.855+0000 7f4491400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad2af0 0x562896ad88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33198/0 (socket says 172.21.15.49:33198) 2024-02-26T06:23:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.855+0000 7f4491400640 1 -- 172.21.15.49:0/1865925054 learned_addr learned my addr 172.21.15.49:0/1865925054 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.855+0000 7f4491400640 1 -- 172.21.15.49:0/1865925054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad1d80 msgr2=0x562896ad83a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:23:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.855+0000 7f4491400640 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad1d80 0x562896ad83a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.855+0000 7f4491400640 1 -- 172.21.15.49:0/1865925054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 msgr2=0x562896ad8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.856+0000 7f4490a00640 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ad8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.856+0000 7f4491400640 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ad8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.856+0000 7f4491400640 1 -- 172.21.15.49:0/1865925054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c11d640 con 0x562896ad2af0 2024-02-26T06:23:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.857+0000 7f4490a00640 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ad8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:23:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.857+0000 7f446ea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:23:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.857+0000 7f4491400640 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad2af0 0x562896ad88e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4480013860 tx=0x7f4480013d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e03b9d595488e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.857+0000 7f446ea00640 1 -- 172.21.15.49:0/1865925054 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480011dd0 con 0x562896ad2af0 2024-02-26T06:23:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.857+0000 7f446ea00640 1 -- 172.21.15.49:0/1865925054 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4480005020 con 0x562896ad2af0 2024-02-26T06:23:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.858+0000 7f4496dc5740 1 -- 172.21.15.49:0/1865925054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562896a0aca0 con 0x562896ad2af0 2024-02-26T06:23:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.858+0000 7f446ea00640 1 -- 172.21.15.49:0/1865925054 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4480011dd0 con 0x562896ad2af0 2024-02-26T06:23:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.858+0000 7f4496dc5740 10 client.25583 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.858+0000 7f4496dc5740 1 -- 172.21.15.49:0/1865925054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562896a36c20 con 0x562896ad2af0 2024-02-26T06:23:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.858+0000 7f446ea00640 1 -- 172.21.15.49:0/1865925054 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44800051e0 con 0x562896ad2af0 2024-02-26T06:23:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.859+0000 7f446ea00640 1 -- 172.21.15.49:0/1865925054 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4480022050 con 0x562896ad2af0 2024-02-26T06:23:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.859+0000 7f4496dc5740 10 client.25583 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.859+0000 7f4496dc5740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.859+0000 7f4496dc5740 20 client.25583 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.859+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562896aeb340 0x562896aed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.859+0000 7f4496dc5740 4 client.25583 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:23:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.859+0000 7f4496dc5740 1 -- 172.21.15.49:0/1865925054 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562896a0aca0 con 0x562896aeb340 2024-02-26T06:23:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.860+0000 7f4490a00640 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562896aeb340 0x562896aed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.861+0000 7f446ea00640 10 client.25583 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.864+0000 7f4490a00640 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562896aeb340 0x562896aed7e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e9091c81089b00 server_cookie=1db034283fa02d50 in_seq=0 out_seq=0 2024-02-26T06:23:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.864+0000 7f446ea00640 1 -- 172.21.15.49:0/1865925054 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562896a0aca0 con 0x562896aeb340 2024-02-26T06:23:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.864+0000 7f446ea00640 10 client.25583 handle_command_reply: tid=0 2024-02-26T06:23:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.865+0000 7f4496dc5740 1 client.25583 shutdown 2024-02-26T06:23:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 -- 172.21.15.49:0/1865925054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad2af0 msgr2=0x562896ad88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad2af0 0x562896ad88e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4480013860 tx=0x7f4480013d20 comp rx=0 tx=0).stop 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 -- 172.21.15.49:0/1865925054 shutdown_connections 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562896aeb340 0x562896aed7e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad2af0 0x562896ad88e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ad8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1865925054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad1d80 0x562896ad83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f446ea00640 0 client.25583 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 -- 172.21.15.49:0/1865925054 >> 172.21.15.49:0/1865925054 conn(0x562896a99480 msgr2=0x562896ace7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 -- 172.21.15.49:0/1865925054 shutdown_connections 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 1 -- 172.21.15.49:0/1865925054 wait complete. 2024-02-26T06:23:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.866+0000 7f4496dc5740 20 client.25583 trim_cache size 0 max 16384 2024-02-26T06:23:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.888+0000 7f4496dc5740 1 Processor -- start 2024-02-26T06:23:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.888+0000 7f4496dc5740 1 -- start start 2024-02-26T06:23:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.888+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 0x562896adaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.888+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ae2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.888+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ae25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.888+0000 7f4496dc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896780e60 con 0x562896adb300 2024-02-26T06:23:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.888+0000 7f4496dc5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896abe670 con 0x562896ad1d80 2024-02-26T06:23:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.888+0000 7f4496dc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896ad81a0 con 0x562896ad2af0 2024-02-26T06:23:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4491400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ae2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4491400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ae2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:56036/0 (socket says 172.21.15.49:56036) 2024-02-26T06:23:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4491400640 1 -- 172.21.15.49:0/2856620377 learned_addr learned my addr 172.21.15.49:0/2856620377 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4491400640 1 -- 172.21.15.49:0/2856620377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 msgr2=0x562896adaad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:23:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4491400640 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 0x562896adaad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4491400640 1 -- 172.21.15.49:0/2856620377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 msgr2=0x562896ae25a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:23:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4490a00640 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ae25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4491400640 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ae25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4491400640 1 -- 172.21.15.49:0/2856620377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628969d3500 con 0x562896ad2af0 2024-02-26T06:23:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.889+0000 7f4490a00640 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ae25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:23:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.890+0000 7f4491400640 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ae2060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4480023df0 tx=0x7f4480009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6e38d7e2f321f64 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.890+0000 7f4479c00640 1 -- 172.21.15.49:0/2856620377 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480011dd0 con 0x562896ad2af0 2024-02-26T06:23:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.890+0000 7f4479c00640 1 -- 172.21.15.49:0/2856620377 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4480023380 con 0x562896ad2af0 2024-02-26T06:23:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.890+0000 7f4479c00640 1 -- 172.21.15.49:0/2856620377 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000e850 con 0x562896ad2af0 2024-02-26T06:23:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.891+0000 7f4496dc5740 1 -- 172.21.15.49:0/2856620377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 msgr2=0x562896ae2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.891+0000 7f4496dc5740 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ae2060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4480023df0 tx=0x7f4480009f70 comp rx=0 tx=0).stop 2024-02-26T06:23:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.892+0000 7f4496dc5740 1 -- 172.21.15.49:0/2856620377 shutdown_connections 2024-02-26T06:23:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.892+0000 7f4496dc5740 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896adb300 0x562896ae25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.892+0000 7f4496dc5740 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896ae2060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.892+0000 7f4496dc5740 1 --2- 172.21.15.49:0/2856620377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896ad1d80 0x562896adaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.892+0000 7f4496dc5740 1 -- 172.21.15.49:0/2856620377 >> 172.21.15.49:0/2856620377 conn(0x562896a99480 msgr2=0x562896acc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.893+0000 7f4496dc5740 1 -- 172.21.15.49:0/2856620377 shutdown_connections 2024-02-26T06:23:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.893+0000 7f4496dc5740 1 -- 172.21.15.49:0/2856620377 wait complete. 2024-02-26T06:23:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.893+0000 7f4496dc5740 1 -- start start 2024-02-26T06:23:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4496dc5740 1 Processor -- start 2024-02-26T06:23:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896ad1d80 0x562896aded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896adf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4496dc5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896adb300 0x562896b41900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4496dc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896adf9e0 con 0x562896ad1d80 2024-02-26T06:23:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4496dc5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896adfb50 con 0x562896adb300 2024-02-26T06:23:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4496dc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562896b41e40 con 0x562896ad2af0 2024-02-26T06:23:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4496dc5740 10 client.0 _ll_register_callbacks cb 0x7f44800098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.894+0000 7f4491400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896adf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.895+0000 7f4491400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896adf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:56042/0 (socket says 172.21.15.49:56042) 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.895+0000 7f4491400640 1 -- 172.21.15.49:0/1565858603 learned_addr learned my addr 172.21.15.49:0/1565858603 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.895+0000 7f4491400640 1 -- 172.21.15.49:0/1565858603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896adb300 msgr2=0x562896b41900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.895+0000 7f4491400640 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896adb300 0x562896b41900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.895+0000 7f4491400640 1 -- 172.21.15.49:0/1565858603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896ad1d80 msgr2=0x562896aded40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.895+0000 7f4490a00640 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896ad1d80 0x562896aded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.895+0000 7f4491400640 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896ad1d80 0x562896aded40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.895+0000 7f4491400640 1 -- 172.21.15.49:0/1565858603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562896a36c20 con 0x562896ad2af0 2024-02-26T06:23:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.896+0000 7f4490a00640 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896ad1d80 0x562896aded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:23:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.896+0000 7f4491400640 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896adf280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4480023e40 tx=0x7f4480013d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdebadf9a02a5a55 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.896+0000 7f446ea00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:23:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.897+0000 7f446ea00640 1 -- 172.21.15.49:0/1565858603 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480033070 con 0x562896ad2af0 2024-02-26T06:23:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.897+0000 7f446ea00640 1 -- 172.21.15.49:0/1565858603 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4480000e40 con 0x562896ad2af0 2024-02-26T06:23:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.897+0000 7f4496dc5740 1 -- 172.21.15.49:0/1565858603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562896a5dd00 con 0x562896ad2af0 2024-02-26T06:23:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.898+0000 7f446ea00640 1 -- 172.21.15.49:0/1565858603 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4480024f10 con 0x562896ad2af0 2024-02-26T06:23:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.898+0000 7f4496dc5740 10 client.25591 fetch_fsmap learned FSMap version 24 2024-02-26T06:23:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.898+0000 7f4496dc5740 1 -- 172.21.15.49:0/1565858603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5628969f9290 con 0x562896ad2af0 2024-02-26T06:23:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.898+0000 7f446ea00640 1 -- 172.21.15.49:0/1565858603 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562896a5dd00 con 0x562896ad2af0 2024-02-26T06:23:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.899+0000 7f446ea00640 1 -- 172.21.15.49:0/1565858603 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 24) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4480036020 con 0x562896ad2af0 2024-02-26T06:23:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.899+0000 7f4496dc5740 10 client.25591 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T06:23:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.899+0000 7f4496dc5740 10 client.25591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:23:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.899+0000 7f4496dc5740 20 client.25591 populate_metadata read hostname 'smithi049' 2024-02-26T06:23:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.899+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562896b43210 0x562896b45630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.899+0000 7f4496dc5740 4 client.25591 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:23:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.899+0000 7f4496dc5740 1 -- 172.21.15.49:0/1565858603 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562896a2ecb0 con 0x562896b43210 2024-02-26T06:23:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.899+0000 7f4490a00640 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562896b43210 0x562896b45630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.900+0000 7f4490a00640 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562896b43210 0x562896b45630 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc9c254aec78415 server_cookie=4d2583deff549131 in_seq=0 out_seq=0 2024-02-26T06:23:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.900+0000 7f446ea00640 10 client.25591 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.932+0000 7f446ea00640 1 -- 172.21.15.49:0/1565858603 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562896a2ecb0 con 0x562896b43210 2024-02-26T06:23:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.932+0000 7f446ea00640 10 client.25591 handle_command_reply: tid=0 2024-02-26T06:23:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.932+0000 7f4496dc5740 1 client.25591 shutdown 2024-02-26T06:23:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f4496dc5740 1 -- 172.21.15.49:0/1565858603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 msgr2=0x562896adf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896adf280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4480023e40 tx=0x7f4480013d20 comp rx=0 tx=0).stop 2024-02-26T06:23:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f4496dc5740 1 -- 172.21.15.49:0/1565858603 shutdown_connections 2024-02-26T06:23:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562896b43210 0x562896b45630 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562896adb300 0x562896b41900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562896ad2af0 0x562896adf280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f4496dc5740 1 --2- 172.21.15.49:0/1565858603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562896ad1d80 0x562896aded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f4496dc5740 1 -- 172.21.15.49:0/1565858603 >> 172.21.15.49:0/1565858603 conn(0x562896a99480 msgr2=0x562896acb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.933+0000 7f446ea00640 0 client.25591 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:23:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.934+0000 7f4496dc5740 1 -- 172.21.15.49:0/1565858603 shutdown_connections 2024-02-26T06:23:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.934+0000 7f4496dc5740 1 -- 172.21.15.49:0/1565858603 wait complete. 2024-02-26T06:23:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.934+0000 7f4496dc5740 20 client.25591 trim_cache size 0 max 16384 2024-02-26T06:23:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:23:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:23:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:23:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:23:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 -- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f44740a23f0 msgr2=0x7f44740a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f44740a23f0 0x7f44740a48b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f44840cc000 tx=0x7f4484004a40 comp rx=0 tx=0).stop 2024-02-26T06:23:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 -- 172.21.15.49:0/540354590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f448c111520 msgr2=0x7f448c1119d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f448c111520 0x7f448c1119d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f448400eb40 tx=0x7f448400c2d0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 -- 172.21.15.49:0/540354590 shutdown_connections 2024-02-26T06:23:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f44740a23f0 0x7f44740a48b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f448c111520 0x7f448c1119d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f448c11a960 0x7f448c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 --2- 172.21.15.49:0/540354590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f448c10f8c0 0x7f448c117f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.935+0000 7f4493e00640 1 -- 172.21.15.49:0/540354590 >> 172.21.15.49:0/540354590 conn(0x7f448c06c220 msgr2=0x7f448c1193d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.936+0000 7f4493e00640 1 -- 172.21.15.49:0/540354590 shutdown_connections 2024-02-26T06:23:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:23:52.936+0000 7f4493e00640 1 -- 172.21.15.49:0/540354590 wait complete. 2024-02-26T06:23:53.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:53 smithi080 ceph-mon[36206]: pgmap v17: 97 pgs: 97 active+clean; 381 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 57 MiB/s wr, 429 op/s 2024-02-26T06:23:53.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:53 smithi115 ceph-mon[36426]: pgmap v17: 97 pgs: 97 active+clean; 381 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 57 MiB/s wr, 429 op/s 2024-02-26T06:23:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:53 smithi049 ceph-mon[31696]: pgmap v17: 97 pgs: 97 active+clean; 381 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 57 MiB/s wr, 429 op/s 2024-02-26T06:23:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:55 smithi080 ceph-mon[36206]: pgmap v18: 97 pgs: 97 active+clean; 381 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 47 MiB/s wr, 298 op/s 2024-02-26T06:23:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:55 smithi115 ceph-mon[36426]: pgmap v18: 97 pgs: 97 active+clean; 381 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 47 MiB/s wr, 298 op/s 2024-02-26T06:23:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:55 smithi049 ceph-mon[31696]: pgmap v18: 97 pgs: 97 active+clean; 381 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 47 MiB/s wr, 298 op/s 2024-02-26T06:23:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 5.0 s, 2998.3 tps, lat 31.955 ms stddev 35.731 2024-02-26T06:23:57.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:57 smithi080 ceph-mon[36206]: pgmap v19: 97 pgs: 97 active+clean; 513 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 64 MiB/s wr, 391 op/s 2024-02-26T06:23:57.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:57 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:23:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:57 smithi115 ceph-mon[36426]: pgmap v19: 97 pgs: 97 active+clean; 513 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 64 MiB/s wr, 391 op/s 2024-02-26T06:23:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:57 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:23:57.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:57 smithi049 ceph-mon[31696]: pgmap v19: 97 pgs: 97 active+clean; 513 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 64 MiB/s wr, 391 op/s 2024-02-26T06:23:57.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:57 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:23:59.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:23:59 smithi080 ceph-mon[36206]: pgmap v20: 97 pgs: 97 active+clean; 513 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 38 MiB/s wr, 193 op/s 2024-02-26T06:23:59.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:23:59 smithi115 ceph-mon[36426]: pgmap v20: 97 pgs: 97 active+clean; 513 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 38 MiB/s wr, 193 op/s 2024-02-26T06:23:59.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:23:59 smithi049 ceph-mon[31696]: pgmap v20: 97 pgs: 97 active+clean; 513 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 38 MiB/s wr, 193 op/s 2024-02-26T06:24:00.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:24:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:24:00.044+0000 7f6e83200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:24:00.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:24:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:24:00.044+0000 7f6e83200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:24:00.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:24:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:24:00.044+0000 7f6e83200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:24:00.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:24:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:24:00.044+0000 7f6e83200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:24:00.419 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:24:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:24:00.044+0000 7f6e83200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:24:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 10.0 s, 2478.1 tps, lat 38.770 ms stddev 57.461 2024-02-26T06:24:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:01 smithi049 ceph-mon[31696]: pgmap v21: 97 pgs: 97 active+clean; 637 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 39 MiB/s wr, 255 op/s 2024-02-26T06:24:01.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:01 smithi080 ceph-mon[36206]: pgmap v21: 97 pgs: 97 active+clean; 637 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 39 MiB/s wr, 255 op/s 2024-02-26T06:24:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:01 smithi115 ceph-mon[36426]: pgmap v21: 97 pgs: 97 active+clean; 637 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 39 MiB/s wr, 255 op/s 2024-02-26T06:24:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:02 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:02.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:02 smithi049 ceph-mon[31696]: pgmap v22: 97 pgs: 97 active+clean; 897 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 34 MiB/s wr, 447 op/s 2024-02-26T06:24:02.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:02 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:02.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:02 smithi080 ceph-mon[36206]: pgmap v22: 97 pgs: 97 active+clean; 897 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 34 MiB/s wr, 447 op/s 2024-02-26T06:24:02.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:02 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:02.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:02 smithi115 ceph-mon[36426]: pgmap v22: 97 pgs: 97 active+clean; 897 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 34 MiB/s wr, 447 op/s 2024-02-26T06:24:03.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:03 smithi049 ceph-mon[31696]: mgrmap e26: x(active, since 38s), standbys: z, y 2024-02-26T06:24:03.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:03 smithi080 ceph-mon[36206]: mgrmap e26: x(active, since 38s), standbys: z, y 2024-02-26T06:24:03.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:03 smithi115 ceph-mon[36426]: mgrmap e26: x(active, since 38s), standbys: z, y 2024-02-26T06:24:04.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:04 smithi049 ceph-mon[31696]: pgmap v23: 97 pgs: 97 active+clean; 897 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 387 op/s 2024-02-26T06:24:04.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:04 smithi080 ceph-mon[36206]: pgmap v23: 97 pgs: 97 active+clean; 897 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 387 op/s 2024-02-26T06:24:04.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:04 smithi115 ceph-mon[36426]: pgmap v23: 97 pgs: 97 active+clean; 897 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 387 op/s 2024-02-26T06:24:05.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:05 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:05.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:05 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:05.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:05 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 -- 172.21.15.49:0/3806137716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544103f50 msgr2=0x7f6544107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 --2- 172.21.15.49:0/3806137716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544103f50 0x7f6544107f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6530009930 tx=0x7f653002bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 -- 172.21.15.49:0/3806137716 shutdown_connections 2024-02-26T06:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 --2- 172.21.15.49:0/3806137716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544103f50 0x7f6544107f90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 --2- 172.21.15.49:0/3806137716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6544108670 0x7f6544110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 --2- 172.21.15.49:0/3806137716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65441035a0 0x7f6544103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 -- 172.21.15.49:0/3806137716 >> 172.21.15.49:0/3806137716 conn(0x7f65440fee00 msgr2=0x7f6544101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 -- 172.21.15.49:0/3806137716 shutdown_connections 2024-02-26T06:24:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.109+0000 7f654b800640 1 -- 172.21.15.49:0/3806137716 wait complete. 2024-02-26T06:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.110+0000 7f654b800640 1 Processor -- start 2024-02-26T06:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f654b800640 1 -- start start 2024-02-26T06:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f654b800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65441035a0 0x7f654419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f654b800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6544108670 0x7f654419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f654b800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544199820 0x7f6544199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f654b800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654419a2d0 con 0x7f6544108670 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f654b800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654419a440 con 0x7f6544199820 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f654b800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654419a5b0 con 0x7f65441035a0 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f6548e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65441035a0 0x7f654419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f6548e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65441035a0 0x7f654419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44900/0 (socket says 172.21.15.49:44900) 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f6543e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6544108670 0x7f654419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.111+0000 7f6548e00640 1 -- 172.21.15.49:0/369378571 learned_addr learned my addr 172.21.15.49:0/369378571 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.112+0000 7f6549800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544199820 0x7f6544199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.112+0000 7f6548e00640 1 -- 172.21.15.49:0/369378571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544199820 msgr2=0x7f6544199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.112+0000 7f6548e00640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544199820 0x7f6544199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.112+0000 7f6548e00640 1 -- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6544108670 msgr2=0x7f654419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.112+0000 7f6548e00640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6544108670 0x7f654419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.112+0000 7f6548e00640 1 -- 172.21.15.49:0/369378571 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538009490 con 0x7f65441035a0 2024-02-26T06:24:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.112+0000 7f6548e00640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65441035a0 0x7f654419f660 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f653800ea60 tx=0x7f653800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.112+0000 7f6541600640 1 -- 172.21.15.49:0/369378571 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653800cde0 con 0x7f65441035a0 2024-02-26T06:24:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.113+0000 7f6541600640 1 -- 172.21.15.49:0/369378571 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65380035f0 con 0x7f65441035a0 2024-02-26T06:24:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.113+0000 7f6541600640 1 -- 172.21.15.49:0/369378571 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538010690 con 0x7f65441035a0 2024-02-26T06:24:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.113+0000 7f654b800640 1 -- 172.21.15.49:0/369378571 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6530009560 con 0x7f65441035a0 2024-02-26T06:24:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.113+0000 7f654b800640 1 -- 172.21.15.49:0/369378571 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65441a6810 con 0x7f65441035a0 2024-02-26T06:24:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.117+0000 7f654e7a8740 1 Processor -- start 2024-02-26T06:24:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f654e7a8740 1 -- start start 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f654e7a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b32835f0 con 0x55c2b32c7d80 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f654e7a8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b3235890 con 0x55c2b32d1300 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f654e7a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b3232dc0 con 0x55c2b32c8af0 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f6549800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f6549800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44906/0 (socket says 172.21.15.49:44906) 2024-02-26T06:24:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f6549800640 1 -- 172.21.15.49:0/3780191809 learned_addr learned my addr 172.21.15.49:0/3780191809 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f6541600640 1 -- 172.21.15.49:0/369378571 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f6538003ad0 con 0x7f65441035a0 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.118+0000 7f6543e00640 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6548e00640 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6549800640 1 -- 172.21.15.49:0/3780191809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 msgr2=0x55c2b32d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6549800640 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6549800640 1 -- 172.21.15.49:0/3780191809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 msgr2=0x55c2b32c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6549800640 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6549800640 1 -- 172.21.15.49:0/3780191809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2b31afad0 con 0x55c2b32c8af0 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6548e00640 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6549800640 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32c7840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f653c009f50 tx=0x7f653c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238b7756a4626a43 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6536000640 1 -- 172.21.15.49:0/3780191809 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c004d20 con 0x55c2b32c8af0 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6536000640 1 -- 172.21.15.49:0/3780191809 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f653c004e80 con 0x55c2b32c8af0 2024-02-26T06:24:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.119+0000 7f6536000640 1 -- 172.21.15.49:0/3780191809 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c005280 con 0x55c2b32c8af0 2024-02-26T06:24:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 -- 172.21.15.49:0/3780191809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 msgr2=0x55c2b32c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32c7840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f653c009f50 tx=0x7f653c0118d0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 -- 172.21.15.49:0/3780191809 shutdown_connections 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3780191809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32c7840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 -- 172.21.15.49:0/3780191809 >> 172.21.15.49:0/3780191809 conn(0x55c2b328f480 msgr2=0x55c2b32c2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f6541600640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f652c0a23f0 0x7f652c0a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 -- 172.21.15.49:0/3780191809 shutdown_connections 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f6541600640 1 -- 172.21.15.49:0/369378571 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6538014070 con 0x7f65441035a0 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.120+0000 7f654e7a8740 1 -- 172.21.15.49:0/3780191809 wait complete. 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.121+0000 7f6549800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f652c0a23f0 0x7f652c0a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.121+0000 7f654e7a8740 1 -- start start 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.121+0000 7f654e7a8740 1 Processor -- start 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.121+0000 7f6549800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f652c0a23f0 0x7f652c0a48b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f653c01b040 tx=0x7f653c000f40 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c7d80 0x55c2b32ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c8af0 0x55c2b32ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32d1300 0x55c2b32ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f654e7a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b322e5a0 con 0x55c2b32c8af0 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f654e7a8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b31d3b20 con 0x55c2b32c7d80 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f654e7a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b31d37c0 con 0x55c2b32d1300 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f654e7a8740 10 client.0 _ll_register_callbacks cb 0x55c2b31e7520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f6548e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c7d80 0x55c2b32ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f6548e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c7d80 0x55c2b32ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60634/0 (socket says 172.21.15.49:60634) 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f6548e00640 1 -- 172.21.15.49:0/3192904389 learned_addr learned my addr 172.21.15.49:0/3192904389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f6549800640 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32d1300 0x55c2b32ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f6543e00640 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c8af0 0x55c2b32ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.122+0000 7f6548e00640 1 -- 172.21.15.49:0/3192904389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32d1300 msgr2=0x55c2b32ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f6548e00640 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32d1300 0x55c2b32ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f6548e00640 1 -- 172.21.15.49:0/3192904389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c8af0 msgr2=0x55c2b32ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f6548e00640 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c8af0 0x55c2b32ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f6548e00640 1 -- 172.21.15.49:0/3192904389 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c009c00 con 0x55c2b32c7d80 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f6543e00640 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c8af0 0x55c2b32ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f6548e00640 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c7d80 0x55c2b32ce310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f653801f000 tx=0x7f653809ac30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cbf8ef57588ab42 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f652aa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f652aa00640 1 -- 172.21.15.49:0/3192904389 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380cc310 con 0x55c2b32c7d80 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f652aa00640 1 -- 172.21.15.49:0/3192904389 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f653809b000 con 0x55c2b32c7d80 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f654e7a8740 1 -- 172.21.15.49:0/3192904389 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2b3200ca0 con 0x55c2b32c7d80 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.123+0000 7f652aa00640 1 -- 172.21.15.49:0/3192904389 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65380cc470 con 0x55c2b32c7d80 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.124+0000 7f654e7a8740 10 client.25601 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.124+0000 7f654e7a8740 1 -- 172.21.15.49:0/3192904389 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c2b322cc20 con 0x55c2b32c7d80 2024-02-26T06:24:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.124+0000 7f652aa00640 1 -- 172.21.15.49:0/3192904389 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f653807d070 con 0x55c2b32c7d80 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.125+0000 7f652aa00640 1 -- 172.21.15.49:0/3192904389 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f653809a450 con 0x55c2b32c7d80 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.125+0000 7f654e7a8740 10 client.25601 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.125+0000 7f654e7a8740 10 client.25601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.125+0000 7f654e7a8740 20 client.25601 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.125+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c2b32e1310 0x55c2b32e3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.125+0000 7f654e7a8740 4 client.25601 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.125+0000 7f654e7a8740 1 -- 172.21.15.49:0/3192904389 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2b3224cb0 con 0x55c2b32e1310 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.125+0000 7f6543e00640 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c2b32e1310 0x55c2b32e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.126+0000 7f6543e00640 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c2b32e1310 0x55c2b32e3750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847e390063a9cd72 server_cookie=aa963642ad3272e4 in_seq=0 out_seq=0 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.126+0000 7f652aa00640 10 client.25601 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.126+0000 7f652aa00640 1 -- 172.21.15.49:0/3192904389 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2b3224cb0 con 0x55c2b32e1310 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.126+0000 7f652aa00640 10 client.25601 handle_command_reply: tid=0 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 client.25601 shutdown 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 -- 172.21.15.49:0/3192904389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c7d80 msgr2=0x55c2b32ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c7d80 0x55c2b32ce310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f653801f000 tx=0x7f653809ac30 comp rx=0 tx=0).stop 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 -- 172.21.15.49:0/3192904389 shutdown_connections 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c2b32e1310 0x55c2b32e3750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32d1300 0x55c2b32ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c8af0 0x55c2b32ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f652aa00640 0 client.25601 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 --2- 172.21.15.49:0/3192904389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c7d80 0x55c2b32ce310 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 -- 172.21.15.49:0/3192904389 >> 172.21.15.49:0/3192904389 conn(0x55c2b328f480 msgr2=0x55c2b32c47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 -- 172.21.15.49:0/3192904389 shutdown_connections 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 1 -- 172.21.15.49:0/3192904389 wait complete. 2024-02-26T06:24:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.127+0000 7f654e7a8740 20 client.25601 trim_cache size 0 max 16384 2024-02-26T06:24:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.148+0000 7f654e7a8740 1 Processor -- start 2024-02-26T06:24:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.148+0000 7f654e7a8740 1 -- start start 2024-02-26T06:24:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.148+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c7d80 0x55c2b32d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.148+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c8af0 0x55c2b32cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.148+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32d1300 0x55c2b32ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.148+0000 7f654e7a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b2f76e60 con 0x55c2b32d1300 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.148+0000 7f654e7a8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b32dfa90 con 0x55c2b32c8af0 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.148+0000 7f654e7a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b32c9fa0 con 0x55c2b32c7d80 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6549800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32d1300 0x55c2b32ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6543e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c8af0 0x55c2b32cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6548e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c7d80 0x55c2b32d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6543e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c8af0 0x55c2b32cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60636/0 (socket says 172.21.15.49:60636) 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6549800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32d1300 0x55c2b32ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45032/0 (socket says 172.21.15.49:45032) 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6548e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c7d80 0x55c2b32d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:44932/0 (socket says 172.21.15.49:44932) 2024-02-26T06:24:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6549800640 1 -- 172.21.15.49:0/545892230 learned_addr learned my addr 172.21.15.49:0/545892230 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6543e00640 1 -- 172.21.15.49:0/545892230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c7d80 msgr2=0x55c2b32d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6543e00640 1 --2- 172.21.15.49:0/545892230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c7d80 0x55c2b32d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6543e00640 1 -- 172.21.15.49:0/545892230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32d1300 msgr2=0x55c2b32ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6543e00640 1 --2- 172.21.15.49:0/545892230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32d1300 0x55c2b32ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6543e00640 1 -- 172.21.15.49:0/545892230 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2b31c9500 con 0x55c2b32c8af0 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6549800640 1 --2- 172.21.15.49:0/545892230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32d1300 0x55c2b32ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.149+0000 7f6543e00640 1 --2- 172.21.15.49:0/545892230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c8af0 0x55c2b32cebb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6530009230 tx=0x7f6530000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94413a90bbd58488 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.150+0000 7f6535600640 1 -- 172.21.15.49:0/545892230 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530043040 con 0x55c2b32c8af0 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.150+0000 7f6535600640 1 -- 172.21.15.49:0/545892230 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6530045040 con 0x55c2b32c8af0 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.150+0000 7f6535600640 1 -- 172.21.15.49:0/545892230 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653004a430 con 0x55c2b32c8af0 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 -- 172.21.15.49:0/545892230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c8af0 msgr2=0x55c2b32cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 --2- 172.21.15.49:0/545892230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c8af0 0x55c2b32cebb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6530009230 tx=0x7f6530000bc0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 -- 172.21.15.49:0/545892230 shutdown_connections 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 --2- 172.21.15.49:0/545892230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32c8af0 0x55c2b32cebb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 --2- 172.21.15.49:0/545892230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32d1300 0x55c2b32ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 --2- 172.21.15.49:0/545892230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c7d80 0x55c2b32d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 -- 172.21.15.49:0/545892230 >> 172.21.15.49:0/545892230 conn(0x55c2b328f480 msgr2=0x55c2b32e80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 -- 172.21.15.49:0/545892230 shutdown_connections 2024-02-26T06:24:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 -- 172.21.15.49:0/545892230 wait complete. 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.151+0000 7f654e7a8740 1 -- start start 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.152+0000 7f654e7a8740 1 Processor -- start 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.152+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32d8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.152+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32d68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.152+0000 7f654e7a8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.152+0000 7f654e7a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b32d7500 con 0x55c2b32c7d80 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.152+0000 7f654e7a8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b32d7670 con 0x55c2b32d1300 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.152+0000 7f654e7a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b33510a0 con 0x55c2b32c8af0 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.152+0000 7f654e7a8740 10 client.0 _ll_register_callbacks cb 0x7f6538004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6548e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32d8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6548e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32d8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45048/0 (socket says 172.21.15.49:45048) 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6548e00640 1 -- 172.21.15.49:0/1620149018 learned_addr learned my addr 172.21.15.49:0/1620149018 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6549800640 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6543e00640 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32d68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6548e00640 1 -- 172.21.15.49:0/1620149018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 msgr2=0x55c2b32d68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6548e00640 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32d68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6548e00640 1 -- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 msgr2=0x55c2b32d6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6548e00640 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6548e00640 1 -- 172.21.15.49:0/1620149018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2b31afad0 con 0x55c2b32c7d80 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.153+0000 7f6549800640 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.154+0000 7f6548e00640 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32d8520 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f65380a3270 tx=0x7f6538097910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a942c2ae19961f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.154+0000 7f652aa00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.154+0000 7f652aa00640 1 -- 172.21.15.49:0/1620149018 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380b94b0 con 0x55c2b32c7d80 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.154+0000 7f652aa00640 1 -- 172.21.15.49:0/1620149018 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f653809b000 con 0x55c2b32c7d80 2024-02-26T06:24:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.154+0000 7f654e7a8740 1 -- 172.21.15.49:0/1620149018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2b3253d00 con 0x55c2b32c7d80 2024-02-26T06:24:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f652aa00640 1 -- 172.21.15.49:0/1620149018 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f653809add0 con 0x55c2b32c7d80 2024-02-26T06:24:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f654e7a8740 10 client.15483 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f654e7a8740 1 -- 172.21.15.49:0/1620149018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f6544110a50 con 0x55c2b32c7d80 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f652aa00640 1 -- 172.21.15.49:0/1620149018 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c2b3253d00 con 0x55c2b32c7d80 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f652aa00640 1 -- 172.21.15.49:0/1620149018 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6538095b70 con 0x55c2b32c7d80 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f654e7a8740 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f654e7a8740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f654e7a8740 20 client.15483 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f654e7a8740 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c2b3352580 0x55c2b33549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f654e7a8740 4 client.15483 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.155+0000 7f654e7a8740 1 -- 172.21.15.49:0/1620149018 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2b2f2e600 con 0x55c2b3352580 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.156+0000 7f6543e00640 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c2b3352580 0x55c2b33549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.156+0000 7f6543e00640 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c2b3352580 0x55c2b33549a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc96e4565a9dcf5 server_cookie=f974842c3b54cc61 in_seq=0 out_seq=0 2024-02-26T06:24:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.156+0000 7f652aa00640 10 client.15483 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.170+0000 7f652aa00640 1 -- 172.21.15.49:0/1620149018 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2b2f2e600 con 0x55c2b3352580 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.170+0000 7f652aa00640 10 client.15483 handle_command_reply: tid=0 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 client.15483 shutdown 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 -- 172.21.15.49:0/1620149018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 msgr2=0x55c2b32d8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32d8520 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f65380a3270 tx=0x7f6538097910 comp rx=0 tx=0).stop 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 -- 172.21.15.49:0/1620149018 shutdown_connections 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c2b3352580 0x55c2b33549a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c2b32d1300 0x55c2b32d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2b32c8af0 0x55c2b32d68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f652aa00640 0 client.15483 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 --2- 172.21.15.49:0/1620149018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2b32c7d80 0x55c2b32d8520 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 -- 172.21.15.49:0/1620149018 >> 172.21.15.49:0/1620149018 conn(0x55c2b328f480 msgr2=0x55c2b32ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 -- 172.21.15.49:0/1620149018 shutdown_connections 2024-02-26T06:24:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.171+0000 7f654e7a8740 1 -- 172.21.15.49:0/1620149018 wait complete. 2024-02-26T06:24:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.172+0000 7f654e7a8740 20 client.15483 trim_cache size 0 max 16384 2024-02-26T06:24:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:24:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:24:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:24:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:24:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.172+0000 7f654b800640 1 -- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f652c0a23f0 msgr2=0x7f652c0a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.172+0000 7f654b800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f652c0a23f0 0x7f652c0a48b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f653c01b040 tx=0x7f653c000f40 comp rx=0 tx=0).stop 2024-02-26T06:24:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.172+0000 7f654b800640 1 -- 172.21.15.49:0/369378571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65441035a0 msgr2=0x7f654419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.172+0000 7f654b800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65441035a0 0x7f654419f660 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f653800ea60 tx=0x7f653800ef20 comp rx=0 tx=0).stop 2024-02-26T06:24:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.173+0000 7f654b800640 1 -- 172.21.15.49:0/369378571 shutdown_connections 2024-02-26T06:24:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.173+0000 7f654b800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f652c0a23f0 0x7f652c0a48b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.173+0000 7f654b800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544199820 0x7f6544199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.173+0000 7f654b800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6544108670 0x7f654419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.173+0000 7f654b800640 1 --2- 172.21.15.49:0/369378571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65441035a0 0x7f654419f660 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.173+0000 7f654b800640 1 -- 172.21.15.49:0/369378571 >> 172.21.15.49:0/369378571 conn(0x7f65440fee00 msgr2=0x7f65441010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.173+0000 7f654b800640 1 -- 172.21.15.49:0/369378571 shutdown_connections 2024-02-26T06:24:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:06.173+0000 7f654b800640 1 -- 172.21.15.49:0/369378571 wait complete. 2024-02-26T06:24:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 15.0 s, 216.6 tps, lat 465.939 ms stddev 266.654 2024-02-26T06:24:06.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:06 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:06.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:06 smithi049 ceph-mon[31696]: pgmap v24: 97 pgs: 97 active+clean; 906 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 22 MiB/s wr, 637 op/s 2024-02-26T06:24:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:06 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:06.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:06 smithi080 ceph-mon[36206]: pgmap v24: 97 pgs: 97 active+clean; 906 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 22 MiB/s wr, 637 op/s 2024-02-26T06:24:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:06 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:06.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:06 smithi115 ceph-mon[36426]: pgmap v24: 97 pgs: 97 active+clean; 906 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 22 MiB/s wr, 637 op/s 2024-02-26T06:24:08.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:08 smithi049 ceph-mon[31696]: pgmap v25: 97 pgs: 97 active+clean; 906 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.4 MiB/s wr, 544 op/s 2024-02-26T06:24:08.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:08 smithi080 ceph-mon[36206]: pgmap v25: 97 pgs: 97 active+clean; 906 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.4 MiB/s wr, 544 op/s 2024-02-26T06:24:08.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:08 smithi115 ceph-mon[36426]: pgmap v25: 97 pgs: 97 active+clean; 906 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.4 MiB/s wr, 544 op/s 2024-02-26T06:24:10.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:10 smithi049 ceph-mon[31696]: pgmap v26: 97 pgs: 97 active+clean; 912 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.1 MiB/s wr, 704 op/s 2024-02-26T06:24:10.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:10 smithi080 ceph-mon[36206]: pgmap v26: 97 pgs: 97 active+clean; 912 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.1 MiB/s wr, 704 op/s 2024-02-26T06:24:10.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:10 smithi115 ceph-mon[36426]: pgmap v26: 97 pgs: 97 active+clean; 912 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.1 MiB/s wr, 704 op/s 2024-02-26T06:24:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 20.0 s, 348.6 tps, lat 289.427 ms stddev 352.805 2024-02-26T06:24:12.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:12 smithi049 ceph-mon[31696]: pgmap v27: 97 pgs: 97 active+clean; 912 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.0 MiB/s wr, 697 op/s 2024-02-26T06:24:12.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:12 smithi080 ceph-mon[36206]: pgmap v27: 97 pgs: 97 active+clean; 912 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.0 MiB/s wr, 697 op/s 2024-02-26T06:24:12.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:12 smithi115 ceph-mon[36426]: pgmap v27: 97 pgs: 97 active+clean; 912 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.0 MiB/s wr, 697 op/s 2024-02-26T06:24:14.678 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:14 smithi049 ceph-mon[31696]: pgmap v28: 97 pgs: 97 active+clean; 912 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.4 MiB/s wr, 464 op/s 2024-02-26T06:24:14.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:14 smithi080 ceph-mon[36206]: pgmap v28: 97 pgs: 97 active+clean; 912 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.4 MiB/s wr, 464 op/s 2024-02-26T06:24:14.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:14 smithi115 ceph-mon[36426]: pgmap v28: 97 pgs: 97 active+clean; 912 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.4 MiB/s wr, 464 op/s 2024-02-26T06:24:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 25.0 s, 3041.2 tps, lat 33.675 ms stddev 38.676 2024-02-26T06:24:16.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:16 smithi049 ceph-mon[31696]: pgmap v29: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 41 MiB/s wr, 1.32k op/s 2024-02-26T06:24:16.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:16 smithi080 ceph-mon[36206]: pgmap v29: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 41 MiB/s wr, 1.32k op/s 2024-02-26T06:24:16.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:16 smithi115 ceph-mon[36426]: pgmap v29: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 41 MiB/s wr, 1.32k op/s 2024-02-26T06:24:17.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:24:17.379 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:24:17.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.679+0000 7f14e0800640 1 -- 172.21.15.49:0/2170124697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81083f0 msgr2=0x7f14d81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.679+0000 7f14e0800640 1 --2- 172.21.15.49:0/2170124697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81083f0 0x7f14d81087d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0085a0 tx=0x7f14cc00ac00 comp rx=0 tx=0).stop 2024-02-26T06:24:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.679+0000 7f14e0800640 1 -- 172.21.15.49:0/2170124697 shutdown_connections 2024-02-26T06:24:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.679+0000 7f14e0800640 1 --2- 172.21.15.49:0/2170124697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14d810cf00 0x7f14d8114a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.679+0000 7f14e0800640 1 --2- 172.21.15.49:0/2170124697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14d8108da0 0x7f14d810c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.679+0000 7f14e0800640 1 --2- 172.21.15.49:0/2170124697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81083f0 0x7f14d81087d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.679+0000 7f14e0800640 1 -- 172.21.15.49:0/2170124697 >> 172.21.15.49:0/2170124697 conn(0x7f14d806c6c0 msgr2=0x7f14d806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.680+0000 7f14e0800640 1 -- 172.21.15.49:0/2170124697 shutdown_connections 2024-02-26T06:24:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.680+0000 7f14e0800640 1 -- 172.21.15.49:0/2170124697 wait complete. 2024-02-26T06:24:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.681+0000 7f14e0800640 1 Processor -- start 2024-02-26T06:24:17.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.681+0000 7f14e0800640 1 -- start start 2024-02-26T06:24:17.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.682+0000 7f14e0800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14d81083f0 0x7f14d819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14e0800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14d810cf00 0x7f14d819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14e0800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81a26b0 0x7f14d81a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14e0800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81a30a0 con 0x7f14d81083f0 2024-02-26T06:24:17.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14e0800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81a3210 con 0x7f14d81a26b0 2024-02-26T06:24:17.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14e0800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81a3380 con 0x7f14d810cf00 2024-02-26T06:24:17.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14dd400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14d810cf00 0x7f14d819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14dd400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14d810cf00 0x7f14d819e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40798/0 (socket says 172.21.15.49:40798) 2024-02-26T06:24:17.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14dd400640 1 -- 172.21.15.49:0/2843334148 learned_addr learned my addr 172.21.15.49:0/2843334148 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:17.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.683+0000 7f14dde00640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14d81083f0 0x7f14d819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.684+0000 7f14de800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81a26b0 0x7f14d81a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.684+0000 7f14dde00640 1 -- 172.21.15.49:0/2843334148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14d810cf00 msgr2=0x7f14d819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.684+0000 7f14dde00640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14d810cf00 0x7f14d819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.684+0000 7f14dde00640 1 -- 172.21.15.49:0/2843334148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81a26b0 msgr2=0x7f14d81a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.684+0000 7f14dde00640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81a26b0 0x7f14d81a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.684+0000 7f14dde00640 1 -- 172.21.15.49:0/2843334148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0081d0 con 0x7f14d81083f0 2024-02-26T06:24:17.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.684+0000 7f14de800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81a26b0 0x7f14d81a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:17.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.684+0000 7f14dd400640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14d810cf00 0x7f14d819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:17.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.686+0000 7f14dde00640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14d81083f0 0x7f14d819ddf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14cc008570 tx=0x7f14cc0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.687+0000 7f14d2a00640 1 -- 172.21.15.49:0/2843334148 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00a9c0 con 0x7f14d81083f0 2024-02-26T06:24:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.687+0000 7f14d2a00640 1 -- 172.21.15.49:0/2843334148 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14cc005300 con 0x7f14d81083f0 2024-02-26T06:24:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.687+0000 7f14e0800640 1 -- 172.21.15.49:0/2843334148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d81aed70 con 0x7f14d81083f0 2024-02-26T06:24:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.687+0000 7f14d2a00640 1 -- 172.21.15.49:0/2843334148 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc003850 con 0x7f14d81083f0 2024-02-26T06:24:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.688+0000 7f14e0800640 1 -- 172.21.15.49:0/2843334148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d81af190 con 0x7f14d81083f0 2024-02-26T06:24:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.693+0000 7f14d2a00640 1 -- 172.21.15.49:0/2843334148 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f14cc0039b0 con 0x7f14d81083f0 2024-02-26T06:24:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.696+0000 7f14e3783740 1 Processor -- start 2024-02-26T06:24:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.696+0000 7f14e3783740 1 -- start start 2024-02-26T06:24:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.697+0000 7f14e3783740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.697+0000 7f14e3783740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05caa0 0x561cac05cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.697+0000 7f14e3783740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 0x561cac068200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.697+0000 7f14e3783740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cabfc3120 con 0x561cac05d780 2024-02-26T06:24:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.697+0000 7f14e3783740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cabfc0650 con 0x561cac065e10 2024-02-26T06:24:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.697+0000 7f14e3783740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cabfbbe30 con 0x561cac05caa0 2024-02-26T06:24:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.698+0000 7f14de800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.698+0000 7f14de800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac05c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44278/0 (socket says 172.21.15.49:44278) 2024-02-26T06:24:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.698+0000 7f14de800640 1 -- 172.21.15.49:0/1286264261 learned_addr learned my addr 172.21.15.49:0/1286264261 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.698+0000 7f14d2a00640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14c00a2560 0x7f14c00a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.698+0000 7f14d2a00640 1 -- 172.21.15.49:0/2843334148 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f14cc0f0070 con 0x7f14d81083f0 2024-02-26T06:24:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.699+0000 7f14dde00640 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05caa0 0x561cac05cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.699+0000 7f14de800640 1 -- 172.21.15.49:0/1286264261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05caa0 msgr2=0x561cac05cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.699+0000 7f14de800640 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05caa0 0x561cac05cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.699+0000 7f14de800640 1 -- 172.21.15.49:0/1286264261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 msgr2=0x561cac068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.699+0000 7f14de800640 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 0x561cac068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.699+0000 7f14de800640 1 -- 172.21.15.49:0/1286264261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cabf75030 con 0x561cac05d780 2024-02-26T06:24:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.699+0000 7f14de800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14c00a2560 0x7f14c00a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.699+0000 7f14dde00640 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05caa0 0x561cac05cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.700+0000 7f14de800640 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac05c4d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f14d400f3e0 tx=0x7f14d400f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6010427196b4f417 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.701+0000 7f14c7000640 1 -- 172.21.15.49:0/1286264261 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40125f0 con 0x561cac05d780 2024-02-26T06:24:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.701+0000 7f14de800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14c00a2560 0x7f14c00a4a20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14d401e000 tx=0x7f14d401c000 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.701+0000 7f14c7000640 1 -- 172.21.15.49:0/1286264261 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14d401cde0 con 0x561cac05d780 2024-02-26T06:24:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.701+0000 7f14c7000640 1 -- 172.21.15.49:0/1286264261 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400d350 con 0x561cac05d780 2024-02-26T06:24:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.702+0000 7f14e3783740 1 -- 172.21.15.49:0/1286264261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 msgr2=0x561cac05c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.702+0000 7f14e3783740 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac05c4d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f14d400f3e0 tx=0x7f14d400f7e0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.702+0000 7f14e3783740 1 -- 172.21.15.49:0/1286264261 shutdown_connections 2024-02-26T06:24:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.702+0000 7f14e3783740 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 0x561cac068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.702+0000 7f14e3783740 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05caa0 0x561cac05cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.702+0000 7f14e3783740 1 --2- 172.21.15.49:0/1286264261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac05c4d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.702+0000 7f14e3783740 1 -- 172.21.15.49:0/1286264261 >> 172.21.15.49:0/1286264261 conn(0x561cac01b570 msgr2=0x561cac0574f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.703+0000 7f14e3783740 1 -- 172.21.15.49:0/1286264261 shutdown_connections 2024-02-26T06:24:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.703+0000 7f14e3783740 1 -- 172.21.15.49:0/1286264261 wait complete. 2024-02-26T06:24:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.704+0000 7f14e3783740 1 -- start start 2024-02-26T06:24:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.704+0000 7f14e3783740 1 Processor -- start 2024-02-26T06:24:17.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.705+0000 7f14e3783740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 0x561cac062e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.705+0000 7f14e3783740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac0633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.705+0000 7f14e3783740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac0638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.705+0000 7f14e3783740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cabf60e90 con 0x561cac05d780 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.705+0000 7f14de800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac0638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14e3783740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cabf60b30 con 0x561cac05caa0 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14e3783740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cabd2a1c0 con 0x561cac065e10 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14de800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac0638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40828/0 (socket says 172.21.15.49:40828) 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14de800640 1 -- 172.21.15.49:0/1824160487 learned_addr learned my addr 172.21.15.49:0/1824160487 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14dde00640 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 0x561cac062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14e3783740 10 client.0 _ll_register_callbacks cb 0x561cabf74930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14de800640 1 -- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 msgr2=0x561cac062e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14de800640 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 0x561cac062e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14de800640 1 -- 172.21.15.49:0/1824160487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 msgr2=0x561cac0633a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14de800640 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac0633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14de800640 1 -- 172.21.15.49:0/1824160487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d4009c10 con 0x561cac065e10 2024-02-26T06:24:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.706+0000 7f14dde00640 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 0x561cac062e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.707+0000 7f14de800640 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac0638e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f14d401bd60 tx=0x7f14d401bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3761b3290121d7e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.707+0000 7f14bb400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.707+0000 7f14bb400640 1 -- 172.21.15.49:0/1824160487 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d401b850 con 0x561cac065e10 2024-02-26T06:24:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.707+0000 7f14e3783740 1 -- 172.21.15.49:0/1824160487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cabcbb7e0 con 0x561cac065e10 2024-02-26T06:24:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.708+0000 7f14bb400640 1 -- 172.21.15.49:0/1824160487 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f14d4018030 con 0x561cac065e10 2024-02-26T06:24:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.708+0000 7f14bb400640 1 -- 172.21.15.49:0/1824160487 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d40125f0 con 0x561cac065e10 2024-02-26T06:24:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.708+0000 7f14e3783740 10 client.25624 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.708+0000 7f14e3783740 1 -- 172.21.15.49:0/1824160487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561cabfba4b0 con 0x561cac065e10 2024-02-26T06:24:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.709+0000 7f14bb400640 1 -- 172.21.15.49:0/1824160487 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561cabcbb7e0 con 0x561cac065e10 2024-02-26T06:24:17.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.709+0000 7f14bb400640 1 -- 172.21.15.49:0/1824160487 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f14d4035070 con 0x561cac065e10 2024-02-26T06:24:17.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.709+0000 7f14e3783740 10 client.25624 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:17.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.709+0000 7f14e3783740 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:17.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.709+0000 7f14e3783740 20 client.25624 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:17.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.709+0000 7f14e3783740 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561cac075c60 0x561cac0780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.709+0000 7f14e3783740 4 client.25624 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:17.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.709+0000 7f14e3783740 1 -- 172.21.15.49:0/1824160487 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cabcf8720 con 0x561cac075c60 2024-02-26T06:24:17.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.711+0000 7f14dde00640 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561cac075c60 0x561cac0780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.712+0000 7f14dde00640 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561cac075c60 0x561cac0780a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a06caf11fd0ab12 server_cookie=9223a9d77e3ac03d in_seq=0 out_seq=0 2024-02-26T06:24:17.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.712+0000 7f14bb400640 10 client.25624 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:17.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.714+0000 7f14bb400640 1 -- 172.21.15.49:0/1824160487 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cabcf8720 con 0x561cac075c60 2024-02-26T06:24:17.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.714+0000 7f14bb400640 10 client.25624 handle_command_reply: tid=0 2024-02-26T06:24:17.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.714+0000 7f14e3783740 1 client.25624 shutdown 2024-02-26T06:24:17.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14e3783740 1 -- 172.21.15.49:0/1824160487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 msgr2=0x561cac0638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14e3783740 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac0638e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f14d401bd60 tx=0x7f14d401bd90 comp rx=0 tx=0).stop 2024-02-26T06:24:17.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14e3783740 1 -- 172.21.15.49:0/1824160487 shutdown_connections 2024-02-26T06:24:17.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14e3783740 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561cac075c60 0x561cac0780a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14e3783740 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac0638e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14e3783740 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac0633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14e3783740 1 --2- 172.21.15.49:0/1824160487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 0x561cac062e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14e3783740 1 -- 172.21.15.49:0/1824160487 >> 172.21.15.49:0/1824160487 conn(0x561cac01b570 msgr2=0x561cac059240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:17.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.715+0000 7f14bb400640 0 client.25624 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:17.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.716+0000 7f14e3783740 1 -- 172.21.15.49:0/1824160487 shutdown_connections 2024-02-26T06:24:17.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.716+0000 7f14e3783740 1 -- 172.21.15.49:0/1824160487 wait complete. 2024-02-26T06:24:17.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.717+0000 7f14e3783740 20 client.25624 trim_cache size 0 max 16384 2024-02-26T06:24:17.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.747+0000 7f14e3783740 1 Processor -- start 2024-02-26T06:24:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.747+0000 7f14e3783740 1 -- start start 2024-02-26T06:24:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.747+0000 7f14e3783740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05caa0 0x561cac065590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14e3783740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05d780 0x561cac06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14e3783740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 0x561cac06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14e3783740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cabcbb7e0 con 0x561cac05caa0 2024-02-26T06:24:17.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14e3783740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cac05bd20 con 0x561cac065e10 2024-02-26T06:24:17.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14e3783740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cac072b10 con 0x561cac05d780 2024-02-26T06:24:17.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14dd400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05d780 0x561cac06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14dd400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05d780 0x561cac06cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40844/0 (socket says 172.21.15.49:40844) 2024-02-26T06:24:17.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14dd400640 1 -- 172.21.15.49:0/2155801029 learned_addr learned my addr 172.21.15.49:0/2155801029 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14de800640 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 0x561cac06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.748+0000 7f14dde00640 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05caa0 0x561cac065590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.749+0000 7f14de800640 1 -- 172.21.15.49:0/2155801029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05d780 msgr2=0x561cac06cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.749+0000 7f14de800640 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05d780 0x561cac06cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.749+0000 7f14de800640 1 -- 172.21.15.49:0/2155801029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05caa0 msgr2=0x561cac065590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.749+0000 7f14de800640 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05caa0 0x561cac065590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.749+0000 7f14de800640 1 -- 172.21.15.49:0/2155801029 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cabf56920 con 0x561cac065e10 2024-02-26T06:24:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.749+0000 7f14de800640 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 0x561cac06d2b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f14d401bb00 tx=0x7f14d400f7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=435730a7951b1297 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.749+0000 7f14c6600640 1 -- 172.21.15.49:0/2155801029 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400d890 con 0x561cac065e10 2024-02-26T06:24:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.750+0000 7f14c6600640 1 -- 172.21.15.49:0/2155801029 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14d40099f0 con 0x561cac065e10 2024-02-26T06:24:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.750+0000 7f14c6600640 1 -- 172.21.15.49:0/2155801029 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4004c00 con 0x561cac065e10 2024-02-26T06:24:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 -- 172.21.15.49:0/2155801029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 msgr2=0x561cac06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 0x561cac06d2b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f14d401bb00 tx=0x7f14d400f7e0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 -- 172.21.15.49:0/2155801029 shutdown_connections 2024-02-26T06:24:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac065e10 0x561cac06d2b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac05d780 0x561cac06cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 --2- 172.21.15.49:0/2155801029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05caa0 0x561cac065590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 -- 172.21.15.49:0/2155801029 >> 172.21.15.49:0/2155801029 conn(0x561cac01b570 msgr2=0x561cac057020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 -- 172.21.15.49:0/2155801029 shutdown_connections 2024-02-26T06:24:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 -- 172.21.15.49:0/2155801029 wait complete. 2024-02-26T06:24:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.751+0000 7f14e3783740 1 -- start start 2024-02-26T06:24:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.752+0000 7f14e3783740 1 Processor -- start 2024-02-26T06:24:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.752+0000 7f14e3783740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 0x561cac06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.752+0000 7f14e3783740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac069310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.752+0000 7f14e3783740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.752+0000 7f14e3783740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cac069dc0 con 0x561cac05d780 2024-02-26T06:24:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.752+0000 7f14e3783740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cac069f30 con 0x561cac05caa0 2024-02-26T06:24:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.752+0000 7f14e3783740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cac06a0f0 con 0x561cac065e10 2024-02-26T06:24:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14e3783740 10 client.0 _ll_register_callbacks cb 0x7f14d401f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14de800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14de800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40850/0 (socket says 172.21.15.49:40850) 2024-02-26T06:24:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14de800640 1 -- 172.21.15.49:0/1132700257 learned_addr learned my addr 172.21.15.49:0/1132700257 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14de800640 1 -- 172.21.15.49:0/1132700257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 msgr2=0x561cac06cf90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14de800640 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 0x561cac06cf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14de800640 1 -- 172.21.15.49:0/1132700257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 msgr2=0x561cac069310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14de800640 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.753+0000 7f14de800640 1 -- 172.21.15.49:0/1132700257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cabfba4b0 con 0x561cac065e10 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.754+0000 7f14de800640 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac069850 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f14d400dac0 tx=0x7f14d401bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=878ed199c48c7976 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.754+0000 7f14bb400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.754+0000 7f14bb400640 1 -- 172.21.15.49:0/1132700257 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400deb0 con 0x561cac065e10 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.754+0000 7f14bb400640 1 -- 172.21.15.49:0/1132700257 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f14d4018040 con 0x561cac065e10 2024-02-26T06:24:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.755+0000 7f14e3783740 1 -- 172.21.15.49:0/1132700257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14d401cde0 con 0x561cac065e10 2024-02-26T06:24:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.755+0000 7f14bb400640 1 -- 172.21.15.49:0/1132700257 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d401b7b0 con 0x561cac065e10 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.755+0000 7f14e3783740 10 client.25630 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.755+0000 7f14e3783740 1 -- 172.21.15.49:0/1132700257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561cabf7c400 con 0x561cac065e10 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.755+0000 7f14bb400640 1 -- 172.21.15.49:0/1132700257 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14d401cde0 con 0x561cac065e10 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.755+0000 7f14bb400640 1 -- 172.21.15.49:0/1132700257 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f14d403d070 con 0x561cac065e10 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.756+0000 7f14e3783740 10 client.25630 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.756+0000 7f14e3783740 10 client.25630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.756+0000 7f14e3783740 20 client.25630 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.756+0000 7f14e3783740 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561cac0cd470 0x561cac0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.756+0000 7f14e3783740 4 client.25630 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.756+0000 7f14e3783740 1 -- 172.21.15.49:0/1132700257 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561cabd47890 con 0x561cac0cd470 2024-02-26T06:24:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.756+0000 7f14dde00640 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561cac0cd470 0x561cac0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.758+0000 7f14dde00640 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561cac0cd470 0x561cac0cf890 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2dc05c92a923b58 server_cookie=d0539255fc0db50a in_seq=0 out_seq=0 2024-02-26T06:24:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.758+0000 7f14bb400640 10 client.25630 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.758+0000 7f14bb400640 1 -- 172.21.15.49:0/1132700257 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561cabd47890 con 0x561cac0cd470 2024-02-26T06:24:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.758+0000 7f14bb400640 10 client.25630 handle_command_reply: tid=0 2024-02-26T06:24:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.758+0000 7f14e3783740 1 client.25630 shutdown 2024-02-26T06:24:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14e3783740 1 -- 172.21.15.49:0/1132700257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 msgr2=0x561cac069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14e3783740 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac069850 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f14d400dac0 tx=0x7f14d401bd90 comp rx=0 tx=0).stop 2024-02-26T06:24:17.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14e3783740 1 -- 172.21.15.49:0/1132700257 shutdown_connections 2024-02-26T06:24:17.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14e3783740 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561cac0cd470 0x561cac0cf890 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14e3783740 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cac065e10 0x561cac069850 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14e3783740 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cac05d780 0x561cac069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14e3783740 1 --2- 172.21.15.49:0/1132700257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561cac05caa0 0x561cac06cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14e3783740 1 -- 172.21.15.49:0/1132700257 >> 172.21.15.49:0/1132700257 conn(0x561cac01b570 msgr2=0x561cac061620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.759+0000 7f14bb400640 0 client.25630 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.760+0000 7f14e3783740 1 -- 172.21.15.49:0/1132700257 shutdown_connections 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.760+0000 7f14e3783740 1 -- 172.21.15.49:0/1132700257 wait complete. 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.760+0000 7f14e3783740 20 client.25630 trim_cache size 0 max 16384 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:24:17.761 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:24:17.762 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:24:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 -- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14c00a2560 msgr2=0x7f14c00a4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14c00a2560 0x7f14c00a4a20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14d401e000 tx=0x7f14d401c000 comp rx=0 tx=0).stop 2024-02-26T06:24:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 -- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14d81083f0 msgr2=0x7f14d819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14d81083f0 0x7f14d819ddf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14cc008570 tx=0x7f14cc0043d0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 -- 172.21.15.49:0/2843334148 shutdown_connections 2024-02-26T06:24:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14c00a2560 0x7f14c00a4a20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14d81a26b0 0x7f14d81a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14d810cf00 0x7f14d819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 --2- 172.21.15.49:0/2843334148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14d81083f0 0x7f14d819ddf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.761+0000 7f14e0800640 1 -- 172.21.15.49:0/2843334148 >> 172.21.15.49:0/2843334148 conn(0x7f14d806c6c0 msgr2=0x7f14d8070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.762+0000 7f14e0800640 1 -- 172.21.15.49:0/2843334148 shutdown_connections 2024-02-26T06:24:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:17.763+0000 7f14e0800640 1 -- 172.21.15.49:0/2843334148 wait complete. 2024-02-26T06:24:17.789 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:24:17.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.070+0000 7f5434a00640 1 -- 172.21.15.49:0/823894825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430100d20 msgr2=0x7f543010ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.070+0000 7f5434a00640 1 --2- 172.21.15.49:0/823894825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430100d20 0x7f543010ceb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5420009930 tx=0x7f542002bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.071+0000 7f5434a00640 1 -- 172.21.15.49:0/823894825 shutdown_connections 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.071+0000 7f5434a00640 1 --2- 172.21.15.49:0/823894825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430100d20 0x7f543010ceb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.071+0000 7f5434a00640 1 --2- 172.21.15.49:0/823894825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5430100360 0x7f54301007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.071+0000 7f5434a00640 1 --2- 172.21.15.49:0/823894825 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5430106130 0x7f5430106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.071+0000 7f5434a00640 1 -- 172.21.15.49:0/823894825 >> 172.21.15.49:0/823894825 conn(0x7f5430077060 msgr2=0x7f5430077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.071+0000 7f5434a00640 1 -- 172.21.15.49:0/823894825 shutdown_connections 2024-02-26T06:24:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.071+0000 7f5434a00640 1 -- 172.21.15.49:0/823894825 wait complete. 2024-02-26T06:24:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.074+0000 7f5434a00640 1 Processor -- start 2024-02-26T06:24:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.074+0000 7f5434a00640 1 -- start start 2024-02-26T06:24:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.075+0000 7f5434a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5430100360 0x7f5430101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.075+0000 7f5434a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5430100d20 0x7f5430102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.075+0000 7f5434a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430106130 0x7f5430102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.075+0000 7f5434a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54301a5ae0 con 0x7f5430100360 2024-02-26T06:24:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.075+0000 7f5434a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54301a5c50 con 0x7f5430100d20 2024-02-26T06:24:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.075+0000 7f5434a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54301a5e10 con 0x7f5430106130 2024-02-26T06:24:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.075+0000 7f542de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5430100360 0x7f5430101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430106130 0x7f5430102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430106130 0x7f5430102760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40874/0 (socket says 172.21.15.49:40874) 2024-02-26T06:24:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542e800640 1 -- 172.21.15.49:0/4173291291 learned_addr learned my addr 172.21.15.49:0/4173291291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542d400640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5430100d20 0x7f5430102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542de00640 1 -- 172.21.15.49:0/4173291291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430106130 msgr2=0x7f5430102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542de00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430106130 0x7f5430102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542de00640 1 -- 172.21.15.49:0/4173291291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5430100d20 msgr2=0x7f5430102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542de00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5430100d20 0x7f5430102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.076+0000 7f542de00640 1 -- 172.21.15.49:0/4173291291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5420009560 con 0x7f5430100360 2024-02-26T06:24:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.077+0000 7f542de00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5430100360 0x7f5430101ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f542800e9f0 tx=0x7f542800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:18.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.078+0000 7f5426a00640 1 -- 172.21.15.49:0/4173291291 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800ccc0 con 0x7f5430100360 2024-02-26T06:24:18.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.078+0000 7f5426a00640 1 -- 172.21.15.49:0/4173291291 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f542800ce20 con 0x7f5430100360 2024-02-26T06:24:18.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.078+0000 7f5426a00640 1 -- 172.21.15.49:0/4173291291 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428010430 con 0x7f5430100360 2024-02-26T06:24:18.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.079+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54301a60f0 con 0x7f5430100360 2024-02-26T06:24:18.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.080+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54301a6580 con 0x7f5430100360 2024-02-26T06:24:18.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.082+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5430101780 con 0x7f5430100360 2024-02-26T06:24:18.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.086+0000 7f5426a00640 1 -- 172.21.15.49:0/4173291291 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f5428005000 con 0x7f5430100360 2024-02-26T06:24:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.088+0000 7f5426a00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54140a23d0 0x7f54140a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.089+0000 7f5426a00640 1 -- 172.21.15.49:0/4173291291 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5428014070 con 0x7f5430100360 2024-02-26T06:24:18.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.091+0000 7f542d400640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54140a23d0 0x7f54140a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.094+0000 7f542d400640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54140a23d0 0x7f54140a4890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5430103770 tx=0x7f54180098b0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:18.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.098+0000 7f5426a00640 1 -- 172.21.15.49:0/4173291291 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5428097580 con 0x7f5430100360 2024-02-26T06:24:18.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.520+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54300683c0 con 0x7f5430100360 2024-02-26T06:24:18.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.521+0000 7f5426a00640 1 -- 172.21.15.49:0/4173291291 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11336 (secure 0 0 0) 0x7f542809ac40 con 0x7f5430100360 2024-02-26T06:24:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:24:18.521 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:24:04.190675+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:24:18.524 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-02-26T06:24:18.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54140a23d0 msgr2=0x7f54140a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54140a23d0 0x7f54140a4890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5430103770 tx=0x7f54180098b0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5430100360 msgr2=0x7f5430101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5430100360 0x7f5430101ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f542800e9f0 tx=0x7f542800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 shutdown_connections 2024-02-26T06:24:18.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54140a23d0 0x7f54140a4890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5430100d20 0x7f5430102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5430106130 0x7f5430102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 --2- 172.21.15.49:0/4173291291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5430100360 0x7f5430101ce0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 >> 172.21.15.49:0/4173291291 conn(0x7f5430077060 msgr2=0x7f5430069890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:18.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 shutdown_connections 2024-02-26T06:24:18.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.526+0000 7f5434a00640 1 -- 172.21.15.49:0/4173291291 wait complete. 2024-02-26T06:24:18.553 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:24:18.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:24:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:18 smithi080 ceph-mon[36206]: pgmap v30: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 1.07k op/s 2024-02-26T06:24:18.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:18 smithi115 ceph-mon[36426]: pgmap v30: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 1.07k op/s 2024-02-26T06:24:18.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 -- 172.21.15.49:0/3726549058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe334101d30 msgr2=0x7fe334105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 --2- 172.21.15.49:0/3726549058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe334101d30 0x7fe334105d70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe320009900 tx=0x7fe32002bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:18.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 -- 172.21.15.49:0/3726549058 shutdown_connections 2024-02-26T06:24:18.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 --2- 172.21.15.49:0/3726549058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe334106450 0x7fe33410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 --2- 172.21.15.49:0/3726549058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe334101d30 0x7fe334105d70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 --2- 172.21.15.49:0/3726549058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe334101380 0x7fe334101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 -- 172.21.15.49:0/3726549058 >> 172.21.15.49:0/3726549058 conn(0x7fe334078580 msgr2=0x7fe334078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:18.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 -- 172.21.15.49:0/3726549058 shutdown_connections 2024-02-26T06:24:18.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.857+0000 7fe338a00640 1 -- 172.21.15.49:0/3726549058 wait complete. 2024-02-26T06:24:18.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe338a00640 1 Processor -- start 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe338a00640 1 -- start start 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe338a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe334101380 0x7fe33419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe338a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe334106450 0x7fe33419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe338a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3341a26c0 0x7fe3341a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe338a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341a30b0 con 0x7fe3341a26c0 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe338a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341a3220 con 0x7fe334101380 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe338a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341a3390 con 0x7fe334106450 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe332800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3341a26c0 0x7fe3341a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe332800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3341a26c0 0x7fe3341a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44346/0 (socket says 172.21.15.49:44346) 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe331e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe334101380 0x7fe33419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe331400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe334106450 0x7fe33419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.859+0000 7fe332800640 1 -- 172.21.15.49:0/2912954869 learned_addr learned my addr 172.21.15.49:0/2912954869 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.860+0000 7fe331e00640 1 -- 172.21.15.49:0/2912954869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe334106450 msgr2=0x7fe33419e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.860+0000 7fe331e00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe334106450 0x7fe33419e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.860+0000 7fe331e00640 1 -- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3341a26c0 msgr2=0x7fe3341a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.860+0000 7fe331e00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3341a26c0 0x7fe3341a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.860+0000 7fe331e00640 1 -- 172.21.15.49:0/2912954869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe320009560 con 0x7fe334101380 2024-02-26T06:24:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.860+0000 7fe331e00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe334101380 0x7fe33419de00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005fd0 tx=0x7fe32c00deb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.861+0000 7fe32aa00640 1 -- 172.21.15.49:0/2912954869 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00cbe0 con 0x7fe334101380 2024-02-26T06:24:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.861+0000 7fe32aa00640 1 -- 172.21.15.49:0/2912954869 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe32c004510 con 0x7fe334101380 2024-02-26T06:24:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.861+0000 7fe32aa00640 1 -- 172.21.15.49:0/2912954869 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c004a00 con 0x7fe334101380 2024-02-26T06:24:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.861+0000 7fe338a00640 1 -- 172.21.15.49:0/2912954869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3341aa970 con 0x7fe334101380 2024-02-26T06:24:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.861+0000 7fe338a00640 1 -- 172.21.15.49:0/2912954869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3341aada0 con 0x7fe334101380 2024-02-26T06:24:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.867+0000 7fe33ba1e740 1 Processor -- start 2024-02-26T06:24:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.867+0000 7fe33ba1e740 1 -- start start 2024-02-26T06:24:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.868+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aaafff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.868+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 0x55c55aab8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.868+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55aabb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.868+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55aa1e1e0 con 0x55c55aaafc10 2024-02-26T06:24:18.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.868+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55aa1b710 con 0x55c55aab0530 2024-02-26T06:24:18.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.868+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55aa16ef0 con 0x55c55aab9110 2024-02-26T06:24:18.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.869+0000 7fe332800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aaafff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.869+0000 7fe331400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55aabb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.869+0000 7fe332800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aaafff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44350/0 (socket says 172.21.15.49:44350) 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.869+0000 7fe332800640 1 -- 172.21.15.49:0/3311811331 learned_addr learned my addr 172.21.15.49:0/3311811331 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.870+0000 7fe332800640 1 -- 172.21.15.49:0/3311811331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 msgr2=0x55c55aabb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.870+0000 7fe332800640 1 --2- 172.21.15.49:0/3311811331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55aabb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.870+0000 7fe332800640 1 -- 172.21.15.49:0/3311811331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 msgr2=0x55c55aab8bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.870+0000 7fe332800640 1 --2- 172.21.15.49:0/3311811331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 0x55c55aab8bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.870+0000 7fe332800640 1 -- 172.21.15.49:0/3311811331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c55a9d8dc0 con 0x55c55aaafc10 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.870+0000 7fe32aa00640 1 -- 172.21.15.49:0/2912954869 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fe32c004b60 con 0x7fe334101380 2024-02-26T06:24:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.870+0000 7fe332800640 1 --2- 172.21.15.49:0/3311811331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aaafff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe324009fb0 tx=0x7fe324011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c68359ca996f71 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.871+0000 7fe31f000640 1 -- 172.21.15.49:0/3311811331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240127d0 con 0x55c55aaafc10 2024-02-26T06:24:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.871+0000 7fe31f000640 1 -- 172.21.15.49:0/3311811331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe324012e10 con 0x55c55aaafc10 2024-02-26T06:24:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.871+0000 7fe31f000640 1 -- 172.21.15.49:0/3311811331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240193a0 con 0x55c55aaafc10 2024-02-26T06:24:18.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3311811331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 msgr2=0x55c55aaafff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/3311811331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aaafff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe324009fb0 tx=0x7fe324011870 comp rx=0 tx=0).stop 2024-02-26T06:24:18.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3311811331 shutdown_connections 2024-02-26T06:24:18.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe32aa00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3180a24a0 0x7fe3180a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/3311811331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 0x55c55aab8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/3311811331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55aabb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/3311811331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aaafff0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3311811331 >> 172.21.15.49:0/3311811331 conn(0x55c55aaaa340 msgr2=0x55c55aaaa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:18.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.874+0000 7fe32aa00640 1 -- 172.21.15.49:0/2912954869 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe32c0cd830 con 0x7fe334101380 2024-02-26T06:24:18.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.875+0000 7fe332800640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3180a24a0 0x7fe3180a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.875+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3311811331 shutdown_connections 2024-02-26T06:24:18.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.875+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3311811331 wait complete. 2024-02-26T06:24:18.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.876+0000 7fe332800640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3180a24a0 0x7fe3180a4960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe32401c040 tx=0x7fe324009b70 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:18.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.876+0000 7fe33ba1e740 1 -- start start 2024-02-26T06:24:18.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.876+0000 7fe33ba1e740 1 Processor -- start 2024-02-26T06:24:18.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.877+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.877+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.877+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.878+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55a9bc120 con 0x55c55aab0530 2024-02-26T06:24:18.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.878+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55a9bbdc0 con 0x55c55aab9110 2024-02-26T06:24:18.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.878+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55a7851c0 con 0x55c55aaafc10 2024-02-26T06:24:18.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.878+0000 7fe33ba1e740 10 client.0 _ll_register_callbacks cb 0x55c55aa0b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:18.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.878+0000 7fe332800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.878+0000 7fe332800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51422/0 (socket says 172.21.15.49:51422) 2024-02-26T06:24:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.878+0000 7fe332800640 1 -- 172.21.15.49:0/1593761375 learned_addr learned my addr 172.21.15.49:0/1593761375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.878+0000 7fe331e00640 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe331400640 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe332800640 1 -- 172.21.15.49:0/1593761375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 msgr2=0x55c55aab60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe332800640 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe332800640 1 -- 172.21.15.49:0/1593761375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 msgr2=0x55c55aab6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe332800640 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe332800640 1 -- 172.21.15.49:0/1593761375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe324009c10 con 0x55c55aab9110 2024-02-26T06:24:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe331400640 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe332800640 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab6b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe324005f10 tx=0x7fe32401a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34dbc4b83d32005a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.879+0000 7fe313400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.880+0000 7fe313400640 1 -- 172.21.15.49:0/1593761375 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32402c070 con 0x55c55aab9110 2024-02-26T06:24:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.880+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1593761375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c55a7167e0 con 0x55c55aab9110 2024-02-26T06:24:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.880+0000 7fe313400640 1 -- 172.21.15.49:0/1593761375 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe3240127e0 con 0x55c55aab9110 2024-02-26T06:24:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.881+0000 7fe313400640 1 -- 172.21.15.49:0/1593761375 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3240129a0 con 0x55c55aab9110 2024-02-26T06:24:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.881+0000 7fe33ba1e740 10 client.25628 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.881+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1593761375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c55aa15570 con 0x55c55aab9110 2024-02-26T06:24:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.882+0000 7fe313400640 1 -- 172.21.15.49:0/1593761375 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe32403e020 con 0x55c55aab9110 2024-02-26T06:24:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.882+0000 7fe313400640 1 -- 172.21.15.49:0/1593761375 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe324022040 con 0x55c55aab9110 2024-02-26T06:24:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.882+0000 7fe33ba1e740 10 client.25628 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.882+0000 7fe33ba1e740 10 client.25628 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:24:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.882+0000 7fe33ba1e740 20 client.25628 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.882+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55c55aac9180 0x55c55aacb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.882+0000 7fe33ba1e740 4 client.25628 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.882+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1593761375 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c55a753720 con 0x55c55aac9180 2024-02-26T06:24:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.883+0000 7fe331e00640 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55c55aac9180 0x55c55aacb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.884+0000 7fe331e00640 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55c55aac9180 0x55c55aacb5a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9283692098ca66b9 server_cookie=fd3a6b7417e38310 in_seq=0 out_seq=0 2024-02-26T06:24:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.884+0000 7fe313400640 10 client.25628 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:24:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe313400640 1 -- 172.21.15.49:0/1593761375 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c55a753720 con 0x55c55aac9180 2024-02-26T06:24:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe313400640 10 client.25628 handle_command_reply: tid=0 2024-02-26T06:24:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 client.25628 shutdown 2024-02-26T06:24:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1593761375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 msgr2=0x55c55aab6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab6b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe324005f10 tx=0x7fe32401a9c0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1593761375 shutdown_connections 2024-02-26T06:24:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55c55aac9180 0x55c55aacb5a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab6b50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe313400640 0 client.25628 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:24:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1593761375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.885+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1593761375 >> 172.21.15.49:0/1593761375 conn(0x55c55aaaa340 msgr2=0x55c55aaac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.886+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1593761375 shutdown_connections 2024-02-26T06:24:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.886+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1593761375 wait complete. 2024-02-26T06:24:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.886+0000 7fe33ba1e740 20 client.25628 trim_cache size 0 max 16384 2024-02-26T06:24:18.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.912+0000 7fe33ba1e740 1 Processor -- start 2024-02-26T06:24:18.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.912+0000 7fe33ba1e740 1 -- start start 2024-02-26T06:24:18.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.912+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.912+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.912+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.912+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55aac5c60 con 0x55c55aab0530 2024-02-26T06:24:18.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.912+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55aabb500 con 0x55c55aab9110 2024-02-26T06:24:18.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.912+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55aac4f00 con 0x55c55aaafc10 2024-02-26T06:24:18.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.913+0000 7fe331400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.913+0000 7fe331e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.913+0000 7fe331e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40920/0 (socket says 172.21.15.49:40920) 2024-02-26T06:24:18.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.913+0000 7fe331400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44374/0 (socket says 172.21.15.49:44374) 2024-02-26T06:24:18.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.913+0000 7fe331e00640 1 -- 172.21.15.49:0/3608630814 learned_addr learned my addr 172.21.15.49:0/3608630814 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:18.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.913+0000 7fe332800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.914+0000 7fe331e00640 1 -- 172.21.15.49:0/3608630814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 msgr2=0x55c55aab18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:18.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.914+0000 7fe331e00640 1 --2- 172.21.15.49:0/3608630814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.914+0000 7fe331e00640 1 -- 172.21.15.49:0/3608630814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 msgr2=0x55c55aab13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:18.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.914+0000 7fe331e00640 1 --2- 172.21.15.49:0/3608630814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.914+0000 7fe331e00640 1 -- 172.21.15.49:0/3608630814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c55a9b1a80 con 0x55c55aaafc10 2024-02-26T06:24:18.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.914+0000 7fe331400640 1 --2- 172.21.15.49:0/3608630814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:18.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.915+0000 7fe331e00640 1 --2- 172.21.15.49:0/3608630814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab8800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005c20 tx=0x7fe32c005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651f63b2cb18d911 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:18.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.915+0000 7fe31e600640 1 -- 172.21.15.49:0/3608630814 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c098390 con 0x55c55aaafc10 2024-02-26T06:24:18.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.915+0000 7fe31e600640 1 -- 172.21.15.49:0/3608630814 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe32c0b1070 con 0x55c55aaafc10 2024-02-26T06:24:18.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.915+0000 7fe31e600640 1 -- 172.21.15.49:0/3608630814 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0b8560 con 0x55c55aaafc10 2024-02-26T06:24:18.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.917+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3608630814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 msgr2=0x55c55aab8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.917+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/3608630814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab8800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005c20 tx=0x7fe32c005bb0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.917+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3608630814 shutdown_connections 2024-02-26T06:24:18.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.917+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/3608630814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab9110 0x55c55aab18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.917+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/3608630814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aab0530 0x55c55aab13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.917+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/3608630814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aaafc10 0x55c55aab8800 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.917+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3608630814 >> 172.21.15.49:0/3608630814 conn(0x55c55aaaa340 msgr2=0x55c55aaaa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:18.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.918+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3608630814 shutdown_connections 2024-02-26T06:24:18.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.919+0000 7fe33ba1e740 1 -- 172.21.15.49:0/3608630814 wait complete. 2024-02-26T06:24:18.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:18 smithi049 ceph-mon[31696]: pgmap v30: 97 pgs: 97 active+clean; 976 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 1.07k op/s 2024-02-26T06:24:18.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.919+0000 7fe33ba1e740 1 -- start start 2024-02-26T06:24:18.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.920+0000 7fe33ba1e740 1 Processor -- start 2024-02-26T06:24:18.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.921+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aabf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.921+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 0x55c55aabf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.921+0000 7fe33ba1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55ab19660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe331400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 0x55c55aabf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe331400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 0x55c55aabf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51442/0 (socket says 172.21.15.49:51442) 2024-02-26T06:24:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe332800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55ab19660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe332800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55ab19660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40932/0 (socket says 172.21.15.49:40932) 2024-02-26T06:24:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55aabfd90 con 0x55c55aaafc10 2024-02-26T06:24:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55ab19ba0 con 0x55c55aab0530 2024-02-26T06:24:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe33ba1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55ab19d60 con 0x55c55aab9110 2024-02-26T06:24:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe33ba1e740 10 client.0 _ll_register_callbacks cb 0x7fe324009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.922+0000 7fe331400640 1 -- 172.21.15.49:0/1730248772 learned_addr learned my addr 172.21.15.49:0/1730248772 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.923+0000 7fe331e00640 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aabf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.923+0000 7fe332800640 1 -- 172.21.15.49:0/1730248772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 msgr2=0x55c55aabf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:18.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.923+0000 7fe332800640 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 0x55c55aabf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.923+0000 7fe332800640 1 -- 172.21.15.49:0/1730248772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 msgr2=0x55c55aabf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:18.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.923+0000 7fe332800640 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aabf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.923+0000 7fe332800640 1 -- 172.21.15.49:0/1730248772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33410e830 con 0x55c55aab9110 2024-02-26T06:24:18.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.924+0000 7fe331e00640 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aabf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:18.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.925+0000 7fe332800640 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55ab19660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe324022ab0 tx=0x7fe324030f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87eb1e069a565857 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:18.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.925+0000 7fe313400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:18.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.925+0000 7fe313400640 1 -- 172.21.15.49:0/1730248772 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32402c070 con 0x55c55aab9110 2024-02-26T06:24:18.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.925+0000 7fe313400640 1 -- 172.21.15.49:0/1730248772 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe32403f080 con 0x55c55aab9110 2024-02-26T06:24:18.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.925+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1730248772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe324012e10 con 0x55c55aab9110 2024-02-26T06:24:18.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.926+0000 7fe313400640 1 -- 172.21.15.49:0/1730248772 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3240093c0 con 0x55c55aab9110 2024-02-26T06:24:18.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.926+0000 7fe33ba1e740 10 client.25654 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:18.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.926+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1730248772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fe3341aada0 con 0x55c55aab9110 2024-02-26T06:24:18.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.926+0000 7fe313400640 1 -- 172.21.15.49:0/1730248772 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe324012e10 con 0x55c55aab9110 2024-02-26T06:24:18.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.927+0000 7fe313400640 1 -- 172.21.15.49:0/1730248772 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe324022040 con 0x55c55aab9110 2024-02-26T06:24:18.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.927+0000 7fe33ba1e740 10 client.25654 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:18.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.927+0000 7fe33ba1e740 10 client.25654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:24:18.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.927+0000 7fe33ba1e740 20 client.25654 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:18.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.927+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55c55aac9180 0x55c55ab346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:18.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.927+0000 7fe33ba1e740 4 client.25654 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:24:18.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.927+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1730248772 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c55a7a2890 con 0x55c55aac9180 2024-02-26T06:24:18.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.928+0000 7fe331e00640 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55c55aac9180 0x55c55ab346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:18.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.928+0000 7fe331e00640 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55c55aac9180 0x55c55ab346a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=164c6a378c71732 server_cookie=9bc22e70e3e7aafa in_seq=0 out_seq=0 2024-02-26T06:24:18.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.928+0000 7fe313400640 10 client.25654 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:24:18.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.928+0000 7fe313400640 1 -- 172.21.15.49:0/1730248772 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c55a7a2890 con 0x55c55aac9180 2024-02-26T06:24:18.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.928+0000 7fe313400640 10 client.25654 handle_command_reply: tid=0 2024-02-26T06:24:18.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.928+0000 7fe33ba1e740 1 client.25654 shutdown 2024-02-26T06:24:18.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.929+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1730248772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 msgr2=0x55c55ab19660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.929+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55ab19660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe324022ab0 tx=0x7fe324030f50 comp rx=0 tx=0).stop 2024-02-26T06:24:18.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.929+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1730248772 shutdown_connections 2024-02-26T06:24:18.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.929+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55c55aac9180 0x55c55ab346a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.929+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c55aab0530 0x55c55aabf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.930+0000 7fe313400640 0 client.25654 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:24:18.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.930+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c55aab9110 0x55c55ab19660 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.930+0000 7fe33ba1e740 1 --2- 172.21.15.49:0/1730248772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c55aaafc10 0x55c55aabf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.930+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1730248772 >> 172.21.15.49:0/1730248772 conn(0x55c55aaaa340 msgr2=0x55c55aab13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:18.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.930+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1730248772 shutdown_connections 2024-02-26T06:24:18.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.930+0000 7fe33ba1e740 1 -- 172.21.15.49:0/1730248772 wait complete. 2024-02-26T06:24:18.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.931+0000 7fe33ba1e740 20 client.25654 trim_cache size 0 max 16384 2024-02-26T06:24:18.931 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:24:18.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 -- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3180a24a0 msgr2=0x7fe3180a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3180a24a0 0x7fe3180a4960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe32401c040 tx=0x7fe324009b70 comp rx=0 tx=0).stop 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 -- 172.21.15.49:0/2912954869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe334101380 msgr2=0x7fe33419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe334101380 0x7fe33419de00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005fd0 tx=0x7fe32c00deb0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 -- 172.21.15.49:0/2912954869 shutdown_connections 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3180a24a0 0x7fe3180a4960 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3341a26c0 0x7fe3341a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe334106450 0x7fe33419e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 --2- 172.21.15.49:0/2912954869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe334101380 0x7fe33419de00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.932+0000 7fe338a00640 1 -- 172.21.15.49:0/2912954869 >> 172.21.15.49:0/2912954869 conn(0x7fe334078580 msgr2=0x7fe3340ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.933+0000 7fe338a00640 1 -- 172.21.15.49:0/2912954869 shutdown_connections 2024-02-26T06:24:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:18.933+0000 7fe338a00640 1 -- 172.21.15.49:0/2912954869 wait complete. 2024-02-26T06:24:18.955 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:24:18.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:24:19.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.255+0000 7fa32d000640 1 -- 172.21.15.49:0/10197625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328103f40 msgr2=0x7fa328107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.255+0000 7fa32d000640 1 --2- 172.21.15.49:0/10197625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328103f40 0x7fa328107f80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa314009900 tx=0x7fa31402bc60 comp rx=0 tx=0).stop 2024-02-26T06:24:19.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.256+0000 7fa32d000640 1 -- 172.21.15.49:0/10197625 shutdown_connections 2024-02-26T06:24:19.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.256+0000 7fa32d000640 1 --2- 172.21.15.49:0/10197625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa328108660 0x7fa328110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.256+0000 7fa32d000640 1 --2- 172.21.15.49:0/10197625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328103f40 0x7fa328107f80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.256+0000 7fa32d000640 1 --2- 172.21.15.49:0/10197625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328103590 0x7fa328103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.256+0000 7fa32d000640 1 -- 172.21.15.49:0/10197625 >> 172.21.15.49:0/10197625 conn(0x7fa3280fee30 msgr2=0x7fa328101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.256+0000 7fa32d000640 1 -- 172.21.15.49:0/10197625 shutdown_connections 2024-02-26T06:24:19.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.256+0000 7fa32d000640 1 -- 172.21.15.49:0/10197625 wait complete. 2024-02-26T06:24:19.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.257+0000 7fa32d000640 1 Processor -- start 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.257+0000 7fa32d000640 1 -- start start 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa32d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328103590 0x7fa32810f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa32d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328108660 0x7fa32810ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa32d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa32810db80 0x7fa32810e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa32d000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32810e6d0 con 0x7fa328103590 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa32d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32810e810 con 0x7fa32810db80 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa32d000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32810e9d0 con 0x7fa328108660 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa326800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa32810db80 0x7fa32810e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa326800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa32810db80 0x7fa32810e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51466/0 (socket says 172.21.15.49:51466) 2024-02-26T06:24:19.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa326800640 1 -- 172.21.15.49:0/4130257946 learned_addr learned my addr 172.21.15.49:0/4130257946 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.258+0000 7fa325e00640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328103590 0x7fa32810f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.259+0000 7fa325400640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328108660 0x7fa32810ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.259+0000 7fa325e00640 1 -- 172.21.15.49:0/4130257946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328108660 msgr2=0x7fa32810ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.259+0000 7fa325e00640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328108660 0x7fa32810ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.259+0000 7fa325e00640 1 -- 172.21.15.49:0/4130257946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa32810db80 msgr2=0x7fa32810e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.259+0000 7fa325e00640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa32810db80 0x7fa32810e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.259+0000 7fa325e00640 1 -- 172.21.15.49:0/4130257946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa314009560 con 0x7fa328103590 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.259+0000 7fa325400640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328108660 0x7fa32810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.260+0000 7fa325e00640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328103590 0x7fa32810f9d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa31800c970 tx=0x7fa31800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.260+0000 7fa31ea00640 1 -- 172.21.15.49:0/4130257946 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318007bb0 con 0x7fa328103590 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.260+0000 7fa31ea00640 1 -- 172.21.15.49:0/4130257946 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa318007d10 con 0x7fa328103590 2024-02-26T06:24:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.260+0000 7fa31ea00640 1 -- 172.21.15.49:0/4130257946 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180187f0 con 0x7fa328103590 2024-02-26T06:24:19.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.260+0000 7fa32d000640 1 -- 172.21.15.49:0/4130257946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32806acc0 con 0x7fa328103590 2024-02-26T06:24:19.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.261+0000 7fa32d000640 1 -- 172.21.15.49:0/4130257946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32806b150 con 0x7fa328103590 2024-02-26T06:24:19.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.269+0000 7fa31ea00640 1 -- 172.21.15.49:0/4130257946 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fa31800f450 con 0x7fa328103590 2024-02-26T06:24:19.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.271+0000 7fa32fffb740 1 Processor -- start 2024-02-26T06:24:19.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.271+0000 7fa32fffb740 1 -- start start 2024-02-26T06:24:19.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c45ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c46530 0x563f05c4ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c4f110 0x563f05c51500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa32fffb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05bb41e0 con 0x563f05c46530 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa32fffb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05bb1710 con 0x563f05c4f110 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa32fffb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05bacef0 con 0x563f05c45c10 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa326800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c45ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa326800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c45ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40958/0 (socket says 172.21.15.49:40958) 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa326800640 1 -- 172.21.15.49:0/4181824888 learned_addr learned my addr 172.21.15.49:0/4181824888 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.272+0000 7fa325400640 1 --2- 172.21.15.49:0/4181824888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c4f110 0x563f05c51500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.273+0000 7fa325400640 1 -- 172.21.15.49:0/4181824888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 msgr2=0x563f05c45ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.273+0000 7fa325400640 1 --2- 172.21.15.49:0/4181824888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c45ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.273+0000 7fa325400640 1 -- 172.21.15.49:0/4181824888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c46530 msgr2=0x563f05c4ebd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.273+0000 7fa325400640 1 --2- 172.21.15.49:0/4181824888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c46530 0x563f05c4ebd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.273+0000 7fa325400640 1 -- 172.21.15.49:0/4181824888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f05b6edc0 con 0x563f05c4f110 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.273+0000 7fa325400640 1 --2- 172.21.15.49:0/4181824888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c4f110 0x563f05c51500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa31402c1a0 tx=0x7fa314004880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13292f1c1d034766 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.273+0000 7fa31ea00640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa30c0a23d0 0x7fa30c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.274+0000 7fa313400640 1 -- 172.21.15.49:0/4181824888 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314004a70 con 0x563f05c4f110 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.274+0000 7fa313400640 1 -- 172.21.15.49:0/4181824888 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3140054e0 con 0x563f05c4f110 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.274+0000 7fa313400640 1 -- 172.21.15.49:0/4181824888 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31403f450 con 0x563f05c4f110 2024-02-26T06:24:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.274+0000 7fa326800640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa30c0a23d0 0x7fa30c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.276+0000 7fa32fffb740 1 -- 172.21.15.49:0/4181824888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c4f110 msgr2=0x563f05c51500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.276+0000 7fa32fffb740 1 --2- 172.21.15.49:0/4181824888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c4f110 0x563f05c51500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa31402c1a0 tx=0x7fa314004880 comp rx=0 tx=0).stop 2024-02-26T06:24:19.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.278+0000 7fa326800640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa30c0a23d0 0x7fa30c0a4890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa320005f40 tx=0x7fa320009680 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.278+0000 7fa31ea00640 1 -- 172.21.15.49:0/4130257946 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fa3180ce2b0 con 0x7fa328103590 2024-02-26T06:24:19.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.278+0000 7fa32fffb740 1 -- 172.21.15.49:0/4181824888 shutdown_connections 2024-02-26T06:24:19.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.278+0000 7fa32fffb740 1 --2- 172.21.15.49:0/4181824888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c4f110 0x563f05c51500 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.278+0000 7fa32fffb740 1 --2- 172.21.15.49:0/4181824888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c46530 0x563f05c4ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.278+0000 7fa32fffb740 1 --2- 172.21.15.49:0/4181824888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c45ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.279+0000 7fa32fffb740 1 -- 172.21.15.49:0/4181824888 >> 172.21.15.49:0/4181824888 conn(0x563f05c40340 msgr2=0x563f05c40720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.279+0000 7fa32fffb740 1 -- 172.21.15.49:0/4181824888 shutdown_connections 2024-02-26T06:24:19.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.279+0000 7fa32fffb740 1 -- 172.21.15.49:0/4181824888 wait complete. 2024-02-26T06:24:19.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.280+0000 7fa32fffb740 1 -- start start 2024-02-26T06:24:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.280+0000 7fa32fffb740 1 Processor -- start 2024-02-26T06:24:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.281+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.281+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.281+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa325e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa325e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40972/0 (socket says 172.21.15.49:40972) 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa325400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa325400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51470/0 (socket says 172.21.15.49:51470) 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa326800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa326800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44404/0 (socket says 172.21.15.49:44404) 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05b52120 con 0x563f05c4f110 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05b51dc0 con 0x563f05c46530 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0591b1c0 con 0x563f05c45c10 2024-02-26T06:24:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.282+0000 7fa325e00640 1 -- 172.21.15.49:0/3850835202 learned_addr learned my addr 172.21.15.49:0/3850835202 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa32fffb740 10 client.0 _ll_register_callbacks cb 0x563f05ba1280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa325e00640 1 -- 172.21.15.49:0/3850835202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 msgr2=0x563f05c4c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa325e00640 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa325e00640 1 -- 172.21.15.49:0/3850835202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 msgr2=0x563f05c4cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa325e00640 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa325e00640 1 -- 172.21.15.49:0/3850835202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa314038040 con 0x563f05c45c10 2024-02-26T06:24:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa325e00640 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4c1a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa3180cc010 tx=0x7fa31809b160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc989f9b1dce077 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa307e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.283+0000 7fa307e00640 1 -- 172.21.15.49:0/3850835202 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180a3200 con 0x563f05c45c10 2024-02-26T06:24:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.284+0000 7fa307e00640 1 -- 172.21.15.49:0/3850835202 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fa31800f700 con 0x563f05c45c10 2024-02-26T06:24:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.284+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f058ac7e0 con 0x563f05c45c10 2024-02-26T06:24:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.285+0000 7fa307e00640 1 -- 172.21.15.49:0/3850835202 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa31800f900 con 0x563f05c45c10 2024-02-26T06:24:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.285+0000 7fa32fffb740 10 client.25666 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.285+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f05bab570 con 0x563f05c45c10 2024-02-26T06:24:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.285+0000 7fa307e00640 1 -- 172.21.15.49:0/3850835202 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563f058ac7e0 con 0x563f05c45c10 2024-02-26T06:24:19.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.285+0000 7fa307e00640 1 -- 172.21.15.49:0/3850835202 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa31804c020 con 0x563f05c45c10 2024-02-26T06:24:19.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.286+0000 7fa32fffb740 10 client.25666 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:19.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.286+0000 7fa32fffb740 10 client.25666 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:24:19.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.286+0000 7fa32fffb740 20 client.25666 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:19.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.286+0000 7fa32fffb740 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x563f05c5f1a0 0x563f05c615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.286+0000 7fa32fffb740 4 client.25666 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:19.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.286+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f058e9720 con 0x563f05c5f1a0 2024-02-26T06:24:19.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.288+0000 7fa325400640 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x563f05c5f1a0 0x563f05c615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.289+0000 7fa307e00640 10 client.25666 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:24:19.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.289+0000 7fa325400640 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x563f05c5f1a0 0x563f05c615c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7281c40180d337e server_cookie=65333edaf9fbec5b in_seq=0 out_seq=0 2024-02-26T06:24:19.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.289+0000 7fa307e00640 1 -- 172.21.15.49:0/3850835202 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563f058e9720 con 0x563f05c5f1a0 2024-02-26T06:24:19.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.289+0000 7fa307e00640 10 client.25666 handle_command_reply: tid=0 2024-02-26T06:24:19.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.289+0000 7fa32fffb740 1 client.25666 shutdown 2024-02-26T06:24:19.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 msgr2=0x563f05c4c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4c1a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa3180cc010 tx=0x7fa31809b160 comp rx=0 tx=0).stop 2024-02-26T06:24:19.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 shutdown_connections 2024-02-26T06:24:19.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x563f05c5f1a0 0x563f05c615c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 --2- 172.21.15.49:0/3850835202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4c1a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 >> 172.21.15.49:0/3850835202 conn(0x563f05c40340 msgr2=0x563f05c42090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa307e00640 0 client.25666 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:24:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.290+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 shutdown_connections 2024-02-26T06:24:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.291+0000 7fa32fffb740 1 -- 172.21.15.49:0/3850835202 wait complete. 2024-02-26T06:24:19.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.291+0000 7fa32fffb740 20 client.25666 trim_cache size 0 max 16384 2024-02-26T06:24:19.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.313+0000 7fa32fffb740 1 Processor -- start 2024-02-26T06:24:19.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.313+0000 7fa32fffb740 1 -- start start 2024-02-26T06:24:19.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.314+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.314+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.314+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.314+0000 7fa32fffb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05c5ef20 con 0x563f05c4f110 2024-02-26T06:24:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.314+0000 7fa32fffb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05c667b0 con 0x563f05c46530 2024-02-26T06:24:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.314+0000 7fa32fffb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05c46330 con 0x563f05c45c10 2024-02-26T06:24:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.314+0000 7fa325400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40988/0 (socket says 172.21.15.49:40988) 2024-02-26T06:24:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325e00640 1 -- 172.21.15.49:0/2610325050 learned_addr learned my addr 172.21.15.49:0/2610325050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51474/0 (socket says 172.21.15.49:51474) 2024-02-26T06:24:19.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa326800640 1 --2- 172.21.15.49:0/2610325050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325e00640 1 -- 172.21.15.49:0/2610325050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 msgr2=0x563f05c4ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325e00640 1 --2- 172.21.15.49:0/2610325050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325e00640 1 -- 172.21.15.49:0/2610325050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 msgr2=0x563f05c4aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325e00640 1 --2- 172.21.15.49:0/2610325050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.315+0000 7fa325e00640 1 -- 172.21.15.49:0/2610325050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f05b47a80 con 0x563f05c45c10 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.316+0000 7fa325e00640 1 --2- 172.21.15.49:0/2610325050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4e8d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa318096ec0 tx=0x7fa31801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f32a5884f58d7102 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.316+0000 7fa312a00640 1 -- 172.21.15.49:0/2610325050 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31809a5a0 con 0x563f05c45c10 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.316+0000 7fa312a00640 1 -- 172.21.15.49:0/2610325050 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3180cb7d0 con 0x563f05c45c10 2024-02-26T06:24:19.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.317+0000 7fa312a00640 1 -- 172.21.15.49:0/2610325050 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180a2d70 con 0x563f05c45c10 2024-02-26T06:24:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.317+0000 7fa32fffb740 1 -- 172.21.15.49:0/2610325050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 msgr2=0x563f05c4e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.317+0000 7fa32fffb740 1 --2- 172.21.15.49:0/2610325050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4e8d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa318096ec0 tx=0x7fa31801c000 comp rx=0 tx=0).stop 2024-02-26T06:24:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.318+0000 7fa32fffb740 1 -- 172.21.15.49:0/2610325050 shutdown_connections 2024-02-26T06:24:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.318+0000 7fa32fffb740 1 --2- 172.21.15.49:0/2610325050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c4ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.318+0000 7fa32fffb740 1 --2- 172.21.15.49:0/2610325050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c4f110 0x563f05c4aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.318+0000 7fa32fffb740 1 --2- 172.21.15.49:0/2610325050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c45c10 0x563f05c4e8d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.318+0000 7fa32fffb740 1 -- 172.21.15.49:0/2610325050 >> 172.21.15.49:0/2610325050 conn(0x563f05c40340 msgr2=0x563f05c474b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.318+0000 7fa32fffb740 1 -- 172.21.15.49:0/2610325050 shutdown_connections 2024-02-26T06:24:19.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.318+0000 7fa32fffb740 1 -- 172.21.15.49:0/2610325050 wait complete. 2024-02-26T06:24:19.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.319+0000 7fa32fffb740 1 -- start start 2024-02-26T06:24:19.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.319+0000 7fa32fffb740 1 Processor -- start 2024-02-26T06:24:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.320+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c45c10 0x563f05c53cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c54200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa32fffb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c4f110 0x563f05cb46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa32fffb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05c549a0 con 0x563f05c45c10 2024-02-26T06:24:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa32fffb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05c54b10 con 0x563f05c46530 2024-02-26T06:24:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa32fffb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f05cb4be0 con 0x563f05c4f110 2024-02-26T06:24:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa32fffb740 10 client.0 _ll_register_callbacks cb 0x7fa3140045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa326800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c4f110 0x563f05cb46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa326800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c4f110 0x563f05cb46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40994/0 (socket says 172.21.15.49:40994) 2024-02-26T06:24:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa326800640 1 -- 172.21.15.49:0/225349174 learned_addr learned my addr 172.21.15.49:0/225349174 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.321+0000 7fa325e00640 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c45c10 0x563f05c53cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.322+0000 7fa325400640 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c54200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.322+0000 7fa325e00640 1 -- 172.21.15.49:0/225349174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c4f110 msgr2=0x563f05cb46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.322+0000 7fa325e00640 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c4f110 0x563f05cb46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.322+0000 7fa325e00640 1 -- 172.21.15.49:0/225349174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 msgr2=0x563f05c54200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.322+0000 7fa325e00640 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c54200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.322+0000 7fa325e00640 1 -- 172.21.15.49:0/225349174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f05bab570 con 0x563f05c45c10 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.323+0000 7fa325e00640 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c45c10 0x563f05c53cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa3180cc010 tx=0x7fa31800fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e118c8203f95f247 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.323+0000 7fa307e00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.324+0000 7fa307e00640 1 -- 172.21.15.49:0/225349174 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31809a5a0 con 0x563f05c45c10 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.324+0000 7fa307e00640 1 -- 172.21.15.49:0/225349174 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fa3180a3200 con 0x563f05c45c10 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.324+0000 7fa32fffb740 1 -- 172.21.15.49:0/225349174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3140054e0 con 0x563f05c45c10 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.325+0000 7fa307e00640 1 -- 172.21.15.49:0/225349174 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa31809a140 con 0x563f05c45c10 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.325+0000 7fa32fffb740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.325+0000 7fa32fffb740 1 -- 172.21.15.49:0/225349174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f05b6d620 con 0x563f05c45c10 2024-02-26T06:24:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.326+0000 7fa307e00640 1 -- 172.21.15.49:0/225349174 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3140054e0 con 0x563f05c45c10 2024-02-26T06:24:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.326+0000 7fa307e00640 1 -- 172.21.15.49:0/225349174 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa318016020 con 0x563f05c45c10 2024-02-26T06:24:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.326+0000 7fa32fffb740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.326+0000 7fa32fffb740 10 client.15528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:24:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.326+0000 7fa32fffb740 20 client.15528 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.326+0000 7fa32fffb740 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x563f05cb5ed0 0x563f05cb82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.326+0000 7fa32fffb740 4 client.15528 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:24:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.326+0000 7fa32fffb740 1 -- 172.21.15.49:0/225349174 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f05938890 con 0x563f05cb5ed0 2024-02-26T06:24:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.327+0000 7fa325400640 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x563f05cb5ed0 0x563f05cb82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.327+0000 7fa307e00640 10 client.15528 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:24:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.328+0000 7fa325400640 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x563f05cb5ed0 0x563f05cb82c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcb90ad9fe0e23e9 server_cookie=437b42d214d41285 in_seq=0 out_seq=0 2024-02-26T06:24:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.328+0000 7fa307e00640 1 -- 172.21.15.49:0/225349174 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f05938890 con 0x563f05cb5ed0 2024-02-26T06:24:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.328+0000 7fa307e00640 10 client.15528 handle_command_reply: tid=0 2024-02-26T06:24:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.328+0000 7fa32fffb740 1 client.15528 shutdown 2024-02-26T06:24:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa32fffb740 1 -- 172.21.15.49:0/225349174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c45c10 msgr2=0x563f05c53cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa32fffb740 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c45c10 0x563f05c53cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa3180cc010 tx=0x7fa31800fa30 comp rx=0 tx=0).stop 2024-02-26T06:24:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa32fffb740 1 -- 172.21.15.49:0/225349174 shutdown_connections 2024-02-26T06:24:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa32fffb740 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x563f05cb5ed0 0x563f05cb82c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa32fffb740 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f05c46530 0x563f05c54200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa32fffb740 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f05c4f110 0x563f05cb46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa32fffb740 1 --2- 172.21.15.49:0/225349174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f05c45c10 0x563f05c53cc0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa3180cc010 tx=0x7fa31800fa30 comp rx=0 tx=0).stop 2024-02-26T06:24:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa32fffb740 1 -- 172.21.15.49:0/225349174 >> 172.21.15.49:0/225349174 conn(0x563f05c40340 msgr2=0x563f05c4aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.329+0000 7fa307e00640 0 client.15528 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:24:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.330+0000 7fa32fffb740 1 -- 172.21.15.49:0/225349174 shutdown_connections 2024-02-26T06:24:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.330+0000 7fa32fffb740 1 -- 172.21.15.49:0/225349174 wait complete. 2024-02-26T06:24:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.331+0000 7fa32fffb740 20 client.15528 trim_cache size 0 max 16384 2024-02-26T06:24:19.331 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:24:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 -- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa30c0a23d0 msgr2=0x7fa30c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa30c0a23d0 0x7fa30c0a4890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa320005f40 tx=0x7fa320009680 comp rx=0 tx=0).stop 2024-02-26T06:24:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 -- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328103590 msgr2=0x7fa32810f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328103590 0x7fa32810f9d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa31800c970 tx=0x7fa31800ce30 comp rx=0 tx=0).stop 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 -- 172.21.15.49:0/4130257946 shutdown_connections 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa30c0a23d0 0x7fa30c0a4890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa32810db80 0x7fa32810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa328108660 0x7fa32810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 --2- 172.21.15.49:0/4130257946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328103590 0x7fa32810f9d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.332+0000 7fa32d000640 1 -- 172.21.15.49:0/4130257946 >> 172.21.15.49:0/4130257946 conn(0x7fa3280fee30 msgr2=0x7fa3281010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.333+0000 7fa32d000640 1 -- 172.21.15.49:0/4130257946 shutdown_connections 2024-02-26T06:24:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.333+0000 7fa32d000640 1 -- 172.21.15.49:0/4130257946 wait complete. 2024-02-26T06:24:19.363 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:24:19.364 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:24:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.385+0000 7f47e0c00640 1 -- 172.21.15.49:0/913807794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc107270 msgr2=0x7f47dc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.385+0000 7f47e0c00640 1 --2- 172.21.15.49:0/913807794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc107270 0x7f47dc107650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009900 tx=0x7f47cc02bcc0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.386+0000 7f47e0c00640 1 -- 172.21.15.49:0/913807794 shutdown_connections 2024-02-26T06:24:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.386+0000 7f47e0c00640 1 --2- 172.21.15.49:0/913807794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47dc068800 0x7f47dc068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.386+0000 7f47e0c00640 1 --2- 172.21.15.49:0/913807794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc0691a0 0x7f47dc10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.386+0000 7f47e0c00640 1 --2- 172.21.15.49:0/913807794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc107270 0x7f47dc107650 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.386+0000 7f47e0c00640 1 -- 172.21.15.49:0/913807794 >> 172.21.15.49:0/913807794 conn(0x7f47dc076ed0 msgr2=0x7f47dc0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.386+0000 7f47e0c00640 1 -- 172.21.15.49:0/913807794 shutdown_connections 2024-02-26T06:24:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.386+0000 7f47e0c00640 1 -- 172.21.15.49:0/913807794 wait complete. 2024-02-26T06:24:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.388+0000 7f47e0c00640 1 Processor -- start 2024-02-26T06:24:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.389+0000 7f47e0c00640 1 -- start start 2024-02-26T06:24:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.389+0000 7f47e0c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc068800 0x7f47dc10e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.389+0000 7f47e0c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47dc0691a0 0x7f47dc10eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.389+0000 7f47e0c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc107270 0x7f47dc10f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.390+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc068800 0x7f47dc10e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.390+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc068800 0x7f47dc10e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41002/0 (socket says 172.21.15.49:41002) 2024-02-26T06:24:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.390+0000 7f47d9e00640 1 -- 172.21.15.49:0/115211062 learned_addr learned my addr 172.21.15.49:0/115211062 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.390+0000 7f47d9400640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47dc0691a0 0x7f47dc10eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.390+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0717a0 con 0x7f47dc107270 2024-02-26T06:24:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.390+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc071910 con 0x7f47dc0691a0 2024-02-26T06:24:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.391+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc071ad0 con 0x7f47dc068800 2024-02-26T06:24:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.391+0000 7f47da800640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc107270 0x7f47dc10f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.391+0000 7f47d9e00640 1 -- 172.21.15.49:0/115211062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47dc0691a0 msgr2=0x7f47dc10eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.391+0000 7f47d9e00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47dc0691a0 0x7f47dc10eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.391+0000 7f47d9e00640 1 -- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc107270 msgr2=0x7f47dc10f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.391+0000 7f47d9e00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc107270 0x7f47dc10f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.391+0000 7f47d9e00640 1 -- 172.21.15.49:0/115211062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009560 con 0x7f47dc068800 2024-02-26T06:24:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.392+0000 7f47d9e00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc068800 0x7f47dc10e620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc02dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.392+0000 7f47d2a00640 1 -- 172.21.15.49:0/115211062 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc02de30 con 0x7f47dc068800 2024-02-26T06:24:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.392+0000 7f47d2a00640 1 -- 172.21.15.49:0/115211062 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47cc0052d0 con 0x7f47dc068800 2024-02-26T06:24:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.392+0000 7f47d2a00640 1 -- 172.21.15.49:0/115211062 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc003ce0 con 0x7f47dc068800 2024-02-26T06:24:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.392+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc071cf0 con 0x7f47dc068800 2024-02-26T06:24:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.393+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc0721b0 con 0x7f47dc068800 2024-02-26T06:24:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.399+0000 7f47d2a00640 1 -- 172.21.15.49:0/115211062 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f47cc004df0 con 0x7f47dc068800 2024-02-26T06:24:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.402+0000 7f47d2a00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f47c00a24c0 0x7f47c00a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.402+0000 7f47d2a00640 1 -- 172.21.15.49:0/115211062 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f47cc0f0a50 con 0x7f47dc068800 2024-02-26T06:24:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.404+0000 7f47d9400640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f47c00a24c0 0x7f47c00a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.403+0000 7f47e3b9b740 1 Processor -- start 2024-02-26T06:24:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.407+0000 7f47e3b9b740 1 -- start start 2024-02-26T06:24:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.407+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a50eaf0 0x55639a50d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.407+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 0x55639a50e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.407+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a517300 0x55639a5196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.407+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a4c95f0 con 0x55639a50eaf0 2024-02-26T06:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.407+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a47b890 con 0x55639a50dd80 2024-02-26T06:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.407+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a478dc0 con 0x55639a517300 2024-02-26T06:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.408+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a50eaf0 0x55639a50d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.408+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a50eaf0 0x55639a50d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44452/0 (socket says 172.21.15.49:44452) 2024-02-26T06:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.408+0000 7f47d9e00640 1 -- 172.21.15.49:0/1963996296 learned_addr learned my addr 172.21.15.49:0/1963996296 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.408+0000 7f47da800640 1 --2- 172.21.15.49:0/1963996296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a517300 0x55639a5196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.409+0000 7f47d9400640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f47c00a24c0 0x7f47c00a4980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47dc10fde0 tx=0x7f47c800a2d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.409+0000 7f47d9e00640 1 -- 172.21.15.49:0/1963996296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a517300 msgr2=0x55639a5196f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.409+0000 7f47d9e00640 1 --2- 172.21.15.49:0/1963996296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a517300 0x55639a5196f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.409+0000 7f47d9e00640 1 -- 172.21.15.49:0/1963996296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 msgr2=0x55639a50e200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.409+0000 7f47d9e00640 1 --2- 172.21.15.49:0/1963996296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 0x55639a50e200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.409+0000 7f47d9e00640 1 -- 172.21.15.49:0/1963996296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55639a3f5ad0 con 0x55639a50eaf0 2024-02-26T06:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.409+0000 7f47d9e00640 1 --2- 172.21.15.49:0/1963996296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a50eaf0 0x55639a50d840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006010 tx=0x7f47cc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252fb79faac25c0c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.410+0000 7f47c7000640 1 -- 172.21.15.49:0/1963996296 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0c1400 con 0x55639a50eaf0 2024-02-26T06:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.410+0000 7f47c7000640 1 -- 172.21.15.49:0/1963996296 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47cc0c5a50 con 0x55639a50eaf0 2024-02-26T06:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.410+0000 7f47c7000640 1 -- 172.21.15.49:0/1963996296 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0c1030 con 0x55639a50eaf0 2024-02-26T06:24:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.412+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1963996296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a50eaf0 msgr2=0x55639a50d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.413+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1963996296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a50eaf0 0x55639a50d840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006010 tx=0x7f47cc009260 comp rx=0 tx=0).stop 2024-02-26T06:24:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.414+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1963996296 shutdown_connections 2024-02-26T06:24:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.414+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1963996296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 0x55639a50e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.414+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1963996296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a517300 0x55639a5196f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.414+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1963996296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a50eaf0 0x55639a50d840 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.414+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1963996296 >> 172.21.15.49:0/1963996296 conn(0x55639a4d5480 msgr2=0x55639a508a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.415+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1963996296 shutdown_connections 2024-02-26T06:24:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.415+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1963996296 wait complete. 2024-02-26T06:24:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.415+0000 7f47e3b9b740 1 -- start start 2024-02-26T06:24:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.416+0000 7f47e3b9b740 1 Processor -- start 2024-02-26T06:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a514380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50eaf0 0x55639a5148c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a514e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a4745a0 con 0x55639a517300 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a419b20 con 0x55639a50eaf0 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a4197c0 con 0x55639a50dd80 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47e3b9b740 10 client.0 _ll_register_callbacks cb 0x55639a42d520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a514380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a514380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41030/0 (socket says 172.21.15.49:41030) 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47d9e00640 1 -- 172.21.15.49:0/1588482052 learned_addr learned my addr 172.21.15.49:0/1588482052 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47d9e00640 1 -- 172.21.15.49:0/1588482052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50eaf0 msgr2=0x55639a5148c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47d9e00640 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50eaf0 0x55639a5148c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47d9e00640 1 -- 172.21.15.49:0/1588482052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 msgr2=0x55639a514e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.418+0000 7f47d9e00640 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a514e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.419+0000 7f47d9e00640 1 -- 172.21.15.49:0/1588482052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc0721b0 con 0x55639a50dd80 2024-02-26T06:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.419+0000 7f47d9e00640 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a514380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f47cc02c2d0 tx=0x7f47cc0bd3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938d905697ca5f66 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.419+0000 7f47bb400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.419+0000 7f47bb400640 1 -- 172.21.15.49:0/1588482052 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0055b0 con 0x55639a50dd80 2024-02-26T06:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.419+0000 7f47bb400640 1 -- 172.21.15.49:0/1588482052 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f47cc0bb350 con 0x55639a50dd80 2024-02-26T06:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.419+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1588482052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55639a446ca0 con 0x55639a50dd80 2024-02-26T06:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.420+0000 7f47bb400640 1 -- 172.21.15.49:0/1588482052 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47cc0ee050 con 0x55639a50dd80 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.420+0000 7f47e3b9b740 10 client.25684 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.420+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1588482052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55639a472c20 con 0x55639a50dd80 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.420+0000 7f47bb400640 1 -- 172.21.15.49:0/1588482052 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55639a446ca0 con 0x55639a50dd80 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.420+0000 7f47bb400640 1 -- 172.21.15.49:0/1588482052 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f47cc03c070 con 0x55639a50dd80 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.421+0000 7f47e3b9b740 10 client.25684 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.421+0000 7f47e3b9b740 10 client.25684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.421+0000 7f47e3b9b740 20 client.25684 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.421+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55639a526d50 0x55639a529170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.421+0000 7f47e3b9b740 4 client.25684 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.421+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1588482052 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55639a46acb0 con 0x55639a526d50 2024-02-26T06:24:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.421+0000 7f47d9400640 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55639a526d50 0x55639a529170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.423+0000 7f47bb400640 10 client.25684 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.423+0000 7f47d9400640 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55639a526d50 0x55639a529170 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a633da5a93c832d server_cookie=d9a2d1ae3a6057ac in_seq=0 out_seq=0 2024-02-26T06:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.424+0000 7f47bb400640 1 -- 172.21.15.49:0/1588482052 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55639a46acb0 con 0x55639a526d50 2024-02-26T06:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.425+0000 7f47bb400640 10 client.25684 handle_command_reply: tid=0 2024-02-26T06:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.425+0000 7f47e3b9b740 1 client.25684 shutdown 2024-02-26T06:24:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.426+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1588482052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 msgr2=0x55639a514380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.426+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a514380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f47cc02c2d0 tx=0x7f47cc0bd3f0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.426+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1588482052 shutdown_connections 2024-02-26T06:24:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.427+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55639a526d50 0x55639a529170 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.427+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50eaf0 0x55639a5148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.427+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a514e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.427+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/1588482052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a514380 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.427+0000 7f47bb400640 0 client.25684 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.427+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1588482052 >> 172.21.15.49:0/1588482052 conn(0x55639a4d5480 msgr2=0x55639a50a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.429+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1588482052 shutdown_connections 2024-02-26T06:24:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.429+0000 7f47e3b9b740 1 -- 172.21.15.49:0/1588482052 wait complete. 2024-02-26T06:24:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.430+0000 7f47e3b9b740 20 client.25684 trim_cache size 0 max 16384 2024-02-26T06:24:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.463+0000 7f47e3b9b740 1 Processor -- start 2024-02-26T06:24:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.464+0000 7f47e3b9b740 1 -- start start 2024-02-26T06:24:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.465+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 0x55639a516ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 0x55639a516ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 0x55639a516ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51506/0 (socket says 172.21.15.49:51506) 2024-02-26T06:24:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50eaf0 0x55639a50d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47d9e00640 1 -- 172.21.15.49:0/2986782502 learned_addr learned my addr 172.21.15.49:0/2986782502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47d9400640 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50eaf0 0x55639a50d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a514c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2986782502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a1bce60 con 0x55639a517300 2024-02-26T06:24:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2986782502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a509aa0 con 0x55639a50dd80 2024-02-26T06:24:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.466+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2986782502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a50d050 con 0x55639a50eaf0 2024-02-26T06:24:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.467+0000 7f47d9400640 1 -- 172.21.15.49:0/2986782502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 msgr2=0x55639a516ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.467+0000 7f47d9400640 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 0x55639a516ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.467+0000 7f47d9400640 1 -- 172.21.15.49:0/2986782502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 msgr2=0x55639a514c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.467+0000 7f47da800640 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a514c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.467+0000 7f47d9400640 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a514c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.467+0000 7f47d9400640 1 -- 172.21.15.49:0/2986782502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55639a40f500 con 0x55639a50eaf0 2024-02-26T06:24:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.467+0000 7f47d9400640 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50eaf0 0x55639a50d760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55639a529a80 tx=0x7f47c8012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3a83232c0520557 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.468+0000 7f47c6600640 1 -- 172.21.15.49:0/2986782502 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8021040 con 0x55639a50eaf0 2024-02-26T06:24:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.468+0000 7f47c6600640 1 -- 172.21.15.49:0/2986782502 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47c8028070 con 0x55639a50eaf0 2024-02-26T06:24:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.468+0000 7f47c6600640 1 -- 172.21.15.49:0/2986782502 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c801c420 con 0x55639a50eaf0 2024-02-26T06:24:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.469+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2986782502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50eaf0 msgr2=0x55639a50d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.469+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50eaf0 0x55639a50d760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55639a529a80 tx=0x7f47c8012000 comp rx=0 tx=0).stop 2024-02-26T06:24:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.469+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2986782502 shutdown_connections 2024-02-26T06:24:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.469+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a514c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.469+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50eaf0 0x55639a50d760 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.469+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2986782502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a50dd80 0x55639a516ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.469+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2986782502 >> 172.21.15.49:0/2986782502 conn(0x55639a4d5480 msgr2=0x55639a512c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.470+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2986782502 shutdown_connections 2024-02-26T06:24:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.470+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2986782502 wait complete. 2024-02-26T06:24:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.470+0000 7f47e3b9b740 1 -- start start 2024-02-26T06:24:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.471+0000 7f47e3b9b740 1 Processor -- start 2024-02-26T06:24:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.471+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a51cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.471+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a51d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.471+0000 7f47e3b9b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a51d640 0x55639a57d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.471+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a57d840 con 0x55639a517300 2024-02-26T06:24:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.471+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a57d9b0 con 0x55639a51d640 2024-02-26T06:24:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.471+0000 7f47e3b9b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55639a57db70 con 0x55639a50dd80 2024-02-26T06:24:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47e3b9b740 10 client.0 _ll_register_callbacks cb 0x7f47cc0bcec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a51cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a51cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41052/0 (socket says 172.21.15.49:41052) 2024-02-26T06:24:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47d9e00640 1 -- 172.21.15.49:0/2564405265 learned_addr learned my addr 172.21.15.49:0/2564405265 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47d9e00640 1 -- 172.21.15.49:0/2564405265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a51d640 msgr2=0x55639a57d260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:24:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47d9e00640 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a51d640 0x55639a57d260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47d9e00640 1 -- 172.21.15.49:0/2564405265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 msgr2=0x55639a51d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47d9e00640 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a51d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.472+0000 7f47d9e00640 1 -- 172.21.15.49:0/2564405265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c800a7b0 con 0x55639a50dd80 2024-02-26T06:24:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.473+0000 7f47d9e00640 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a51cb70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0bb800 tx=0x7f47cc0bc8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54dfcafd0d9181ca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.473+0000 7f47bb400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.473+0000 7f47bb400640 1 -- 172.21.15.49:0/2564405265 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0bc9c0 con 0x55639a50dd80 2024-02-26T06:24:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.473+0000 7f47bb400640 1 -- 172.21.15.49:0/2564405265 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f47cc0c1400 con 0x55639a50dd80 2024-02-26T06:24:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.474+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2564405265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55639a499d00 con 0x55639a50dd80 2024-02-26T06:24:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.474+0000 7f47bb400640 1 -- 172.21.15.49:0/2564405265 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47cc0ee020 con 0x55639a50dd80 2024-02-26T06:24:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.474+0000 7f47e3b9b740 10 client.25690 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.474+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2564405265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55639a435290 con 0x55639a50dd80 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47bb400640 1 -- 172.21.15.49:0/2564405265 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55639a499d00 con 0x55639a50dd80 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47bb400640 1 -- 172.21.15.49:0/2564405265 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f47cc03c070 con 0x55639a50dd80 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47e3b9b740 10 client.25690 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47e3b9b740 10 client.25690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47e3b9b740 20 client.25690 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55639a57ee70 0x55639a581260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47e3b9b740 4 client.25690 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2564405265 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55639a174600 con 0x55639a57ee70 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.475+0000 7f47d9400640 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55639a57ee70 0x55639a581260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.476+0000 7f47bb400640 10 client.25690 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.477+0000 7f47d9400640 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55639a57ee70 0x55639a581260 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4cb18ec1079b50a server_cookie=fbab451363dec66c in_seq=0 out_seq=0 2024-02-26T06:24:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.491+0000 7f47bb400640 1 -- 172.21.15.49:0/2564405265 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55639a174600 con 0x55639a57ee70 2024-02-26T06:24:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.491+0000 7f47bb400640 10 client.25690 handle_command_reply: tid=0 2024-02-26T06:24:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.491+0000 7f47e3b9b740 1 client.25690 shutdown 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2564405265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 msgr2=0x55639a51cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a51cb70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0bb800 tx=0x7f47cc0bc8d0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2564405265 shutdown_connections 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55639a57ee70 0x55639a581260 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55639a51d640 0x55639a57d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55639a517300 0x55639a51d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47e3b9b740 1 --2- 172.21.15.49:0/2564405265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55639a50dd80 0x55639a51cb70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2564405265 >> 172.21.15.49:0/2564405265 conn(0x55639a4d5480 msgr2=0x55639a507480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.492+0000 7f47bb400640 0 client.25690 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.493+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2564405265 shutdown_connections 2024-02-26T06:24:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.493+0000 7f47e3b9b740 1 -- 172.21.15.49:0/2564405265 wait complete. 2024-02-26T06:24:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.493+0000 7f47e3b9b740 20 client.25690 trim_cache size 0 max 16384 2024-02-26T06:24:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:24:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:24:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:24:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:24:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.495+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f47c00a24c0 msgr2=0x7f47c00a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.495+0000 7f47e0c00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f47c00a24c0 0x7f47c00a4980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47dc10fde0 tx=0x7f47c800a2d0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.495+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc068800 msgr2=0x7f47dc10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.495+0000 7f47e0c00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc068800 0x7f47dc10e620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc02dab0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.496+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 shutdown_connections 2024-02-26T06:24:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.496+0000 7f47e0c00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f47c00a24c0 0x7f47c00a4980 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.496+0000 7f47e0c00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47dc0691a0 0x7f47dc10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.496+0000 7f47e0c00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc107270 0x7f47dc10f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.496+0000 7f47e0c00640 1 --2- 172.21.15.49:0/115211062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47dc068800 0x7f47dc10e620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.496+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 >> 172.21.15.49:0/115211062 conn(0x7f47dc076ed0 msgr2=0x7f47dc0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.496+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 shutdown_connections 2024-02-26T06:24:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:19.497+0000 7f47e0c00640 1 -- 172.21.15.49:0/115211062 wait complete. 2024-02-26T06:24:19.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:19 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4173291291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 -- 172.21.15.49:0/3694543386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c74106080 msgr2=0x7f6c740696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 --2- 172.21.15.49:0/3694543386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c74106080 0x7f6c740696f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009900 tx=0x7f6c6402bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 -- 172.21.15.49:0/3694543386 shutdown_connections 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 --2- 172.21.15.49:0/3694543386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c7406a310 0x7f6c74103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 --2- 172.21.15.49:0/3694543386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c741064f0 0x7f6c74069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 --2- 172.21.15.49:0/3694543386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c74106080 0x7f6c740696f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 -- 172.21.15.49:0/3694543386 >> 172.21.15.49:0/3694543386 conn(0x7f6c74077060 msgr2=0x7f6c74077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 -- 172.21.15.49:0/3694543386 shutdown_connections 2024-02-26T06:24:19.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.687+0000 7f6c79400640 1 -- 172.21.15.49:0/3694543386 wait complete. 2024-02-26T06:24:19.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.689+0000 7f6c79400640 1 Processor -- start 2024-02-26T06:24:19.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.689+0000 7f6c79400640 1 -- start start 2024-02-26T06:24:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.689+0000 7f6c79400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74106080 0x7f6c7419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.689+0000 7f6c79400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c741064f0 0x7f6c7419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.689+0000 7f6c79400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c741a2680 0x7f6c741a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.689+0000 7f6c79400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c741a3070 con 0x7f6c741064f0 2024-02-26T06:24:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.689+0000 7f6c79400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c741a31e0 con 0x7f6c741a2680 2024-02-26T06:24:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.689+0000 7f6c79400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c741a3350 con 0x7f6c74106080 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c72800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74106080 0x7f6c7419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c72800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74106080 0x7f6c7419ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41078/0 (socket says 172.21.15.49:41078) 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c72800640 1 -- 172.21.15.49:0/2721104437 learned_addr learned my addr 172.21.15.49:0/2721104437 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c73200640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c741a2680 0x7f6c741a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c71e00640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c741064f0 0x7f6c7419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c72800640 1 -- 172.21.15.49:0/2721104437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c741a2680 msgr2=0x7f6c741a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c72800640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c741a2680 0x7f6c741a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c72800640 1 -- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c741064f0 msgr2=0x7f6c7419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c72800640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c741064f0 0x7f6c7419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.690+0000 7f6c72800640 1 -- 172.21.15.49:0/2721104437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64009560 con 0x7f6c74106080 2024-02-26T06:24:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.691+0000 7f6c72800640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74106080 0x7f6c7419ddc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000c00 tx=0x7f6c6402c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.691+0000 7f6c6b400640 1 -- 172.21.15.49:0/2721104437 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64004410 con 0x7f6c74106080 2024-02-26T06:24:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.691+0000 7f6c79400640 1 -- 172.21.15.49:0/2721104437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c741aa930 con 0x7f6c74106080 2024-02-26T06:24:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.691+0000 7f6c6b400640 1 -- 172.21.15.49:0/2721104437 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c640052d0 con 0x7f6c74106080 2024-02-26T06:24:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.691+0000 7f6c6b400640 1 -- 172.21.15.49:0/2721104437 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64003ca0 con 0x7f6c74106080 2024-02-26T06:24:19.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.692+0000 7f6c79400640 1 -- 172.21.15.49:0/2721104437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c741aadb0 con 0x7f6c74106080 2024-02-26T06:24:19.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.695+0000 7f6c7c22c740 1 Processor -- start 2024-02-26T06:24:19.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.695+0000 7f6c7c22c740 1 -- start start 2024-02-26T06:24:19.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.695+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa851ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.695+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 0x5583fa85abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.695+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa85d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.695+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa7c01e0 con 0x5583fa852530 2024-02-26T06:24:19.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.695+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa7bd710 con 0x5583fa85b110 2024-02-26T06:24:19.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.695+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa7b8ef0 con 0x5583fa851c10 2024-02-26T06:24:19.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.696+0000 7f6c73200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa851ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.696+0000 7f6c73200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa851ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41090/0 (socket says 172.21.15.49:41090) 2024-02-26T06:24:19.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.696+0000 7f6c73200640 1 -- 172.21.15.49:0/2912664505 learned_addr learned my addr 172.21.15.49:0/2912664505 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.696+0000 7f6c6b400640 1 -- 172.21.15.49:0/2721104437 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f6c64005440 con 0x7f6c74106080 2024-02-26T06:24:19.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.696+0000 7f6c71e00640 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa85d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.696+0000 7f6c73200640 1 -- 172.21.15.49:0/2912664505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 msgr2=0x5583fa85d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.696+0000 7f6c73200640 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa85d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.696+0000 7f6c73200640 1 -- 172.21.15.49:0/2912664505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 msgr2=0x5583fa85abd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:19.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.697+0000 7f6c72800640 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 0x5583fa85abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.697+0000 7f6c73200640 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 0x5583fa85abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.697+0000 7f6c73200640 1 -- 172.21.15.49:0/2912664505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583fa77adc0 con 0x5583fa851c10 2024-02-26T06:24:19.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.697+0000 7f6c71e00640 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa85d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:19.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.698+0000 7f6c73200640 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa851ff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009fc0 tx=0x7f6c6c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9071e3580b22d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.698+0000 7f6c5f400640 1 -- 172.21.15.49:0/2912664505 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004d20 con 0x5583fa851c10 2024-02-26T06:24:19.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.698+0000 7f6c6b400640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c580a2350 0x7f6c580a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.698+0000 7f6c5f400640 1 -- 172.21.15.49:0/2912664505 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c004e80 con 0x5583fa851c10 2024-02-26T06:24:19.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.698+0000 7f6c5f400640 1 -- 172.21.15.49:0/2912664505 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004d20 con 0x5583fa851c10 2024-02-26T06:24:19.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.699+0000 7f6c6b400640 1 -- 172.21.15.49:0/2721104437 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6c640f09e0 con 0x7f6c74106080 2024-02-26T06:24:19.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.700+0000 7f6c72800640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c580a2350 0x7f6c580a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.701+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2912664505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 msgr2=0x5583fa851ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.701+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa851ff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009fc0 tx=0x7f6c6c010800 comp rx=0 tx=0).stop 2024-02-26T06:24:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.701+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2912664505 shutdown_connections 2024-02-26T06:24:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.701+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa85d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.701+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 0x5583fa85abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.701+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2912664505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa851ff0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.701+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2912664505 >> 172.21.15.49:0/2912664505 conn(0x5583fa84c340 msgr2=0x5583fa84c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.702+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2912664505 shutdown_connections 2024-02-26T06:24:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.702+0000 7f6c72800640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c580a2350 0x7f6c580a4810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c640ee110 tx=0x7f6c640b9040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.702+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2912664505 wait complete. 2024-02-26T06:24:19.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.702+0000 7f6c7c22c740 1 -- start start 2024-02-26T06:24:19.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.703+0000 7f6c7c22c740 1 Processor -- start 2024-02-26T06:24:19.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.703+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa858160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa85b110 0x5583fa858be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c71e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c71e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41100/0 (socket says 172.21.15.49:41100) 2024-02-26T06:24:19.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c73200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa858160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c73200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa858160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51544/0 (socket says 172.21.15.49:51544) 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa75e120 con 0x5583fa85b110 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa75ddc0 con 0x5583fa851c10 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.704+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa5271c0 con 0x5583fa852530 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.705+0000 7f6c71e00640 1 -- 172.21.15.49:0/3081431645 learned_addr learned my addr 172.21.15.49:0/3081431645 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.705+0000 7f6c71e00640 1 -- 172.21.15.49:0/3081431645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 msgr2=0x5583fa858160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.705+0000 7f6c71e00640 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa858160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.705+0000 7f6c71e00640 1 -- 172.21.15.49:0/3081431645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa85b110 msgr2=0x5583fa858be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.705+0000 7f6c71e00640 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa85b110 0x5583fa858be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.705+0000 7f6c71e00640 1 -- 172.21.15.49:0/3081431645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c009bf0 con 0x5583fa852530 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.705+0000 7f6c73200640 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa858160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-26T06:24:19.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.705+0000 7f6c71e00640 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8586a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c60012730 tx=0x7f6c60012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf0b06c51ddf902 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.706+0000 7f6c53e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.706+0000 7f6c53e00640 1 -- 172.21.15.49:0/3081431645 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60004460 con 0x5583fa852530 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.706+0000 7f6c53e00640 1 -- 172.21.15.49:0/3081431645 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6c600109b0 con 0x5583fa852530 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.706+0000 7f6c7c22c740 10 client.0 _ll_register_callbacks cb 0x5583fa7ad280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.706+0000 7f6c7c22c740 1 -- 172.21.15.49:0/3081431645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583fa4f5720 con 0x5583fa852530 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c53e00640 1 -- 172.21.15.49:0/3081431645 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c60010b70 con 0x5583fa852530 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c7c22c740 10 client.25702 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c7c22c740 1 -- 172.21.15.49:0/3081431645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x5583fa7b7570 con 0x5583fa852530 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c53e00640 1 -- 172.21.15.49:0/3081431645 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6c60012d20 con 0x5583fa852530 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c7c22c740 10 client.25702 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c7c22c740 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c7c22c740 20 client.25702 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583fa86ad30 0x5583fa86d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c7c22c740 4 client.25702 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:19.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.707+0000 7f6c7c22c740 1 -- 172.21.15.49:0/3081431645 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583fa544890 con 0x5583fa86ad30 2024-02-26T06:24:19.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.708+0000 7f6c73200640 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583fa86ad30 0x5583fa86d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.708+0000 7f6c73200640 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583fa86ad30 0x5583fa86d1d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba6ca315fade32c6 server_cookie=e9a2f4b36c56582d in_seq=0 out_seq=0 2024-02-26T06:24:19.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.709+0000 7f6c53e00640 10 client.25702 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:19.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.709+0000 7f6c53e00640 1 -- 172.21.15.49:0/3081431645 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583fa544890 con 0x5583fa86ad30 2024-02-26T06:24:19.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.710+0000 7f6c53e00640 10 client.25702 handle_command_reply: tid=0 2024-02-26T06:24:19.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.710+0000 7f6c7c22c740 1 client.25702 shutdown 2024-02-26T06:24:19.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.710+0000 7f6c7c22c740 1 -- 172.21.15.49:0/3081431645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 msgr2=0x5583fa8586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.710+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8586a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c60012730 tx=0x7f6c60012bf0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.710+0000 7f6c7c22c740 1 -- 172.21.15.49:0/3081431645 shutdown_connections 2024-02-26T06:24:19.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.710+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583fa86ad30 0x5583fa86d1d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.711+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa85b110 0x5583fa858be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.711+0000 7f6c53e00640 0 client.25702 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.711+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8586a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.711+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/3081431645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa858160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.711+0000 7f6c7c22c740 1 -- 172.21.15.49:0/3081431645 >> 172.21.15.49:0/3081431645 conn(0x5583fa84c340 msgr2=0x5583fa84e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.711+0000 7f6c7c22c740 1 -- 172.21.15.49:0/3081431645 shutdown_connections 2024-02-26T06:24:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.711+0000 7f6c7c22c740 1 -- 172.21.15.49:0/3081431645 wait complete. 2024-02-26T06:24:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.711+0000 7f6c7c22c740 20 client.25702 trim_cache size 0 max 16384 2024-02-26T06:24:19.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.734+0000 7f6c7c22c740 1 Processor -- start 2024-02-26T06:24:19.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.734+0000 7f6c7c22c740 1 -- start start 2024-02-26T06:24:19.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.734+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa85a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.734+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 0x5583fa8553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.734+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa855920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.734+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa85ae70 con 0x5583fa852530 2024-02-26T06:24:19.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.734+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa544890 con 0x5583fa85b110 2024-02-26T06:24:19.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.734+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa867640 con 0x5583fa851c10 2024-02-26T06:24:19.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.735+0000 7f6c73200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa85a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.735+0000 7f6c73200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa85a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41106/0 (socket says 172.21.15.49:41106) 2024-02-26T06:24:19.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.735+0000 7f6c73200640 1 -- 172.21.15.49:0/1402218761 learned_addr learned my addr 172.21.15.49:0/1402218761 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.735+0000 7f6c73200640 1 --2- 172.21.15.49:0/1402218761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa855920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.736+0000 7f6c73200640 1 -- 172.21.15.49:0/1402218761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 msgr2=0x5583fa855920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.736+0000 7f6c73200640 1 --2- 172.21.15.49:0/1402218761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa855920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.736+0000 7f6c73200640 1 -- 172.21.15.49:0/1402218761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 msgr2=0x5583fa8553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.736+0000 7f6c73200640 1 --2- 172.21.15.49:0/1402218761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 0x5583fa8553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.736+0000 7f6c73200640 1 -- 172.21.15.49:0/1402218761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583fa753a80 con 0x5583fa851c10 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.736+0000 7f6c73200640 1 --2- 172.21.15.49:0/1402218761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa85a890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005300 tx=0x7f6c6c009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b1a270e97c2604 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.736+0000 7f6c5ea00640 1 -- 172.21.15.49:0/1402218761 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c01e040 con 0x5583fa851c10 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.737+0000 7f6c5ea00640 1 -- 172.21.15.49:0/1402218761 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c0298e0 con 0x5583fa851c10 2024-02-26T06:24:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.737+0000 7f6c5ea00640 1 -- 172.21.15.49:0/1402218761 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c012b80 con 0x5583fa851c10 2024-02-26T06:24:19.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.738+0000 7f6c7c22c740 1 -- 172.21.15.49:0/1402218761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 msgr2=0x5583fa85a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.738+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/1402218761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa85a890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005300 tx=0x7f6c6c009800 comp rx=0 tx=0).stop 2024-02-26T06:24:19.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.738+0000 7f6c7c22c740 1 -- 172.21.15.49:0/1402218761 shutdown_connections 2024-02-26T06:24:19.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.738+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/1402218761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa85b110 0x5583fa855920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.738+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/1402218761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa852530 0x5583fa8553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.738+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/1402218761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa851c10 0x5583fa85a890 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.738+0000 7f6c7c22c740 1 -- 172.21.15.49:0/1402218761 >> 172.21.15.49:0/1402218761 conn(0x5583fa84c340 msgr2=0x5583fa871b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.739+0000 7f6c7c22c740 1 -- 172.21.15.49:0/1402218761 shutdown_connections 2024-02-26T06:24:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.740+0000 7f6c7c22c740 1 -- 172.21.15.49:0/1402218761 wait complete. 2024-02-26T06:24:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.740+0000 7f6c7c22c740 1 -- start start 2024-02-26T06:24:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.741+0000 7f6c7c22c740 1 Processor -- start 2024-02-26T06:24:19.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.743+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa8bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c7c22c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa85b110 0x5583fa861420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa861af0 con 0x5583fa85b110 2024-02-26T06:24:19.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa861c60 con 0x5583fa851c10 2024-02-26T06:24:19.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c7c22c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fa861e20 con 0x5583fa852530 2024-02-26T06:24:19.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c73200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa8bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c73200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa8bb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51564/0 (socket says 172.21.15.49:51564) 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c73200640 1 -- 172.21.15.49:0/2538571889 learned_addr learned my addr 172.21.15.49:0/2538571889 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c7c22c740 10 client.0 _ll_register_callbacks cb 0x7f6c6c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.744+0000 7f6c71e00640 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c73200640 1 -- 172.21.15.49:0/2538571889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 msgr2=0x5583fa8bbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c73200640 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8bbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c73200640 1 -- 172.21.15.49:0/2538571889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa85b110 msgr2=0x5583fa861420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c73200640 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa85b110 0x5583fa861420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c73200640 1 -- 172.21.15.49:0/2538571889 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583fa77adc0 con 0x5583fa851c10 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c71e00640 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:19.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c73200640 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa8bb8d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c012680 tx=0x7f6c6c005290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2604df417bd499 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:19.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c53e00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:19.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c53e00640 1 -- 172.21.15.49:0/2538571889 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c01e040 con 0x5583fa851c10 2024-02-26T06:24:19.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c53e00640 1 -- 172.21.15.49:0/2538571889 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6c6c003c40 con 0x5583fa851c10 2024-02-26T06:24:19.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.745+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2538571889 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583fa4b87e0 con 0x5583fa851c10 2024-02-26T06:24:19.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.746+0000 7f6c53e00640 1 -- 172.21.15.49:0/2538571889 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c6c003e00 con 0x5583fa851c10 2024-02-26T06:24:19.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.746+0000 7f6c7c22c740 10 client.25670 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:19.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.746+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2538571889 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5583fa779620 con 0x5583fa851c10 2024-02-26T06:24:19.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.746+0000 7f6c53e00640 1 -- 172.21.15.49:0/2538571889 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c039030 con 0x5583fa851c10 2024-02-26T06:24:19.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.747+0000 7f6c53e00640 1 -- 172.21.15.49:0/2538571889 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6c6c004040 con 0x5583fa851c10 2024-02-26T06:24:19.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.747+0000 7f6c7c22c740 10 client.25670 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:19.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.747+0000 7f6c7c22c740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:19.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.747+0000 7f6c7c22c740 20 client.25670 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:19.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.747+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583fa8c32a0 0x5583fa8c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:19.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.747+0000 7f6c7c22c740 4 client.25670 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:24:19.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.747+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2538571889 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5583fa4a74e0 con 0x5583fa8c32a0 2024-02-26T06:24:19.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.747+0000 7f6c71e00640 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583fa8c32a0 0x5583fa8c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:19.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.748+0000 7f6c53e00640 10 client.25670 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:19.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.750+0000 7f6c71e00640 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583fa8c32a0 0x5583fa8c5690 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06750545bfc3ea8 server_cookie=885636d186fb6473 in_seq=0 out_seq=0 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c53e00640 1 -- 172.21.15.49:0/2538571889 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5583fa4a74e0 con 0x5583fa8c32a0 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c53e00640 10 client.25670 handle_command_reply: tid=0 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 client.25670 shutdown 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2538571889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 msgr2=0x5583fa8bb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa8bb8d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c012680 tx=0x7f6c6c005290 comp rx=0 tx=0).stop 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2538571889 shutdown_connections 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583fa8c32a0 0x5583fa8c5690 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583fa85b110 0x5583fa861420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583fa852530 0x5583fa8bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 --2- 172.21.15.49:0/2538571889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583fa851c10 0x5583fa8bb8d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c53e00640 0 client.25670 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:19.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.751+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2538571889 >> 172.21.15.49:0/2538571889 conn(0x5583fa84c340 msgr2=0x5583fa84abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.752+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2538571889 shutdown_connections 2024-02-26T06:24:19.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.752+0000 7f6c7c22c740 1 -- 172.21.15.49:0/2538571889 wait complete. 2024-02-26T06:24:19.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.752+0000 7f6c7c22c740 20 client.25670 trim_cache size 0 max 16384 2024-02-26T06:24:19.753 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:24:19.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 -- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c580a2350 msgr2=0x7f6c580a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c580a2350 0x7f6c580a4810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c640ee110 tx=0x7f6c640b9040 comp rx=0 tx=0).stop 2024-02-26T06:24:19.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 -- 172.21.15.49:0/2721104437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74106080 msgr2=0x7f6c7419ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:19.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74106080 0x7f6c7419ddc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000c00 tx=0x7f6c6402c6a0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 -- 172.21.15.49:0/2721104437 shutdown_connections 2024-02-26T06:24:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c580a2350 0x7f6c580a4810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c741a2680 0x7f6c741a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c741064f0 0x7f6c7419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 --2- 172.21.15.49:0/2721104437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74106080 0x7f6c7419ddc0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000c00 tx=0x7f6c6402c6a0 comp rx=0 tx=0).stop 2024-02-26T06:24:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 -- 172.21.15.49:0/2721104437 >> 172.21.15.49:0/2721104437 conn(0x7f6c74077060 msgr2=0x7f6c74102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 -- 172.21.15.49:0/2721104437 shutdown_connections 2024-02-26T06:24:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:19.753+0000 7f6c79400640 1 -- 172.21.15.49:0/2721104437 wait complete. 2024-02-26T06:24:19.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:19 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4173291291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:19.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:19 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4173291291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:20.783 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:24:20.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:24:20.784 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:24:20.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:20 smithi080 ceph-mon[36206]: pgmap v31: 97 pgs: 97 active+clean; 991 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 MiB/s wr, 1.34k op/s 2024-02-26T06:24:20.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:20 smithi115 ceph-mon[36426]: pgmap v31: 97 pgs: 97 active+clean; 991 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 MiB/s wr, 1.34k op/s 2024-02-26T06:24:20.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:20 smithi049 ceph-mon[31696]: pgmap v31: 97 pgs: 97 active+clean; 991 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 MiB/s wr, 1.34k op/s 2024-02-26T06:24:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.075+0000 7fea52a00640 1 -- 172.21.15.49:0/4063075434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 msgr2=0x7fea4c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.075+0000 7fea52a00640 1 --2- 172.21.15.49:0/4063075434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 0x7fea4c10db10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fea38009900 tx=0x7fea3802bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.076+0000 7fea52a00640 1 -- 172.21.15.49:0/4063075434 shutdown_connections 2024-02-26T06:24:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.077+0000 7fea52a00640 1 --2- 172.21.15.49:0/4063075434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea4c1080e0 0x7fea4c1145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.077+0000 7fea52a00640 1 --2- 172.21.15.49:0/4063075434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4c107720 0x7fea4c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.077+0000 7fea52a00640 1 --2- 172.21.15.49:0/4063075434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 0x7fea4c10db10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.077+0000 7fea52a00640 1 -- 172.21.15.49:0/4063075434 >> 172.21.15.49:0/4063075434 conn(0x7fea4c0756c0 msgr2=0x7fea4c077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.077+0000 7fea52a00640 1 -- 172.21.15.49:0/4063075434 shutdown_connections 2024-02-26T06:24:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.077+0000 7fea52a00640 1 -- 172.21.15.49:0/4063075434 wait complete. 2024-02-26T06:24:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.078+0000 7fea52a00640 1 Processor -- start 2024-02-26T06:24:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.078+0000 7fea52a00640 1 -- start start 2024-02-26T06:24:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea52a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4c107720 0x7fea4c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea52a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea4c1080e0 0x7fea4c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea52a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 0x7fea4c1a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea52a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c1a2bf0 con 0x7fea4c1080e0 2024-02-26T06:24:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea52a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c1a2d60 con 0x7fea4c10d730 2024-02-26T06:24:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea52a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c1a2f20 con 0x7fea4c107720 2024-02-26T06:24:21.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea4be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4c107720 0x7fea4c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea50a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 0x7fea4c1a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea4be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4c107720 0x7fea4c19dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46920/0 (socket says 172.21.15.49:46920) 2024-02-26T06:24:21.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea4be00640 1 -- 172.21.15.49:0/832951311 learned_addr learned my addr 172.21.15.49:0/832951311 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea4b400640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea4c1080e0 0x7fea4c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea50a00640 1 -- 172.21.15.49:0/832951311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4c107720 msgr2=0x7fea4c19dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea50a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4c107720 0x7fea4c19dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea50a00640 1 -- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea4c1080e0 msgr2=0x7fea4c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea50a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea4c1080e0 0x7fea4c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea50a00640 1 -- 172.21.15.49:0/832951311 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38009560 con 0x7fea4c10d730 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.079+0000 7fea4be00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4c107720 0x7fea4c19dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.080+0000 7fea50a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 0x7fea4c1a25f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fea4400cab0 tx=0x7fea4400cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.080+0000 7fea48c00640 1 -- 172.21.15.49:0/832951311 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4400ae20 con 0x7fea4c10d730 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.080+0000 7fea48c00640 1 -- 172.21.15.49:0/832951311 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea4400f930 con 0x7fea4c10d730 2024-02-26T06:24:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.080+0000 7fea52a00640 1 -- 172.21.15.49:0/832951311 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c1a3200 con 0x7fea4c10d730 2024-02-26T06:24:21.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.080+0000 7fea48c00640 1 -- 172.21.15.49:0/832951311 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44018a50 con 0x7fea4c10d730 2024-02-26T06:24:21.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.080+0000 7fea52a00640 1 -- 172.21.15.49:0/832951311 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c1aaa80 con 0x7fea4c10d730 2024-02-26T06:24:21.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.085+0000 7fea48c00640 1 -- 172.21.15.49:0/832951311 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fea440213f0 con 0x7fea4c10d730 2024-02-26T06:24:21.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.088+0000 7fea559a2740 1 Processor -- start 2024-02-26T06:24:21.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.088+0000 7fea559a2740 1 -- start start 2024-02-26T06:24:21.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea559a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d319852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea559a2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d3198ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea559a2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31990800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea559a2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d319402d0 con 0x557d3198e410 2024-02-26T06:24:21.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea559a2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d318f2e20 con 0x557d31985830 2024-02-26T06:24:21.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea559a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d318f0350 con 0x557d31984f10 2024-02-26T06:24:21.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea48c00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea300a2450 0x7fea300a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea4b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d319852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea4b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d319852f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46934/0 (socket says 172.21.15.49:46934) 2024-02-26T06:24:21.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea4b400640 1 -- 172.21.15.49:0/292686230 learned_addr learned my addr 172.21.15.49:0/292686230 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea48c00640 1 -- 172.21.15.49:0/832951311 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fea440cea00 con 0x7fea4c10d730 2024-02-26T06:24:21.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.089+0000 7fea50a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea300a2450 0x7fea300a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.090+0000 7fea4b400640 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31990800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.090+0000 7fea4be00640 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d3198ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.091+0000 7fea4b400640 1 -- 172.21.15.49:0/292686230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 msgr2=0x557d3198ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.091+0000 7fea4b400640 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d3198ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.092+0000 7fea50a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea300a2450 0x7fea300a4910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea440cbfe0 tx=0x7fea44018bf0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.092+0000 7fea4b400640 1 -- 172.21.15.49:0/292686230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 msgr2=0x557d31990800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.092+0000 7fea4b400640 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31990800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.092+0000 7fea4b400640 1 -- 172.21.15.49:0/292686230 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d318b4c50 con 0x557d31984f10 2024-02-26T06:24:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.093+0000 7fea4b400640 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d319852f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0127c0 tx=0x7fea3c00b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d3c890d6dc963f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.094+0000 7fea41200640 1 -- 172.21.15.49:0/292686230 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c012e00 con 0x557d31984f10 2024-02-26T06:24:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.094+0000 7fea41200640 1 -- 172.21.15.49:0/292686230 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea3c010c50 con 0x557d31984f10 2024-02-26T06:24:21.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.094+0000 7fea41200640 1 -- 172.21.15.49:0/292686230 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c01eac0 con 0x557d31984f10 2024-02-26T06:24:21.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.095+0000 7fea559a2740 1 -- 172.21.15.49:0/292686230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 msgr2=0x557d319852f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.095+0000 7fea559a2740 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d319852f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0127c0 tx=0x7fea3c00b7c0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.095+0000 7fea559a2740 1 -- 172.21.15.49:0/292686230 shutdown_connections 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.095+0000 7fea559a2740 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d3198ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.095+0000 7fea559a2740 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31990800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.095+0000 7fea559a2740 1 --2- 172.21.15.49:0/292686230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d319852f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.095+0000 7fea559a2740 1 -- 172.21.15.49:0/292686230 >> 172.21.15.49:0/292686230 conn(0x557d3197f640 msgr2=0x557d3197fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.095+0000 7fea559a2740 1 -- 172.21.15.49:0/292686230 shutdown_connections 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.096+0000 7fea559a2740 1 -- 172.21.15.49:0/292686230 wait complete. 2024-02-26T06:24:21.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.096+0000 7fea559a2740 1 -- start start 2024-02-26T06:24:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.096+0000 7fea559a2740 1 Processor -- start 2024-02-26T06:24:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea559a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea559a2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d31985830 0x557d3198b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea559a2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d3198e410 0x557d3198be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea559a2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d318ebb30 con 0x557d31985830 2024-02-26T06:24:21.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea559a2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d31890fa0 con 0x557d3198e410 2024-02-26T06:24:21.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea559a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d31890c40 con 0x557d31984f10 2024-02-26T06:24:21.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea559a2740 10 client.0 _ll_register_callbacks cb 0x557d315fcc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:21.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea4b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea4b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46948/0 (socket says 172.21.15.49:46948) 2024-02-26T06:24:21.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea4b400640 1 -- 172.21.15.49:0/2884442045 learned_addr learned my addr 172.21.15.49:0/2884442045 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea4b400640 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d3198e410 0x557d3198be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.097+0000 7fea4be00640 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d31985830 0x557d3198b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea4b400640 1 -- 172.21.15.49:0/2884442045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d3198e410 msgr2=0x557d3198be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea4b400640 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d3198e410 0x557d3198be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea4b400640 1 -- 172.21.15.49:0/2884442045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d31985830 msgr2=0x557d3198b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea4b400640 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d31985830 0x557d3198b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea4b400640 1 -- 172.21.15.49:0/2884442045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c01a040 con 0x557d31984f10 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea4b400640 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198b3f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00be50 tx=0x7fea3c01e620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b4cdfc81ac1761 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea35600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea35600640 1 -- 172.21.15.49:0/2884442045 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c010830 con 0x557d31984f10 2024-02-26T06:24:21.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea35600640 1 -- 172.21.15.49:0/2884442045 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fea3c010990 con 0x557d31984f10 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.098+0000 7fea559a2740 1 -- 172.21.15.49:0/2884442045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d315eb860 con 0x557d31984f10 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea35600640 1 -- 172.21.15.49:0/2884442045 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea3c005020 con 0x557d31984f10 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea559a2740 10 client.25720 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea559a2740 1 -- 172.21.15.49:0/2884442045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557d318ea1b0 con 0x557d31984f10 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea35600640 1 -- 172.21.15.49:0/2884442045 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557d315eb860 con 0x557d31984f10 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea35600640 1 -- 172.21.15.49:0/2884442045 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fea3c01ed00 con 0x557d31984f10 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea559a2740 10 client.25720 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea559a2740 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea559a2740 20 client.25720 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea559a2740 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557d3199e7c0 0x557d319a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea559a2740 4 client.25720 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.099+0000 7fea559a2740 1 -- 172.21.15.49:0/2884442045 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d3163c410 con 0x557d3199e7c0 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.100+0000 7fea4be00640 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557d3199e7c0 0x557d319a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.100+0000 7fea4be00640 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557d3199e7c0 0x557d319a0be0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5235bfd344b40890 server_cookie=7732e3077ee4392e in_seq=0 out_seq=0 2024-02-26T06:24:21.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.100+0000 7fea35600640 10 client.25720 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.101+0000 7fea35600640 1 -- 172.21.15.49:0/2884442045 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d3163c410 con 0x557d3199e7c0 2024-02-26T06:24:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.101+0000 7fea35600640 10 client.25720 handle_command_reply: tid=0 2024-02-26T06:24:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.101+0000 7fea559a2740 1 client.25720 shutdown 2024-02-26T06:24:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.102+0000 7fea559a2740 1 -- 172.21.15.49:0/2884442045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 msgr2=0x557d3198b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.102+0000 7fea559a2740 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198b3f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00be50 tx=0x7fea3c01e620 comp rx=0 tx=0).stop 2024-02-26T06:24:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.103+0000 7fea559a2740 1 -- 172.21.15.49:0/2884442045 shutdown_connections 2024-02-26T06:24:21.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.103+0000 7fea559a2740 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557d3199e7c0 0x557d319a0be0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.103+0000 7fea559a2740 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d3198e410 0x557d3198be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.103+0000 7fea35600640 0 client.25720 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.103+0000 7fea559a2740 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d31985830 0x557d3198b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.103+0000 7fea559a2740 1 --2- 172.21.15.49:0/2884442045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198b3f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.103+0000 7fea559a2740 1 -- 172.21.15.49:0/2884442045 >> 172.21.15.49:0/2884442045 conn(0x557d3197f640 msgr2=0x557d31981390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.105+0000 7fea559a2740 1 -- 172.21.15.49:0/2884442045 shutdown_connections 2024-02-26T06:24:21.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.107+0000 7fea559a2740 1 -- 172.21.15.49:0/2884442045 wait complete. 2024-02-26T06:24:21.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.107+0000 7fea559a2740 20 client.25720 trim_cache size 0 max 16384 2024-02-26T06:24:21.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.162+0000 7fea559a2740 1 Processor -- start 2024-02-26T06:24:21.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.162+0000 7fea559a2740 1 -- start start 2024-02-26T06:24:21.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.163+0000 7fea559a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.164+0000 7fea559a2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31986750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.164+0000 7fea559a2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31988670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.164+0000 7fea559a2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d316340c0 con 0x557d3198e410 2024-02-26T06:24:21.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.164+0000 7fea559a2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d3199e560 con 0x557d31985830 2024-02-26T06:24:21.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.164+0000 7fea559a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d319a5db0 con 0x557d31984f10 2024-02-26T06:24:21.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.164+0000 7fea4be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31986750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.165+0000 7fea4be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31986750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51066/0 (socket says 172.21.15.49:51066) 2024-02-26T06:24:21.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.165+0000 7fea4be00640 1 -- 172.21.15.49:0/1450497164 learned_addr learned my addr 172.21.15.49:0/1450497164 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.165+0000 7fea4b400640 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.166+0000 7fea4b400640 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31988670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.166+0000 7fea4be00640 1 -- 172.21.15.49:0/1450497164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 msgr2=0x557d3198db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.166+0000 7fea4be00640 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.166+0000 7fea4be00640 1 -- 172.21.15.49:0/1450497164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 msgr2=0x557d31988670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.166+0000 7fea4be00640 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31988670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.167+0000 7fea4be00640 1 -- 172.21.15.49:0/1450497164 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d31886a60 con 0x557d31985830 2024-02-26T06:24:21.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.167+0000 7fea4b400640 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31988670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:21.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.167+0000 7fea4be00640 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31986750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fea38000c00 tx=0x7fea38005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5ad6caba2c64c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.168+0000 7fea37e00640 1 -- 172.21.15.49:0/1450497164 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38044040 con 0x557d31985830 2024-02-26T06:24:21.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.168+0000 7fea37e00640 1 -- 172.21.15.49:0/1450497164 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea38046040 con 0x557d31985830 2024-02-26T06:24:21.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.168+0000 7fea37e00640 1 -- 172.21.15.49:0/1450497164 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3802dea0 con 0x557d31985830 2024-02-26T06:24:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.169+0000 7fea559a2740 1 -- 172.21.15.49:0/1450497164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 msgr2=0x557d31986750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.169+0000 7fea559a2740 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31986750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fea38000c00 tx=0x7fea38005a20 comp rx=0 tx=0).stop 2024-02-26T06:24:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.170+0000 7fea559a2740 1 -- 172.21.15.49:0/1450497164 shutdown_connections 2024-02-26T06:24:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.170+0000 7fea559a2740 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31986750 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.170+0000 7fea559a2740 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d31988670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.170+0000 7fea559a2740 1 --2- 172.21.15.49:0/1450497164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d3198db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.170+0000 7fea559a2740 1 -- 172.21.15.49:0/1450497164 >> 172.21.15.49:0/1450497164 conn(0x557d3197f640 msgr2=0x557d31985c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.170+0000 7fea559a2740 1 -- 172.21.15.49:0/1450497164 shutdown_connections 2024-02-26T06:24:21.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.170+0000 7fea559a2740 1 -- 172.21.15.49:0/1450497164 wait complete. 2024-02-26T06:24:21.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.170+0000 7fea559a2740 1 -- start start 2024-02-26T06:24:21.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.171+0000 7fea559a2740 1 Processor -- start 2024-02-26T06:24:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea559a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d31993e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea559a2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31994390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea559a2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d319948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea559a2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d319f4c90 con 0x557d3198e410 2024-02-26T06:24:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea559a2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d319f4dd0 con 0x557d31985830 2024-02-26T06:24:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea559a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d31a05fa0 con 0x557d31984f10 2024-02-26T06:24:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea559a2740 10 client.0 _ll_register_callbacks cb 0x7fea3c00bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea4b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d31993e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea4b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d31993e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46972/0 (socket says 172.21.15.49:46972) 2024-02-26T06:24:21.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.172+0000 7fea4b400640 1 -- 172.21.15.49:0/1461117782 learned_addr learned my addr 172.21.15.49:0/1461117782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.173+0000 7fea4b400640 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d319948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.173+0000 7fea4be00640 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31994390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.173+0000 7fea4b400640 1 -- 172.21.15.49:0/1461117782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 msgr2=0x557d31994390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.173+0000 7fea4b400640 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31994390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.173+0000 7fea4b400640 1 -- 172.21.15.49:0/1461117782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 msgr2=0x557d319948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.173+0000 7fea4b400640 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d319948d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.173+0000 7fea4b400640 1 -- 172.21.15.49:0/1461117782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c114ef0 con 0x557d31984f10 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.174+0000 7fea4b400640 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d31993e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fea3c001eb0 tx=0x7fea3c01e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317a394dd8a83099 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.174+0000 7fea35600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.174+0000 7fea35600640 1 -- 172.21.15.49:0/1461117782 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c013e80 con 0x557d31984f10 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.174+0000 7fea35600640 1 -- 172.21.15.49:0/1461117782 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fea3c000e40 con 0x557d31984f10 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.175+0000 7fea559a2740 1 -- 172.21.15.49:0/1461117782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea3c010c50 con 0x557d31984f10 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.175+0000 7fea35600640 1 -- 172.21.15.49:0/1461117782 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea3c013e80 con 0x557d31984f10 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.175+0000 7fea559a2740 10 client.25726 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.175+0000 7fea559a2740 1 -- 172.21.15.49:0/1461117782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557d318ac670 con 0x557d31984f10 2024-02-26T06:24:21.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.176+0000 7fea35600640 1 -- 172.21.15.49:0/1461117782 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea3c044030 con 0x557d31984f10 2024-02-26T06:24:21.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.176+0000 7fea35600640 1 -- 172.21.15.49:0/1461117782 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fea3c01e8f0 con 0x557d31984f10 2024-02-26T06:24:21.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.176+0000 7fea559a2740 10 client.25726 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:21.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.176+0000 7fea559a2740 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:21.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.176+0000 7fea559a2740 20 client.25726 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:21.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.176+0000 7fea559a2740 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557d3199e7c0 0x557d31a09540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.176+0000 7fea559a2740 4 client.25726 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:24:21.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.176+0000 7fea559a2740 1 -- 172.21.15.49:0/1461117782 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557d316287b0 con 0x557d3199e7c0 2024-02-26T06:24:21.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.177+0000 7fea4be00640 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557d3199e7c0 0x557d31a09540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.178+0000 7fea4be00640 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557d3199e7c0 0x557d31a09540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3330c170bf157579 server_cookie=c48ec60a609fb6ff in_seq=0 out_seq=0 2024-02-26T06:24:21.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.178+0000 7fea35600640 10 client.25726 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:21.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.179+0000 7fea35600640 1 -- 172.21.15.49:0/1461117782 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557d316287b0 con 0x557d3199e7c0 2024-02-26T06:24:21.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.179+0000 7fea35600640 10 client.25726 handle_command_reply: tid=0 2024-02-26T06:24:21.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.179+0000 7fea559a2740 1 client.25726 shutdown 2024-02-26T06:24:21.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 -- 172.21.15.49:0/1461117782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 msgr2=0x557d31993e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d31993e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fea3c001eb0 tx=0x7fea3c01e6c0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 -- 172.21.15.49:0/1461117782 shutdown_connections 2024-02-26T06:24:21.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557d3199e7c0 0x557d31a09540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d31985830 0x557d31994390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d3198e410 0x557d319948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 --2- 172.21.15.49:0/1461117782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d31984f10 0x557d31993e50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 -- 172.21.15.49:0/1461117782 >> 172.21.15.49:0/1461117782 conn(0x557d3197f640 msgr2=0x557d31986750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea35600640 0 client.25726 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:21.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.180+0000 7fea559a2740 1 -- 172.21.15.49:0/1461117782 shutdown_connections 2024-02-26T06:24:21.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.181+0000 7fea559a2740 1 -- 172.21.15.49:0/1461117782 wait complete. 2024-02-26T06:24:21.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.181+0000 7fea559a2740 20 client.25726 trim_cache size 0 max 16384 2024-02-26T06:24:21.182 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:24:21.182 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:24:21.182 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "8ba093cd-910f-44f5-9ebd-79afe4fb2526", 2024-02-26T06:24:21.182 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:24:21.182 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.182+0000 7fea52a00640 1 -- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea300a2450 msgr2=0x7fea300a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.182+0000 7fea52a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea300a2450 0x7fea300a4910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea440cbfe0 tx=0x7fea44018bf0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.182+0000 7fea52a00640 1 -- 172.21.15.49:0/832951311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 msgr2=0x7fea4c1a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.182+0000 7fea52a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 0x7fea4c1a25f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fea4400cab0 tx=0x7fea4400cf70 comp rx=0 tx=0).stop 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.183+0000 7fea52a00640 1 -- 172.21.15.49:0/832951311 shutdown_connections 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.183+0000 7fea52a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea300a2450 0x7fea300a4910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.183+0000 7fea52a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea4c10d730 0x7fea4c1a25f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.183+0000 7fea52a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea4c1080e0 0x7fea4c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.183+0000 7fea52a00640 1 --2- 172.21.15.49:0/832951311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4c107720 0x7fea4c19dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.183+0000 7fea52a00640 1 -- 172.21.15.49:0/832951311 >> 172.21.15.49:0/832951311 conn(0x7fea4c0756c0 msgr2=0x7fea4c075d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.183+0000 7fea52a00640 1 -- 172.21.15.49:0/832951311 shutdown_connections 2024-02-26T06:24:21.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.183+0000 7fea52a00640 1 -- 172.21.15.49:0/832951311 wait complete. 2024-02-26T06:24:21.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:24:21.214 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:24:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 30.0 s, 2785.1 tps, lat 35.730 ms stddev 41.999 2024-02-26T06:24:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.501+0000 7f1acfc00640 1 -- 172.21.15.49:0/706948711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8101380 msgr2=0x7f1ac8101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.501+0000 7f1acfc00640 1 --2- 172.21.15.49:0/706948711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8101380 0x7f1ac8101760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009930 tx=0x7f1ab402bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:21.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.502+0000 7f1acfc00640 1 -- 172.21.15.49:0/706948711 shutdown_connections 2024-02-26T06:24:21.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.502+0000 7f1acfc00640 1 --2- 172.21.15.49:0/706948711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ac8106450 0x7f1ac810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.502+0000 7f1acfc00640 1 --2- 172.21.15.49:0/706948711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac8101d30 0x7f1ac8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.502+0000 7f1acfc00640 1 --2- 172.21.15.49:0/706948711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8101380 0x7f1ac8101760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.502+0000 7f1acfc00640 1 -- 172.21.15.49:0/706948711 >> 172.21.15.49:0/706948711 conn(0x7f1ac8078580 msgr2=0x7f1ac8078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.502+0000 7f1acfc00640 1 -- 172.21.15.49:0/706948711 shutdown_connections 2024-02-26T06:24:21.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.502+0000 7f1acfc00640 1 -- 172.21.15.49:0/706948711 wait complete. 2024-02-26T06:24:21.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.504+0000 7f1acfc00640 1 Processor -- start 2024-02-26T06:24:21.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acfc00640 1 -- start start 2024-02-26T06:24:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acfc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ac8101380 0x7f1ac819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acfc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac8106450 0x7f1ac819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acfc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8199850 0x7f1ac8199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acfc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac819a300 con 0x7f1ac8101380 2024-02-26T06:24:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acfc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac819a470 con 0x7f1ac8199850 2024-02-26T06:24:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acfc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac819a5e0 con 0x7f1ac8106450 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac8106450 0x7f1ac819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acdc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8199850 0x7f1ac8199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac8106450 0x7f1ac819fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46984/0 (socket says 172.21.15.49:46984) 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1acdc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8199850 0x7f1ac8199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51082/0 (socket says 172.21.15.49:51082) 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.505+0000 7f1ac7e00640 1 -- 172.21.15.49:0/2708580705 learned_addr learned my addr 172.21.15.49:0/2708580705 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.506+0000 7f1acd200640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ac8101380 0x7f1ac819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.506+0000 7f1acdc00640 1 -- 172.21.15.49:0/2708580705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac8106450 msgr2=0x7f1ac819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.506+0000 7f1acdc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac8106450 0x7f1ac819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.506+0000 7f1acdc00640 1 -- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ac8101380 msgr2=0x7f1ac819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.506+0000 7f1acdc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ac8101380 0x7f1ac819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.506+0000 7f1acdc00640 1 -- 172.21.15.49:0/2708580705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009560 con 0x7f1ac8199850 2024-02-26T06:24:21.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.506+0000 7f1acd200640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ac8101380 0x7f1ac819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:21.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.506+0000 7f1acdc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8199850 0x7f1ac8199d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e940 tx=0x7f1ac000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.507+0000 7f1ac5600640 1 -- 172.21.15.49:0/2708580705 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004d20 con 0x7f1ac8199850 2024-02-26T06:24:21.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.507+0000 7f1ac5600640 1 -- 172.21.15.49:0/2708580705 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ac0004e80 con 0x7f1ac8199850 2024-02-26T06:24:21.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.507+0000 7f1ac5600640 1 -- 172.21.15.49:0/2708580705 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00116d0 con 0x7f1ac8199850 2024-02-26T06:24:21.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.507+0000 7f1acfc00640 1 -- 172.21.15.49:0/2708580705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac81a6590 con 0x7f1ac8199850 2024-02-26T06:24:21.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.507+0000 7f1acfc00640 1 -- 172.21.15.49:0/2708580705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac81a6a20 con 0x7f1ac8199850 2024-02-26T06:24:21.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.512+0000 7f1ac5600640 1 -- 172.21.15.49:0/2708580705 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f1ac00059c0 con 0x7f1ac8199850 2024-02-26T06:24:21.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.515+0000 7f1ac5600640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1ab00a23d0 0x7f1ab00a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.515+0000 7f1ac5600640 1 -- 172.21.15.49:0/2708580705 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1ac00ce450 con 0x7f1ac8199850 2024-02-26T06:24:21.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.516+0000 7f1acd200640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1ab00a23d0 0x7f1ab00a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.516+0000 7f1ad2b79740 1 Processor -- start 2024-02-26T06:24:21.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.516+0000 7f1ad2b79740 1 -- start start 2024-02-26T06:24:21.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 0x562e509f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 0x562e509f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509fde10 0x562e50a00200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5095b120 con 0x562e509f4aa0 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e50958650 con 0x562e509fde10 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e50953e30 con 0x562e509f5780 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1acdc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509fde10 0x562e50a00200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1acdc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509fde10 0x562e50a00200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51090/0 (socket says 172.21.15.49:51090) 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.517+0000 7f1acdc00640 1 -- 172.21.15.49:0/940394741 learned_addr learned my addr 172.21.15.49:0/940394741 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1acdc00640 1 -- 172.21.15.49:0/940394741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 msgr2=0x562e509f44d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1acdc00640 1 --2- 172.21.15.49:0/940394741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 0x562e509f44d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1acdc00640 1 -- 172.21.15.49:0/940394741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 msgr2=0x562e509f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1acdc00640 1 --2- 172.21.15.49:0/940394741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 0x562e509f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1acdc00640 1 -- 172.21.15.49:0/940394741 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e5090d030 con 0x562e509fde10 2024-02-26T06:24:21.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1acdc00640 1 --2- 172.21.15.49:0/940394741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509fde10 0x562e50a00200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ac009a900 tx=0x7f1ac009a870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876dfe9d6f18887d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1acd200640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1ab00a23d0 0x7f1ab00a4890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009900 tx=0x7f1ab4005fd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1aba000640 1 -- 172.21.15.49:0/940394741 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a33a0 con 0x562e509fde10 2024-02-26T06:24:21.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1aba000640 1 -- 172.21.15.49:0/940394741 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ac0085050 con 0x562e509fde10 2024-02-26T06:24:21.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.518+0000 7f1aba000640 1 -- 172.21.15.49:0/940394741 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00cbf70 con 0x562e509fde10 2024-02-26T06:24:21.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 -- 172.21.15.49:0/940394741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509fde10 msgr2=0x562e50a00200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/940394741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509fde10 0x562e50a00200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ac009a900 tx=0x7f1ac009a870 comp rx=0 tx=0).stop 2024-02-26T06:24:21.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 -- 172.21.15.49:0/940394741 shutdown_connections 2024-02-26T06:24:21.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/940394741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509fde10 0x562e50a00200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/940394741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 0x562e509f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/940394741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 0x562e509f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 -- 172.21.15.49:0/940394741 >> 172.21.15.49:0/940394741 conn(0x562e509b3570 msgr2=0x562e509ef4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 -- 172.21.15.49:0/940394741 shutdown_connections 2024-02-26T06:24:21.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.520+0000 7f1ad2b79740 1 -- 172.21.15.49:0/940394741 wait complete. 2024-02-26T06:24:21.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.521+0000 7f1ad2b79740 1 -- start start 2024-02-26T06:24:21.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.521+0000 7f1ad2b79740 1 Processor -- start 2024-02-26T06:24:21.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 0x562e509fae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f5780 0x562e509fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e509fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e508f8e90 con 0x562e509f4aa0 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e508f8b30 con 0x562e509f5780 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e506c21c0 con 0x562e509fde10 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ad2b79740 10 client.0 _ll_register_callbacks cb 0x562e5090c930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acdc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e509fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f5780 0x562e509fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acdc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e509fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46988/0 (socket says 172.21.15.49:46988) 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acdc00640 1 -- 172.21.15.49:0/1676671623 learned_addr learned my addr 172.21.15.49:0/1676671623 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acd200640 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 0x562e509fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acdc00640 1 -- 172.21.15.49:0/1676671623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f5780 msgr2=0x562e509fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acdc00640 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f5780 0x562e509fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acdc00640 1 -- 172.21.15.49:0/1676671623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 msgr2=0x562e509fae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acdc00640 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 0x562e509fae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1acdc00640 1 -- 172.21.15.49:0/1676671623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac81a6a20 con 0x562e509fde10 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.522+0000 7f1ac7e00640 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f5780 0x562e509fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.523+0000 7f1acdc00640 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e509fb8c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c640 tx=0x7f1ac0097540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea315816f25afe02 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.523+0000 7f1aaea00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.523+0000 7f1aaea00640 1 -- 172.21.15.49:0/1676671623 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a33a0 con 0x562e509fde10 2024-02-26T06:24:21.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.523+0000 7f1aaea00640 1 -- 172.21.15.49:0/1676671623 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1ac00052b0 con 0x562e509fde10 2024-02-26T06:24:21.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.523+0000 7f1ad2b79740 1 -- 172.21.15.49:0/1676671623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e506537e0 con 0x562e509fde10 2024-02-26T06:24:21.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.523+0000 7f1aaea00640 1 -- 172.21.15.49:0/1676671623 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac009a050 con 0x562e509fde10 2024-02-26T06:24:21.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.523+0000 7f1ad2b79740 10 client.25735 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.523+0000 7f1ad2b79740 1 -- 172.21.15.49:0/1676671623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562e509524b0 con 0x562e509fde10 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.524+0000 7f1aaea00640 1 -- 172.21.15.49:0/1676671623 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562e506537e0 con 0x562e509fde10 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.524+0000 7f1aaea00640 1 -- 172.21.15.49:0/1676671623 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1ac0018030 con 0x562e509fde10 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.524+0000 7f1ad2b79740 10 client.25735 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.524+0000 7f1ad2b79740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.525+0000 7f1ad2b79740 20 client.25735 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.525+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562e50a0db80 0x562e50a0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.525+0000 7f1ad2b79740 4 client.25735 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.525+0000 7f1ad2b79740 1 -- 172.21.15.49:0/1676671623 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e50690720 con 0x562e50a0db80 2024-02-26T06:24:21.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.525+0000 7f1acd200640 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562e50a0db80 0x562e50a0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.527+0000 7f1acd200640 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562e50a0db80 0x562e50a0ffa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4ce8f8a97438aa server_cookie=4bdda8dbdb7cb803 in_seq=0 out_seq=0 2024-02-26T06:24:21.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.527+0000 7f1aaea00640 10 client.25735 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:21.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1aaea00640 1 -- 172.21.15.49:0/1676671623 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e50690720 con 0x562e50a0db80 2024-02-26T06:24:21.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1aaea00640 10 client.25735 handle_command_reply: tid=0 2024-02-26T06:24:21.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 client.25735 shutdown 2024-02-26T06:24:21.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 -- 172.21.15.49:0/1676671623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 msgr2=0x562e509fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e509fb8c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c640 tx=0x7f1ac0097540 comp rx=0 tx=0).stop 2024-02-26T06:24:21.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 -- 172.21.15.49:0/1676671623 shutdown_connections 2024-02-26T06:24:21.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562e50a0db80 0x562e50a0ffa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f5780 0x562e509fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e509fb8c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/1676671623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f4aa0 0x562e509fae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1aaea00640 0 client.25735 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:21.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.528+0000 7f1ad2b79740 1 -- 172.21.15.49:0/1676671623 >> 172.21.15.49:0/1676671623 conn(0x562e509b3570 msgr2=0x562e509f1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.529+0000 7f1ad2b79740 1 -- 172.21.15.49:0/1676671623 shutdown_connections 2024-02-26T06:24:21.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.529+0000 7f1ad2b79740 1 -- 172.21.15.49:0/1676671623 wait complete. 2024-02-26T06:24:21.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.529+0000 7f1ad2b79740 20 client.25735 trim_cache size 0 max 16384 2024-02-26T06:24:21.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.588+0000 7f1ad2b79740 1 Processor -- start 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.588+0000 7f1ad2b79740 1 -- start start 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e509fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 0x562e509fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509fde10 0x562e50a04cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e509f3d20 con 0x562e509fde10 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e50a151b0 con 0x562e509f4aa0 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e509f3ad0 con 0x562e509f5780 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1acd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e509fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 0x562e509fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1acd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e509fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51108/0 (socket says 172.21.15.49:51108) 2024-02-26T06:24:21.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.589+0000 7f1acd200640 1 -- 172.21.15.49:0/947654337 learned_addr learned my addr 172.21.15.49:0/947654337 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.590+0000 7f1acd200640 1 -- 172.21.15.49:0/947654337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 msgr2=0x562e509fb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.590+0000 7f1acd200640 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 0x562e509fb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.590+0000 7f1acd200640 1 -- 172.21.15.49:0/947654337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509fde10 msgr2=0x562e50a04cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.590+0000 7f1acdc00640 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509fde10 0x562e50a04cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.591+0000 7f1acd200640 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509fde10 0x562e50a04cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.591+0000 7f1acd200640 1 -- 172.21.15.49:0/947654337 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e508ee920 con 0x562e509f4aa0 2024-02-26T06:24:21.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.591+0000 7f1acdc00640 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509fde10 0x562e50a04cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:21.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.593+0000 7f1acd200640 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e509fd570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4000c00 tx=0x7f1ab4042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f8f5000b592b82 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.593+0000 7f1ab9600640 1 -- 172.21.15.49:0/947654337 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab402cc40 con 0x562e509f4aa0 2024-02-26T06:24:21.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.593+0000 7f1ab9600640 1 -- 172.21.15.49:0/947654337 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ab4004510 con 0x562e509f4aa0 2024-02-26T06:24:21.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.593+0000 7f1ab9600640 1 -- 172.21.15.49:0/947654337 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4005120 con 0x562e509f4aa0 2024-02-26T06:24:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.594+0000 7f1ad2b79740 1 -- 172.21.15.49:0/947654337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 msgr2=0x562e509fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.594+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e509fd570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4000c00 tx=0x7f1ab4042000 comp rx=0 tx=0).stop 2024-02-26T06:24:21.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.594+0000 7f1ad2b79740 1 -- 172.21.15.49:0/947654337 shutdown_connections 2024-02-26T06:24:21.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.595+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509fde10 0x562e50a04cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.595+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509f5780 0x562e509fb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.595+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/947654337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e509fd570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.595+0000 7f1ad2b79740 1 -- 172.21.15.49:0/947654337 >> 172.21.15.49:0/947654337 conn(0x562e509b3570 msgr2=0x562e509ef020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.595+0000 7f1ad2b79740 1 -- 172.21.15.49:0/947654337 shutdown_connections 2024-02-26T06:24:21.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.595+0000 7f1ad2b79740 1 -- 172.21.15.49:0/947654337 wait complete. 2024-02-26T06:24:21.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.596+0000 7f1ad2b79740 1 -- start start 2024-02-26T06:24:21.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.596+0000 7f1ad2b79740 1 Processor -- start 2024-02-26T06:24:21.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.597+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e50a04f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.597+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f5780 0x562e50a5f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.598+0000 7f1ad2b79740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e50a5f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.598+0000 7f1acd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e50a04f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.598+0000 7f1acd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e50a04f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51120/0 (socket says 172.21.15.49:51120) 2024-02-26T06:24:21.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.598+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e50a5fd30 con 0x562e509f5780 2024-02-26T06:24:21.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e50a5fea0 con 0x562e509f4aa0 2024-02-26T06:24:21.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1ad2b79740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e50a5e070 con 0x562e509fde10 2024-02-26T06:24:21.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1ad2b79740 10 client.0 _ll_register_callbacks cb 0x7f1ac009a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:21.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.598+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f5780 0x562e50a5f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f5780 0x562e50a5f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54684/0 (socket says 172.21.15.49:54684) 2024-02-26T06:24:21.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1ac7e00640 1 -- 172.21.15.49:0/577081182 learned_addr learned my addr 172.21.15.49:0/577081182 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:21.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1acdc00640 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e50a5f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1acdc00640 1 -- 172.21.15.49:0/577081182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 msgr2=0x562e50a04f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1acdc00640 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e50a04f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1acdc00640 1 -- 172.21.15.49:0/577081182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f5780 msgr2=0x562e50a5f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:21.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1acdc00640 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f5780 0x562e50a5f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.599+0000 7f1acdc00640 1 -- 172.21.15.49:0/577081182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac810e830 con 0x562e509fde10 2024-02-26T06:24:21.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.600+0000 7f1acdc00640 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e50a5f770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x562e509f4120 tx=0x7f1ac0096fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc8922846c513a81 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:21.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.600+0000 7f1aaea00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:21.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.600+0000 7f1aaea00640 1 -- 172.21.15.49:0/577081182 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac009ac80 con 0x562e509fde10 2024-02-26T06:24:21.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.600+0000 7f1aaea00640 1 -- 172.21.15.49:0/577081182 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1ac00a33a0 con 0x562e509fde10 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.600+0000 7f1acd200640 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e50a04f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.601+0000 7f1ad2b79740 1 -- 172.21.15.49:0/577081182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ac0085050 con 0x562e509fde10 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.601+0000 7f1aaea00640 1 -- 172.21.15.49:0/577081182 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac0097270 con 0x562e509fde10 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.601+0000 7f1ad2b79740 10 client.25741 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.601+0000 7f1ad2b79740 1 -- 172.21.15.49:0/577081182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562e50914400 con 0x562e509fde10 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.601+0000 7f1aaea00640 1 -- 172.21.15.49:0/577081182 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ac0085050 con 0x562e509fde10 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.602+0000 7f1aaea00640 1 -- 172.21.15.49:0/577081182 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1ac0018040 con 0x562e509fde10 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.602+0000 7f1ad2b79740 10 client.25741 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.602+0000 7f1ad2b79740 10 client.25741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.602+0000 7f1ad2b79740 20 client.25741 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.602+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562e50a04060 0x562e50a04470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.602+0000 7f1ad2b79740 4 client.25741 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.602+0000 7f1ad2b79740 1 -- 172.21.15.49:0/577081182 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562e506df890 con 0x562e50a04060 2024-02-26T06:24:21.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.602+0000 7f1acd200640 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562e50a04060 0x562e50a04470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:21.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.603+0000 7f1acd200640 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562e50a04060 0x562e50a04470 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4dfb8d2046eec6 server_cookie=ed2fac0974d45cad in_seq=0 out_seq=0 2024-02-26T06:24:21.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.603+0000 7f1aaea00640 10 client.25741 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:21.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.604+0000 7f1aaea00640 1 -- 172.21.15.49:0/577081182 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562e506df890 con 0x562e50a04060 2024-02-26T06:24:21.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.604+0000 7f1aaea00640 10 client.25741 handle_command_reply: tid=0 2024-02-26T06:24:21.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.605+0000 7f1ad2b79740 1 client.25741 shutdown 2024-02-26T06:24:21.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.605+0000 7f1ad2b79740 1 -- 172.21.15.49:0/577081182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 msgr2=0x562e50a5f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.605+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e50a5f770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x562e509f4120 tx=0x7f1ac0096fd0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.606+0000 7f1ad2b79740 1 -- 172.21.15.49:0/577081182 shutdown_connections 2024-02-26T06:24:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.606+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562e50a04060 0x562e50a04470 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.606+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e509fde10 0x562e50a5f770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.606+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e509f5780 0x562e50a5f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.606+0000 7f1ad2b79740 1 --2- 172.21.15.49:0/577081182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562e509f4aa0 0x562e50a04f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.606+0000 7f1aaea00640 0 client.25741 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.606+0000 7f1ad2b79740 1 -- 172.21.15.49:0/577081182 >> 172.21.15.49:0/577081182 conn(0x562e509b3570 msgr2=0x562e509f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.607+0000 7f1ad2b79740 1 -- 172.21.15.49:0/577081182 shutdown_connections 2024-02-26T06:24:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.607+0000 7f1ad2b79740 1 -- 172.21.15.49:0/577081182 wait complete. 2024-02-26T06:24:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.607+0000 7f1ad2b79740 20 client.25741 trim_cache size 0 max 16384 2024-02-26T06:24:21.607 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:24:21.607 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:24:21.608 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:24:21.608 INFO:teuthology.orchestra.run.smithi049.stdout: "8ba093cd-910f-44f5-9ebd-79afe4fb2526": { 2024-02-26T06:24:21.608 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:24:21.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "8ba093cd-910f-44f5-9ebd-79afe4fb2526", 2024-02-26T06:24:21.608 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:24:21.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:24:21.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:24:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:24:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.608+0000 7f1acfc00640 1 -- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1ab00a23d0 msgr2=0x7f1ab00a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.608+0000 7f1acfc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1ab00a23d0 0x7f1ab00a4890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009900 tx=0x7f1ab4005fd0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.608+0000 7f1acfc00640 1 -- 172.21.15.49:0/2708580705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8199850 msgr2=0x7f1ac8199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.608+0000 7f1acfc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8199850 0x7f1ac8199d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e940 tx=0x7f1ac000ee00 comp rx=0 tx=0).stop 2024-02-26T06:24:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.609+0000 7f1acfc00640 1 -- 172.21.15.49:0/2708580705 shutdown_connections 2024-02-26T06:24:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.609+0000 7f1acfc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1ab00a23d0 0x7f1ab00a4890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.609+0000 7f1acfc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac8199850 0x7f1ac8199d00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.609+0000 7f1acfc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac8106450 0x7f1ac819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.609+0000 7f1acfc00640 1 --2- 172.21.15.49:0/2708580705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ac8101380 0x7f1ac819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.609+0000 7f1acfc00640 1 -- 172.21.15.49:0/2708580705 >> 172.21.15.49:0/2708580705 conn(0x7f1ac8078580 msgr2=0x7f1ac80feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.609+0000 7f1acfc00640 1 -- 172.21.15.49:0/2708580705 shutdown_connections 2024-02-26T06:24:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:21.609+0000 7f1acfc00640 1 -- 172.21.15.49:0/2708580705 wait complete. 2024-02-26T06:24:21.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:21 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:21.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:21 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:21 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:21.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:21 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:21.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:21 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:21 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:21.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:21 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:21 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:21.633 INFO:tasks.cephfs.filesystem:scrub status for tag:8ba093cd-910f-44f5-9ebd-79afe4fb2526 - {'path': '/', 'tag': '8ba093cd-910f-44f5-9ebd-79afe4fb2526', 'options': 'recursive,force'} 2024-02-26T06:24:21.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:21 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:21.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:21 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:21 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:21.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:21 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:21.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:21 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:21 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:21.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:21 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:21 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:21.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:21 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:21.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:21 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:21 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:21.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:21 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:21.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:21 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:21 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:21.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:21 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:21.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:21 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:22 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:24:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:22 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:22 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:22 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:22 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:22 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:22 smithi080 ceph-mon[36206]: pgmap v32: 97 pgs: 97 active+clean; 993 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 1.21k op/s 2024-02-26T06:24:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:22 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:24:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:22 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:22 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:22 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:22 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:22 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:22 smithi115 ceph-mon[36426]: pgmap v32: 97 pgs: 97 active+clean; 993 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 1.21k op/s 2024-02-26T06:24:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:22 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:24:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:22 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:22 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:22 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:22 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:22 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:22.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:22 smithi049 ceph-mon[31696]: pgmap v32: 97 pgs: 97 active+clean; 993 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 1.21k op/s 2024-02-26T06:24:24.722 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:24 smithi049 ceph-mon[31696]: pgmap v33: 97 pgs: 97 active+clean; 993 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 1.16k op/s 2024-02-26T06:24:24.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:24 smithi080 ceph-mon[36206]: pgmap v33: 97 pgs: 97 active+clean; 993 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 1.16k op/s 2024-02-26T06:24:24.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:24 smithi115 ceph-mon[36426]: pgmap v33: 97 pgs: 97 active+clean; 993 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 1.16k op/s 2024-02-26T06:24:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 35.0 s, 3020.5 tps, lat 33.079 ms stddev 36.826 2024-02-26T06:24:26.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:26 smithi080 ceph-mon[36206]: pgmap v34: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 52 MiB/s wr, 1.60k op/s 2024-02-26T06:24:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:26 smithi115 ceph-mon[36426]: pgmap v34: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 52 MiB/s wr, 1.60k op/s 2024-02-26T06:24:26.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:26 smithi049 ceph-mon[31696]: pgmap v34: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 52 MiB/s wr, 1.60k op/s 2024-02-26T06:24:28.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:28 smithi080 ceph-mon[36206]: pgmap v35: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 742 op/s 2024-02-26T06:24:28.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:28 smithi115 ceph-mon[36426]: pgmap v35: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 742 op/s 2024-02-26T06:24:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:28 smithi049 ceph-mon[31696]: pgmap v35: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 742 op/s 2024-02-26T06:24:30.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:30 smithi080 ceph-mon[36206]: pgmap v36: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 1.04k op/s 2024-02-26T06:24:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:30 smithi115 ceph-mon[36426]: pgmap v36: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 1.04k op/s 2024-02-26T06:24:30.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:30 smithi049 ceph-mon[31696]: pgmap v36: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 1.04k op/s 2024-02-26T06:24:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 40.0 s, 3038.2 tps, lat 32.854 ms stddev 34.938 2024-02-26T06:24:31.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:31 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:24:31.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:31 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:24:31.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:31 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:24:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:31 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:24:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:31 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:24:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:31 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:24:31.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:31 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:24:31.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:31 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:24:31.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:31 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:24:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.712+0000 7fc780400640 1 -- 172.21.15.49:0/725729524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778103f70 msgr2=0x7fc778107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.712+0000 7fc780400640 1 --2- 172.21.15.49:0/725729524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778103f70 0x7fc778107fb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc768009930 tx=0x7fc76802bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.712+0000 7fc780400640 1 -- 172.21.15.49:0/725729524 shutdown_connections 2024-02-26T06:24:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.712+0000 7fc780400640 1 --2- 172.21.15.49:0/725729524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778103f70 0x7fc778107fb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.712+0000 7fc780400640 1 --2- 172.21.15.49:0/725729524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc778108690 0x7fc778110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.712+0000 7fc780400640 1 --2- 172.21.15.49:0/725729524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7781035c0 0x7fc7781039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.712+0000 7fc780400640 1 -- 172.21.15.49:0/725729524 >> 172.21.15.49:0/725729524 conn(0x7fc7780fee40 msgr2=0x7fc778101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.713+0000 7fc780400640 1 -- 172.21.15.49:0/725729524 shutdown_connections 2024-02-26T06:24:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.713+0000 7fc780400640 1 -- 172.21.15.49:0/725729524 wait complete. 2024-02-26T06:24:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.714+0000 7fc780400640 1 Processor -- start 2024-02-26T06:24:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.714+0000 7fc780400640 1 -- start start 2024-02-26T06:24:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.714+0000 7fc780400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7781035c0 0x7fc77819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.714+0000 7fc780400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778108690 0x7fc77819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.714+0000 7fc780400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc778199800 0x7fc778199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.714+0000 7fc780400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77819a2b0 con 0x7fc778199800 2024-02-26T06:24:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.714+0000 7fc780400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77819a420 con 0x7fc778108690 2024-02-26T06:24:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.714+0000 7fc780400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77819a590 con 0x7fc7781035c0 2024-02-26T06:24:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7781035c0 0x7fc77819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778108690 0x7fc77819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7781035c0 0x7fc77819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46946/0 (socket says 172.21.15.49:46946) 2024-02-26T06:24:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778108690 0x7fc77819fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39364/0 (socket says 172.21.15.49:39364) 2024-02-26T06:24:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77da00640 1 -- 172.21.15.49:0/2261005610 learned_addr learned my addr 172.21.15.49:0/2261005610 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77d000640 1 -- 172.21.15.49:0/2261005610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7781035c0 msgr2=0x7fc77819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77d000640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7781035c0 0x7fc77819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77d000640 1 -- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc778199800 msgr2=0x7fc778199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77d000640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc778199800 0x7fc778199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.715+0000 7fc77d000640 1 -- 172.21.15.49:0/2261005610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc768009560 con 0x7fc778108690 2024-02-26T06:24:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.716+0000 7fc77d000640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778108690 0x7fc77819fb80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc768009230 tx=0x7fc76802c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.716+0000 7fc776000640 1 -- 172.21.15.49:0/2261005610 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680042d0 con 0x7fc778108690 2024-02-26T06:24:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.716+0000 7fc776000640 1 -- 172.21.15.49:0/2261005610 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc768004430 con 0x7fc778108690 2024-02-26T06:24:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.716+0000 7fc780400640 1 -- 172.21.15.49:0/2261005610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7781a6490 con 0x7fc778108690 2024-02-26T06:24:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.716+0000 7fc776000640 1 -- 172.21.15.49:0/2261005610 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768003ca0 con 0x7fc778108690 2024-02-26T06:24:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.717+0000 7fc780400640 1 -- 172.21.15.49:0/2261005610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7780755c0 con 0x7fc778108690 2024-02-26T06:24:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.721+0000 7fc776000640 1 -- 172.21.15.49:0/2261005610 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fc768004df0 con 0x7fc778108690 2024-02-26T06:24:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.723+0000 7fc776000640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc7600a2470 0x7fc7600a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.723+0000 7fc776000640 1 -- 172.21.15.49:0/2261005610 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc7680f0880 con 0x7fc778108690 2024-02-26T06:24:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.723+0000 7fc77da00640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc7600a2470 0x7fc7600a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.724+0000 7fc77da00640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc7600a2470 0x7fc7600a4930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009fd0 tx=0x7fc76c009500 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.725+0000 7fc783439740 1 Processor -- start 2024-02-26T06:24:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc783439740 1 -- start start 2024-02-26T06:24:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc783439740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc783439740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc783439740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc783439740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864887e5f0 con 0x5586488cc300 2024-02-26T06:24:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc783439740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558648830890 con 0x5586488c2d80 2024-02-26T06:24:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc783439740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864882ddc0 con 0x5586488c3af0 2024-02-26T06:24:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc77d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc77d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39374/0 (socket says 172.21.15.49:39374) 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc77d000640 1 -- 172.21.15.49:0/820690252 learned_addr learned my addr 172.21.15.49:0/820690252 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc77e400640 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.726+0000 7fc77da00640 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.727+0000 7fc77d000640 1 -- 172.21.15.49:0/820690252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 msgr2=0x5586488c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.727+0000 7fc77d000640 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.727+0000 7fc77d000640 1 -- 172.21.15.49:0/820690252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 msgr2=0x5586488ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.727+0000 7fc77d000640 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.727+0000 7fc77d000640 1 -- 172.21.15.49:0/820690252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586487aaad0 con 0x5586488c2d80 2024-02-26T06:24:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.728+0000 7fc77da00640 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.728+0000 7fc77d000640 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c3200 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc7680bd0d0 tx=0x7fc7680bcb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deee00af3b297664 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.729+0000 7fc766a00640 1 -- 172.21.15.49:0/820690252 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680c1290 con 0x5586488c2d80 2024-02-26T06:24:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.729+0000 7fc766a00640 1 -- 172.21.15.49:0/820690252 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7680c58e0 con 0x5586488c2d80 2024-02-26T06:24:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.729+0000 7fc766a00640 1 -- 172.21.15.49:0/820690252 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680c56b0 con 0x5586488c2d80 2024-02-26T06:24:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.730+0000 7fc783439740 1 -- 172.21.15.49:0/820690252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 msgr2=0x5586488c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.731+0000 7fc783439740 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c3200 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc7680bd0d0 tx=0x7fc7680bcb60 comp rx=0 tx=0).stop 2024-02-26T06:24:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.731+0000 7fc783439740 1 -- 172.21.15.49:0/820690252 shutdown_connections 2024-02-26T06:24:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.731+0000 7fc783439740 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c3200 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.731+0000 7fc783439740 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.732+0000 7fc783439740 1 --2- 172.21.15.49:0/820690252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.732+0000 7fc783439740 1 -- 172.21.15.49:0/820690252 >> 172.21.15.49:0/820690252 conn(0x55864888a480 msgr2=0x5586488bda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.732+0000 7fc783439740 1 -- 172.21.15.49:0/820690252 shutdown_connections 2024-02-26T06:24:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.732+0000 7fc783439740 1 -- 172.21.15.49:0/820690252 wait complete. 2024-02-26T06:24:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.733+0000 7fc783439740 1 -- start start 2024-02-26T06:24:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.733+0000 7fc783439740 1 Processor -- start 2024-02-26T06:24:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.736+0000 7fc783439740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc783439740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc783439740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc783439740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586488295a0 con 0x5586488cc300 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc783439740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586487ceb20 con 0x5586488c2d80 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc783439740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586487ce7c0 con 0x5586488c3af0 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc783439740 10 client.0 _ll_register_callbacks cb 0x5586487e2520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46970/0 (socket says 172.21.15.49:46970) 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39390/0 (socket says 172.21.15.49:39390) 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77d000640 1 -- 172.21.15.49:0/3808717668 learned_addr learned my addr 172.21.15.49:0/3808717668 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77e400640 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77da00640 1 -- 172.21.15.49:0/3808717668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 msgr2=0x5586488c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77da00640 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77da00640 1 -- 172.21.15.49:0/3808717668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 msgr2=0x5586488c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77da00640 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77da00640 1 -- 172.21.15.49:0/3808717668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7780755c0 con 0x5586488c2d80 2024-02-26T06:24:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77e400640 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.737+0000 7fc77d000640 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.738+0000 7fc77da00640 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c9390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009aa0 tx=0x7fc76c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20355cee1dfb4b22 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.738+0000 7fc75b400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.739+0000 7fc75b400640 1 -- 172.21.15.49:0/3808717668 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c023070 con 0x5586488c2d80 2024-02-26T06:24:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.739+0000 7fc75b400640 1 -- 172.21.15.49:0/3808717668 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc76c01eb00 con 0x5586488c2d80 2024-02-26T06:24:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.739+0000 7fc783439740 1 -- 172.21.15.49:0/3808717668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586487fbca0 con 0x5586488c2d80 2024-02-26T06:24:32.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.740+0000 7fc75b400640 1 -- 172.21.15.49:0/3808717668 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc76c01ecc0 con 0x5586488c2d80 2024-02-26T06:24:32.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.740+0000 7fc783439740 10 client.25715 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:32.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.740+0000 7fc783439740 1 -- 172.21.15.49:0/3808717668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558648827c20 con 0x5586488c2d80 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.741+0000 7fc75b400640 1 -- 172.21.15.49:0/3808717668 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5586487fbca0 con 0x5586488c2d80 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.741+0000 7fc75b400640 1 -- 172.21.15.49:0/3808717668 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc76c031580 con 0x5586488c2d80 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.741+0000 7fc783439740 10 client.25715 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.741+0000 7fc783439740 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.741+0000 7fc783439740 20 client.25715 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.741+0000 7fc783439740 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5586488dc130 0x5586488de590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.741+0000 7fc783439740 4 client.25715 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.741+0000 7fc783439740 1 -- 172.21.15.49:0/3808717668 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55864881fcb0 con 0x5586488dc130 2024-02-26T06:24:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.742+0000 7fc77d000640 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5586488dc130 0x5586488de590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.742+0000 7fc77d000640 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5586488dc130 0x5586488de590 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89954259261b1165 server_cookie=af708c4c4e5674c3 in_seq=0 out_seq=0 2024-02-26T06:24:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.742+0000 7fc75b400640 10 client.25715 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.743+0000 7fc75b400640 1 -- 172.21.15.49:0/3808717668 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55864881fcb0 con 0x5586488dc130 2024-02-26T06:24:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.743+0000 7fc75b400640 10 client.25715 handle_command_reply: tid=0 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.743+0000 7fc783439740 1 client.25715 shutdown 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.743+0000 7fc783439740 1 -- 172.21.15.49:0/3808717668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 msgr2=0x5586488c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.743+0000 7fc783439740 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c9390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009aa0 tx=0x7fc76c004ab0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 1 -- 172.21.15.49:0/3808717668 shutdown_connections 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5586488dc130 0x5586488de590 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 1 --2- 172.21.15.49:0/3808717668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488c9390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 1 -- 172.21.15.49:0/3808717668 >> 172.21.15.49:0/3808717668 conn(0x55864888a480 msgr2=0x5586488bf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc75b400640 0 client.25715 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 1 -- 172.21.15.49:0/3808717668 shutdown_connections 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 1 -- 172.21.15.49:0/3808717668 wait complete. 2024-02-26T06:24:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.744+0000 7fc783439740 20 client.25715 trim_cache size 0 max 16384 2024-02-26T06:24:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.769+0000 7fc783439740 1 Processor -- start 2024-02-26T06:24:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.769+0000 7fc783439740 1 -- start start 2024-02-26T06:24:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.769+0000 7fc783439740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c2d80 0x5586488c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.769+0000 7fc783439740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488c3af0 0x5586488c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.769+0000 7fc783439740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488cc300 0x5586488c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.770+0000 7fc783439740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558648571e60 con 0x5586488c3af0 2024-02-26T06:24:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.770+0000 7fc783439740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586488da950 con 0x5586488cc300 2024-02-26T06:24:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.770+0000 7fc783439740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586488c8740 con 0x5586488c2d80 2024-02-26T06:24:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.770+0000 7fc77da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c2d80 0x5586488c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.770+0000 7fc77e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488cc300 0x5586488c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.771+0000 7fc77e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488cc300 0x5586488c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39396/0 (socket says 172.21.15.49:39396) 2024-02-26T06:24:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.771+0000 7fc77d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488c3af0 0x5586488c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.771+0000 7fc77da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c2d80 0x5586488c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46980/0 (socket says 172.21.15.49:46980) 2024-02-26T06:24:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.771+0000 7fc77e400640 1 -- 172.21.15.49:0/2583079011 learned_addr learned my addr 172.21.15.49:0/2583079011 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.772+0000 7fc77e400640 1 -- 172.21.15.49:0/2583079011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c2d80 msgr2=0x5586488c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.772+0000 7fc77e400640 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c2d80 0x5586488c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.772+0000 7fc77e400640 1 -- 172.21.15.49:0/2583079011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488c3af0 msgr2=0x5586488c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.772+0000 7fc77e400640 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488c3af0 0x5586488c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.772+0000 7fc77e400640 1 -- 172.21.15.49:0/2583079011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586487c4500 con 0x5586488cc300 2024-02-26T06:24:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.773+0000 7fc77da00640 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c2d80 0x5586488c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.773+0000 7fc77d000640 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488c3af0 0x5586488c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.774+0000 7fc77e400640 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488cc300 0x5586488c6670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc77000aa30 tx=0x7fc77000aef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2e9fd9ad4781ff server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.775+0000 7fc766000640 1 -- 172.21.15.49:0/2583079011 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770011ed0 con 0x5586488cc300 2024-02-26T06:24:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.775+0000 7fc766000640 1 -- 172.21.15.49:0/2583079011 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc770005020 con 0x5586488cc300 2024-02-26T06:24:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.775+0000 7fc766000640 1 -- 172.21.15.49:0/2583079011 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770004ca0 con 0x5586488cc300 2024-02-26T06:24:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.775+0000 7fc783439740 1 -- 172.21.15.49:0/2583079011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488cc300 msgr2=0x5586488c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.775+0000 7fc783439740 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488cc300 0x5586488c6670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc77000aa30 tx=0x7fc77000aef0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.776+0000 7fc783439740 1 -- 172.21.15.49:0/2583079011 shutdown_connections 2024-02-26T06:24:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.776+0000 7fc783439740 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488cc300 0x5586488c6670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.776+0000 7fc783439740 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488c3af0 0x5586488c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.776+0000 7fc783439740 1 --2- 172.21.15.49:0/2583079011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c2d80 0x5586488c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.776+0000 7fc783439740 1 -- 172.21.15.49:0/2583079011 >> 172.21.15.49:0/2583079011 conn(0x55864888a480 msgr2=0x5586488bd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.776+0000 7fc783439740 1 -- 172.21.15.49:0/2583079011 shutdown_connections 2024-02-26T06:24:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.776+0000 7fc783439740 1 -- 172.21.15.49:0/2583079011 wait complete. 2024-02-26T06:24:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 1 -- start start 2024-02-26T06:24:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 1 Processor -- start 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586488d37b0 con 0x5586488cc300 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558648932680 con 0x5586488c2d80 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586489327f0 con 0x5586488c3af0 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.777+0000 7fc783439740 10 client.0 _ll_register_callbacks cb 0x7fc76c00f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.778+0000 7fc77da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.778+0000 7fc77da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488d1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39402/0 (socket says 172.21.15.49:39402) 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.778+0000 7fc77da00640 1 -- 172.21.15.49:0/1826804497 learned_addr learned my addr 172.21.15.49:0/1826804497 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.778+0000 7fc77d000640 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.778+0000 7fc77d000640 1 -- 172.21.15.49:0/1826804497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 msgr2=0x5586488d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.778+0000 7fc77d000640 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.778+0000 7fc77d000640 1 -- 172.21.15.49:0/1826804497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 msgr2=0x5586488d2290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.779+0000 7fc77e400640 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.779+0000 7fc77d000640 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.779+0000 7fc77d000640 1 -- 172.21.15.49:0/1826804497 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558648827c20 con 0x5586488c3af0 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.780+0000 7fc77e400640 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.780+0000 7fc77d000640 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488d1d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc7680ee5e0 tx=0x7fc768003760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578286729688d4f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.780+0000 7fc75b400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.780+0000 7fc75b400640 1 -- 172.21.15.49:0/1826804497 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680db040 con 0x5586488c3af0 2024-02-26T06:24:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.780+0000 7fc75b400640 1 -- 172.21.15.49:0/1826804497 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc7680d2030 con 0x5586488c3af0 2024-02-26T06:24:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.781+0000 7fc783439740 1 -- 172.21.15.49:0/1826804497 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55864884ed00 con 0x5586488c3af0 2024-02-26T06:24:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.781+0000 7fc75b400640 1 -- 172.21.15.49:0/1826804497 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7680c1290 con 0x5586488c3af0 2024-02-26T06:24:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.781+0000 7fc783439740 10 client.25756 fetch_fsmap learned FSMap version 28 2024-02-26T06:24:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.781+0000 7fc783439740 1 -- 172.21.15.49:0/1826804497 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fc7680d5e70 con 0x5586488c3af0 2024-02-26T06:24:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.781+0000 7fc75b400640 1 -- 172.21.15.49:0/1826804497 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55864884ed00 con 0x5586488c3af0 2024-02-26T06:24:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.782+0000 7fc75b400640 1 -- 172.21.15.49:0/1826804497 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc76803c070 con 0x5586488c3af0 2024-02-26T06:24:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.783+0000 7fc783439740 10 client.25756 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:24:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.783+0000 7fc783439740 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.783+0000 7fc783439740 20 client.25756 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.783+0000 7fc783439740 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558648933c80 0x5586489360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.783+0000 7fc783439740 4 client.25756 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:24:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.783+0000 7fc783439740 1 -- 172.21.15.49:0/1826804497 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558648529600 con 0x558648933c80 2024-02-26T06:24:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.783+0000 7fc77da00640 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558648933c80 0x5586489360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.784+0000 7fc77da00640 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558648933c80 0x5586489360a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c53f2dc85a26bf server_cookie=56c8af4847b62413 in_seq=0 out_seq=0 2024-02-26T06:24:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.784+0000 7fc75b400640 10 client.25756 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.798+0000 7fc75b400640 1 -- 172.21.15.49:0/1826804497 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558648529600 con 0x558648933c80 2024-02-26T06:24:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.799+0000 7fc75b400640 10 client.25756 handle_command_reply: tid=0 2024-02-26T06:24:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.799+0000 7fc783439740 1 client.25756 shutdown 2024-02-26T06:24:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.799+0000 7fc783439740 1 -- 172.21.15.49:0/1826804497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 msgr2=0x5586488d1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.799+0000 7fc783439740 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488d1d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc7680ee5e0 tx=0x7fc768003760 comp rx=0 tx=0).stop 2024-02-26T06:24:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.800+0000 7fc783439740 1 -- 172.21.15.49:0/1826804497 shutdown_connections 2024-02-26T06:24:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.800+0000 7fc783439740 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558648933c80 0x5586489360a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.800+0000 7fc783439740 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586488cc300 0x5586488d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.800+0000 7fc783439740 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586488c3af0 0x5586488d1d50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.800+0000 7fc75b400640 0 client.25756 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.800+0000 7fc783439740 1 --2- 172.21.15.49:0/1826804497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5586488c2d80 0x5586488d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.800+0000 7fc783439740 1 -- 172.21.15.49:0/1826804497 >> 172.21.15.49:0/1826804497 conn(0x55864888a480 msgr2=0x5586488bd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.800+0000 7fc783439740 1 -- 172.21.15.49:0/1826804497 shutdown_connections 2024-02-26T06:24:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.801+0000 7fc783439740 1 -- 172.21.15.49:0/1826804497 wait complete. 2024-02-26T06:24:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.801+0000 7fc783439740 20 client.25756 trim_cache size 0 max 16384 2024-02-26T06:24:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:24:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:24:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:24:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:24:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 -- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc7600a2470 msgr2=0x7fc7600a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc7600a2470 0x7fc7600a4930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009fd0 tx=0x7fc76c009500 comp rx=0 tx=0).stop 2024-02-26T06:24:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 -- 172.21.15.49:0/2261005610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778108690 msgr2=0x7fc77819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778108690 0x7fc77819fb80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc768009230 tx=0x7fc76802c640 comp rx=0 tx=0).stop 2024-02-26T06:24:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 -- 172.21.15.49:0/2261005610 shutdown_connections 2024-02-26T06:24:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc7600a2470 0x7fc7600a4930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc778108690 0x7fc77819fb80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc778199800 0x7fc778199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 --2- 172.21.15.49:0/2261005610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7781035c0 0x7fc77819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.802+0000 7fc780400640 1 -- 172.21.15.49:0/2261005610 >> 172.21.15.49:0/2261005610 conn(0x7fc7780fee40 msgr2=0x7fc7780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.803+0000 7fc780400640 1 -- 172.21.15.49:0/2261005610 shutdown_connections 2024-02-26T06:24:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:32.803+0000 7fc780400640 1 -- 172.21.15.49:0/2261005610 wait complete. 2024-02-26T06:24:32.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:32 smithi080 ceph-mon[36206]: pgmap v37: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 14 MiB/s wr, 769 op/s 2024-02-26T06:24:32.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:32 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:32.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:32 smithi115 ceph-mon[36426]: pgmap v37: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 14 MiB/s wr, 769 op/s 2024-02-26T06:24:32.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:32 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:32.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:32 smithi049 ceph-mon[31696]: pgmap v37: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 14 MiB/s wr, 769 op/s 2024-02-26T06:24:32.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:32 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:34.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:34 smithi080 ceph-mon[36206]: pgmap v38: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 13 MiB/s wr, 741 op/s 2024-02-26T06:24:34.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:34 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:34.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:34 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:34.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:34 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:24:34.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:34 smithi115 ceph-mon[36426]: pgmap v38: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 13 MiB/s wr, 741 op/s 2024-02-26T06:24:34.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:34 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:34.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:34 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:34.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:34 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:24:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:34 smithi049 ceph-mon[31696]: pgmap v38: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 13 MiB/s wr, 741 op/s 2024-02-26T06:24:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:34 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:34 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:34 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:24:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 45.0 s, 3080.2 tps, lat 32.396 ms stddev 35.129 2024-02-26T06:24:36.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:36 smithi049 ceph-mon[31696]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 18 MiB/s wr, 1.05k op/s 2024-02-26T06:24:36.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:36 smithi080 ceph-mon[36206]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 18 MiB/s wr, 1.05k op/s 2024-02-26T06:24:36.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:36 smithi115 ceph-mon[36426]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 18 MiB/s wr, 1.05k op/s 2024-02-26T06:24:38.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:38 smithi080 ceph-mon[36206]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-02-26T06:24:38.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:38 smithi115 ceph-mon[36426]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-02-26T06:24:38.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:38 smithi049 ceph-mon[31696]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-02-26T06:24:40.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:40 smithi080 ceph-mon[36206]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 915 op/s 2024-02-26T06:24:40.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:40 smithi115 ceph-mon[36426]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 915 op/s 2024-02-26T06:24:40.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:40 smithi049 ceph-mon[31696]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 915 op/s 2024-02-26T06:24:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 50.0 s, 2944.0 tps, lat 33.973 ms stddev 42.724 2024-02-26T06:24:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:42 smithi080 ceph-mon[36206]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-02-26T06:24:42.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:42 smithi115 ceph-mon[36426]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-02-26T06:24:42.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:42 smithi049 ceph-mon[31696]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-02-26T06:24:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:44 smithi080 ceph-mon[36206]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-02-26T06:24:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:44 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:44 smithi115 ceph-mon[36426]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-02-26T06:24:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:44 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:44 smithi049 ceph-mon[31696]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-02-26T06:24:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:44 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:24:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 -- 172.21.15.49:0/3465293782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0468106150 msgr2=0x7f0468069920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 --2- 172.21.15.49:0/3465293782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0468106150 0x7f0468069920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0458009900 tx=0x7f045802bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 -- 172.21.15.49:0/3465293782 shutdown_connections 2024-02-26T06:24:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 --2- 172.21.15.49:0/3465293782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0468103bb0 0x7f0468104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 --2- 172.21.15.49:0/3465293782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0468069e60 0x7f046806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 --2- 172.21.15.49:0/3465293782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0468106150 0x7f0468069920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 -- 172.21.15.49:0/3465293782 >> 172.21.15.49:0/3465293782 conn(0x7f0468077060 msgr2=0x7f0468077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 -- 172.21.15.49:0/3465293782 shutdown_connections 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.992+0000 7f046cc00640 1 -- 172.21.15.49:0/3465293782 wait complete. 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.993+0000 7f046cc00640 1 Processor -- start 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.993+0000 7f046cc00640 1 -- start start 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.994+0000 7f046cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0468069e60 0x7f04681a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.994+0000 7f046cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0468103bb0 0x7f04681a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.994+0000 7f046cc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0468106150 0x7f046819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.994+0000 7f046cc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046819e140 con 0x7f0468069e60 2024-02-26T06:24:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.994+0000 7f046cc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046819e2b0 con 0x7f0468106150 2024-02-26T06:24:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.994+0000 7f046cc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046819e470 con 0x7f0468103bb0 2024-02-26T06:24:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.994+0000 7f0466800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0468106150 0x7f046819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0465e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0468069e60 0x7f04681a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0466800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0468106150 0x7f046819daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43042/0 (socket says 172.21.15.49:43042) 2024-02-26T06:24:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0466800640 1 -- 172.21.15.49:0/3986710389 learned_addr learned my addr 172.21.15.49:0/3986710389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0465400640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0468103bb0 0x7f04681a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0465e00640 1 -- 172.21.15.49:0/3986710389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0468103bb0 msgr2=0x7f04681a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0465e00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0468103bb0 0x7f04681a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0465e00640 1 -- 172.21.15.49:0/3986710389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0468106150 msgr2=0x7f046819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0465e00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0468106150 0x7f046819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0465e00640 1 -- 172.21.15.49:0/3986710389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0454009490 con 0x7f0468069e60 2024-02-26T06:24:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.995+0000 7f0466800640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0468106150 0x7f046819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.996+0000 7f0465e00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0468069e60 0x7f04681a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f045802c170 tx=0x7f045802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.996+0000 7f045ea00640 1 -- 172.21.15.49:0/3986710389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458004410 con 0x7f0468069e60 2024-02-26T06:24:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.997+0000 7f046cc00640 1 -- 172.21.15.49:0/3986710389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0458009560 con 0x7f0468069e60 2024-02-26T06:24:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.997+0000 7f045ea00640 1 -- 172.21.15.49:0/3986710389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04580052d0 con 0x7f0468069e60 2024-02-26T06:24:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.997+0000 7f045ea00640 1 -- 172.21.15.49:0/3986710389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458003bf0 con 0x7f0468069e60 2024-02-26T06:24:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:45.997+0000 7f046cc00640 1 -- 172.21.15.49:0/3986710389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04681aa800 con 0x7f0468069e60 2024-02-26T06:24:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.002+0000 7f046f9f8740 1 Processor -- start 2024-02-26T06:24:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.006+0000 7f046f9f8740 1 -- start start 2024-02-26T06:24:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.006+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41379840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.006+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4137a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.006+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a413856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.006+0000 7f046f9f8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a413355f0 con 0x561a41379d80 2024-02-26T06:24:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.006+0000 7f046f9f8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a412e7890 con 0x561a41383300 2024-02-26T06:24:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.006+0000 7f046f9f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a412e4dc0 con 0x561a4137aaf0 2024-02-26T06:24:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.007+0000 7f045ea00640 1 -- 172.21.15.49:0/3986710389 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f0458003d50 con 0x7f0468069e60 2024-02-26T06:24:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.008+0000 7f0466800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41379840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.008+0000 7f0466800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41379840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42918/0 (socket says 172.21.15.49:42918) 2024-02-26T06:24:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.008+0000 7f0466800640 1 -- 172.21.15.49:0/3334798389 learned_addr learned my addr 172.21.15.49:0/3334798389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.009+0000 7f0465e00640 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4137a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.009+0000 7f0465400640 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a413856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.011+0000 7f0466800640 1 -- 172.21.15.49:0/3334798389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 msgr2=0x561a413856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.011+0000 7f0466800640 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a413856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.011+0000 7f0466800640 1 -- 172.21.15.49:0/3334798389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 msgr2=0x561a4137a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.011+0000 7f0466800640 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4137a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.011+0000 7f0466800640 1 -- 172.21.15.49:0/3334798389 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a41261ad0 con 0x561a4137aaf0 2024-02-26T06:24:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.012+0000 7f045ea00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f044c0a24f0 0x7f044c0a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.012+0000 7f045ea00640 1 -- 172.21.15.49:0/3986710389 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f04580f04d0 con 0x7f0468069e60 2024-02-26T06:24:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.013+0000 7f0465400640 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a413856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.013+0000 7f0465e00640 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4137a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.015+0000 7f0465e00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f044c0a24f0 0x7f044c0a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.016+0000 7f0465e00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f044c0a24f0 0x7f044c0a49b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04580edbc0 tx=0x7f04580f3ea0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.020+0000 7f0466800640 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41379840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0460011850 tx=0x7f0460011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97fafff6df8bc9d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.020+0000 7f0453000640 1 -- 172.21.15.49:0/3334798389 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460011330 con 0x561a4137aaf0 2024-02-26T06:24:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.021+0000 7f0453000640 1 -- 172.21.15.49:0/3334798389 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f046000fe80 con 0x561a4137aaf0 2024-02-26T06:24:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.021+0000 7f0453000640 1 -- 172.21.15.49:0/3334798389 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600193e0 con 0x561a4137aaf0 2024-02-26T06:24:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.023+0000 7f046f9f8740 1 -- 172.21.15.49:0/3334798389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 msgr2=0x561a41379840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.023+0000 7f046f9f8740 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41379840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0460011850 tx=0x7f0460011d10 comp rx=0 tx=0).stop 2024-02-26T06:24:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.024+0000 7f046f9f8740 1 -- 172.21.15.49:0/3334798389 shutdown_connections 2024-02-26T06:24:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.024+0000 7f046f9f8740 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a413856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.024+0000 7f046f9f8740 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4137a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.024+0000 7f046f9f8740 1 --2- 172.21.15.49:0/3334798389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41379840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.024+0000 7f046f9f8740 1 -- 172.21.15.49:0/3334798389 >> 172.21.15.49:0/3334798389 conn(0x561a41341480 msgr2=0x561a41374a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.024+0000 7f046f9f8740 1 -- 172.21.15.49:0/3334798389 shutdown_connections 2024-02-26T06:24:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.024+0000 7f046f9f8740 1 -- 172.21.15.49:0/3334798389 wait complete. 2024-02-26T06:24:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.025+0000 7f046f9f8740 1 -- start start 2024-02-26T06:24:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.025+0000 7f046f9f8740 1 Processor -- start 2024-02-26T06:24:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.026+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41379d80 0x561a413802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.027+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a413807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.027+0000 7f0466800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41379d80 0x561a413802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.027+0000 7f0466800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41379d80 0x561a413802a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43070/0 (socket says 172.21.15.49:43070) 2024-02-26T06:24:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.027+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41383300 0x561a41380d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.027+0000 7f0466800640 1 -- 172.21.15.49:0/233128441 learned_addr learned my addr 172.21.15.49:0/233128441 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.027+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a412e05a0 con 0x561a41383300 2024-02-26T06:24:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.027+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a41285b20 con 0x561a41379d80 2024-02-26T06:24:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.027+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a412857c0 con 0x561a4137aaf0 2024-02-26T06:24:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.028+0000 7f046f9f8740 10 client.0 _ll_register_callbacks cb 0x561a41299520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.028+0000 7f0465400640 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a413807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.028+0000 7f0466800640 1 -- 172.21.15.49:0/233128441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 msgr2=0x561a413807e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.028+0000 7f0466800640 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a413807e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.028+0000 7f0466800640 1 -- 172.21.15.49:0/233128441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41383300 msgr2=0x561a41380d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.028+0000 7f0466800640 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41383300 0x561a41380d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.028+0000 7f0466800640 1 -- 172.21.15.49:0/233128441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0460011500 con 0x561a41379d80 2024-02-26T06:24:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.029+0000 7f0466800640 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41379d80 0x561a413802a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0460009a70 tx=0x7f0460011390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a74adfee2c3e3d96 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.029+0000 7f0447400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.029+0000 7f0447400640 1 -- 172.21.15.49:0/233128441 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460004a40 con 0x561a41379d80 2024-02-26T06:24:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.029+0000 7f0447400640 1 -- 172.21.15.49:0/233128441 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f0460004ba0 con 0x561a41379d80 2024-02-26T06:24:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.030+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a412b2ca0 con 0x561a41379d80 2024-02-26T06:24:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.032+0000 7f0447400640 1 -- 172.21.15.49:0/233128441 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0460004d60 con 0x561a41379d80 2024-02-26T06:24:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.032+0000 7f046f9f8740 10 client.25733 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.032+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561a412dec20 con 0x561a41379d80 2024-02-26T06:24:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.032+0000 7f0447400640 1 -- 172.21.15.49:0/233128441 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561a412b2ca0 con 0x561a41379d80 2024-02-26T06:24:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.032+0000 7f0447400640 1 -- 172.21.15.49:0/233128441 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0460022070 con 0x561a41379d80 2024-02-26T06:24:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.033+0000 7f046f9f8740 10 client.25733 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.033+0000 7f046f9f8740 10 client.25733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.033+0000 7f046f9f8740 20 client.25733 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.033+0000 7f046f9f8740 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561a413931c0 0x561a413955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.033+0000 7f046f9f8740 4 client.25733 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.033+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a412d6cb0 con 0x561a413931c0 2024-02-26T06:24:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.033+0000 7f0465400640 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561a413931c0 0x561a413955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.034+0000 7f0465400640 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561a413931c0 0x561a413955e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b2ef2ffe1eda8b9 server_cookie=a5b72fb52460207e in_seq=0 out_seq=0 2024-02-26T06:24:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.034+0000 7f0447400640 10 client.25733 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.035+0000 7f0447400640 1 -- 172.21.15.49:0/233128441 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a412d6cb0 con 0x561a413931c0 2024-02-26T06:24:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.035+0000 7f0447400640 10 client.25733 handle_command_reply: tid=0 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.035+0000 7f046f9f8740 1 client.25733 shutdown 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41379d80 msgr2=0x561a413802a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f046f9f8740 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41379d80 0x561a413802a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0460009a70 tx=0x7f0460011390 comp rx=0 tx=0).stop 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 shutdown_connections 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f046f9f8740 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561a413931c0 0x561a413955e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f046f9f8740 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41383300 0x561a41380d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f046f9f8740 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a413807e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f046f9f8740 1 --2- 172.21.15.49:0/233128441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41379d80 0x561a413802a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 >> 172.21.15.49:0/233128441 conn(0x561a41341480 msgr2=0x561a413767c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.036+0000 7f0447400640 0 client.25733 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.037+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 shutdown_connections 2024-02-26T06:24:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.037+0000 7f046f9f8740 1 -- 172.21.15.49:0/233128441 wait complete. 2024-02-26T06:24:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.037+0000 7f046f9f8740 20 client.25733 trim_cache size 0 max 16384 2024-02-26T06:24:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.073+0000 7f046f9f8740 1 Processor -- start 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.073+0000 7f046f9f8740 1 -- start start 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a413829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41380b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a4137d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f046f9f8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a41028e60 con 0x561a41379d80 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f046f9f8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a41374ff0 con 0x561a41383300 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f046f9f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a41385f10 con 0x561a4137aaf0 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f0466800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a413829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f0466800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a413829d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44692/0 (socket says 172.21.15.49:44692) 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f0466800640 1 -- 172.21.15.49:0/421242416 learned_addr learned my addr 172.21.15.49:0/421242416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.074+0000 7f0466800640 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a4137d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.075+0000 7f0465400640 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41380b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.075+0000 7f0466800640 1 -- 172.21.15.49:0/421242416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 msgr2=0x561a41380b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.075+0000 7f0466800640 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41380b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.075+0000 7f0466800640 1 -- 172.21.15.49:0/421242416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 msgr2=0x561a4137d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.075+0000 7f0466800640 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a4137d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.075+0000 7f0466800640 1 -- 172.21.15.49:0/421242416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a4127b500 con 0x561a41379d80 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.075+0000 7f0465400640 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41380b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.075+0000 7f0466800640 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a413829d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f04600114d0 tx=0x7f04600021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a60d9b62f68b58f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.076+0000 7f0452600640 1 -- 172.21.15.49:0/421242416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600049d0 con 0x561a41379d80 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.076+0000 7f0452600640 1 -- 172.21.15.49:0/421242416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0460005520 con 0x561a41379d80 2024-02-26T06:24:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.076+0000 7f0452600640 1 -- 172.21.15.49:0/421242416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046000fb10 con 0x561a41379d80 2024-02-26T06:24:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 -- 172.21.15.49:0/421242416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 msgr2=0x561a413829d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a413829d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f04600114d0 tx=0x7f04600021f0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 -- 172.21.15.49:0/421242416 shutdown_connections 2024-02-26T06:24:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a4137d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41380b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 --2- 172.21.15.49:0/421242416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a413829d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 -- 172.21.15.49:0/421242416 >> 172.21.15.49:0/421242416 conn(0x561a41341480 msgr2=0x561a4137eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 -- 172.21.15.49:0/421242416 shutdown_connections 2024-02-26T06:24:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.077+0000 7f046f9f8740 1 -- 172.21.15.49:0/421242416 wait complete. 2024-02-26T06:24:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 1 -- start start 2024-02-26T06:24:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 1 Processor -- start 2024-02-26T06:24:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4138a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41388700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a41388c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a4138aa40 con 0x561a41379d80 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a41389180 con 0x561a41383300 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a41389340 con 0x561a4137aaf0 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.078+0000 7f046f9f8740 10 client.0 _ll_register_callbacks cb 0x7f0460011250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.079+0000 7f0465400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41388700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.079+0000 7f0465400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41388700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42942/0 (socket says 172.21.15.49:42942) 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.079+0000 7f0465400640 1 -- 172.21.15.49:0/1472889483 learned_addr learned my addr 172.21.15.49:0/1472889483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.079+0000 7f0466800640 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a41388c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.079+0000 7f0466800640 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4138a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.080+0000 7f0465400640 1 -- 172.21.15.49:0/1472889483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 msgr2=0x561a41388c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.080+0000 7f0465400640 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a41388c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.080+0000 7f0465400640 1 -- 172.21.15.49:0/1472889483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 msgr2=0x561a4138a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.080+0000 7f0465400640 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4138a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.080+0000 7f0465400640 1 -- 172.21.15.49:0/1472889483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a412dec20 con 0x561a4137aaf0 2024-02-26T06:24:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.080+0000 7f0466800640 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4138a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.081+0000 7f0465400640 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41388700 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x561a4137a820 tx=0x7f0454012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adae6a5174d52f67 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.081+0000 7f0447400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.082+0000 7f0447400640 1 -- 172.21.15.49:0/1472889483 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0454018040 con 0x561a4137aaf0 2024-02-26T06:24:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.082+0000 7f0447400640 1 -- 172.21.15.49:0/1472889483 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f045401d7e0 con 0x561a4137aaf0 2024-02-26T06:24:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.082+0000 7f046f9f8740 1 -- 172.21.15.49:0/1472889483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a41305d00 con 0x561a4137aaf0 2024-02-26T06:24:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.083+0000 7f0447400640 1 -- 172.21.15.49:0/1472889483 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0454005030 con 0x561a4137aaf0 2024-02-26T06:24:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.084+0000 7f046f9f8740 10 client.25774 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.084+0000 7f046f9f8740 1 -- 172.21.15.49:0/1472889483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561a412a1290 con 0x561a4137aaf0 2024-02-26T06:24:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.084+0000 7f0447400640 1 -- 172.21.15.49:0/1472889483 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561a41305d00 con 0x561a4137aaf0 2024-02-26T06:24:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.085+0000 7f0447400640 1 -- 172.21.15.49:0/1472889483 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0454026450 con 0x561a4137aaf0 2024-02-26T06:24:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.085+0000 7f046f9f8740 10 client.25774 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.085+0000 7f046f9f8740 10 client.25774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.085+0000 7f046f9f8740 20 client.25774 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.085+0000 7f046f9f8740 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561a413e4bf0 0x561a413e5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.085+0000 7f046f9f8740 4 client.25774 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:24:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.085+0000 7f046f9f8740 1 -- 172.21.15.49:0/1472889483 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a40fe0600 con 0x561a413e4bf0 2024-02-26T06:24:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.085+0000 7f0466800640 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561a413e4bf0 0x561a413e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.086+0000 7f0466800640 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561a413e4bf0 0x561a413e5000 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=858e572d71c3d031 server_cookie=6cdef49d8fa1060 in_seq=0 out_seq=0 2024-02-26T06:24:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.086+0000 7f0447400640 10 client.25774 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.120+0000 7f0447400640 1 -- 172.21.15.49:0/1472889483 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a40fe0600 con 0x561a413e4bf0 2024-02-26T06:24:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.121+0000 7f0447400640 10 client.25774 handle_command_reply: tid=0 2024-02-26T06:24:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.121+0000 7f046f9f8740 1 client.25774 shutdown 2024-02-26T06:24:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.121+0000 7f046f9f8740 1 -- 172.21.15.49:0/1472889483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 msgr2=0x561a41388700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.121+0000 7f046f9f8740 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41388700 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x561a4137a820 tx=0x7f0454012820 comp rx=0 tx=0).stop 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 1 -- 172.21.15.49:0/1472889483 shutdown_connections 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561a413e4bf0 0x561a413e5000 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561a41383300 0x561a41388c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4137aaf0 0x561a41388700 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 1 --2- 172.21.15.49:0/1472889483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a41379d80 0x561a4138a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 1 -- 172.21.15.49:0/1472889483 >> 172.21.15.49:0/1472889483 conn(0x561a41341480 msgr2=0x561a41373480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f0447400640 0 client.25774 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 1 -- 172.21.15.49:0/1472889483 shutdown_connections 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 1 -- 172.21.15.49:0/1472889483 wait complete. 2024-02-26T06:24:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.122+0000 7f046f9f8740 20 client.25774 trim_cache size 0 max 16384 2024-02-26T06:24:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:24:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:24:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:24:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:24:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.123+0000 7f046cc00640 1 -- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f044c0a24f0 msgr2=0x7f044c0a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.123+0000 7f046cc00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f044c0a24f0 0x7f044c0a49b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04580edbc0 tx=0x7f04580f3ea0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.123+0000 7f046cc00640 1 -- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0468069e60 msgr2=0x7f04681a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.123+0000 7f046cc00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0468069e60 0x7f04681a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f045802c170 tx=0x7f045802c650 comp rx=0 tx=0).stop 2024-02-26T06:24:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.124+0000 7f046cc00640 1 -- 172.21.15.49:0/3986710389 shutdown_connections 2024-02-26T06:24:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.124+0000 7f046cc00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f044c0a24f0 0x7f044c0a49b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.124+0000 7f046cc00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0468106150 0x7f046819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.124+0000 7f046cc00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0468103bb0 0x7f04681a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.124+0000 7f046cc00640 1 --2- 172.21.15.49:0/3986710389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0468069e60 0x7f04681a3940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.124+0000 7f046cc00640 1 -- 172.21.15.49:0/3986710389 >> 172.21.15.49:0/3986710389 conn(0x7f0468077060 msgr2=0x7f0468102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.125+0000 7f046cc00640 1 -- 172.21.15.49:0/3986710389 shutdown_connections 2024-02-26T06:24:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:46.125+0000 7f046cc00640 1 -- 172.21.15.49:0/3986710389 wait complete. 2024-02-26T06:24:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 55.0 s, 3062.0 tps, lat 32.728 ms stddev 41.575 2024-02-26T06:24:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:46 smithi080 ceph-mon[36206]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-02-26T06:24:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:46 smithi115 ceph-mon[36426]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-02-26T06:24:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:46 smithi049 ceph-mon[31696]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-02-26T06:24:48.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:48 smithi080 ceph-mon[36206]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.9 MiB/s wr, 632 op/s 2024-02-26T06:24:48.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:48 smithi115 ceph-mon[36426]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.9 MiB/s wr, 632 op/s 2024-02-26T06:24:48.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:48 smithi049 ceph-mon[31696]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.9 MiB/s wr, 632 op/s 2024-02-26T06:24:50.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:50 smithi080 ceph-mon[36206]: pgmap v46: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 11 MiB/s wr, 925 op/s 2024-02-26T06:24:50.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:50 smithi115 ceph-mon[36426]: pgmap v46: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 11 MiB/s wr, 925 op/s 2024-02-26T06:24:50.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:50 smithi049 ceph-mon[31696]: pgmap v46: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 11 MiB/s wr, 925 op/s 2024-02-26T06:24:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 60.0 s, 2721.2 tps, lat 36.762 ms stddev 40.401 2024-02-26T06:24:51.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:24:51.633 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:24:51.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.941+0000 7f619b000640 1 -- 172.21.15.49:0/1077534308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194103580 msgr2=0x7f6194103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.941+0000 7f619b000640 1 --2- 172.21.15.49:0/1077534308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194103580 0x7f6194103960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f617c013180 tx=0x7f617c033f90 comp rx=0 tx=0).stop 2024-02-26T06:24:51.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.942+0000 7f619b000640 1 -- 172.21.15.49:0/1077534308 shutdown_connections 2024-02-26T06:24:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.942+0000 7f619b000640 1 --2- 172.21.15.49:0/1077534308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6194108650 0x7f6194110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.942+0000 7f619b000640 1 --2- 172.21.15.49:0/1077534308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6194103f30 0x7f6194107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.943+0000 7f619b000640 1 --2- 172.21.15.49:0/1077534308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194103580 0x7f6194103960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.943+0000 7f619b000640 1 -- 172.21.15.49:0/1077534308 >> 172.21.15.49:0/1077534308 conn(0x7f61940fedc0 msgr2=0x7f61941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:51.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.943+0000 7f619b000640 1 -- 172.21.15.49:0/1077534308 shutdown_connections 2024-02-26T06:24:51.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.943+0000 7f619b000640 1 -- 172.21.15.49:0/1077534308 wait complete. 2024-02-26T06:24:51.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.945+0000 7f619b000640 1 Processor -- start 2024-02-26T06:24:51.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.945+0000 7f619b000640 1 -- start start 2024-02-26T06:24:51.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.946+0000 7f619b000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6194103580 0x7f6194195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f619b000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194108650 0x7f61941960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f6193e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6194103580 0x7f6194195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f619b000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6194199fa0 0x7f619419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f619b000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419a960 con 0x7f6194103580 2024-02-26T06:24:51.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f619b000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419aad0 con 0x7f6194108650 2024-02-26T06:24:51.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f619b000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419ac90 con 0x7f6194199fa0 2024-02-26T06:24:51.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f6193e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6194103580 0x7f6194195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50232/0 (socket says 172.21.15.49:50232) 2024-02-26T06:24:51.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f6193e00640 1 -- 172.21.15.49:0/2517053280 learned_addr learned my addr 172.21.15.49:0/2517053280 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:51.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.947+0000 7f6199000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6194199fa0 0x7f619419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.948+0000 7f6199000640 1 -- 172.21.15.49:0/2517053280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194108650 msgr2=0x7f61941960b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:24:51.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.948+0000 7f6193400640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194108650 0x7f61941960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.948+0000 7f6199000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194108650 0x7f61941960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.948+0000 7f6199000640 1 -- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6194103580 msgr2=0x7f6194195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:51.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.948+0000 7f6199000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6194103580 0x7f6194195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.948+0000 7f6199000640 1 -- 172.21.15.49:0/2517053280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c009490 con 0x7f6194199fa0 2024-02-26T06:24:51.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.948+0000 7f6193e00640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6194103580 0x7f6194195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:51.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.948+0000 7f6193400640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194108650 0x7f61941960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:51.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.950+0000 7f6199000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6194199fa0 0x7f619419a420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e970 tx=0x7f618c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:51.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.950+0000 7f6190c00640 1 -- 172.21.15.49:0/2517053280 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c00ccc0 con 0x7f6194199fa0 2024-02-26T06:24:51.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.950+0000 7f6190c00640 1 -- 172.21.15.49:0/2517053280 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f618c00ce20 con 0x7f6194199fa0 2024-02-26T06:24:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.950+0000 7f6190c00640 1 -- 172.21.15.49:0/2517053280 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c010430 con 0x7f6194199fa0 2024-02-26T06:24:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.950+0000 7f619b000640 1 -- 172.21.15.49:0/2517053280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617c011d70 con 0x7f6194199fa0 2024-02-26T06:24:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.950+0000 7f619b000640 1 -- 172.21.15.49:0/2517053280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61941a25a0 con 0x7f6194199fa0 2024-02-26T06:24:51.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.955+0000 7f619dde4740 1 Processor -- start 2024-02-26T06:24:51.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.956+0000 7f6190c00640 1 -- 172.21.15.49:0/2517053280 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f618c004020 con 0x7f6194199fa0 2024-02-26T06:24:51.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.956+0000 7f619dde4740 1 -- start start 2024-02-26T06:24:51.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.957+0000 7f619dde4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f553cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.957+0000 7f619dde4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f554710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.957+0000 7f6193400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f553cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.957+0000 7f6193400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f553cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46240/0 (socket says 172.21.15.49:46240) 2024-02-26T06:24:51.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.957+0000 7f6193e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f554710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.958+0000 7f6193e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f554710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50248/0 (socket says 172.21.15.49:50248) 2024-02-26T06:24:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.958+0000 7f6190c00640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f61780a23d0 0x7f61780a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.959+0000 7f6190c00640 1 -- 172.21.15.49:0/2517053280 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f618c014070 con 0x7f6194199fa0 2024-02-26T06:24:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.960+0000 7f619dde4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f55f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.960+0000 7f619dde4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f4c3120 con 0x55e63f554290 2024-02-26T06:24:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.960+0000 7f619dde4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f4c0650 con 0x55e63f55d600 2024-02-26T06:24:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.960+0000 7f619dde4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f4bbe30 con 0x55e63f554f70 2024-02-26T06:24:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.960+0000 7f6193400640 1 -- 172.21.15.49:0/3102033976 learned_addr learned my addr 172.21.15.49:0/3102033976 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.960+0000 7f6199000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f61780a23d0 0x7f61780a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.960+0000 7f6193400640 1 --2- 172.21.15.49:0/3102033976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f55f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6193400640 1 -- 172.21.15.49:0/3102033976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 msgr2=0x55e63f55f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6193400640 1 --2- 172.21.15.49:0/3102033976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f55f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6193400640 1 -- 172.21.15.49:0/3102033976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 msgr2=0x55e63f554710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6193400640 1 --2- 172.21.15.49:0/3102033976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f554710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6193400640 1 -- 172.21.15.49:0/3102033976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63f475030 con 0x55e63f554f70 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6193400640 1 --2- 172.21.15.49:0/3102033976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f553cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6184015870 tx=0x7f6184009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c521045760c720d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6189600640 1 -- 172.21.15.49:0/3102033976 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618401a070 con 0x55e63f554f70 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6189600640 1 -- 172.21.15.49:0/3102033976 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61840150e0 con 0x55e63f554f70 2024-02-26T06:24:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.961+0000 7f6189600640 1 -- 172.21.15.49:0/3102033976 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184005280 con 0x55e63f554f70 2024-02-26T06:24:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.963+0000 7f619dde4740 1 -- 172.21.15.49:0/3102033976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 msgr2=0x55e63f553cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.963+0000 7f6199000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f61780a23d0 0x7f61780a4890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f618c0cbca0 tx=0x7f618c0097a0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.963+0000 7f619dde4740 1 --2- 172.21.15.49:0/3102033976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f553cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6184015870 tx=0x7f6184009bd0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.963+0000 7f619dde4740 1 -- 172.21.15.49:0/3102033976 shutdown_connections 2024-02-26T06:24:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.963+0000 7f619dde4740 1 --2- 172.21.15.49:0/3102033976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f55f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.963+0000 7f619dde4740 1 --2- 172.21.15.49:0/3102033976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f554710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.963+0000 7f619dde4740 1 --2- 172.21.15.49:0/3102033976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f553cc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.963+0000 7f619dde4740 1 -- 172.21.15.49:0/3102033976 >> 172.21.15.49:0/3102033976 conn(0x55e63f51b570 msgr2=0x55e63f54ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.964+0000 7f619dde4740 1 -- 172.21.15.49:0/3102033976 shutdown_connections 2024-02-26T06:24:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.964+0000 7f619dde4740 1 -- 172.21.15.49:0/3102033976 wait complete. 2024-02-26T06:24:51.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.965+0000 7f619dde4740 1 -- start start 2024-02-26T06:24:51.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.965+0000 7f619dde4740 1 Processor -- start 2024-02-26T06:24:51.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.967+0000 7f619dde4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f55a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.967+0000 7f619dde4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554f70 0x55e63f55ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.967+0000 7f6193400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f55a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.967+0000 7f619dde4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f55b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.967+0000 7f619dde4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f460e90 con 0x55e63f554f70 2024-02-26T06:24:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.968+0000 7f619dde4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f460b30 con 0x55e63f55d600 2024-02-26T06:24:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.968+0000 7f619dde4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f22a1c0 con 0x55e63f554290 2024-02-26T06:24:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.968+0000 7f619dde4740 10 client.0 _ll_register_callbacks cb 0x55e63f474930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.968+0000 7f6193e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554f70 0x55e63f55ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.968+0000 7f6193400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f55a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46256/0 (socket says 172.21.15.49:46256) 2024-02-26T06:24:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.968+0000 7f6193e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554f70 0x55e63f55ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50254/0 (socket says 172.21.15.49:50254) 2024-02-26T06:24:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.968+0000 7f6193400640 1 -- 172.21.15.49:0/1253618120 learned_addr learned my addr 172.21.15.49:0/1253618120 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.969+0000 7f6193400640 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f55b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.969+0000 7f6193400640 1 -- 172.21.15.49:0/1253618120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 msgr2=0x55e63f55b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.969+0000 7f6193400640 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f55b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.969+0000 7f6193400640 1 -- 172.21.15.49:0/1253618120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554f70 msgr2=0x55e63f55ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.969+0000 7f6193400640 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554f70 0x55e63f55ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.969+0000 7f6193400640 1 -- 172.21.15.49:0/1253618120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61840154a0 con 0x55e63f554290 2024-02-26T06:24:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.970+0000 7f6193400640 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f55a650 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55e63f54e5c0 tx=0x7f618400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4d3874b836384ff server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:51.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.970+0000 7f6182000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.970+0000 7f6182000640 1 -- 172.21.15.49:0/1253618120 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618401a040 con 0x55e63f554290 2024-02-26T06:24:51.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.970+0000 7f6182000640 1 -- 172.21.15.49:0/1253618120 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6184003b70 con 0x55e63f554290 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.970+0000 7f619dde4740 1 -- 172.21.15.49:0/1253618120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e63f1bb7e0 con 0x55e63f554290 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.970+0000 7f6182000640 1 -- 172.21.15.49:0/1253618120 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6184003d30 con 0x55e63f554290 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.970+0000 7f619dde4740 10 client.25786 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f619dde4740 1 -- 172.21.15.49:0/1253618120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e63f4ba4b0 con 0x55e63f554290 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f6182000640 1 -- 172.21.15.49:0/1253618120 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f618402f020 con 0x55e63f554290 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f6182000640 1 -- 172.21.15.49:0/1253618120 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6184011600 con 0x55e63f554290 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f619dde4740 10 client.25786 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f619dde4740 10 client.25786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f619dde4740 20 client.25786 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f619dde4740 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e63f56d730 0x55e63f56fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f619dde4740 4 client.25786 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f619dde4740 1 -- 172.21.15.49:0/1253618120 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e63f1f8720 con 0x55e63f56d730 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.971+0000 7f6193e00640 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e63f56d730 0x55e63f56fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.973+0000 7f6193e00640 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e63f56d730 0x55e63f56fb50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8189bde3edfe9ead server_cookie=a140e545d217892f in_seq=0 out_seq=0 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.973+0000 7f6182000640 10 client.25786 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.974+0000 7f6182000640 1 -- 172.21.15.49:0/1253618120 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e63f1f8720 con 0x55e63f56d730 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.974+0000 7f6182000640 10 client.25786 handle_command_reply: tid=0 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.974+0000 7f619dde4740 1 client.25786 shutdown 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.975+0000 7f619dde4740 1 -- 172.21.15.49:0/1253618120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 msgr2=0x55e63f55a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.975+0000 7f619dde4740 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f55a650 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55e63f54e5c0 tx=0x7f618400ed10 comp rx=0 tx=0).stop 2024-02-26T06:24:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 1 -- 172.21.15.49:0/1253618120 shutdown_connections 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e63f56d730 0x55e63f56fb50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f55b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554f70 0x55e63f55ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 1 --2- 172.21.15.49:0/1253618120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f55a650 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f6182000640 0 client.25786 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 1 -- 172.21.15.49:0/1253618120 >> 172.21.15.49:0/1253618120 conn(0x55e63f51b570 msgr2=0x55e63f54d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 1 -- 172.21.15.49:0/1253618120 shutdown_connections 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 1 -- 172.21.15.49:0/1253618120 wait complete. 2024-02-26T06:24:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:51.976+0000 7f619dde4740 20 client.25786 trim_cache size 0 max 16384 2024-02-26T06:24:52.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.008+0000 7f619dde4740 1 Processor -- start 2024-02-26T06:24:52.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.008+0000 7f619dde4740 1 -- start start 2024-02-26T06:24:52.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.009+0000 7f619dde4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f55aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.009+0000 7f619dde4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f558f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.010+0000 7f619dde4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f559460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.010+0000 7f619dde4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f55fe40 con 0x55e63f554290 2024-02-26T06:24:52.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.010+0000 7f619dde4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f559a00 con 0x55e63f55d600 2024-02-26T06:24:52.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.010+0000 7f619dde4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f553340 con 0x55e63f554f70 2024-02-26T06:24:52.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.010+0000 7f6193e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f558f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.010+0000 7f6193e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f558f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46272/0 (socket says 172.21.15.49:46272) 2024-02-26T06:24:52.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.010+0000 7f6193e00640 1 -- 172.21.15.49:0/2015413182 learned_addr learned my addr 172.21.15.49:0/2015413182 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:52.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.011+0000 7f6193400640 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f55aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.011+0000 7f6193e00640 1 -- 172.21.15.49:0/2015413182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 msgr2=0x55e63f559460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:52.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.011+0000 7f6193e00640 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f559460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.011+0000 7f6193e00640 1 -- 172.21.15.49:0/2015413182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 msgr2=0x55e63f55aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:52.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.011+0000 7f6193e00640 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f55aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.011+0000 7f6193e00640 1 -- 172.21.15.49:0/2015413182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63f456920 con 0x55e63f554f70 2024-02-26T06:24:52.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.012+0000 7f6193400640 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f55aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:52.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.012+0000 7f6193e00640 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f558f20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f617c034c20 tx=0x7f617c036040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6feae0510ab00575 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:52.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.012+0000 7f6188c00640 1 -- 172.21.15.49:0/2015413182 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c04eb00 con 0x55e63f554f70 2024-02-26T06:24:52.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.013+0000 7f6188c00640 1 -- 172.21.15.49:0/2015413182 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f617c04ec60 con 0x55e63f554f70 2024-02-26T06:24:52.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.013+0000 7f6188c00640 1 -- 172.21.15.49:0/2015413182 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c04e4b0 con 0x55e63f554f70 2024-02-26T06:24:52.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.014+0000 7f619dde4740 1 -- 172.21.15.49:0/2015413182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 msgr2=0x55e63f558f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.014+0000 7f619dde4740 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f558f20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f617c034c20 tx=0x7f617c036040 comp rx=0 tx=0).stop 2024-02-26T06:24:52.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.014+0000 7f619dde4740 1 -- 172.21.15.49:0/2015413182 shutdown_connections 2024-02-26T06:24:52.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.014+0000 7f619dde4740 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f55d600 0x55e63f559460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.016+0000 7f619dde4740 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554f70 0x55e63f558f20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.016+0000 7f619dde4740 1 --2- 172.21.15.49:0/2015413182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f554290 0x55e63f55aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.016+0000 7f619dde4740 1 -- 172.21.15.49:0/2015413182 >> 172.21.15.49:0/2015413182 conn(0x55e63f51b570 msgr2=0x55e63f54e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:52.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.017+0000 7f619dde4740 1 -- 172.21.15.49:0/2015413182 shutdown_connections 2024-02-26T06:24:52.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.017+0000 7f619dde4740 1 -- 172.21.15.49:0/2015413182 wait complete. 2024-02-26T06:24:52.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.018+0000 7f619dde4740 1 -- start start 2024-02-26T06:24:52.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.018+0000 7f619dde4740 1 Processor -- start 2024-02-26T06:24:52.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f619dde4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f5610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f619dde4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f554f70 0x55e63f561630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f619dde4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f55d600 0x55e63f561b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f619dde4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f5c3cf0 con 0x55e63f55d600 2024-02-26T06:24:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f619dde4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f5c3e60 con 0x55e63f554f70 2024-02-26T06:24:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f619dde4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63f5c4020 con 0x55e63f554290 2024-02-26T06:24:52.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f6193400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f5610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f6193400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f5610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46278/0 (socket says 172.21.15.49:46278) 2024-02-26T06:24:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.019+0000 7f6193400640 1 -- 172.21.15.49:0/2593290964 learned_addr learned my addr 172.21.15.49:0/2593290964 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.020+0000 7f6193e00640 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f554f70 0x55e63f561630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.020+0000 7f6193400640 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f55d600 0x55e63f561b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.020+0000 7f619dde4740 10 client.0 _ll_register_callbacks cb 0x7f617c013050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.021+0000 7f6193e00640 1 -- 172.21.15.49:0/2593290964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 msgr2=0x55e63f5610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:52.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.021+0000 7f6193e00640 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f5610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.021+0000 7f6193e00640 1 -- 172.21.15.49:0/2593290964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f55d600 msgr2=0x55e63f561b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:52.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.021+0000 7f6193e00640 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f55d600 0x55e63f561b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.021+0000 7f6193e00640 1 -- 172.21.15.49:0/2593290964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c040040 con 0x55e63f554f70 2024-02-26T06:24:52.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.021+0000 7f6193400640 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f55d600 0x55e63f561b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:52.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.022+0000 7f6193e00640 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f554f70 0x55e63f561630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55e63f55b830 tx=0x7f617c011a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6efcbb98690efdac server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:52.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.022+0000 7f6182000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:52.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.022+0000 7f6182000640 1 -- 172.21.15.49:0/2593290964 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c004310 con 0x55e63f554f70 2024-02-26T06:24:52.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.023+0000 7f6182000640 1 -- 172.21.15.49:0/2593290964 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f617c004470 con 0x55e63f554f70 2024-02-26T06:24:52.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.023+0000 7f619dde4740 1 -- 172.21.15.49:0/2593290964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61840150e0 con 0x55e63f554f70 2024-02-26T06:24:52.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.023+0000 7f6182000640 1 -- 172.21.15.49:0/2593290964 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f617c0110d0 con 0x55e63f554f70 2024-02-26T06:24:52.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.023+0000 7f619dde4740 10 client.25748 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:52.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.023+0000 7f619dde4740 1 -- 172.21.15.49:0/2593290964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e63f47c400 con 0x55e63f554f70 2024-02-26T06:24:52.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.023+0000 7f6182000640 1 -- 172.21.15.49:0/2593290964 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61840150e0 con 0x55e63f554f70 2024-02-26T06:24:52.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.024+0000 7f6182000640 1 -- 172.21.15.49:0/2593290964 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f617c05d020 con 0x55e63f554f70 2024-02-26T06:24:52.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.024+0000 7f619dde4740 10 client.25748 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:52.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.024+0000 7f619dde4740 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:52.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.024+0000 7f619dde4740 20 client.25748 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:52.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.024+0000 7f619dde4740 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e63f5c56e0 0x55e63f5c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.024+0000 7f619dde4740 4 client.25748 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:24:52.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.024+0000 7f619dde4740 1 -- 172.21.15.49:0/2593290964 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e63f247890 con 0x55e63f5c56e0 2024-02-26T06:24:52.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.025+0000 7f6193400640 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e63f5c56e0 0x55e63f5c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.025+0000 7f6193400640 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e63f5c56e0 0x55e63f5c7b00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf05dabb36d9a39 server_cookie=bb724dc82cdae2ec in_seq=0 out_seq=0 2024-02-26T06:24:52.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.025+0000 7f6182000640 10 client.25748 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:52.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.026+0000 7f6182000640 1 -- 172.21.15.49:0/2593290964 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e63f247890 con 0x55e63f5c56e0 2024-02-26T06:24:52.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.026+0000 7f6182000640 10 client.25748 handle_command_reply: tid=0 2024-02-26T06:24:52.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.026+0000 7f619dde4740 1 client.25748 shutdown 2024-02-26T06:24:52.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.026+0000 7f619dde4740 1 -- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f554f70 msgr2=0x55e63f561630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.026+0000 7f619dde4740 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f554f70 0x55e63f561630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55e63f55b830 tx=0x7f617c011a70 comp rx=0 tx=0).stop 2024-02-26T06:24:52.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f619dde4740 1 -- 172.21.15.49:0/2593290964 shutdown_connections 2024-02-26T06:24:52.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f619dde4740 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e63f5c56e0 0x55e63f5c7b00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f619dde4740 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e63f554f70 0x55e63f561630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f6182000640 0 client.25748 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:52.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f619dde4740 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e63f55d600 0x55e63f561b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f619dde4740 1 --2- 172.21.15.49:0/2593290964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e63f554290 0x55e63f5610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f619dde4740 1 -- 172.21.15.49:0/2593290964 >> 172.21.15.49:0/2593290964 conn(0x55e63f51b570 msgr2=0x55e63f54d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f619dde4740 1 -- 172.21.15.49:0/2593290964 shutdown_connections 2024-02-26T06:24:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.027+0000 7f619dde4740 1 -- 172.21.15.49:0/2593290964 wait complete. 2024-02-26T06:24:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.028+0000 7f619dde4740 20 client.25748 trim_cache size 0 max 16384 2024-02-26T06:24:52.029 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:24:52.029 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:24:52.029 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:24:52.029 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:24:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.028+0000 7f619b000640 1 -- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f61780a23d0 msgr2=0x7f61780a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.028+0000 7f619b000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f61780a23d0 0x7f61780a4890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f618c0cbca0 tx=0x7f618c0097a0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.029+0000 7f619b000640 1 -- 172.21.15.49:0/2517053280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6194199fa0 msgr2=0x7f619419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.029+0000 7f619b000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6194199fa0 0x7f619419a420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e970 tx=0x7f618c00ee30 comp rx=0 tx=0).stop 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.029+0000 7f619b000640 1 -- 172.21.15.49:0/2517053280 shutdown_connections 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.029+0000 7f619b000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f61780a23d0 0x7f61780a4890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.029+0000 7f619b000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6194108650 0x7f61941960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.029+0000 7f619b000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6194199fa0 0x7f619419a420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.029+0000 7f619b000640 1 --2- 172.21.15.49:0/2517053280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6194103580 0x7f6194195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.029+0000 7f619b000640 1 -- 172.21.15.49:0/2517053280 >> 172.21.15.49:0/2517053280 conn(0x7f61940fedc0 msgr2=0x7f6194106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.030+0000 7f619b000640 1 -- 172.21.15.49:0/2517053280 shutdown_connections 2024-02-26T06:24:52.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.030+0000 7f619b000640 1 -- 172.21.15.49:0/2517053280 wait complete. 2024-02-26T06:24:52.052 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:24:52.052 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:52.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.346+0000 7fb807e00640 1 -- 172.21.15.49:0/3914946502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb808108490 msgr2=0x7fb808110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.346+0000 7fb807e00640 1 --2- 172.21.15.49:0/3914946502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb808108490 0x7fb808110060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb800009ca0 tx=0x7fb80002c0f0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.346+0000 7fb807e00640 1 -- 172.21.15.49:0/3914946502 shutdown_connections 2024-02-26T06:24:52.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.346+0000 7fb807e00640 1 --2- 172.21.15.49:0/3914946502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb808103d70 0x7fb808107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.346+0000 7fb807e00640 1 --2- 172.21.15.49:0/3914946502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb808108490 0x7fb808110060 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.346+0000 7fb807e00640 1 --2- 172.21.15.49:0/3914946502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8081033c0 0x7fb8081037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.346+0000 7fb807e00640 1 -- 172.21.15.49:0/3914946502 >> 172.21.15.49:0/3914946502 conn(0x7fb8080fec20 msgr2=0x7fb808101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:52.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.347+0000 7fb807e00640 1 -- 172.21.15.49:0/3914946502 shutdown_connections 2024-02-26T06:24:52.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.347+0000 7fb807e00640 1 -- 172.21.15.49:0/3914946502 wait complete. 2024-02-26T06:24:52.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.347+0000 7fb807e00640 1 Processor -- start 2024-02-26T06:24:52.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb807e00640 1 -- start start 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb807e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8081033c0 0x7fb80810fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb807e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb808108490 0x7fb80810ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb807e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb80810d3b0 0x7fb80810d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb807e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081aa1f0 con 0x7fb80810d3b0 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb807e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806aba0 con 0x7fb808108490 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb807e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806ad60 con 0x7fb8081033c0 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb806a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8081033c0 0x7fb80810fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb806a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8081033c0 0x7fb80810fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46296/0 (socket says 172.21.15.49:46296) 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb806a00640 1 -- 172.21.15.49:0/3764994812 learned_addr learned my addr 172.21.15.49:0/3764994812 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb807400640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb80810d3b0 0x7fb80810d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.348+0000 7fb806000640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb808108490 0x7fb80810ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.349+0000 7fb806a00640 1 -- 172.21.15.49:0/3764994812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb808108490 msgr2=0x7fb80810ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.349+0000 7fb806a00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb808108490 0x7fb80810ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.349+0000 7fb806a00640 1 -- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb80810d3b0 msgr2=0x7fb80810d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.349+0000 7fb806a00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb80810d3b0 0x7fb80810d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.349+0000 7fb806a00640 1 -- 172.21.15.49:0/3764994812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb800009900 con 0x7fb8081033c0 2024-02-26T06:24:52.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.349+0000 7fb806a00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8081033c0 0x7fb80810fd80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800b4d0 tx=0x7fb7f800b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:52.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.350+0000 7fb7ff400640 1 -- 172.21.15.49:0/3764994812 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8010870 con 0x7fb8081033c0 2024-02-26T06:24:52.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.350+0000 7fb7ff400640 1 -- 172.21.15.49:0/3764994812 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7f800f3c0 con 0x7fb8081033c0 2024-02-26T06:24:52.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.350+0000 7fb7ff400640 1 -- 172.21.15.49:0/3764994812 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80175c0 con 0x7fb8081033c0 2024-02-26T06:24:52.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.350+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80806b040 con 0x7fb8081033c0 2024-02-26T06:24:52.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.350+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80806b5a0 con 0x7fb8081033c0 2024-02-26T06:24:52.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.352+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb808104890 con 0x7fb8081033c0 2024-02-26T06:24:52.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.356+0000 7fb7ff400640 1 -- 172.21.15.49:0/3764994812 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fb7f800f8a0 con 0x7fb8081033c0 2024-02-26T06:24:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.358+0000 7fb7ff400640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7ec0a23d0 0x7fb7ec0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.358+0000 7fb7ff400640 1 -- 172.21.15.49:0/3764994812 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb7f80cdb70 con 0x7fb8081033c0 2024-02-26T06:24:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.359+0000 7fb806000640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7ec0a23d0 0x7fb7ec0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.360+0000 7fb806000640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7ec0a23d0 0x7fb7ec0a4890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb80810e5e0 tx=0x7fb7f400b410 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.362+0000 7fb7ff400640 1 -- 172.21.15.49:0/3764994812 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb7f8096720 con 0x7fb8081033c0 2024-02-26T06:24:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.784+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8080682e0 con 0x7fb8081033c0 2024-02-26T06:24:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.785+0000 7fb7ff400640 1 -- 172.21.15.49:0/3764994812 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11338 (secure 0 0 0) 0x7fb7f800f530 con 0x7fb8081033c0 2024-02-26T06:24:52.785 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:24:52.786 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:24:43.229679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:24:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 29 2024-02-26T06:24:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.788+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7ec0a23d0 msgr2=0x7fb7ec0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.788+0000 7fb807e00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7ec0a23d0 0x7fb7ec0a4890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb80810e5e0 tx=0x7fb7f400b410 comp rx=0 tx=0).stop 2024-02-26T06:24:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.788+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8081033c0 msgr2=0x7fb80810fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.788+0000 7fb807e00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8081033c0 0x7fb80810fd80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800b4d0 tx=0x7fb7f800b990 comp rx=0 tx=0).stop 2024-02-26T06:24:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.789+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 shutdown_connections 2024-02-26T06:24:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.789+0000 7fb807e00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7ec0a23d0 0x7fb7ec0a4890 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.789+0000 7fb807e00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb808108490 0x7fb80810ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.789+0000 7fb807e00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb80810d3b0 0x7fb80810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.789+0000 7fb807e00640 1 --2- 172.21.15.49:0/3764994812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8081033c0 0x7fb80810fd80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.789+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 >> 172.21.15.49:0/3764994812 conn(0x7fb8080fec20 msgr2=0x7fb808100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.789+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 shutdown_connections 2024-02-26T06:24:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:52.789+0000 7fb807e00640 1 -- 172.21.15.49:0/3764994812 wait complete. 2024-02-26T06:24:52.814 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:52 smithi049 ceph-mon[31696]: pgmap v47: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-02-26T06:24:52.819 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:24:52.819 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:24:52.829 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:52 smithi080 ceph-mon[36206]: pgmap v47: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-02-26T06:24:52.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:52 smithi115 ceph-mon[36426]: pgmap v47: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-02-26T06:24:53.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.108+0000 7f85d5200640 1 -- 172.21.15.49:0/3805085388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 msgr2=0x7f85d010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.108+0000 7f85d5200640 1 --2- 172.21.15.49:0/3805085388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 0x7f85d010e020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009930 tx=0x7f85c802bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.108+0000 7f85d5200640 1 -- 172.21.15.49:0/3805085388 shutdown_connections 2024-02-26T06:24:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.108+0000 7f85d5200640 1 --2- 172.21.15.49:0/3805085388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 0x7f85d010e020 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.108+0000 7f85d5200640 1 --2- 172.21.15.49:0/3805085388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85d0101d30 0x7f85d0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.108+0000 7f85d5200640 1 --2- 172.21.15.49:0/3805085388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d0101380 0x7f85d0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.108+0000 7f85d5200640 1 -- 172.21.15.49:0/3805085388 >> 172.21.15.49:0/3805085388 conn(0x7f85d0078580 msgr2=0x7f85d0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.109+0000 7f85d5200640 1 -- 172.21.15.49:0/3805085388 shutdown_connections 2024-02-26T06:24:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.109+0000 7f85d5200640 1 -- 172.21.15.49:0/3805085388 wait complete. 2024-02-26T06:24:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.110+0000 7f85d5200640 1 Processor -- start 2024-02-26T06:24:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.110+0000 7f85d5200640 1 -- start start 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85d5200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d0101380 0x7f85d010d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85d5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 0x7f85d010d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85d5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85d01a26c0 0x7f85d01a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85d5200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d01a30b0 con 0x7f85d0101380 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85d5200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d01a3220 con 0x7f85d0106450 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85d5200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d01a3390 con 0x7f85d01a26c0 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85c5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 0x7f85d010d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85c5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 0x7f85d010d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46230/0 (socket says 172.21.15.49:46230) 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85c5400640 1 -- 172.21.15.49:0/2471975054 learned_addr learned my addr 172.21.15.49:0/2471975054 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85ce800640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85d01a26c0 0x7f85d01a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85cde00640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d0101380 0x7f85d010d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85c5400640 1 -- 172.21.15.49:0/2471975054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85d01a26c0 msgr2=0x7f85d01a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85c5400640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85d01a26c0 0x7f85d01a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85c5400640 1 -- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d0101380 msgr2=0x7f85d010d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85c5400640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d0101380 0x7f85d010d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.111+0000 7f85c5400640 1 -- 172.21.15.49:0/2471975054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8009560 con 0x7f85d0106450 2024-02-26T06:24:53.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.112+0000 7f85c5400640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 0x7f85d010d7c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f85b800e9c0 tx=0x7f85b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.112+0000 7f85c7400640 1 -- 172.21.15.49:0/2471975054 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8009700 con 0x7f85d0106450 2024-02-26T06:24:53.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.112+0000 7f85d5200640 1 -- 172.21.15.49:0/2471975054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d01aa970 con 0x7f85d0106450 2024-02-26T06:24:53.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.112+0000 7f85d5200640 1 -- 172.21.15.49:0/2471975054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d01aae70 con 0x7f85d0106450 2024-02-26T06:24:53.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.113+0000 7f85c7400640 1 -- 172.21.15.49:0/2471975054 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85b8005500 con 0x7f85d0106450 2024-02-26T06:24:53.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.113+0000 7f85c7400640 1 -- 172.21.15.49:0/2471975054 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8010430 con 0x7f85d0106450 2024-02-26T06:24:53.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.117+0000 7f85d804d740 1 Processor -- start 2024-02-26T06:24:53.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.117+0000 7f85d804d740 1 -- start start 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.117+0000 7f85d804d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x5618499ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.117+0000 7f85d804d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a08bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.118+0000 7f85d804d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 0x561849a0b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.118+0000 7f85d804d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184996e1e0 con 0x561849a09110 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.118+0000 7f85d804d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184996b710 con 0x561849a00530 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.118+0000 7f85d804d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561849966ef0 con 0x5618499ffc10 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.118+0000 7f85cde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a08bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.118+0000 7f85cde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a08bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46232/0 (socket says 172.21.15.49:46232) 2024-02-26T06:24:53.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.118+0000 7f85cde00640 1 -- 172.21.15.49:0/1284820971 learned_addr learned my addr 172.21.15.49:0/1284820971 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.118+0000 7f85ce800640 1 --2- 172.21.15.49:0/1284820971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x5618499ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.119+0000 7f85cde00640 1 -- 172.21.15.49:0/1284820971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 msgr2=0x5618499ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.119+0000 7f85cde00640 1 --2- 172.21.15.49:0/1284820971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x5618499ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.119+0000 7f85cde00640 1 -- 172.21.15.49:0/1284820971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 msgr2=0x561849a0b500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:53.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.119+0000 7f85cde00640 1 --2- 172.21.15.49:0/1284820971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 0x561849a0b500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.119+0000 7f85cde00640 1 -- 172.21.15.49:0/1284820971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561849928dc0 con 0x561849a00530 2024-02-26T06:24:53.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.119+0000 7f85c7400640 1 -- 172.21.15.49:0/2471975054 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f85b8010590 con 0x7f85d0106450 2024-02-26T06:24:53.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.119+0000 7f85cde00640 1 --2- 172.21.15.49:0/1284820971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a08bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f85c0010400 tx=0x7f85c00108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306193736aa96ed0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.119+0000 7f85bf400640 1 -- 172.21.15.49:0/1284820971 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0003b70 con 0x561849a00530 2024-02-26T06:24:53.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.120+0000 7f85bf400640 1 -- 172.21.15.49:0/1284820971 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85c0015070 con 0x561849a00530 2024-02-26T06:24:53.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.120+0000 7f85bf400640 1 -- 172.21.15.49:0/1284820971 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0011920 con 0x561849a00530 2024-02-26T06:24:53.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.121+0000 7f85d804d740 1 -- 172.21.15.49:0/1284820971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 msgr2=0x561849a08bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.121+0000 7f85d804d740 1 --2- 172.21.15.49:0/1284820971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a08bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f85c0010400 tx=0x7f85c00108c0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.121+0000 7f85d804d740 1 -- 172.21.15.49:0/1284820971 shutdown_connections 2024-02-26T06:24:53.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.121+0000 7f85d804d740 1 --2- 172.21.15.49:0/1284820971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a08bd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.121+0000 7f85d804d740 1 --2- 172.21.15.49:0/1284820971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 0x561849a0b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.121+0000 7f85d804d740 1 --2- 172.21.15.49:0/1284820971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x5618499ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.121+0000 7f85d804d740 1 -- 172.21.15.49:0/1284820971 >> 172.21.15.49:0/1284820971 conn(0x5618499fa340 msgr2=0x5618499fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.122+0000 7f85c7400640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f85b40a2420 0x7f85b40a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.122+0000 7f85c7400640 1 -- 172.21.15.49:0/2471975054 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f85b8014070 con 0x7f85d0106450 2024-02-26T06:24:53.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.122+0000 7f85cde00640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f85b40a2420 0x7f85b40a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.122+0000 7f85d804d740 1 -- 172.21.15.49:0/1284820971 shutdown_connections 2024-02-26T06:24:53.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.123+0000 7f85d804d740 1 -- 172.21.15.49:0/1284820971 wait complete. 2024-02-26T06:24:53.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.123+0000 7f85d804d740 1 -- start start 2024-02-26T06:24:53.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.124+0000 7f85cde00640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f85b40a2420 0x7f85b40a48e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85c00049b0 tx=0x7f85c0004910 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.124+0000 7f85d804d740 1 Processor -- start 2024-02-26T06:24:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.124+0000 7f85d804d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618499ffc10 0x561849a06110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.124+0000 7f85d804d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a00530 0x561849a06650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85d804d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a06b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85d804d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184990c120 con 0x561849a00530 2024-02-26T06:24:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85d804d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184990bdc0 con 0x5618499ffc10 2024-02-26T06:24:53.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85d804d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618496d51c0 con 0x561849a09110 2024-02-26T06:24:53.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85d804d740 10 client.0 _ll_register_callbacks cb 0x56184995b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:53.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85ce800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a06b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85cde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618499ffc10 0x561849a06110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85ce800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a06b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46334/0 (socket says 172.21.15.49:46334) 2024-02-26T06:24:53.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85ce800640 1 -- 172.21.15.49:0/3196689355 learned_addr learned my addr 172.21.15.49:0/3196689355 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.125+0000 7f85c5400640 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a00530 0x561849a06650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.126+0000 7f85ce800640 1 -- 172.21.15.49:0/3196689355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618499ffc10 msgr2=0x561849a06110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.126+0000 7f85ce800640 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618499ffc10 0x561849a06110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.126+0000 7f85ce800640 1 -- 172.21.15.49:0/3196689355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a00530 msgr2=0x561849a06650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.126+0000 7f85ce800640 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a00530 0x561849a06650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.126+0000 7f85ce800640 1 -- 172.21.15.49:0/3196689355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c0009c50 con 0x561849a09110 2024-02-26T06:24:53.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.126+0000 7f85ce800640 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a06b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005ab0 tx=0x7f85c802ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af72642840542cb7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.126+0000 7f85b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.127+0000 7f85b3e00640 1 -- 172.21.15.49:0/3196689355 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c802cc40 con 0x561849a09110 2024-02-26T06:24:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.127+0000 7f85b3e00640 1 -- 172.21.15.49:0/3196689355 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f85c8046070 con 0x561849a09110 2024-02-26T06:24:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.127+0000 7f85d804d740 1 -- 172.21.15.49:0/3196689355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618496667e0 con 0x561849a09110 2024-02-26T06:24:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.127+0000 7f85b3e00640 1 -- 172.21.15.49:0/3196689355 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85c8004250 con 0x561849a09110 2024-02-26T06:24:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.127+0000 7f85d804d740 10 client.25810 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.127+0000 7f85d804d740 1 -- 172.21.15.49:0/3196689355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561849965570 con 0x561849a09110 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.128+0000 7f85b3e00640 1 -- 172.21.15.49:0/3196689355 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5618496667e0 con 0x561849a09110 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.128+0000 7f85b3e00640 1 -- 172.21.15.49:0/3196689355 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f85c803c040 con 0x561849a09110 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.128+0000 7f85d804d740 10 client.25810 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.128+0000 7f85d804d740 10 client.25810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.128+0000 7f85d804d740 20 client.25810 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.128+0000 7f85d804d740 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x561849a29fd0 0x561849a2c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.128+0000 7f85d804d740 4 client.25810 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.128+0000 7f85d804d740 1 -- 172.21.15.49:0/3196689355 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618496a3720 con 0x561849a29fd0 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.129+0000 7f85cde00640 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x561849a29fd0 0x561849a2c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.129+0000 7f85b3e00640 10 client.25810 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:24:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.129+0000 7f85cde00640 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x561849a29fd0 0x561849a2c3f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90ac35fe716b3702 server_cookie=4149fc69bb8e3c90 in_seq=0 out_seq=0 2024-02-26T06:24:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.130+0000 7f85b3e00640 1 -- 172.21.15.49:0/3196689355 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5618496a3720 con 0x561849a29fd0 2024-02-26T06:24:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.130+0000 7f85b3e00640 10 client.25810 handle_command_reply: tid=0 2024-02-26T06:24:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.130+0000 7f85d804d740 1 client.25810 shutdown 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.130+0000 7f85d804d740 1 -- 172.21.15.49:0/3196689355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 msgr2=0x561849a06b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.130+0000 7f85d804d740 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a06b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005ab0 tx=0x7f85c802ca10 comp rx=0 tx=0).stop 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 1 -- 172.21.15.49:0/3196689355 shutdown_connections 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x561849a29fd0 0x561849a2c3f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a06b90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a00530 0x561849a06650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 1 --2- 172.21.15.49:0/3196689355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5618499ffc10 0x561849a06110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 1 -- 172.21.15.49:0/3196689355 >> 172.21.15.49:0/3196689355 conn(0x5618499fa340 msgr2=0x5618499fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85b3e00640 0 client.25810 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:24:53.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 1 -- 172.21.15.49:0/3196689355 shutdown_connections 2024-02-26T06:24:53.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 1 -- 172.21.15.49:0/3196689355 wait complete. 2024-02-26T06:24:53.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.131+0000 7f85d804d740 20 client.25810 trim_cache size 0 max 16384 2024-02-26T06:24:53.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.157+0000 7f85d804d740 1 Processor -- start 2024-02-26T06:24:53.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.158+0000 7f85d804d740 1 -- start start 2024-02-26T06:24:53.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.159+0000 7f85d804d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x561849a08840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.159+0000 7f85d804d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a01430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.160+0000 7f85d804d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 0x561849a01970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.160+0000 7f85cde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x561849a08840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.161+0000 7f85cde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x561849a08840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46340/0 (socket says 172.21.15.49:46340) 2024-02-26T06:24:53.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.161+0000 7f85ce800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 0x561849a01970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.161+0000 7f85d804d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561849a1ce20 con 0x561849a09110 2024-02-26T06:24:53.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.161+0000 7f85d804d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618499f5cb0 con 0x561849a00530 2024-02-26T06:24:53.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.161+0000 7f85d804d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561849a31600 con 0x5618499ffc10 2024-02-26T06:24:53.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.161+0000 7f85cde00640 1 -- 172.21.15.49:0/3581572133 learned_addr learned my addr 172.21.15.49:0/3581572133 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.162+0000 7f85cde00640 1 -- 172.21.15.49:0/3581572133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 msgr2=0x561849a01430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.162+0000 7f85cde00640 1 --2- 172.21.15.49:0/3581572133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a01430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.162+0000 7f85cde00640 1 -- 172.21.15.49:0/3581572133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 msgr2=0x561849a01970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.162+0000 7f85cde00640 1 --2- 172.21.15.49:0/3581572133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 0x561849a01970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.162+0000 7f85cde00640 1 -- 172.21.15.49:0/3581572133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561849901a80 con 0x5618499ffc10 2024-02-26T06:24:53.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.162+0000 7f85ce800640 1 --2- 172.21.15.49:0/3581572133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 0x561849a01970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:53.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.163+0000 7f85cde00640 1 --2- 172.21.15.49:0/3581572133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x561849a08840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f85c0010400 tx=0x7f85c0010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4325c2503b5a6699 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.163+0000 7f85bea00640 1 -- 172.21.15.49:0/3581572133 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0003a50 con 0x5618499ffc10 2024-02-26T06:24:53.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.163+0000 7f85bea00640 1 -- 172.21.15.49:0/3581572133 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85c0020040 con 0x5618499ffc10 2024-02-26T06:24:53.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.163+0000 7f85bea00640 1 -- 172.21.15.49:0/3581572133 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0036580 con 0x5618499ffc10 2024-02-26T06:24:53.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.164+0000 7f85d804d740 1 -- 172.21.15.49:0/3581572133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 msgr2=0x561849a08840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.164+0000 7f85d804d740 1 --2- 172.21.15.49:0/3581572133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x561849a08840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f85c0010400 tx=0x7f85c0010fd0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.165+0000 7f85d804d740 1 -- 172.21.15.49:0/3581572133 shutdown_connections 2024-02-26T06:24:53.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.165+0000 7f85d804d740 1 --2- 172.21.15.49:0/3581572133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a01430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.165+0000 7f85d804d740 1 --2- 172.21.15.49:0/3581572133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561849a09110 0x561849a01970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.165+0000 7f85d804d740 1 --2- 172.21.15.49:0/3581572133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618499ffc10 0x561849a08840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.165+0000 7f85d804d740 1 -- 172.21.15.49:0/3581572133 >> 172.21.15.49:0/3581572133 conn(0x5618499fa340 msgr2=0x561849a00910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.165+0000 7f85d804d740 1 -- 172.21.15.49:0/3581572133 shutdown_connections 2024-02-26T06:24:53.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.165+0000 7f85d804d740 1 -- 172.21.15.49:0/3581572133 wait complete. 2024-02-26T06:24:53.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.166+0000 7f85d804d740 1 -- start start 2024-02-26T06:24:53.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.166+0000 7f85d804d740 1 Processor -- start 2024-02-26T06:24:53.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.167+0000 7f85d804d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5618499ffc10 0x561849a79990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.167+0000 7f85d804d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.167+0000 7f85d804d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.167+0000 7f85d804d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561849a20be0 con 0x5618499ffc10 2024-02-26T06:24:53.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.167+0000 7f85d804d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561849a20d50 con 0x561849a00530 2024-02-26T06:24:53.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.167+0000 7f85d804d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561849a20f10 con 0x561849a09110 2024-02-26T06:24:53.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.168+0000 7f85c5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.168+0000 7f85c5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a79ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46264/0 (socket says 172.21.15.49:46264) 2024-02-26T06:24:53.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.168+0000 7f85c5400640 1 -- 172.21.15.49:0/3072044600 learned_addr learned my addr 172.21.15.49:0/3072044600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.169+0000 7f85ce800640 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.169+0000 7f85d804d740 10 client.0 _ll_register_callbacks cb 0x7f85c0009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:53.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.169+0000 7f85cde00640 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5618499ffc10 0x561849a79990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.169+0000 7f85ce800640 1 -- 172.21.15.49:0/3072044600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 msgr2=0x561849a79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.169+0000 7f85ce800640 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a79ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.169+0000 7f85ce800640 1 -- 172.21.15.49:0/3072044600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5618499ffc10 msgr2=0x561849a79990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.169+0000 7f85ce800640 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5618499ffc10 0x561849a79990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.169+0000 7f85ce800640 1 -- 172.21.15.49:0/3072044600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561849965570 con 0x561849a09110 2024-02-26T06:24:53.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.170+0000 7f85ce800640 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a204e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005a80 tx=0x7f85c8004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62765d4787f43a69 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.170+0000 7f85b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:53.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.171+0000 7f85b3e00640 1 -- 172.21.15.49:0/3072044600 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8005200 con 0x561849a09110 2024-02-26T06:24:53.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.172+0000 7f85b3e00640 1 -- 172.21.15.49:0/3072044600 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f85c8046040 con 0x561849a09110 2024-02-26T06:24:53.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.172+0000 7f85d804d740 1 -- 172.21.15.49:0/3072044600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561849a05f40 con 0x561849a09110 2024-02-26T06:24:53.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.172+0000 7f85b3e00640 1 -- 172.21.15.49:0/3072044600 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85c8004b10 con 0x561849a09110 2024-02-26T06:24:53.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.173+0000 7f85d804d740 10 client.25816 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:53.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.173+0000 7f85d804d740 1 -- 172.21.15.49:0/3072044600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f85c002e040 con 0x561849a09110 2024-02-26T06:24:53.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.174+0000 7f85b3e00640 1 -- 172.21.15.49:0/3072044600 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561849a05f40 con 0x561849a09110 2024-02-26T06:24:53.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.175+0000 7f85b3e00640 1 -- 172.21.15.49:0/3072044600 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f85c803c040 con 0x561849a09110 2024-02-26T06:24:53.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.175+0000 7f85d804d740 10 client.25816 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:53.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.175+0000 7f85d804d740 10 client.25816 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:24:53.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.175+0000 7f85d804d740 20 client.25816 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:53.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.175+0000 7f85d804d740 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x561849a826c0 0x561849a84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.175+0000 7f85d804d740 4 client.25816 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:24:53.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.175+0000 7f85d804d740 1 -- 172.21.15.49:0/3072044600 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f85c0015070 con 0x561849a826c0 2024-02-26T06:24:53.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.178+0000 7f85cde00640 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x561849a826c0 0x561849a84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.179+0000 7f85cde00640 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x561849a826c0 0x561849a84ae0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4eb7be32c79aef9 server_cookie=8c5f3118c9a6161c in_seq=0 out_seq=0 2024-02-26T06:24:53.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85b3e00640 10 client.25816 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:24:53.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85b3e00640 1 -- 172.21.15.49:0/3072044600 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f85c0015070 con 0x561849a826c0 2024-02-26T06:24:53.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85b3e00640 10 client.25816 handle_command_reply: tid=0 2024-02-26T06:24:53.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 client.25816 shutdown 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 -- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 msgr2=0x561849a204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a204e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005a80 tx=0x7f85c8004480 comp rx=0 tx=0).stop 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 -- 172.21.15.49:0/3072044600 shutdown_connections 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x561849a826c0 0x561849a84ae0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561849a00530 0x561849a79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561849a09110 0x561849a204e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 --2- 172.21.15.49:0/3072044600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5618499ffc10 0x561849a79990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.180+0000 7f85d804d740 1 -- 172.21.15.49:0/3072044600 >> 172.21.15.49:0/3072044600 conn(0x5618499fa340 msgr2=0x561849a01430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.181+0000 7f85b3e00640 0 client.25816 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:24:53.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.181+0000 7f85d804d740 1 -- 172.21.15.49:0/3072044600 shutdown_connections 2024-02-26T06:24:53.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.182+0000 7f85d804d740 1 -- 172.21.15.49:0/3072044600 wait complete. 2024-02-26T06:24:53.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.182+0000 7f85d804d740 20 client.25816 trim_cache size 0 max 16384 2024-02-26T06:24:53.183 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:24:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.185+0000 7f85d5200640 1 -- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f85b40a2420 msgr2=0x7f85b40a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.185+0000 7f85d5200640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f85b40a2420 0x7f85b40a48e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85c00049b0 tx=0x7f85c0004910 comp rx=0 tx=0).stop 2024-02-26T06:24:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.185+0000 7f85d5200640 1 -- 172.21.15.49:0/2471975054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 msgr2=0x7f85d010d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.185+0000 7f85d5200640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 0x7f85d010d7c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f85b800e9c0 tx=0x7f85b800ee80 comp rx=0 tx=0).stop 2024-02-26T06:24:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.186+0000 7f85d5200640 1 -- 172.21.15.49:0/2471975054 shutdown_connections 2024-02-26T06:24:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.186+0000 7f85d5200640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f85b40a2420 0x7f85b40a48e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.186+0000 7f85d5200640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85d0106450 0x7f85d010d7c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.186+0000 7f85d5200640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85d01a26c0 0x7f85d01a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.186+0000 7f85d5200640 1 --2- 172.21.15.49:0/2471975054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d0101380 0x7f85d010d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.186+0000 7f85d5200640 1 -- 172.21.15.49:0/2471975054 >> 172.21.15.49:0/2471975054 conn(0x7f85d0078580 msgr2=0x7f85d00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.186+0000 7f85d5200640 1 -- 172.21.15.49:0/2471975054 shutdown_connections 2024-02-26T06:24:53.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.186+0000 7f85d5200640 1 -- 172.21.15.49:0/2471975054 wait complete. 2024-02-26T06:24:53.224 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:24:53.224 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:24:53.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 -- 172.21.15.49:0/3433715500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 msgr2=0x7f496c100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 --2- 172.21.15.49:0/3433715500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 0x7f496c100530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f495c009900 tx=0x7f495c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:53.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 -- 172.21.15.49:0/3433715500 shutdown_connections 2024-02-26T06:24:53.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 --2- 172.21.15.49:0/3433715500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f496c105220 0x7f496c0682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 --2- 172.21.15.49:0/3433715500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f496c100b00 0x7f496c104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 --2- 172.21.15.49:0/3433715500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 0x7f496c100530 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 -- 172.21.15.49:0/3433715500 >> 172.21.15.49:0/3433715500 conn(0x7f496c0fb9d0 msgr2=0x7f496c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 -- 172.21.15.49:0/3433715500 shutdown_connections 2024-02-26T06:24:53.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.512+0000 7f4972000640 1 -- 172.21.15.49:0/3433715500 wait complete. 2024-02-26T06:24:53.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.514+0000 7f4972000640 1 Processor -- start 2024-02-26T06:24:53.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.515+0000 7f4972000640 1 -- start start 2024-02-26T06:24:53.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.515+0000 7f4972000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 0x7f496c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.515+0000 7f4972000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f496c105220 0x7f496c19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.515+0000 7f4972000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f496c1996b0 0x7f496c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.515+0000 7f4972000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c19a180 con 0x7f496c1996b0 2024-02-26T06:24:53.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.515+0000 7f4972000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c19a2f0 con 0x7f496c105220 2024-02-26T06:24:53.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.515+0000 7f4972000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c19a4b0 con 0x7f496c100150 2024-02-26T06:24:53.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.516+0000 7f4971600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f496c1996b0 0x7f496c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.516+0000 7f4970c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 0x7f496c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.516+0000 7f4970c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 0x7f496c19f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46368/0 (socket says 172.21.15.49:46368) 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.516+0000 7f4970c00640 1 -- 172.21.15.49:0/3579152946 learned_addr learned my addr 172.21.15.49:0/3579152946 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.516+0000 7f496be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f496c105220 0x7f496c19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.517+0000 7f4970c00640 1 -- 172.21.15.49:0/3579152946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f496c105220 msgr2=0x7f496c19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.517+0000 7f4970c00640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f496c105220 0x7f496c19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.517+0000 7f4970c00640 1 -- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f496c1996b0 msgr2=0x7f496c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.517+0000 7f4970c00640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f496c1996b0 0x7f496c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.517+0000 7f4970c00640 1 -- 172.21.15.49:0/3579152946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c009560 con 0x7f496c100150 2024-02-26T06:24:53.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.517+0000 7f4970c00640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 0x7f496c19f5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f495c005940 tx=0x7f495c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.517+0000 7f4969600640 1 -- 172.21.15.49:0/3579152946 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c004440 con 0x7f496c100150 2024-02-26T06:24:53.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.518+0000 7f4969600640 1 -- 172.21.15.49:0/3579152946 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f495c0052d0 con 0x7f496c100150 2024-02-26T06:24:53.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.518+0000 7f4969600640 1 -- 172.21.15.49:0/3579152946 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c003cb0 con 0x7f496c100150 2024-02-26T06:24:53.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.518+0000 7f4972000640 1 -- 172.21.15.49:0/3579152946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496c1a63a0 con 0x7f496c100150 2024-02-26T06:24:53.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.518+0000 7f4972000640 1 -- 172.21.15.49:0/3579152946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c1a6770 con 0x7f496c100150 2024-02-26T06:24:53.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.527+0000 7f4969600640 1 -- 172.21.15.49:0/3579152946 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f495c004df0 con 0x7f496c100150 2024-02-26T06:24:53.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.528+0000 7f49765a7740 1 Processor -- start 2024-02-26T06:24:53.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.528+0000 7f49765a7740 1 -- start start 2024-02-26T06:24:53.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f49765a7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b32fdc0 0x56481b3301a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f49765a7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b338d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f49765a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3392c0 0x56481b33b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f49765a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b29e1e0 con 0x56481b3392c0 2024-02-26T06:24:53.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f49765a7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b29b710 con 0x56481b32fdc0 2024-02-26T06:24:53.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f49765a7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b296ef0 con 0x56481b3306e0 2024-02-26T06:24:53.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f4970c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b338d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f4971600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b32fdc0 0x56481b3301a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f4971600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b32fdc0 0x56481b3301a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46298/0 (socket says 172.21.15.49:46298) 2024-02-26T06:24:53.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.529+0000 7f4971600640 1 -- 172.21.15.49:0/3990428441 learned_addr learned my addr 172.21.15.49:0/3990428441 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.530+0000 7f4971600640 1 -- 172.21.15.49:0/3990428441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 msgr2=0x56481b338d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.530+0000 7f4971600640 1 --2- 172.21.15.49:0/3990428441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b338d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.530+0000 7f4971600640 1 -- 172.21.15.49:0/3990428441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3392c0 msgr2=0x56481b33b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.530+0000 7f4971600640 1 --2- 172.21.15.49:0/3990428441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3392c0 0x56481b33b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.530+0000 7f4971600640 1 -- 172.21.15.49:0/3990428441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56481b258dc0 con 0x56481b32fdc0 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.531+0000 7f4971600640 1 --2- 172.21.15.49:0/3990428441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b32fdc0 0x56481b3301a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4964009e60 tx=0x7f4964010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=739ab737967c1b5e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.531+0000 7f4969600640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49500a23d0 0x7f49500a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.531+0000 7f4961c00640 1 -- 172.21.15.49:0/3990428441 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964004d20 con 0x56481b32fdc0 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.531+0000 7f4961c00640 1 -- 172.21.15.49:0/3990428441 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f496401d070 con 0x56481b32fdc0 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.531+0000 7f4961c00640 1 -- 172.21.15.49:0/3990428441 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964022800 con 0x56481b32fdc0 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.531+0000 7f4969600640 1 -- 172.21.15.49:0/3579152946 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f495c0f0410 con 0x7f496c100150 2024-02-26T06:24:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.531+0000 7f4970c00640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49500a23d0 0x7f49500a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.532+0000 7f49765a7740 1 -- 172.21.15.49:0/3990428441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b32fdc0 msgr2=0x56481b3301a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.532+0000 7f49765a7740 1 --2- 172.21.15.49:0/3990428441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b32fdc0 0x56481b3301a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4964009e60 tx=0x7f4964010740 comp rx=0 tx=0).stop 2024-02-26T06:24:53.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.533+0000 7f49765a7740 1 -- 172.21.15.49:0/3990428441 shutdown_connections 2024-02-26T06:24:53.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.533+0000 7f49765a7740 1 --2- 172.21.15.49:0/3990428441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3392c0 0x56481b33b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.533+0000 7f49765a7740 1 --2- 172.21.15.49:0/3990428441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b338d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.533+0000 7f49765a7740 1 --2- 172.21.15.49:0/3990428441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b32fdc0 0x56481b3301a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.533+0000 7f49765a7740 1 -- 172.21.15.49:0/3990428441 >> 172.21.15.49:0/3990428441 conn(0x56481b32a4f0 msgr2=0x56481b32a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.534+0000 7f4970c00640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49500a23d0 0x7f49500a4890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f495c0edb60 tx=0x7f495c0f28e0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.534+0000 7f49765a7740 1 -- 172.21.15.49:0/3990428441 shutdown_connections 2024-02-26T06:24:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.535+0000 7f49765a7740 1 -- 172.21.15.49:0/3990428441 wait complete. 2024-02-26T06:24:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.535+0000 7f49765a7740 1 -- start start 2024-02-26T06:24:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.536+0000 7f49765a7740 1 Processor -- start 2024-02-26T06:24:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.537+0000 7f49765a7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b336300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.537+0000 7f49765a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 0x56481b336840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.537+0000 7f49765a7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b336d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.537+0000 7f49765a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b23c120 con 0x56481b3306e0 2024-02-26T06:24:53.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.537+0000 7f49765a7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b23bdc0 con 0x56481b3392c0 2024-02-26T06:24:53.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.537+0000 7f49765a7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b0051c0 con 0x56481b32fdc0 2024-02-26T06:24:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.537+0000 7f49765a7740 10 client.0 _ll_register_callbacks cb 0x56481b28b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.538+0000 7f4971600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b336300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f4971600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b336300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46398/0 (socket says 172.21.15.49:46398) 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f4971600640 1 -- 172.21.15.49:0/3417104277 learned_addr learned my addr 172.21.15.49:0/3417104277 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f4971600640 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b336d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f4971600640 1 -- 172.21.15.49:0/3417104277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 msgr2=0x56481b336d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f4971600640 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b336d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f4971600640 1 -- 172.21.15.49:0/3417104277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 msgr2=0x56481b336840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f496be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 0x56481b336840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f4971600640 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 0x56481b336840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f4971600640 1 -- 172.21.15.49:0/3417104277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4964009ac0 con 0x56481b32fdc0 2024-02-26T06:24:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.539+0000 7f496be00640 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 0x56481b336840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.540+0000 7f4971600640 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b336300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4964011010 tx=0x7f4964009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da0769b8d1531ac server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.540+0000 7f4956000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.540+0000 7f4956000640 1 -- 172.21.15.49:0/3417104277 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964005430 con 0x56481b32fdc0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.540+0000 7f4956000640 1 -- 172.21.15.49:0/3417104277 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4964005590 con 0x56481b32fdc0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.540+0000 7f49765a7740 1 -- 172.21.15.49:0/3417104277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56481af967e0 con 0x56481b32fdc0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.540+0000 7f4956000640 1 -- 172.21.15.49:0/3417104277 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4964022580 con 0x56481b32fdc0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f49765a7740 10 client.25828 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f49765a7740 1 -- 172.21.15.49:0/3417104277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56481b295570 con 0x56481b32fdc0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f4956000640 1 -- 172.21.15.49:0/3417104277 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56481af967e0 con 0x56481b32fdc0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f4956000640 1 -- 172.21.15.49:0/3417104277 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f49640127b0 con 0x56481b32fdc0 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f49765a7740 10 client.25828 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f49765a7740 10 client.25828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f49765a7740 20 client.25828 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:53.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f49765a7740 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56481b349320 0x56481b34b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f49765a7740 4 client.25828 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:53.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.541+0000 7f49765a7740 1 -- 172.21.15.49:0/3417104277 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56481afd3720 con 0x56481b349320 2024-02-26T06:24:53.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.542+0000 7f496be00640 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56481b349320 0x56481b34b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.543+0000 7f4956000640 10 client.25828 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:24:53.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.543+0000 7f496be00640 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56481b349320 0x56481b34b740 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5b54422484727a5 server_cookie=bceb5d270cb6f2e5 in_seq=0 out_seq=0 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f4956000640 1 -- 172.21.15.49:0/3417104277 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56481afd3720 con 0x56481b349320 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f4956000640 10 client.25828 handle_command_reply: tid=0 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 client.25828 shutdown 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 -- 172.21.15.49:0/3417104277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 msgr2=0x56481b336300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b336300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4964011010 tx=0x7f4964009fb0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 -- 172.21.15.49:0/3417104277 shutdown_connections 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56481b349320 0x56481b34b740 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b336d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 0x56481b336840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 --2- 172.21.15.49:0/3417104277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b336300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 -- 172.21.15.49:0/3417104277 >> 172.21.15.49:0/3417104277 conn(0x56481b32a4f0 msgr2=0x56481b32c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f4956000640 0 client.25828 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:24:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 -- 172.21.15.49:0/3417104277 shutdown_connections 2024-02-26T06:24:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.544+0000 7f49765a7740 1 -- 172.21.15.49:0/3417104277 wait complete. 2024-02-26T06:24:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.545+0000 7f49765a7740 20 client.25828 trim_cache size 0 max 16384 2024-02-26T06:24:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.570+0000 7f49765a7740 1 Processor -- start 2024-02-26T06:24:53.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.571+0000 7f49765a7740 1 -- start start 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.571+0000 7f49765a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b32fdc0 0x56481b338a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.571+0000 7f49765a7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b3315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.571+0000 7f49765a7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b331b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.571+0000 7f49765a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b3360f0 con 0x56481b32fdc0 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.571+0000 7f49765a7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b350910 con 0x56481b3392c0 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.571+0000 7f49765a7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b34bda0 con 0x56481b3306e0 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.572+0000 7f496be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b3315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.572+0000 7f496be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b3315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46402/0 (socket says 172.21.15.49:46402) 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.572+0000 7f496be00640 1 -- 172.21.15.49:0/1900421082 learned_addr learned my addr 172.21.15.49:0/1900421082 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.573+0000 7f4971600640 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b32fdc0 0x56481b338a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.574+0000 7f4971600640 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b331b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.574+0000 7f496be00640 1 -- 172.21.15.49:0/1900421082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 msgr2=0x56481b331b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.574+0000 7f496be00640 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b331b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.574+0000 7f496be00640 1 -- 172.21.15.49:0/1900421082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b32fdc0 msgr2=0x56481b338a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.574+0000 7f496be00640 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b32fdc0 0x56481b338a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.574+0000 7f496be00640 1 -- 172.21.15.49:0/1900421082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56481b231a80 con 0x56481b3306e0 2024-02-26T06:24:53.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.575+0000 7f496be00640 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b3315c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f495800baf0 tx=0x7f49580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3df7de2bc080041 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.577+0000 7f4961200640 1 -- 172.21.15.49:0/1900421082 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958017040 con 0x56481b3306e0 2024-02-26T06:24:53.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.577+0000 7f4961200640 1 -- 172.21.15.49:0/1900421082 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f495801a330 con 0x56481b3306e0 2024-02-26T06:24:53.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.577+0000 7f4961200640 1 -- 172.21.15.49:0/1900421082 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49580119d0 con 0x56481b3306e0 2024-02-26T06:24:53.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.578+0000 7f49765a7740 1 -- 172.21.15.49:0/1900421082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 msgr2=0x56481b3315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.578+0000 7f49765a7740 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b3315c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f495800baf0 tx=0x7f49580046c0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.578+0000 7f49765a7740 1 -- 172.21.15.49:0/1900421082 shutdown_connections 2024-02-26T06:24:53.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.578+0000 7f49765a7740 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b331b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.578+0000 7f49765a7740 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b3306e0 0x56481b3315c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.578+0000 7f49765a7740 1 --2- 172.21.15.49:0/1900421082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b32fdc0 0x56481b338a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.578+0000 7f49765a7740 1 -- 172.21.15.49:0/1900421082 >> 172.21.15.49:0/1900421082 conn(0x56481b32a4f0 msgr2=0x56481b325520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.579+0000 7f49765a7740 1 -- 172.21.15.49:0/1900421082 shutdown_connections 2024-02-26T06:24:53.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.579+0000 7f49765a7740 1 -- 172.21.15.49:0/1900421082 wait complete. 2024-02-26T06:24:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.580+0000 7f49765a7740 1 -- start start 2024-02-26T06:24:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.580+0000 7f49765a7740 1 Processor -- start 2024-02-26T06:24:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.581+0000 7f49765a7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b398aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f49765a7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 0x56481b398fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f49765a7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b33f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f49765a7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b33f980 con 0x56481b3306e0 2024-02-26T06:24:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f49765a7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b33faf0 con 0x56481b3392c0 2024-02-26T06:24:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f49765a7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481b33fcb0 con 0x56481b32fdc0 2024-02-26T06:24:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f49765a7740 10 client.0 _ll_register_callbacks cb 0x7f4964009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f4971600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b398aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f4971600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b398aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46416/0 (socket says 172.21.15.49:46416) 2024-02-26T06:24:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f4971600640 1 -- 172.21.15.49:0/3611222271 learned_addr learned my addr 172.21.15.49:0/3611222271 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.582+0000 7f4971600640 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b33f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.583+0000 7f4971600640 1 -- 172.21.15.49:0/3611222271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 msgr2=0x56481b33f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.583+0000 7f4971600640 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b33f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.583+0000 7f4971600640 1 -- 172.21.15.49:0/3611222271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 msgr2=0x56481b398fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.583+0000 7f4971600640 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 0x56481b398fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.583+0000 7f4971600640 1 -- 172.21.15.49:0/3611222271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495800bc00 con 0x56481b32fdc0 2024-02-26T06:24:53.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.584+0000 7f4971600640 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b398aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4964002230 tx=0x7f4964005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10427bfd38aaa076 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.584+0000 7f4956000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.584+0000 7f4956000640 1 -- 172.21.15.49:0/3611222271 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964012450 con 0x56481b32fdc0 2024-02-26T06:24:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.584+0000 7f4956000640 1 -- 172.21.15.49:0/3611222271 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f49640125b0 con 0x56481b32fdc0 2024-02-26T06:24:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.584+0000 7f49765a7740 1 -- 172.21.15.49:0/3611222271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f496401d070 con 0x56481b32fdc0 2024-02-26T06:24:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.584+0000 7f4956000640 1 -- 172.21.15.49:0/3611222271 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4964012770 con 0x56481b32fdc0 2024-02-26T06:24:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.584+0000 7f49765a7740 10 client.25834 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.584+0000 7f49765a7740 1 -- 172.21.15.49:0/3611222271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56481b257620 con 0x56481b32fdc0 2024-02-26T06:24:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.585+0000 7f4956000640 1 -- 172.21.15.49:0/3611222271 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4964046020 con 0x56481b32fdc0 2024-02-26T06:24:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.585+0000 7f4956000640 1 -- 172.21.15.49:0/3611222271 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4964019910 con 0x56481b32fdc0 2024-02-26T06:24:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.585+0000 7f49765a7740 10 client.25834 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.585+0000 7f49765a7740 10 client.25834 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:24:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.585+0000 7f49765a7740 20 client.25834 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.585+0000 7f49765a7740 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56481b3a17c0 0x56481b3a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.585+0000 7f49765a7740 4 client.25834 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:24:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.585+0000 7f49765a7740 1 -- 172.21.15.49:0/3611222271 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56481b022890 con 0x56481b3a17c0 2024-02-26T06:24:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.586+0000 7f496be00640 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56481b3a17c0 0x56481b3a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f496be00640 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56481b3a17c0 0x56481b3a3be0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bca0db9ad6e29a7b server_cookie=bef3515734f81ada in_seq=0 out_seq=0 2024-02-26T06:24:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f4956000640 10 client.25834 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:24:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f4956000640 1 -- 172.21.15.49:0/3611222271 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56481b022890 con 0x56481b3a17c0 2024-02-26T06:24:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f4956000640 10 client.25834 handle_command_reply: tid=0 2024-02-26T06:24:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 client.25834 shutdown 2024-02-26T06:24:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 -- 172.21.15.49:0/3611222271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 msgr2=0x56481b398aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b398aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4964002230 tx=0x7f4964005890 comp rx=0 tx=0).stop 2024-02-26T06:24:53.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 -- 172.21.15.49:0/3611222271 shutdown_connections 2024-02-26T06:24:53.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56481b3a17c0 0x56481b3a3be0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56481b3392c0 0x56481b33f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56481b3306e0 0x56481b398fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f4956000640 0 client.25834 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:24:53.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 --2- 172.21.15.49:0/3611222271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56481b32fdc0 0x56481b398aa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.587+0000 7f49765a7740 1 -- 172.21.15.49:0/3611222271 >> 172.21.15.49:0/3611222271 conn(0x56481b32a4f0 msgr2=0x56481b3315c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.588+0000 7f49765a7740 1 -- 172.21.15.49:0/3611222271 shutdown_connections 2024-02-26T06:24:53.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.588+0000 7f49765a7740 1 -- 172.21.15.49:0/3611222271 wait complete. 2024-02-26T06:24:53.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.589+0000 7f49765a7740 20 client.25834 trim_cache size 0 max 16384 2024-02-26T06:24:53.596 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:24:53.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 -- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49500a23d0 msgr2=0x7f49500a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49500a23d0 0x7f49500a4890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f495c0edb60 tx=0x7f495c0f28e0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 -- 172.21.15.49:0/3579152946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 msgr2=0x7f496c19f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 0x7f496c19f5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f495c005940 tx=0x7f495c004040 comp rx=0 tx=0).stop 2024-02-26T06:24:53.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 -- 172.21.15.49:0/3579152946 shutdown_connections 2024-02-26T06:24:53.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49500a23d0 0x7f49500a4890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f496c105220 0x7f496c19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f496c1996b0 0x7f496c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 --2- 172.21.15.49:0/3579152946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f496c100150 0x7f496c19f5a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.591+0000 7f4972000640 1 -- 172.21.15.49:0/3579152946 >> 172.21.15.49:0/3579152946 conn(0x7f496c0fb9d0 msgr2=0x7f496c10cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.592+0000 7f4972000640 1 -- 172.21.15.49:0/3579152946 shutdown_connections 2024-02-26T06:24:53.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.592+0000 7f4972000640 1 -- 172.21.15.49:0/3579152946 wait complete. 2024-02-26T06:24:53.630 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:24:53.630 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:24:53.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:53 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3764994812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:53.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:53 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3764994812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:53.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.915+0000 7f9160400640 1 -- 172.21.15.49:0/682248520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f915810a260 msgr2=0x7f915810c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.915+0000 7f9160400640 1 --2- 172.21.15.49:0/682248520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f915810a260 0x7f915810c650 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9150009ca0 tx=0x7f915002c100 comp rx=0 tx=0).stop 2024-02-26T06:24:53.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.915+0000 7f9160400640 1 -- 172.21.15.49:0/682248520 shutdown_connections 2024-02-26T06:24:53.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.915+0000 7f9160400640 1 --2- 172.21.15.49:0/682248520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f915810a260 0x7f915810c650 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.915+0000 7f9160400640 1 --2- 172.21.15.49:0/682248520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91580ffef0 0x7f9158104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.915+0000 7f9160400640 1 --2- 172.21.15.49:0/682248520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9158067600 0x7f9158103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.915+0000 7f9160400640 1 -- 172.21.15.49:0/682248520 >> 172.21.15.49:0/682248520 conn(0x7f91580761d0 msgr2=0x7f91580fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.915+0000 7f9160400640 1 -- 172.21.15.49:0/682248520 shutdown_connections 2024-02-26T06:24:53.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.916+0000 7f9160400640 1 -- 172.21.15.49:0/682248520 wait complete. 2024-02-26T06:24:53.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.917+0000 7f9160400640 1 Processor -- start 2024-02-26T06:24:53.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.918+0000 7f9160400640 1 -- start start 2024-02-26T06:24:53.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.918+0000 7f9160400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9158067600 0x7f915819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.918+0000 7f9160400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91580ffef0 0x7f9158108df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.918+0000 7f9160400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f915810a260 0x7f9158109330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.918+0000 7f9160400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9158109930 con 0x7f915810a260 2024-02-26T06:24:53.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.918+0000 7f9160400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9158109aa0 con 0x7f91580ffef0 2024-02-26T06:24:53.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.918+0000 7f9160400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9158109c60 con 0x7f9158067600 2024-02-26T06:24:53.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.918+0000 7f915da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9158067600 0x7f915819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9158067600 0x7f915819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46436/0 (socket says 172.21.15.49:46436) 2024-02-26T06:24:53.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91580ffef0 0x7f9158108df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915da00640 1 -- 172.21.15.49:0/4281347881 learned_addr learned my addr 172.21.15.49:0/4281347881 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:53 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3764994812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:53.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915e400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f915810a260 0x7f9158109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915da00640 1 -- 172.21.15.49:0/4281347881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91580ffef0 msgr2=0x7f9158108df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915da00640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91580ffef0 0x7f9158108df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915da00640 1 -- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f915810a260 msgr2=0x7f9158109330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915da00640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f915810a260 0x7f9158109330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915da00640 1 -- 172.21.15.49:0/4281347881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150009900 con 0x7f9158067600 2024-02-26T06:24:53.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.919+0000 7f915d000640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91580ffef0 0x7f9158108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:53.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.920+0000 7f915da00640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9158067600 0x7f915819a3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.921+0000 7f9156000640 1 -- 172.21.15.49:0/4281347881 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144009720 con 0x7f9158067600 2024-02-26T06:24:53.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.921+0000 7f9156000640 1 -- 172.21.15.49:0/4281347881 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f914400cd80 con 0x7f9158067600 2024-02-26T06:24:53.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.921+0000 7f9156000640 1 -- 172.21.15.49:0/4281347881 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144010690 con 0x7f9158067600 2024-02-26T06:24:53.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.922+0000 7f9160400640 1 -- 172.21.15.49:0/4281347881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91581aa5d0 con 0x7f9158067600 2024-02-26T06:24:53.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.922+0000 7f9160400640 1 -- 172.21.15.49:0/4281347881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91581aaa60 con 0x7f9158067600 2024-02-26T06:24:53.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.927+0000 7f91631fa740 1 Processor -- start 2024-02-26T06:24:53.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.927+0000 7f91631fa740 1 -- start start 2024-02-26T06:24:53.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f91631fa740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e10eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f91631fa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 0x55f26e117bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f91631fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 0x55f26e11a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f91631fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e07d1e0 con 0x55f26e118110 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f91631fa740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e07a710 con 0x55f26e10ec10 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f91631fa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e075ef0 con 0x55f26e10f530 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f915e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e10eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f915e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e10eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46354/0 (socket says 172.21.15.49:46354) 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f915e400640 1 -- 172.21.15.49:0/3605486447 learned_addr learned my addr 172.21.15.49:0/3605486447 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f915e400640 1 -- 172.21.15.49:0/3605486447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 msgr2=0x55f26e117bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f915e400640 1 --2- 172.21.15.49:0/3605486447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 0x55f26e117bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f915e400640 1 -- 172.21.15.49:0/3605486447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 msgr2=0x55f26e11a500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f915e400640 1 --2- 172.21.15.49:0/3605486447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 0x55f26e11a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.928+0000 7f915e400640 1 -- 172.21.15.49:0/3605486447 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f26e037dc0 con 0x55f26e10ec10 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.929+0000 7f915e400640 1 --2- 172.21.15.49:0/3605486447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e10eff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f915806a9f0 tx=0x7f9150005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11451016d532bf04 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.929+0000 7f914aa00640 1 -- 172.21.15.49:0/3605486447 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150032540 con 0x55f26e10ec10 2024-02-26T06:24:53.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.929+0000 7f9156000640 1 -- 172.21.15.49:0/4281347881 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f91440046c0 con 0x7f9158067600 2024-02-26T06:24:53.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.929+0000 7f914aa00640 1 -- 172.21.15.49:0/3605486447 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9150032e90 con 0x55f26e10ec10 2024-02-26T06:24:53.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.929+0000 7f914aa00640 1 -- 172.21.15.49:0/3605486447 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150003e10 con 0x55f26e10ec10 2024-02-26T06:24:53.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.930+0000 7f91631fa740 1 -- 172.21.15.49:0/3605486447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 msgr2=0x55f26e10eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.930+0000 7f91631fa740 1 --2- 172.21.15.49:0/3605486447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e10eff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f915806a9f0 tx=0x7f9150005ef0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.931+0000 7f9156000640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91400a24a0 0x7f91400a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.932+0000 7f9156000640 1 -- 172.21.15.49:0/4281347881 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9144014070 con 0x7f9158067600 2024-02-26T06:24:53.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.932+0000 7f915e400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91400a24a0 0x7f91400a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.933+0000 7f91631fa740 1 -- 172.21.15.49:0/3605486447 shutdown_connections 2024-02-26T06:24:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.933+0000 7f91631fa740 1 --2- 172.21.15.49:0/3605486447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 0x55f26e11a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.933+0000 7f91631fa740 1 --2- 172.21.15.49:0/3605486447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 0x55f26e117bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.933+0000 7f91631fa740 1 --2- 172.21.15.49:0/3605486447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e10eff0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.933+0000 7f91631fa740 1 -- 172.21.15.49:0/3605486447 >> 172.21.15.49:0/3605486447 conn(0x55f26e109340 msgr2=0x55f26e109720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.933+0000 7f91631fa740 1 -- 172.21.15.49:0/3605486447 shutdown_connections 2024-02-26T06:24:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.933+0000 7f91631fa740 1 -- 172.21.15.49:0/3605486447 wait complete. 2024-02-26T06:24:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.934+0000 7f915e400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91400a24a0 0x7f91400a4960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9150009600 tx=0x7f915002c100 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.934+0000 7f91631fa740 1 -- start start 2024-02-26T06:24:53.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.934+0000 7f91631fa740 1 Processor -- start 2024-02-26T06:24:53.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.935+0000 7f91631fa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e115240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.935+0000 7f91631fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 0x55f26e115780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.935+0000 7f91631fa740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.935+0000 7f91631fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e01b120 con 0x55f26e10f530 2024-02-26T06:24:53.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.935+0000 7f91631fa740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e01adc0 con 0x55f26e118110 2024-02-26T06:24:53.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.935+0000 7f91631fa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26dde41c0 con 0x55f26e10ec10 2024-02-26T06:24:53.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.935+0000 7f91631fa740 10 client.0 _ll_register_callbacks cb 0x55f26e06a280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:53.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.936+0000 7f915da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e115240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.936+0000 7f915da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e115240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46442/0 (socket says 172.21.15.49:46442) 2024-02-26T06:24:53.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.936+0000 7f915da00640 1 -- 172.21.15.49:0/404681756 learned_addr learned my addr 172.21.15.49:0/404681756 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.936+0000 7f915d000640 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 0x55f26e115780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.936+0000 7f915e400640 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f915da00640 1 -- 172.21.15.49:0/404681756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 msgr2=0x55f26e115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f915da00640 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f915da00640 1 -- 172.21.15.49:0/404681756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 msgr2=0x55f26e115780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f915da00640 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 0x55f26e115780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f915da00640 1 -- 172.21.15.49:0/404681756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91580ff950 con 0x55f26e10ec10 2024-02-26T06:24:53.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f915d000640 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 0x55f26e115780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:53.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f915da00640 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e115240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f91440cc440 tx=0x7f914409b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a472c77e4569c12 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f913f400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:53.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f913f400640 1 -- 172.21.15.49:0/404681756 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914409efe0 con 0x55f26e10ec10 2024-02-26T06:24:53.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.937+0000 7f913f400640 1 -- 172.21.15.49:0/404681756 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f91440a3630 con 0x55f26e10ec10 2024-02-26T06:24:53.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.938+0000 7f91631fa740 1 -- 172.21.15.49:0/404681756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f26dd757e0 con 0x55f26e10ec10 2024-02-26T06:24:53.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.938+0000 7f913f400640 1 -- 172.21.15.49:0/404681756 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91440b9300 con 0x55f26e10ec10 2024-02-26T06:24:53.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.938+0000 7f91631fa740 10 client.25843 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:53.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.938+0000 7f91631fa740 1 -- 172.21.15.49:0/404681756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f26e074570 con 0x55f26e10ec10 2024-02-26T06:24:53.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.939+0000 7f913f400640 1 -- 172.21.15.49:0/404681756 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f26dd757e0 con 0x55f26e10ec10 2024-02-26T06:24:53.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.939+0000 7f913f400640 1 -- 172.21.15.49:0/404681756 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f914401f060 con 0x55f26e10ec10 2024-02-26T06:24:53.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.939+0000 7f91631fa740 10 client.25843 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:53.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.939+0000 7f91631fa740 10 client.25843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:53.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.939+0000 7f91631fa740 20 client.25843 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:53.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.939+0000 7f91631fa740 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f26e128190 0x55f26e12a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.939+0000 7f91631fa740 4 client.25843 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:53.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.939+0000 7f91631fa740 1 -- 172.21.15.49:0/404681756 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f26ddb2720 con 0x55f26e128190 2024-02-26T06:24:53.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.940+0000 7f915d000640 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f26e128190 0x55f26e12a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.942+0000 7f913f400640 10 client.25843 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:53.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.942+0000 7f915d000640 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f26e128190 0x55f26e12a5b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8908b0f48f129c48 server_cookie=afb8d3ce5fd578d in_seq=0 out_seq=0 2024-02-26T06:24:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.943+0000 7f913f400640 1 -- 172.21.15.49:0/404681756 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f26ddb2720 con 0x55f26e128190 2024-02-26T06:24:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.943+0000 7f913f400640 10 client.25843 handle_command_reply: tid=0 2024-02-26T06:24:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.943+0000 7f91631fa740 1 client.25843 shutdown 2024-02-26T06:24:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 -- 172.21.15.49:0/404681756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 msgr2=0x55f26e115240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e115240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f91440cc440 tx=0x7f914409b590 comp rx=0 tx=0).stop 2024-02-26T06:24:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 -- 172.21.15.49:0/404681756 shutdown_connections 2024-02-26T06:24:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f26e128190 0x55f26e12a5b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 0x55f26e115780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 --2- 172.21.15.49:0/404681756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e115240 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 -- 172.21.15.49:0/404681756 >> 172.21.15.49:0/404681756 conn(0x55f26e109340 msgr2=0x55f26e10b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f913f400640 0 client.25843 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:53.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 -- 172.21.15.49:0/404681756 shutdown_connections 2024-02-26T06:24:53.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.944+0000 7f91631fa740 1 -- 172.21.15.49:0/404681756 wait complete. 2024-02-26T06:24:53.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.945+0000 7f91631fa740 20 client.25843 trim_cache size 0 max 16384 2024-02-26T06:24:53.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.966+0000 7f91631fa740 1 Processor -- start 2024-02-26T06:24:53.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.966+0000 7f91631fa740 1 -- start start 2024-02-26T06:24:53.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.967+0000 7f91631fa740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e117970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.967+0000 7f91631fa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 0x55f26e115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.967+0000 7f91631fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 0x55f26e1124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.967+0000 7f91631fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e10a340 con 0x55f26e118110 2024-02-26T06:24:53.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.967+0000 7f91631fa740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e124d40 con 0x55f26e10ec10 2024-02-26T06:24:53.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.967+0000 7f91631fa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e10e650 con 0x55f26e10f530 2024-02-26T06:24:53.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.968+0000 7f915da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e117970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.968+0000 7f915da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e117970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46370/0 (socket says 172.21.15.49:46370) 2024-02-26T06:24:53.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.968+0000 7f915da00640 1 -- 172.21.15.49:0/204440211 learned_addr learned my addr 172.21.15.49:0/204440211 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.968+0000 7f915d000640 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 0x55f26e115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.968+0000 7f915e400640 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 0x55f26e1124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.968+0000 7f915da00640 1 -- 172.21.15.49:0/204440211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 msgr2=0x55f26e115ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.968+0000 7f915da00640 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 0x55f26e115ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.968+0000 7f915da00640 1 -- 172.21.15.49:0/204440211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 msgr2=0x55f26e1124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.969+0000 7f915da00640 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 0x55f26e1124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.969+0000 7f915da00640 1 -- 172.21.15.49:0/204440211 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f26e010a80 con 0x55f26e10ec10 2024-02-26T06:24:53.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.969+0000 7f915e400640 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 0x55f26e1124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:53.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.969+0000 7f915da00640 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e117970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91440972f0 tx=0x7f9144005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8873a06c9cbc99e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.970+0000 7f914a000640 1 -- 172.21.15.49:0/204440211 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914409a9d0 con 0x55f26e10ec10 2024-02-26T06:24:53.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.970+0000 7f914a000640 1 -- 172.21.15.49:0/204440211 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91440a3630 con 0x55f26e10ec10 2024-02-26T06:24:53.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.970+0000 7f914a000640 1 -- 172.21.15.49:0/204440211 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91440cc200 con 0x55f26e10ec10 2024-02-26T06:24:53.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.970+0000 7f91631fa740 1 -- 172.21.15.49:0/204440211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 msgr2=0x55f26e117970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.970+0000 7f91631fa740 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e117970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91440972f0 tx=0x7f9144005720 comp rx=0 tx=0).stop 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.971+0000 7f91631fa740 1 -- 172.21.15.49:0/204440211 shutdown_connections 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.971+0000 7f91631fa740 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e118110 0x55f26e1124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.971+0000 7f91631fa740 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10f530 0x55f26e115ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.971+0000 7f91631fa740 1 --2- 172.21.15.49:0/204440211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e10ec10 0x55f26e117970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.971+0000 7f91631fa740 1 -- 172.21.15.49:0/204440211 >> 172.21.15.49:0/204440211 conn(0x55f26e109340 msgr2=0x55f26e110590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.971+0000 7f91631fa740 1 -- 172.21.15.49:0/204440211 shutdown_connections 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.971+0000 7f91631fa740 1 -- 172.21.15.49:0/204440211 wait complete. 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.971+0000 7f91631fa740 1 -- start start 2024-02-26T06:24:53.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.972+0000 7f91631fa740 1 Processor -- start 2024-02-26T06:24:53.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.972+0000 7f91631fa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e11dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.972+0000 7f91631fa740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 0x55f26e11e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.972+0000 7f91631fa740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e11e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.972+0000 7f91631fa740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e17e790 con 0x55f26e10f530 2024-02-26T06:24:53.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.972+0000 7f91631fa740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e17e900 con 0x55f26e118110 2024-02-26T06:24:53.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.972+0000 7f91631fa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26e18fad0 con 0x55f26e10ec10 2024-02-26T06:24:53.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.972+0000 7f91631fa740 10 client.0 _ll_register_callbacks cb 0x7f9150037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:53.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.973+0000 7f915da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e11dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.973+0000 7f915e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e11e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.973+0000 7f915e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e11e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46372/0 (socket says 172.21.15.49:46372) 2024-02-26T06:24:53.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.973+0000 7f915e400640 1 -- 172.21.15.49:0/1507163460 learned_addr learned my addr 172.21.15.49:0/1507163460 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:53.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.974+0000 7f915e400640 1 -- 172.21.15.49:0/1507163460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 msgr2=0x55f26e11dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.974+0000 7f915e400640 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e11dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.974+0000 7f915e400640 1 -- 172.21.15.49:0/1507163460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 msgr2=0x55f26e11e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:53.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.974+0000 7f915e400640 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 0x55f26e11e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.974+0000 7f915e400640 1 -- 172.21.15.49:0/1507163460 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f26e074570 con 0x55f26e118110 2024-02-26T06:24:53.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.974+0000 7f913f400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:53.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.974+0000 7f915e400640 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e11e620 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9150003fa0 tx=0x7f9150003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec9b3d8bd577ff7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.974+0000 7f913f400640 1 -- 172.21.15.49:0/1507163460 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150034910 con 0x55f26e118110 2024-02-26T06:24:53.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.975+0000 7f913f400640 1 -- 172.21.15.49:0/1507163460 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9150004000 con 0x55f26e118110 2024-02-26T06:24:53.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.975+0000 7f91631fa740 1 -- 172.21.15.49:0/1507163460 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9150032e90 con 0x55f26e118110 2024-02-26T06:24:53.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.975+0000 7f913f400640 1 -- 172.21.15.49:0/1507163460 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91500041c0 con 0x55f26e118110 2024-02-26T06:24:53.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.976+0000 7f91631fa740 10 client.25805 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:53.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.976+0000 7f91631fa740 1 -- 172.21.15.49:0/1507163460 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f26e036620 con 0x55f26e118110 2024-02-26T06:24:53.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.976+0000 7f913f400640 1 -- 172.21.15.49:0/1507163460 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9150034910 con 0x55f26e118110 2024-02-26T06:24:53.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.976+0000 7f913f400640 1 -- 172.21.15.49:0/1507163460 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9150051040 con 0x55f26e118110 2024-02-26T06:24:53.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.976+0000 7f91631fa740 10 client.25805 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:53.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.977+0000 7f91631fa740 10 client.25805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:53.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.977+0000 7f91631fa740 20 client.25805 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:53.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.977+0000 7f91631fa740 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f26e190f40 0x55f26e193360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.977+0000 7f91631fa740 4 client.25805 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:24:53.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.977+0000 7f91631fa740 1 -- 172.21.15.49:0/1507163460 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f9150032e90 con 0x55f26e190f40 2024-02-26T06:24:53.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.977+0000 7f915da00640 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f26e190f40 0x55f26e193360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.978+0000 7f913f400640 10 client.25805 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.978+0000 7f915da00640 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f26e190f40 0x55f26e193360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d0656b6625aa6e server_cookie=ca5416c803fa4dd7 in_seq=0 out_seq=0 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f913f400640 1 -- 172.21.15.49:0/1507163460 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f9150032e90 con 0x55f26e190f40 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f913f400640 10 client.25805 handle_command_reply: tid=0 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 client.25805 shutdown 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 -- 172.21.15.49:0/1507163460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 msgr2=0x55f26e11e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e11e620 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9150003fa0 tx=0x7f9150003a10 comp rx=0 tx=0).stop 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 -- 172.21.15.49:0/1507163460 shutdown_connections 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f26e190f40 0x55f26e193360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f913f400640 0 client.25805 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:53.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f26e118110 0x55f26e11e620 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f26e10f530 0x55f26e11e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 --2- 172.21.15.49:0/1507163460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f26e10ec10 0x55f26e11dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 -- 172.21.15.49:0/1507163460 >> 172.21.15.49:0/1507163460 conn(0x55f26e109340 msgr2=0x55f26e107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 -- 172.21.15.49:0/1507163460 shutdown_connections 2024-02-26T06:24:53.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.979+0000 7f91631fa740 1 -- 172.21.15.49:0/1507163460 wait complete. 2024-02-26T06:24:53.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.980+0000 7f91631fa740 20 client.25805 trim_cache size 0 max 16384 2024-02-26T06:24:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.980+0000 7f9160400640 1 -- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91400a24a0 msgr2=0x7f91400a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.980+0000 7f9160400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91400a24a0 0x7f91400a4960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9150009600 tx=0x7f915002c100 comp rx=0 tx=0).stop 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.980+0000 7f9160400640 1 -- 172.21.15.49:0/4281347881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9158067600 msgr2=0x7f915819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.980+0000 7f9160400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9158067600 0x7f915819a3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400edf0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.981+0000 7f9160400640 1 -- 172.21.15.49:0/4281347881 shutdown_connections 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.981+0000 7f9160400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f91400a24a0 0x7f91400a4960 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.981+0000 7f9160400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91580ffef0 0x7f9158108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.981+0000 7f9160400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f915810a260 0x7f9158109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.981+0000 7f9160400640 1 --2- 172.21.15.49:0/4281347881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9158067600 0x7f915819a3c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.981+0000 7f9160400640 1 -- 172.21.15.49:0/4281347881 >> 172.21.15.49:0/4281347881 conn(0x7f91580761d0 msgr2=0x7f915810b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.981+0000 7f9160400640 1 -- 172.21.15.49:0/4281347881 shutdown_connections 2024-02-26T06:24:53.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:53.981+0000 7f9160400640 1 -- 172.21.15.49:0/4281347881 wait complete. 2024-02-26T06:24:54.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:54 smithi080 ceph-mon[36206]: pgmap v48: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-02-26T06:24:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:54 smithi115 ceph-mon[36426]: pgmap v48: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-02-26T06:24:54.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:54 smithi049 ceph-mon[31696]: pgmap v48: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-02-26T06:24:55.008 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:24:55.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:24:55.009 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:24:55.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.317+0000 7fb50e200640 1 -- 172.21.15.49:0/1116113935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb50811a960 msgr2=0x7fb50811cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.317+0000 7fb50e200640 1 --2- 172.21.15.49:0/1116113935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb50811a960 0x7fb50811cd50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb500009be0 tx=0x7fb50002bf10 comp rx=0 tx=0).stop 2024-02-26T06:24:55.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.318+0000 7fb50e200640 1 -- 172.21.15.49:0/1116113935 shutdown_connections 2024-02-26T06:24:55.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.318+0000 7fb50e200640 1 --2- 172.21.15.49:0/1116113935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb50811a960 0x7fb50811cd50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.318+0000 7fb50e200640 1 --2- 172.21.15.49:0/1116113935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50810d9d0 0x7fb50810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.318+0000 7fb50e200640 1 --2- 172.21.15.49:0/1116113935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb50810f9a0 0x7fb50810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.318+0000 7fb50e200640 1 -- 172.21.15.49:0/1116113935 >> 172.21.15.49:0/1116113935 conn(0x7fb50806c260 msgr2=0x7fb50806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.318+0000 7fb50e200640 1 -- 172.21.15.49:0/1116113935 shutdown_connections 2024-02-26T06:24:55.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.318+0000 7fb50e200640 1 -- 172.21.15.49:0/1116113935 wait complete. 2024-02-26T06:24:55.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.319+0000 7fb50e200640 1 Processor -- start 2024-02-26T06:24:55.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.320+0000 7fb50e200640 1 -- start start 2024-02-26T06:24:55.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.320+0000 7fb50e200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50810d9d0 0x7fb5081a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.320+0000 7fb50e200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb50810f9a0 0x7fb5081a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.320+0000 7fb50e200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb50811a960 0x7fb5081aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.320+0000 7fb50e200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081ab4d0 con 0x7fb50810f9a0 2024-02-26T06:24:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.320+0000 7fb50e200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081ab640 con 0x7fb50811a960 2024-02-26T06:24:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.320+0000 7fb50e200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081ab800 con 0x7fb50810d9d0 2024-02-26T06:24:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50810d9d0 0x7fb5081a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50810d9d0 0x7fb5081a6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46464/0 (socket says 172.21.15.49:46464) 2024-02-26T06:24:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507400640 1 -- 172.21.15.49:0/1945594740 learned_addr learned my addr 172.21.15.49:0/1945594740 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb506a00640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb50810f9a0 0x7fb5081a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507e00640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb50811a960 0x7fb5081aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507400640 1 -- 172.21.15.49:0/1945594740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb50811a960 msgr2=0x7fb5081aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507400640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb50811a960 0x7fb5081aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507400640 1 -- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb50810f9a0 msgr2=0x7fb5081a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507400640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb50810f9a0 0x7fb5081a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.321+0000 7fb507400640 1 -- 172.21.15.49:0/1945594740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb500009840 con 0x7fb50810d9d0 2024-02-26T06:24:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.322+0000 7fb507400640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50810d9d0 0x7fb5081a6610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b450 tx=0x7fb4f800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.322+0000 7fb4ffe00640 1 -- 172.21.15.49:0/1945594740 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8011890 con 0x7fb50810d9d0 2024-02-26T06:24:55.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.322+0000 7fb4ffe00640 1 -- 172.21.15.49:0/1945594740 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4f80103e0 con 0x7fb50810d9d0 2024-02-26T06:24:55.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.322+0000 7fb4ffe00640 1 -- 172.21.15.49:0/1945594740 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f800f5a0 con 0x7fb50810d9d0 2024-02-26T06:24:55.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.322+0000 7fb50e200640 1 -- 172.21.15.49:0/1945594740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5081abae0 con 0x7fb50810d9d0 2024-02-26T06:24:55.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.322+0000 7fb50e200640 1 -- 172.21.15.49:0/1945594740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5081b3360 con 0x7fb50810d9d0 2024-02-26T06:24:55.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.327+0000 7fb4ffe00640 1 -- 172.21.15.49:0/1945594740 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fb4f800f700 con 0x7fb50810d9d0 2024-02-26T06:24:55.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.328+0000 7fb510fda740 1 Processor -- start 2024-02-26T06:24:55.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.329+0000 7fb4ffe00640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb4ec0a2450 0x7fb4ec0a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.329+0000 7fb510fda740 1 -- start start 2024-02-26T06:24:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.330+0000 7fb4ffe00640 1 -- 172.21.15.49:0/1945594740 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb4f80ce060 con 0x7fb50810d9d0 2024-02-26T06:24:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.330+0000 7fb507e00640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb4ec0a2450 0x7fb4ec0a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.330+0000 7fb510fda740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5db0f10 0x557aa5db12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.330+0000 7fb510fda740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb507400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5db0f10 0x557aa5db12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb507400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5db0f10 0x557aa5db12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50468/0 (socket says 172.21.15.49:50468) 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb507e00640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb4ec0a2450 0x7fb4ec0a4910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb500005f00 tx=0x7fb50002c8b0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb506a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb506a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46478/0 (socket says 172.21.15.49:46478) 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb507400640 1 -- 172.21.15.49:0/1636981953 learned_addr learned my addr 172.21.15.49:0/1636981953 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb510fda740 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5dba410 0x557aa5dbc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb510fda740 1 -- 172.21.15.49:0/1636981953 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5d6c2d0 con 0x557aa5db0f10 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb510fda740 1 -- 172.21.15.49:0/1636981953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5d1ee20 con 0x557aa5dba410 2024-02-26T06:24:55.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb510fda740 1 -- 172.21.15.49:0/1636981953 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5d1c350 con 0x557aa5db1830 2024-02-26T06:24:55.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.331+0000 7fb507e00640 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5dba410 0x557aa5dbc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb506a00640 1 -- 172.21.15.49:0/1636981953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5dba410 msgr2=0x557aa5dbc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb506a00640 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5dba410 0x557aa5dbc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb506a00640 1 -- 172.21.15.49:0/1636981953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5db0f10 msgr2=0x557aa5db12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb506a00640 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5db0f10 0x557aa5db12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb506a00640 1 -- 172.21.15.49:0/1636981953 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa5ce0c50 con 0x557aa5db1830 2024-02-26T06:24:55.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb507400640 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5db0f10 0x557aa5db12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:55.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb506a00640 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db9ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40045d0 tx=0x7fb4f400bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeab6ec637401797 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb4f3e00640 1 -- 172.21.15.49:0/1636981953 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4003d70 con 0x557aa5db1830 2024-02-26T06:24:55.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb4f3e00640 1 -- 172.21.15.49:0/1636981953 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4f400be30 con 0x557aa5db1830 2024-02-26T06:24:55.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.332+0000 7fb4f3e00640 1 -- 172.21.15.49:0/1636981953 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4011840 con 0x557aa5db1830 2024-02-26T06:24:55.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.333+0000 7fb510fda740 1 -- 172.21.15.49:0/1636981953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 msgr2=0x557aa5db9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.333+0000 7fb510fda740 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db9ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40045d0 tx=0x7fb4f400bb20 comp rx=0 tx=0).stop 2024-02-26T06:24:55.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.334+0000 7fb510fda740 1 -- 172.21.15.49:0/1636981953 shutdown_connections 2024-02-26T06:24:55.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.334+0000 7fb510fda740 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5dba410 0x557aa5dbc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.334+0000 7fb510fda740 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db9ed0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.334+0000 7fb510fda740 1 --2- 172.21.15.49:0/1636981953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5db0f10 0x557aa5db12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.334+0000 7fb510fda740 1 -- 172.21.15.49:0/1636981953 >> 172.21.15.49:0/1636981953 conn(0x557aa5dab640 msgr2=0x557aa5daba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.334+0000 7fb510fda740 1 -- 172.21.15.49:0/1636981953 shutdown_connections 2024-02-26T06:24:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.335+0000 7fb510fda740 1 -- 172.21.15.49:0/1636981953 wait complete. 2024-02-26T06:24:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.335+0000 7fb510fda740 1 -- start start 2024-02-26T06:24:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.336+0000 7fb510fda740 1 Processor -- start 2024-02-26T06:24:55.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.336+0000 7fb510fda740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db0f10 0x557aa5db7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.336+0000 7fb510fda740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb510fda740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb510fda740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5d17b30 con 0x557aa5dba410 2024-02-26T06:24:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb510fda740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5cbcfa0 con 0x557aa5db0f10 2024-02-26T06:24:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb510fda740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5cbcc40 con 0x557aa5db1830 2024-02-26T06:24:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb510fda740 10 client.0 _ll_register_callbacks cb 0x557aa5a28c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb507400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db0f10 0x557aa5db7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb507400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db0f10 0x557aa5db7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46420/0 (socket says 172.21.15.49:46420) 2024-02-26T06:24:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb507400640 1 -- 172.21.15.49:0/956801094 learned_addr learned my addr 172.21.15.49:0/956801094 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb506a00640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.337+0000 7fb507400640 1 -- 172.21.15.49:0/956801094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 msgr2=0x557aa5db7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb507e00640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb507400640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb507400640 1 -- 172.21.15.49:0/956801094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 msgr2=0x557aa5db7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb507400640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb507400640 1 -- 172.21.15.49:0/956801094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f4004230 con 0x557aa5db0f10 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb507e00640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb506a00640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb507400640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db0f10 0x557aa5db7440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800be10 tx=0x7fb4f809ea10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e22b83cc25093719 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb4f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.338+0000 7fb4f0c00640 1 -- 172.21.15.49:0/956801094 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8010550 con 0x557aa5db0f10 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.339+0000 7fb4f0c00640 1 -- 172.21.15.49:0/956801094 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb4f80106b0 con 0x557aa5db0f10 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.339+0000 7fb510fda740 1 -- 172.21.15.49:0/956801094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557aa5a17860 con 0x557aa5db0f10 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.339+0000 7fb4f0c00640 1 -- 172.21.15.49:0/956801094 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f80b3220 con 0x557aa5db0f10 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.339+0000 7fb510fda740 10 client.25817 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.339+0000 7fb510fda740 1 -- 172.21.15.49:0/956801094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557aa5d161b0 con 0x557aa5db0f10 2024-02-26T06:24:55.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.339+0000 7fb4f0c00640 1 -- 172.21.15.49:0/956801094 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557aa5a17860 con 0x557aa5db0f10 2024-02-26T06:24:55.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.340+0000 7fb4f0c00640 1 -- 172.21.15.49:0/956801094 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb4f80cba30 con 0x557aa5db0f10 2024-02-26T06:24:55.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.340+0000 7fb510fda740 10 client.25817 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:55.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.340+0000 7fb510fda740 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:55.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.340+0000 7fb510fda740 20 client.25817 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:55.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.340+0000 7fb510fda740 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557aa5dca860 0x557aa5dccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.340+0000 7fb510fda740 4 client.25817 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:55.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.340+0000 7fb510fda740 1 -- 172.21.15.49:0/956801094 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557aa5a68410 con 0x557aa5dca860 2024-02-26T06:24:55.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.341+0000 7fb506a00640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557aa5dca860 0x557aa5dccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.341+0000 7fb506a00640 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557aa5dca860 0x557aa5dccca0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b421d943e3f95f9e server_cookie=6057650091f6af4 in_seq=0 out_seq=0 2024-02-26T06:24:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.341+0000 7fb4f0c00640 10 client.25817 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.342+0000 7fb4f0c00640 1 -- 172.21.15.49:0/956801094 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557aa5a68410 con 0x557aa5dca860 2024-02-26T06:24:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.342+0000 7fb4f0c00640 10 client.25817 handle_command_reply: tid=0 2024-02-26T06:24:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.342+0000 7fb510fda740 1 client.25817 shutdown 2024-02-26T06:24:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb510fda740 1 -- 172.21.15.49:0/956801094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db0f10 msgr2=0x557aa5db7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb510fda740 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db0f10 0x557aa5db7440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800be10 tx=0x7fb4f809ea10 comp rx=0 tx=0).stop 2024-02-26T06:24:55.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb510fda740 1 -- 172.21.15.49:0/956801094 shutdown_connections 2024-02-26T06:24:55.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb510fda740 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557aa5dca860 0x557aa5dccca0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb510fda740 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb510fda740 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db1830 0x557aa5db7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb510fda740 1 --2- 172.21.15.49:0/956801094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db0f10 0x557aa5db7440 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb510fda740 1 -- 172.21.15.49:0/956801094 >> 172.21.15.49:0/956801094 conn(0x557aa5dab640 msgr2=0x557aa5dad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.343+0000 7fb4f0c00640 0 client.25817 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:55.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.344+0000 7fb510fda740 1 -- 172.21.15.49:0/956801094 shutdown_connections 2024-02-26T06:24:55.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.344+0000 7fb510fda740 1 -- 172.21.15.49:0/956801094 wait complete. 2024-02-26T06:24:55.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.344+0000 7fb510fda740 20 client.25817 trim_cache size 0 max 16384 2024-02-26T06:24:55.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.370+0000 7fb510fda740 1 Processor -- start 2024-02-26T06:24:55.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb510fda740 1 -- start start 2024-02-26T06:24:55.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb510fda740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5db9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb510fda740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5db46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb510fda740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb510fda740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5a600c0 con 0x557aa5dba410 2024-02-26T06:24:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb510fda740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5a17860 con 0x557aa5db1830 2024-02-26T06:24:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb510fda740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5dc7350 con 0x557aa5db0f10 2024-02-26T06:24:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb507400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5db9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.371+0000 7fb507400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5db9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46500/0 (socket says 172.21.15.49:46500) 2024-02-26T06:24:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.372+0000 7fb507400640 1 -- 172.21.15.49:0/853022671 learned_addr learned my addr 172.21.15.49:0/853022671 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.372+0000 7fb506a00640 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5db46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.372+0000 7fb507e00640 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.373+0000 7fb507400640 1 -- 172.21.15.49:0/853022671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 msgr2=0x557aa5db46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.373+0000 7fb507400640 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5db46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.373+0000 7fb507400640 1 -- 172.21.15.49:0/853022671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 msgr2=0x557aa5db4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.373+0000 7fb507400640 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.373+0000 7fb507400640 1 -- 172.21.15.49:0/853022671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa5cb2a60 con 0x557aa5db0f10 2024-02-26T06:24:55.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.373+0000 7fb507e00640 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:55.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.373+0000 7fb506a00640 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5db46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:55.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.373+0000 7fb507400640 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5db9b70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80a3220 tx=0x7fb4f8098dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ae041cb269839d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.374+0000 7fb4f3400640 1 -- 172.21.15.49:0/853022671 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8010550 con 0x557aa5db0f10 2024-02-26T06:24:55.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.374+0000 7fb4f3400640 1 -- 172.21.15.49:0/853022671 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4f80106b0 con 0x557aa5db0f10 2024-02-26T06:24:55.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.374+0000 7fb4f3400640 1 -- 172.21.15.49:0/853022671 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8096610 con 0x557aa5db0f10 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 -- 172.21.15.49:0/853022671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 msgr2=0x557aa5db9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5db9b70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80a3220 tx=0x7fb4f8098dd0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 -- 172.21.15.49:0/853022671 shutdown_connections 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5db46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5db4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 --2- 172.21.15.49:0/853022671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5db9b70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 -- 172.21.15.49:0/853022671 >> 172.21.15.49:0/853022671 conn(0x557aa5dab640 msgr2=0x557aa5db26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 -- 172.21.15.49:0/853022671 shutdown_connections 2024-02-26T06:24:55.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.375+0000 7fb510fda740 1 -- 172.21.15.49:0/853022671 wait complete. 2024-02-26T06:24:55.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.376+0000 7fb510fda740 1 -- start start 2024-02-26T06:24:55.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.377+0000 7fb510fda740 1 Processor -- start 2024-02-26T06:24:55.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.377+0000 7fb510fda740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5dc0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.377+0000 7fb510fda740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5dc08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.377+0000 7fb510fda740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5e20d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb510fda740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5dc0fe0 con 0x557aa5dba410 2024-02-26T06:24:55.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb510fda740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5e212d0 con 0x557aa5db1830 2024-02-26T06:24:55.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb510fda740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa5e21460 con 0x557aa5db0f10 2024-02-26T06:24:55.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb510fda740 10 client.0 _ll_register_callbacks cb 0x7fb4f4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:55.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb507400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5dc0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb506a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5dc08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb506a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5dc08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46434/0 (socket says 172.21.15.49:46434) 2024-02-26T06:24:55.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb506a00640 1 -- 172.21.15.49:0/3049165631 learned_addr learned my addr 172.21.15.49:0/3049165631 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.378+0000 7fb507e00640 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5e20d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.379+0000 7fb506a00640 1 -- 172.21.15.49:0/3049165631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 msgr2=0x557aa5dc0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.379+0000 7fb506a00640 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5dc0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.379+0000 7fb506a00640 1 -- 172.21.15.49:0/3049165631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 msgr2=0x557aa5e20d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.379+0000 7fb506a00640 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5e20d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.379+0000 7fb506a00640 1 -- 172.21.15.49:0/3049165631 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f4021040 con 0x557aa5db1830 2024-02-26T06:24:55.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.379+0000 7fb507e00640 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5e20d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:55.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.379+0000 7fb506a00640 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5dc08a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40039c0 tx=0x7fb4f4005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a0e5fab435bcb9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.379+0000 7fb4f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:55.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.380+0000 7fb4f0c00640 1 -- 172.21.15.49:0/3049165631 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4028630 con 0x557aa5db1830 2024-02-26T06:24:55.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.380+0000 7fb4f0c00640 1 -- 172.21.15.49:0/3049165631 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb4f4028790 con 0x557aa5db1830 2024-02-26T06:24:55.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.380+0000 7fb510fda740 1 -- 172.21.15.49:0/3049165631 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4f400be30 con 0x557aa5db1830 2024-02-26T06:24:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.380+0000 7fb4f0c00640 1 -- 172.21.15.49:0/3049165631 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f4005040 con 0x557aa5db1830 2024-02-26T06:24:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.380+0000 7fb510fda740 10 client.25823 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.380+0000 7fb510fda740 1 -- 172.21.15.49:0/3049165631 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557aa5d161b0 con 0x557aa5db1830 2024-02-26T06:24:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.382+0000 7fb4f0c00640 1 -- 172.21.15.49:0/3049165631 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4f400be30 con 0x557aa5db1830 2024-02-26T06:24:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.383+0000 7fb4f0c00640 1 -- 172.21.15.49:0/3049165631 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb4f4024030 con 0x557aa5db1830 2024-02-26T06:24:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.383+0000 7fb510fda740 10 client.25823 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.383+0000 7fb510fda740 10 client.25823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.383+0000 7fb510fda740 20 client.25823 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.383+0000 7fb510fda740 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557aa5e22d80 0x557aa5e251a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.383+0000 7fb510fda740 4 client.25823 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:24:55.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.383+0000 7fb510fda740 1 -- 172.21.15.49:0/3049165631 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557aa5a547b0 con 0x557aa5e22d80 2024-02-26T06:24:55.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.383+0000 7fb507400640 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557aa5e22d80 0x557aa5e251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.384+0000 7fb507400640 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557aa5e22d80 0x557aa5e251a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d4bd345c50a5120 server_cookie=4e379ce7115ee69a in_seq=0 out_seq=0 2024-02-26T06:24:55.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.384+0000 7fb4f0c00640 10 client.25823 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:55.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.385+0000 7fb4f0c00640 1 -- 172.21.15.49:0/3049165631 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557aa5a547b0 con 0x557aa5e22d80 2024-02-26T06:24:55.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.385+0000 7fb4f0c00640 10 client.25823 handle_command_reply: tid=0 2024-02-26T06:24:55.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.385+0000 7fb510fda740 1 client.25823 shutdown 2024-02-26T06:24:55.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.385+0000 7fb510fda740 1 -- 172.21.15.49:0/3049165631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 msgr2=0x557aa5dc08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.385+0000 7fb510fda740 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5dc08a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40039c0 tx=0x7fb4f4005b70 comp rx=0 tx=0).stop 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 1 -- 172.21.15.49:0/3049165631 shutdown_connections 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557aa5e22d80 0x557aa5e251a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557aa5db1830 0x557aa5dc08a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aa5dba410 0x557aa5e20d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 1 --2- 172.21.15.49:0/3049165631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557aa5db0f10 0x557aa5dc0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 1 -- 172.21.15.49:0/3049165631 >> 172.21.15.49:0/3049165631 conn(0x557aa5dab640 msgr2=0x557aa5daca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb4f0c00640 0 client.25823 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 1 -- 172.21.15.49:0/3049165631 shutdown_connections 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 1 -- 172.21.15.49:0/3049165631 wait complete. 2024-02-26T06:24:55.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.386+0000 7fb510fda740 20 client.25823 trim_cache size 0 max 16384 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "9315427a-1ad5-4809-90e0-3151881c1f69", 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.387+0000 7fb50e200640 1 -- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb4ec0a2450 msgr2=0x7fb4ec0a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.387+0000 7fb50e200640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb4ec0a2450 0x7fb4ec0a4910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb500005f00 tx=0x7fb50002c8b0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.387+0000 7fb50e200640 1 -- 172.21.15.49:0/1945594740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50810d9d0 msgr2=0x7fb5081a6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.387+0000 7fb50e200640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50810d9d0 0x7fb5081a6610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b450 tx=0x7fb4f800b910 comp rx=0 tx=0).stop 2024-02-26T06:24:55.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.388+0000 7fb50e200640 1 -- 172.21.15.49:0/1945594740 shutdown_connections 2024-02-26T06:24:55.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.388+0000 7fb50e200640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb4ec0a2450 0x7fb4ec0a4910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.388+0000 7fb50e200640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb50811a960 0x7fb5081aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.388+0000 7fb50e200640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb50810f9a0 0x7fb5081a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.388+0000 7fb50e200640 1 --2- 172.21.15.49:0/1945594740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50810d9d0 0x7fb5081a6610 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.388+0000 7fb50e200640 1 -- 172.21.15.49:0/1945594740 >> 172.21.15.49:0/1945594740 conn(0x7fb50806c260 msgr2=0x7fb508071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.388+0000 7fb50e200640 1 -- 172.21.15.49:0/1945594740 shutdown_connections 2024-02-26T06:24:55.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.388+0000 7fb50e200640 1 -- 172.21.15.49:0/1945594740 wait complete. 2024-02-26T06:24:55.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:24:55.419 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:24:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:55 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:24:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:55 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:55 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 -- 172.21.15.49:0/1830193930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c108540 msgr2=0x7f645c110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 --2- 172.21.15.49:0/1830193930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c108540 0x7f645c110110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6454009cd0 tx=0x7f645402c0f0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 -- 172.21.15.49:0/1830193930 shutdown_connections 2024-02-26T06:24:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 --2- 172.21.15.49:0/1830193930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c108540 0x7f645c110110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 --2- 172.21.15.49:0/1830193930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c103e20 0x7f645c107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 --2- 172.21.15.49:0/1830193930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645c103470 0x7f645c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 -- 172.21.15.49:0/1830193930 >> 172.21.15.49:0/1830193930 conn(0x7f645c0fec90 msgr2=0x7f645c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 -- 172.21.15.49:0/1830193930 shutdown_connections 2024-02-26T06:24:55.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.712+0000 7f6462800640 1 -- 172.21.15.49:0/1830193930 wait complete. 2024-02-26T06:24:55.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.714+0000 7f6462800640 1 Processor -- start 2024-02-26T06:24:55.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.714+0000 7f6462800640 1 -- start start 2024-02-26T06:24:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6462800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c103470 0x7f645c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6462800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645c108540 0x7f645c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6462800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c199790 0x7f645c199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6462800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c19a260 con 0x7f645c108540 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6462800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c19a3d0 con 0x7f645c103470 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6462800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c19a590 con 0x7f645c199790 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6461400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c103470 0x7f645c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6461e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c199790 0x7f645c199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6461400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c103470 0x7f645c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46440/0 (socket says 172.21.15.49:46440) 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6461400640 1 -- 172.21.15.49:0/3486251063 learned_addr learned my addr 172.21.15.49:0/3486251063 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6461e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c199790 0x7f645c199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46536/0 (socket says 172.21.15.49:46536) 2024-02-26T06:24:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.715+0000 7f6460a00640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645c108540 0x7f645c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6461400640 1 -- 172.21.15.49:0/3486251063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c199790 msgr2=0x7f645c199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6461400640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c199790 0x7f645c199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6461400640 1 -- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645c108540 msgr2=0x7f645c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6461400640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645c108540 0x7f645c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6461400640 1 -- 172.21.15.49:0/3486251063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6454009900 con 0x7f645c103470 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6461e00640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c199790 0x7f645c199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6461400640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c103470 0x7f645c19f680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f645000e9f0 tx=0x7f645000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f645a000640 1 -- 172.21.15.49:0/3486251063 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450009700 con 0x7f645c103470 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f645a000640 1 -- 172.21.15.49:0/3486251063 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f645000cd80 con 0x7f645c103470 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6462800640 1 -- 172.21.15.49:0/3486251063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c1a64e0 con 0x7f645c103470 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f645a000640 1 -- 172.21.15.49:0/3486251063 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450018920 con 0x7f645c103470 2024-02-26T06:24:55.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.716+0000 7f6462800640 1 -- 172.21.15.49:0/3486251063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c1a6a10 con 0x7f645c103470 2024-02-26T06:24:55.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.722+0000 7f645a000640 1 -- 172.21.15.49:0/3486251063 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6450018a80 con 0x7f645c103470 2024-02-26T06:24:55.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6466db4740 1 Processor -- start 2024-02-26T06:24:55.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6466db4740 1 -- start start 2024-02-26T06:24:55.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f645a000640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f64440a2330 0x7f64440a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6466db4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af91c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6466db4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af91d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6466db4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af925fe0 0x5567af9283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6466db4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af883120 con 0x5567af925fe0 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6466db4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af880650 con 0x5567af91cc70 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6466db4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af87be30 con 0x5567af91d950 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f645a000640 1 -- 172.21.15.49:0/3486251063 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6450014070 con 0x7f645c103470 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.723+0000 7f6460a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af91d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.724+0000 7f6460a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af91d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46450/0 (socket says 172.21.15.49:46450) 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.724+0000 7f6460a00640 1 -- 172.21.15.49:0/2909497139 learned_addr learned my addr 172.21.15.49:0/2909497139 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.724+0000 7f6461400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af91c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.724+0000 7f6461400640 1 -- 172.21.15.49:0/2909497139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 msgr2=0x5567af91d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.724+0000 7f6461400640 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af91d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.724+0000 7f6461400640 1 -- 172.21.15.49:0/2909497139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af925fe0 msgr2=0x5567af9283d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.724+0000 7f6461e00640 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af925fe0 0x5567af9283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.725+0000 7f6461400640 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af925fe0 0x5567af9283d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.725+0000 7f6461400640 1 -- 172.21.15.49:0/2909497139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567af835030 con 0x5567af91d950 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.725+0000 7f6460a00640 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af91d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.725+0000 7f6461e00640 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af925fe0 0x5567af9283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.725+0000 7f6461e00640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f64440a2330 0x7f64440a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.726+0000 7f6461400640 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af91c6a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f64500cbae0 tx=0x7f645009bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5415307125eb3082 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.726+0000 7f644e600640 1 -- 172.21.15.49:0/2909497139 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450059330 con 0x5567af91d950 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.726+0000 7f644e600640 1 -- 172.21.15.49:0/2909497139 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6450010b80 con 0x5567af91d950 2024-02-26T06:24:55.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.726+0000 7f644e600640 1 -- 172.21.15.49:0/2909497139 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450059790 con 0x5567af91d950 2024-02-26T06:24:55.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.728+0000 7f6466db4740 1 -- 172.21.15.49:0/2909497139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 msgr2=0x5567af91c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.728+0000 7f6466db4740 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af91c6a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f64500cbae0 tx=0x7f645009bd40 comp rx=0 tx=0).stop 2024-02-26T06:24:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.728+0000 7f6461e00640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f64440a2330 0x7f64440a47f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6454005cc0 tx=0x7f6454005c50 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.728+0000 7f6466db4740 1 -- 172.21.15.49:0/2909497139 shutdown_connections 2024-02-26T06:24:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.728+0000 7f6466db4740 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af91d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.728+0000 7f6466db4740 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af925fe0 0x5567af9283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.728+0000 7f6466db4740 1 --2- 172.21.15.49:0/2909497139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af91c6a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.728+0000 7f6466db4740 1 -- 172.21.15.49:0/2909497139 >> 172.21.15.49:0/2909497139 conn(0x5567af8db570 msgr2=0x5567af9176c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.729+0000 7f6466db4740 1 -- 172.21.15.49:0/2909497139 shutdown_connections 2024-02-26T06:24:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.729+0000 7f6466db4740 1 -- 172.21.15.49:0/2909497139 wait complete. 2024-02-26T06:24:55.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.729+0000 7f6466db4740 1 -- start start 2024-02-26T06:24:55.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.730+0000 7f6466db4740 1 Processor -- start 2024-02-26T06:24:55.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.730+0000 7f6466db4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af922f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.730+0000 7f6466db4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91d950 0x5567af923490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.730+0000 7f6466db4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af9239d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.731+0000 7f6466db4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af820e90 con 0x5567af91d950 2024-02-26T06:24:55.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.731+0000 7f6466db4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af820b30 con 0x5567af91cc70 2024-02-26T06:24:55.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.731+0000 7f6466db4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af5ea1c0 con 0x5567af925fe0 2024-02-26T06:24:55.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.732+0000 7f6466db4740 10 client.0 _ll_register_callbacks cb 0x5567af834930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:55.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.732+0000 7f6461e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af9239d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.732+0000 7f6461e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af9239d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46546/0 (socket says 172.21.15.49:46546) 2024-02-26T06:24:55.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.732+0000 7f6461e00640 1 -- 172.21.15.49:0/1253205221 learned_addr learned my addr 172.21.15.49:0/1253205221 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.733+0000 7f6461400640 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af922f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.734+0000 7f6461e00640 1 -- 172.21.15.49:0/1253205221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 msgr2=0x5567af922f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.734+0000 7f6461e00640 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af922f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.734+0000 7f6460a00640 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91d950 0x5567af923490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.735+0000 7f6461e00640 1 -- 172.21.15.49:0/1253205221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91d950 msgr2=0x5567af923490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.735+0000 7f6461e00640 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91d950 0x5567af923490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.735+0000 7f6461e00640 1 -- 172.21.15.49:0/1253205221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c1a6a10 con 0x5567af925fe0 2024-02-26T06:24:55.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.735+0000 7f6460a00640 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91d950 0x5567af923490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:55.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.736+0000 7f6461e00640 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af9239d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6454009cd0 tx=0x7f645402c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5ff65b82d1cf99 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.736+0000 7f6442a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:55.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.737+0000 7f6442a00640 1 -- 172.21.15.49:0/1253205221 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454003d80 con 0x5567af925fe0 2024-02-26T06:24:55.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.737+0000 7f6442a00640 1 -- 172.21.15.49:0/1253205221 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6454004040 con 0x5567af925fe0 2024-02-26T06:24:55.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.738+0000 7f6466db4740 1 -- 172.21.15.49:0/1253205221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567af57b7e0 con 0x5567af925fe0 2024-02-26T06:24:55.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.739+0000 7f6442a00640 1 -- 172.21.15.49:0/1253205221 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6454004240 con 0x5567af925fe0 2024-02-26T06:24:55.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.739+0000 7f6466db4740 10 client.25876 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:55.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.739+0000 7f6466db4740 1 -- 172.21.15.49:0/1253205221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5567af87a4b0 con 0x5567af925fe0 2024-02-26T06:24:55.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.740+0000 7f6442a00640 1 -- 172.21.15.49:0/1253205221 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5567af57b7e0 con 0x5567af925fe0 2024-02-26T06:24:55.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.742+0000 7f6442a00640 1 -- 172.21.15.49:0/1253205221 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6454050020 con 0x5567af925fe0 2024-02-26T06:24:55.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.742+0000 7f6466db4740 10 client.25876 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:55.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.742+0000 7f6466db4740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:55.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.742+0000 7f6466db4740 20 client.25876 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:55.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.742+0000 7f6466db4740 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5567af936090 0x5567af9384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.742+0000 7f6466db4740 4 client.25876 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:55.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.742+0000 7f6466db4740 1 -- 172.21.15.49:0/1253205221 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567af5b8720 con 0x5567af936090 2024-02-26T06:24:55.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.742+0000 7f6461400640 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5567af936090 0x5567af9384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.743+0000 7f6461400640 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5567af936090 0x5567af9384b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c440e1a1ee3a36fa server_cookie=303980505d48fd89 in_seq=0 out_seq=0 2024-02-26T06:24:55.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.744+0000 7f6442a00640 10 client.25876 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.744+0000 7f6442a00640 1 -- 172.21.15.49:0/1253205221 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567af5b8720 con 0x5567af936090 2024-02-26T06:24:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.744+0000 7f6442a00640 10 client.25876 handle_command_reply: tid=0 2024-02-26T06:24:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.744+0000 7f6466db4740 1 client.25876 shutdown 2024-02-26T06:24:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.745+0000 7f6466db4740 1 -- 172.21.15.49:0/1253205221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 msgr2=0x5567af9239d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.745+0000 7f6466db4740 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af9239d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6454009cd0 tx=0x7f645402c0f0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.745+0000 7f6466db4740 1 -- 172.21.15.49:0/1253205221 shutdown_connections 2024-02-26T06:24:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.745+0000 7f6466db4740 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5567af936090 0x5567af9384b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.745+0000 7f6466db4740 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af9239d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.745+0000 7f6466db4740 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91d950 0x5567af923490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.745+0000 7f6466db4740 1 --2- 172.21.15.49:0/1253205221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91cc70 0x5567af922f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.745+0000 7f6442a00640 0 client.25876 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:55.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.746+0000 7f6466db4740 1 -- 172.21.15.49:0/1253205221 >> 172.21.15.49:0/1253205221 conn(0x5567af8db570 msgr2=0x5567af919410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.746+0000 7f6466db4740 1 -- 172.21.15.49:0/1253205221 shutdown_connections 2024-02-26T06:24:55.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.746+0000 7f6466db4740 1 -- 172.21.15.49:0/1253205221 wait complete. 2024-02-26T06:24:55.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.747+0000 7f6466db4740 20 client.25876 trim_cache size 0 max 16384 2024-02-26T06:24:55.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.773+0000 7f6466db4740 1 Processor -- start 2024-02-26T06:24:55.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.773+0000 7f6466db4740 1 -- start start 2024-02-26T06:24:55.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.774+0000 7f6466db4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af925680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.774+0000 7f6466db4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af92cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.774+0000 7f6466db4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af925fe0 0x5567af92d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.774+0000 7f6466db4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af909300 con 0x5567af91cc70 2024-02-26T06:24:55.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.774+0000 7f6466db4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af91c4e0 con 0x5567af925fe0 2024-02-26T06:24:55.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.774+0000 7f6466db4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af90a860 con 0x5567af91d950 2024-02-26T06:24:55.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.774+0000 7f6460a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af92cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6460a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af92cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46556/0 (socket says 172.21.15.49:46556) 2024-02-26T06:24:55.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6460a00640 1 -- 172.21.15.49:0/4272966166 learned_addr learned my addr 172.21.15.49:0/4272966166 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6461400640 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af925680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6461e00640 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af925fe0 0x5567af92d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6460a00640 1 -- 172.21.15.49:0/4272966166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af925fe0 msgr2=0x5567af92d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6460a00640 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af925fe0 0x5567af92d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6460a00640 1 -- 172.21.15.49:0/4272966166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 msgr2=0x5567af925680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6460a00640 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af925680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.775+0000 7f6460a00640 1 -- 172.21.15.49:0/4272966166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567af816920 con 0x5567af91d950 2024-02-26T06:24:55.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.776+0000 7f6460a00640 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af92cf40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6448009c00 tx=0x7f6448010430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f561805dd2e30302 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.776+0000 7f644dc00640 1 -- 172.21.15.49:0/4272966166 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448010de0 con 0x5567af91d950 2024-02-26T06:24:55.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.776+0000 7f644dc00640 1 -- 172.21.15.49:0/4272966166 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64480125b0 con 0x5567af91d950 2024-02-26T06:24:55.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.776+0000 7f644dc00640 1 -- 172.21.15.49:0/4272966166 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644801bae0 con 0x5567af91d950 2024-02-26T06:24:55.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.777+0000 7f6466db4740 1 -- 172.21.15.49:0/4272966166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 msgr2=0x5567af92cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.777+0000 7f6466db4740 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af92cf40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6448009c00 tx=0x7f6448010430 comp rx=0 tx=0).stop 2024-02-26T06:24:55.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 -- 172.21.15.49:0/4272966166 shutdown_connections 2024-02-26T06:24:55.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af925fe0 0x5567af92d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af91d950 0x5567af92cf40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 --2- 172.21.15.49:0/4272966166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af925680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 -- 172.21.15.49:0/4272966166 >> 172.21.15.49:0/4272966166 conn(0x5567af8db570 msgr2=0x5567af921820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 -- 172.21.15.49:0/4272966166 shutdown_connections 2024-02-26T06:24:55.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 -- 172.21.15.49:0/4272966166 wait complete. 2024-02-26T06:24:55.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 -- start start 2024-02-26T06:24:55.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.778+0000 7f6466db4740 1 Processor -- start 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6466db4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af92cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6466db4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91d950 0x5567af92d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6466db4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af92a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6466db4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af92a760 con 0x5567af91cc70 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6466db4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af92a8d0 con 0x5567af91d950 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6466db4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567af92aa90 con 0x5567af925fe0 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6466db4740 10 client.0 _ll_register_callbacks cb 0x7f6450005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6461400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af92cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6460a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91d950 0x5567af92d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6461400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af92cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50556/0 (socket says 172.21.15.49:50556) 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6461400640 1 -- 172.21.15.49:0/3054326547 learned_addr learned my addr 172.21.15.49:0/3054326547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6460a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91d950 0x5567af92d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46464/0 (socket says 172.21.15.49:46464) 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.779+0000 7f6461e00640 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af92a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.780+0000 7f6460a00640 1 -- 172.21.15.49:0/3054326547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 msgr2=0x5567af92a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.780+0000 7f6460a00640 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af92a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.780+0000 7f6460a00640 1 -- 172.21.15.49:0/3054326547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 msgr2=0x5567af92cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.780+0000 7f6460a00640 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af92cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.780+0000 7f6460a00640 1 -- 172.21.15.49:0/3054326547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6448009850 con 0x5567af91d950 2024-02-26T06:24:55.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.780+0000 7f6461400640 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af92cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:55.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.780+0000 7f6460a00640 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91d950 0x5567af92d1b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5567af91ef80 tx=0x7f6448012930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559080caadae7a0c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.780+0000 7f6442a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:24:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.781+0000 7f6442a00640 1 -- 172.21.15.49:0/3054326547 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644801b7e0 con 0x5567af91d950 2024-02-26T06:24:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.781+0000 7f6442a00640 1 -- 172.21.15.49:0/3054326547 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f644801b940 con 0x5567af91d950 2024-02-26T06:24:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.781+0000 7f6466db4740 1 -- 172.21.15.49:0/3054326547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6450010b80 con 0x5567af91d950 2024-02-26T06:24:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.781+0000 7f6442a00640 1 -- 172.21.15.49:0/3054326547 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f644801bb00 con 0x5567af91d950 2024-02-26T06:24:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.781+0000 7f6466db4740 10 client.25835 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.781+0000 7f6466db4740 1 -- 172.21.15.49:0/3054326547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5567af83c400 con 0x5567af91d950 2024-02-26T06:24:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.782+0000 7f6442a00640 1 -- 172.21.15.49:0/3054326547 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6450010b80 con 0x5567af91d950 2024-02-26T06:24:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.782+0000 7f6442a00640 1 -- 172.21.15.49:0/3054326547 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6448004c00 con 0x5567af91d950 2024-02-26T06:24:55.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.783+0000 7f6466db4740 10 client.25835 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:55.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.783+0000 7f6466db4740 10 client.25835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:55.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.783+0000 7f6466db4740 20 client.25835 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:55.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.783+0000 7f6466db4740 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5567af98e7b0 0x5567af990bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.783+0000 7f6466db4740 4 client.25835 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:24:55.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.783+0000 7f6466db4740 1 -- 172.21.15.49:0/3054326547 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5567af607890 con 0x5567af98e7b0 2024-02-26T06:24:55.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.783+0000 7f6461400640 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5567af98e7b0 0x5567af990bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.783+0000 7f6442a00640 10 client.25835 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:55.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.784+0000 7f6461400640 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5567af98e7b0 0x5567af990bd0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb7c6e6e1ea8391 server_cookie=71a75aa3fdc3dc3b in_seq=0 out_seq=0 2024-02-26T06:24:55.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.784+0000 7f6442a00640 1 -- 172.21.15.49:0/3054326547 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5567af607890 con 0x5567af98e7b0 2024-02-26T06:24:55.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.784+0000 7f6442a00640 10 client.25835 handle_command_reply: tid=0 2024-02-26T06:24:55.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.784+0000 7f6466db4740 1 client.25835 shutdown 2024-02-26T06:24:55.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.785+0000 7f6466db4740 1 -- 172.21.15.49:0/3054326547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91d950 msgr2=0x5567af92d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.785+0000 7f6466db4740 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91d950 0x5567af92d1b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5567af91ef80 tx=0x7f6448012930 comp rx=0 tx=0).stop 2024-02-26T06:24:55.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6466db4740 1 -- 172.21.15.49:0/3054326547 shutdown_connections 2024-02-26T06:24:55.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6466db4740 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5567af98e7b0 0x5567af990bd0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6466db4740 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5567af91d950 0x5567af92d1b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6466db4740 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5567af925fe0 0x5567af92a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6466db4740 1 --2- 172.21.15.49:0/3054326547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5567af91cc70 0x5567af92cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6466db4740 1 -- 172.21.15.49:0/3054326547 >> 172.21.15.49:0/3054326547 conn(0x5567af8db570 msgr2=0x5567af90d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6442a00640 0 client.25835 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:55.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6466db4740 1 -- 172.21.15.49:0/3054326547 shutdown_connections 2024-02-26T06:24:55.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.786+0000 7f6466db4740 1 -- 172.21.15.49:0/3054326547 wait complete. 2024-02-26T06:24:55.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.787+0000 7f6466db4740 20 client.25835 trim_cache size 0 max 16384 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout: "9315427a-1ad5-4809-90e0-3151881c1f69": { 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "9315427a-1ad5-4809-90e0-3151881c1f69", 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:24:55.787 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 -- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f64440a2330 msgr2=0x7f64440a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f64440a2330 0x7f64440a47f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6454005cc0 tx=0x7f6454005c50 comp rx=0 tx=0).stop 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 -- 172.21.15.49:0/3486251063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c103470 msgr2=0x7f645c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c103470 0x7f645c19f680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f645000e9f0 tx=0x7f645000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 -- 172.21.15.49:0/3486251063 shutdown_connections 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f64440a2330 0x7f64440a47f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c199790 0x7f645c199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645c108540 0x7f645c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 --2- 172.21.15.49:0/3486251063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f645c103470 0x7f645c19f680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.788+0000 7f6462800640 1 -- 172.21.15.49:0/3486251063 >> 172.21.15.49:0/3486251063 conn(0x7f645c0fec90 msgr2=0x7f645c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.789+0000 7f6462800640 1 -- 172.21.15.49:0/3486251063 shutdown_connections 2024-02-26T06:24:55.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:24:55.789+0000 7f6462800640 1 -- 172.21.15.49:0/3486251063 wait complete. 2024-02-26T06:24:55.809 INFO:tasks.cephfs.filesystem:scrub status for tag:9315427a-1ad5-4809-90e0-3151881c1f69 - {'path': '/', 'tag': '9315427a-1ad5-4809-90e0-3151881c1f69', 'options': 'recursive,force'} 2024-02-26T06:24:55.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:55 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:24:55.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:55 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:55.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:55 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:56.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:55 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:24:56.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:55 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:55 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 65.0 s, 2838.6 tps, lat 35.196 ms stddev 41.981 2024-02-26T06:24:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: pgmap v49: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 10 MiB/s wr, 921 op/s 2024-02-26T06:24:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:56.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:56.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:24:56.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:56 smithi115 ceph-mon[36426]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: pgmap v49: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 10 MiB/s wr, 921 op/s 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:56.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:56 smithi049 ceph-mon[31696]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-02-26T06:24:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: pgmap v49: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 10 MiB/s wr, 921 op/s 2024-02-26T06:24:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:57.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:24:57.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:57.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:57.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:24:57.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:56 smithi080 ceph-mon[36206]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-02-26T06:24:58.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:24:58 smithi049 ceph-mon[31696]: pgmap v50: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-02-26T06:24:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:24:58 smithi080 ceph-mon[36206]: pgmap v50: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-02-26T06:24:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:24:58 smithi115 ceph-mon[36426]: pgmap v50: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-02-26T06:24:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.310+0000 7f57bc600640 1 -- 172.21.15.49:0/2677797500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b4111ff0 msgr2=0x7f57b41143e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.310+0000 7f57bc600640 1 --2- 172.21.15.49:0/2677797500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b4111ff0 0x7f57b41143e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009c10 tx=0x7f57ac02bee0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.311+0000 7f57bc600640 1 -- 172.21.15.49:0/2677797500 shutdown_connections 2024-02-26T06:24:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.311+0000 7f57bc600640 1 --2- 172.21.15.49:0/2677797500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b4111ff0 0x7f57b41143e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.311+0000 7f57bc600640 1 --2- 172.21.15.49:0/2677797500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b4075fd0 0x7f57b4111ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.311+0000 7f57bc600640 1 --2- 172.21.15.49:0/2677797500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57b40756b0 0x7f57b4075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.311+0000 7f57bc600640 1 -- 172.21.15.49:0/2677797500 >> 172.21.15.49:0/2677797500 conn(0x7f57b40fee40 msgr2=0x7f57b4101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.312+0000 7f57bc600640 1 -- 172.21.15.49:0/2677797500 shutdown_connections 2024-02-26T06:24:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.312+0000 7f57bc600640 1 -- 172.21.15.49:0/2677797500 wait complete. 2024-02-26T06:24:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.314+0000 7f57bc600640 1 Processor -- start 2024-02-26T06:24:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.314+0000 7f57bc600640 1 -- start start 2024-02-26T06:24:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.314+0000 7f57bc600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b40756b0 0x7f57b419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.315+0000 7f57bc600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57b4075fd0 0x7f57b419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.315+0000 7f57bc600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b4111ff0 0x7f57b41a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.315+0000 7f57b9c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b40756b0 0x7f57b419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.315+0000 7f57b9c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b40756b0 0x7f57b419dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46482/0 (socket says 172.21.15.49:46482) 2024-02-26T06:24:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.315+0000 7f57ba600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b4111ff0 0x7f57b41a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.315+0000 7f57b9200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57b4075fd0 0x7f57b419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.315+0000 7f57ba600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b4111ff0 0x7f57b41a2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46594/0 (socket says 172.21.15.49:46594) 2024-02-26T06:24:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.316+0000 7f57b9200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57b4075fd0 0x7f57b419e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50578/0 (socket says 172.21.15.49:50578) 2024-02-26T06:24:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.316+0000 7f57bc600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b41a2c20 con 0x7f57b4075fd0 2024-02-26T06:24:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.316+0000 7f57bc600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b41a2d90 con 0x7f57b40756b0 2024-02-26T06:24:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.316+0000 7f57bc600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b41a2f50 con 0x7f57b4111ff0 2024-02-26T06:24:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.316+0000 7f57b9c00640 1 -- 172.21.15.49:0/963996712 learned_addr learned my addr 172.21.15.49:0/963996712 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.317+0000 7f57ba600640 1 -- 172.21.15.49:0/963996712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b40756b0 msgr2=0x7f57b419dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.317+0000 7f57ba600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b40756b0 0x7f57b419dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.317+0000 7f57ba600640 1 -- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57b4075fd0 msgr2=0x7f57b419e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.317+0000 7f57ba600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57b4075fd0 0x7f57b419e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.317+0000 7f57ba600640 1 -- 172.21.15.49:0/963996712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8009560 con 0x7f57b4111ff0 2024-02-26T06:24:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.319+0000 7f57ba600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b4111ff0 0x7f57b41a2620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009be0 tx=0x7f57ac033980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.319+0000 7f57b2000640 1 -- 172.21.15.49:0/963996712 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac003b80 con 0x7f57b4111ff0 2024-02-26T06:24:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.320+0000 7f57bc600640 1 -- 172.21.15.49:0/963996712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac009840 con 0x7f57b4111ff0 2024-02-26T06:24:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.320+0000 7f57bc600640 1 -- 172.21.15.49:0/963996712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b41aa8d0 con 0x7f57b4111ff0 2024-02-26T06:24:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.320+0000 7f57b2000640 1 -- 172.21.15.49:0/963996712 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57ac003ce0 con 0x7f57b4111ff0 2024-02-26T06:24:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.320+0000 7f57b2000640 1 -- 172.21.15.49:0/963996712 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac03e910 con 0x7f57b4111ff0 2024-02-26T06:24:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.329+0000 7f57b2000640 1 -- 172.21.15.49:0/963996712 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f57ac03ea70 con 0x7f57b4111ff0 2024-02-26T06:24:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.329+0000 7f57bf476740 1 Processor -- start 2024-02-26T06:24:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.330+0000 7f57b2000640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f579c0a2350 0x7f579c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.331+0000 7f57b2000640 1 -- 172.21.15.49:0/963996712 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f57ac004080 con 0x7f57b4111ff0 2024-02-26T06:24:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.331+0000 7f57b9200640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f579c0a2350 0x7f579c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.331+0000 7f57bf476740 1 -- start start 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57bf476740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0ad3af0 0x558cc0ad2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57bf476740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0ad3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57bf476740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0adc300 0x558cc0ade6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57bf476740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0a8e5f0 con 0x558cc0ad2d80 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57bf476740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0a40890 con 0x558cc0adc300 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57bf476740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0a3ddc0 con 0x558cc0ad3af0 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57ba600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0adc300 0x558cc0ade6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57ba600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0adc300 0x558cc0ade6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46484/0 (socket says 172.21.15.49:46484) 2024-02-26T06:24:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.332+0000 7f57ba600640 1 -- 172.21.15.49:0/1527405844 learned_addr learned my addr 172.21.15.49:0/1527405844 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57ba600640 1 -- 172.21.15.49:0/1527405844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0ad3af0 msgr2=0x558cc0ad2840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:24:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57ba600640 1 --2- 172.21.15.49:0/1527405844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0ad3af0 0x558cc0ad2840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57ba600640 1 -- 172.21.15.49:0/1527405844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 msgr2=0x558cc0ad3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57ba600640 1 --2- 172.21.15.49:0/1527405844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0ad3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57ba600640 1 -- 172.21.15.49:0/1527405844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cc09baad0 con 0x558cc0adc300 2024-02-26T06:24:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57ba600640 1 --2- 172.21.15.49:0/1527405844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0adc300 0x558cc0ade6f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009c10 tx=0x7f57ac0be310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e83544efe1b07db3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57a6a00640 1 -- 172.21.15.49:0/1527405844 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0be7d0 con 0x558cc0adc300 2024-02-26T06:24:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57a6a00640 1 -- 172.21.15.49:0/1527405844 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57ac0c2920 con 0x558cc0adc300 2024-02-26T06:24:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.333+0000 7f57a6a00640 1 -- 172.21.15.49:0/1527405844 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac03ed20 con 0x558cc0adc300 2024-02-26T06:24:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.334+0000 7f57b9200640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f579c0a2350 0x7f579c0a4810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57b4078690 tx=0x7f57a000b390 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 -- 172.21.15.49:0/1527405844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0adc300 msgr2=0x558cc0ade6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 --2- 172.21.15.49:0/1527405844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0adc300 0x558cc0ade6f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009c10 tx=0x7f57ac0be310 comp rx=0 tx=0).stop 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 -- 172.21.15.49:0/1527405844 shutdown_connections 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 --2- 172.21.15.49:0/1527405844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0adc300 0x558cc0ade6f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 --2- 172.21.15.49:0/1527405844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0ad3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 --2- 172.21.15.49:0/1527405844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0ad3af0 0x558cc0ad2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 -- 172.21.15.49:0/1527405844 >> 172.21.15.49:0/1527405844 conn(0x558cc0a9a480 msgr2=0x558cc0acda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 -- 172.21.15.49:0/1527405844 shutdown_connections 2024-02-26T06:24:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.335+0000 7f57bf476740 1 -- 172.21.15.49:0/1527405844 wait complete. 2024-02-26T06:24:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.336+0000 7f57bf476740 1 -- start start 2024-02-26T06:24:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.336+0000 7f57bf476740 1 Processor -- start 2024-02-26T06:24:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.338+0000 7f57bf476740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0ad92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.338+0000 7f57bf476740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.338+0000 7f57bf476740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.339+0000 7f57bf476740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0a395a0 con 0x558cc0ad2d80 2024-02-26T06:24:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.339+0000 7f57bf476740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc09deb20 con 0x558cc0ad3af0 2024-02-26T06:24:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.339+0000 7f57bf476740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc09de7c0 con 0x558cc0adc300 2024-02-26T06:24:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.339+0000 7f57bf476740 10 client.0 _ll_register_callbacks cb 0x558cc09f2520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.340+0000 7f57b9200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.340+0000 7f57b9200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46500/0 (socket says 172.21.15.49:46500) 2024-02-26T06:24:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.340+0000 7f57b9200640 1 -- 172.21.15.49:0/2839281417 learned_addr learned my addr 172.21.15.49:0/2839281417 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.340+0000 7f57b9c00640 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0ad92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.341+0000 7f57ba600640 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.341+0000 7f57b9c00640 1 -- 172.21.15.49:0/2839281417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 msgr2=0x558cc0ad9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.341+0000 7f57b9c00640 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.341+0000 7f57b9c00640 1 -- 172.21.15.49:0/2839281417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 msgr2=0x558cc0ad97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.341+0000 7f57b9c00640 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.342+0000 7f57b9c00640 1 -- 172.21.15.49:0/2839281417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b41aa8d0 con 0x558cc0ad2d80 2024-02-26T06:24:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.342+0000 7f579b400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:24:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.343+0000 7f57b9c00640 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0ad92a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f57a8011750 tx=0x7f57a8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ef3f97e2b4ee3b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.343+0000 7f579b400640 1 -- 172.21.15.49:0/2839281417 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8004520 con 0x558cc0ad2d80 2024-02-26T06:24:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.343+0000 7f57bf476740 1 -- 172.21.15.49:0/2839281417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cc0a0bca0 con 0x558cc0ad2d80 2024-02-26T06:24:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.344+0000 7f579b400640 1 -- 172.21.15.49:0/2839281417 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f57a800f920 con 0x558cc0ad2d80 2024-02-26T06:24:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.344+0000 7f579b400640 1 -- 172.21.15.49:0/2839281417 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57a8004520 con 0x558cc0ad2d80 2024-02-26T06:24:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.345+0000 7f57bf476740 10 client.15675 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.345+0000 7f57bf476740 1 -- 172.21.15.49:0/2839281417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558cc0a37c20 con 0x558cc0ad2d80 2024-02-26T06:24:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.346+0000 7f579b400640 1 -- 172.21.15.49:0/2839281417 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558cc0a0bca0 con 0x558cc0ad2d80 2024-02-26T06:24:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.346+0000 7f579b400640 1 -- 172.21.15.49:0/2839281417 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f57a8024470 con 0x558cc0ad2d80 2024-02-26T06:24:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.347+0000 7f57bf476740 10 client.15675 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.347+0000 7f57bf476740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.347+0000 7f57bf476740 20 client.15675 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.347+0000 7f57bf476740 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558cc0afd1b0 0x558cc0aff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.347+0000 7f57bf476740 4 client.15675 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:24:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.347+0000 7f57bf476740 1 -- 172.21.15.49:0/2839281417 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cc0a2fcb0 con 0x558cc0afd1b0 2024-02-26T06:24:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.348+0000 7f57b9200640 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558cc0afd1b0 0x558cc0aff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.349+0000 7f57b9200640 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558cc0afd1b0 0x558cc0aff5f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=561688686a52469c server_cookie=daa9cfd61a74484a in_seq=0 out_seq=0 2024-02-26T06:24:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.349+0000 7f579b400640 10 client.15675 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.350+0000 7f579b400640 1 -- 172.21.15.49:0/2839281417 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558cc0a2fcb0 con 0x558cc0afd1b0 2024-02-26T06:24:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.350+0000 7f579b400640 10 client.15675 handle_command_reply: tid=0 2024-02-26T06:24:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.350+0000 7f57bf476740 1 client.15675 shutdown 2024-02-26T06:24:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.351+0000 7f57bf476740 1 -- 172.21.15.49:0/2839281417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 msgr2=0x558cc0ad92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.351+0000 7f57bf476740 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0ad92a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f57a8011750 tx=0x7f57a8011c10 comp rx=0 tx=0).stop 2024-02-26T06:24:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.352+0000 7f57bf476740 1 -- 172.21.15.49:0/2839281417 shutdown_connections 2024-02-26T06:24:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.352+0000 7f57bf476740 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558cc0afd1b0 0x558cc0aff5f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.352+0000 7f57bf476740 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.352+0000 7f57bf476740 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.352+0000 7f57bf476740 1 --2- 172.21.15.49:0/2839281417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0ad92a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.352+0000 7f57bf476740 1 -- 172.21.15.49:0/2839281417 >> 172.21.15.49:0/2839281417 conn(0x558cc0a9a480 msgr2=0x558cc0acf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.353+0000 7f579b400640 0 client.15675 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.354+0000 7f57bf476740 1 -- 172.21.15.49:0/2839281417 shutdown_connections 2024-02-26T06:24:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.354+0000 7f57bf476740 1 -- 172.21.15.49:0/2839281417 wait complete. 2024-02-26T06:24:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.354+0000 7f57bf476740 20 client.15675 trim_cache size 0 max 16384 2024-02-26T06:24:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.385+0000 7f57bf476740 1 Processor -- start 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57bf476740 1 -- start start 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57bf476740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0adb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57bf476740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57bf476740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57bf476740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0781e60 con 0x558cc0ad2d80 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57bf476740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0ad2050 con 0x558cc0ad3af0 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57bf476740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0ad4e70 con 0x558cc0adc300 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57ba600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57b9200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57b9200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46506/0 (socket says 172.21.15.49:46506) 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57b9200640 1 -- 172.21.15.49:0/2236450846 learned_addr learned my addr 172.21.15.49:0/2236450846 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57ba600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46610/0 (socket says 172.21.15.49:46610) 2024-02-26T06:24:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.386+0000 7f57b9c00640 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0adb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.387+0000 7f57ba600640 1 -- 172.21.15.49:0/2236450846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 msgr2=0x558cc0ad9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.387+0000 7f57ba600640 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.387+0000 7f57ba600640 1 -- 172.21.15.49:0/2236450846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 msgr2=0x558cc0adb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.387+0000 7f57ba600640 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0adb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.387+0000 7f57ba600640 1 -- 172.21.15.49:0/2236450846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cc0acdc80 con 0x558cc0adc300 2024-02-26T06:24:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.387+0000 7f57b9200640 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.387+0000 7f57b9c00640 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0adb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:24:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.387+0000 7f57ba600640 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad7b70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0c7130 tx=0x7f57ac004930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=195f93f3be56fc36 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.388+0000 7f57a6000640 1 -- 172.21.15.49:0/2236450846 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0c2920 con 0x558cc0adc300 2024-02-26T06:24:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.388+0000 7f57a6000640 1 -- 172.21.15.49:0/2236450846 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57ac0c6f70 con 0x558cc0adc300 2024-02-26T06:24:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.388+0000 7f57a6000640 1 -- 172.21.15.49:0/2236450846 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0eea00 con 0x558cc0adc300 2024-02-26T06:24:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.390+0000 7f57bf476740 1 -- 172.21.15.49:0/2236450846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 msgr2=0x558cc0ad7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.390+0000 7f57bf476740 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad7b70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0c7130 tx=0x7f57ac004930 comp rx=0 tx=0).stop 2024-02-26T06:24:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.390+0000 7f57bf476740 1 -- 172.21.15.49:0/2236450846 shutdown_connections 2024-02-26T06:24:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.390+0000 7f57bf476740 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad3af0 0x558cc0ad9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.390+0000 7f57bf476740 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0ad7b70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.390+0000 7f57bf476740 1 --2- 172.21.15.49:0/2236450846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad2d80 0x558cc0adb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.390+0000 7f57bf476740 1 -- 172.21.15.49:0/2236450846 >> 172.21.15.49:0/2236450846 conn(0x558cc0a9a480 msgr2=0x558cc0b03f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.390+0000 7f57bf476740 1 -- 172.21.15.49:0/2236450846 shutdown_connections 2024-02-26T06:24:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.391+0000 7f57bf476740 1 -- 172.21.15.49:0/2236450846 wait complete. 2024-02-26T06:24:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.391+0000 7f57bf476740 1 -- start start 2024-02-26T06:24:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.392+0000 7f57bf476740 1 Processor -- start 2024-02-26T06:24:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.392+0000 7f57bf476740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad2d80 0x558cc0af3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.392+0000 7f57bf476740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad3af0 0x558cc0af2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.392+0000 7f57bf476740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0af2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.392+0000 7f57bf476740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0af4830 con 0x558cc0ad3af0 2024-02-26T06:24:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.392+0000 7f57bf476740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0af2d80 con 0x558cc0ad2d80 2024-02-26T06:24:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.392+0000 7f57bf476740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0af2f40 con 0x558cc0adc300 2024-02-26T06:24:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.392+0000 7f57bf476740 10 client.0 _ll_register_callbacks cb 0x7f57a8009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:24:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57b9c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad2d80 0x558cc0af3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57ba600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0af2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57b9c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad2d80 0x558cc0af3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46512/0 (socket says 172.21.15.49:46512) 2024-02-26T06:24:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57b9c00640 1 -- 172.21.15.49:0/1093351405 learned_addr learned my addr 172.21.15.49:0/1093351405 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:24:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57b9200640 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad3af0 0x558cc0af2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57ba600640 1 -- 172.21.15.49:0/1093351405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad2d80 msgr2=0x558cc0af3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57ba600640 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad2d80 0x558cc0af3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57ba600640 1 -- 172.21.15.49:0/1093351405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad3af0 msgr2=0x558cc0af2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:24:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57ba600640 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad3af0 0x558cc0af2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.393+0000 7f57ba600640 1 -- 172.21.15.49:0/1093351405 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a0010cb0 con 0x558cc0adc300 2024-02-26T06:24:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.394+0000 7f57ba600640 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0af2840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0be990 tx=0x7f57ac0be230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f310395190d3a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.394+0000 7f579b400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:24:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.394+0000 7f579b400640 1 -- 172.21.15.49:0/1093351405 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac03ed90 con 0x558cc0adc300 2024-02-26T06:24:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.395+0000 7f579b400640 1 -- 172.21.15.49:0/1093351405 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f57ac0c6ae0 con 0x558cc0adc300 2024-02-26T06:24:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.395+0000 7f57bf476740 1 -- 172.21.15.49:0/1093351405 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cc0a5ed00 con 0x558cc0adc300 2024-02-26T06:24:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.395+0000 7f579b400640 1 -- 172.21.15.49:0/1093351405 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57ac03eef0 con 0x558cc0adc300 2024-02-26T06:24:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.395+0000 7f57bf476740 10 client.25897 fetch_fsmap learned FSMap version 29 2024-02-26T06:24:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.395+0000 7f57bf476740 1 -- 172.21.15.49:0/1093351405 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558cc09baad0 con 0x558cc0adc300 2024-02-26T06:24:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.396+0000 7f579b400640 1 -- 172.21.15.49:0/1093351405 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558cc0a5ed00 con 0x558cc0adc300 2024-02-26T06:24:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.396+0000 7f579b400640 1 -- 172.21.15.49:0/1093351405 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f57ac03c070 con 0x558cc0adc300 2024-02-26T06:24:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.396+0000 7f57bf476740 10 client.25897 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:24:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.396+0000 7f57bf476740 10 client.25897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:24:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.396+0000 7f57bf476740 20 client.25897 populate_metadata read hostname 'smithi049' 2024-02-26T06:24:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.396+0000 7f57bf476740 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558cc0b4e580 0x558cc0b4e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.396+0000 7f57bf476740 4 client.25897 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:24:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.396+0000 7f57bf476740 1 -- 172.21.15.49:0/1093351405 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558cc0739600 con 0x558cc0b4e580 2024-02-26T06:24:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.397+0000 7f57b9c00640 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558cc0b4e580 0x558cc0b4e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.398+0000 7f57b9c00640 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558cc0b4e580 0x558cc0b4e990 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45b31006c7283e6 server_cookie=fcba4cfbe3430a37 in_seq=0 out_seq=0 2024-02-26T06:24:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.398+0000 7f579b400640 10 client.25897 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.411+0000 7f579b400640 1 -- 172.21.15.49:0/1093351405 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558cc0739600 con 0x558cc0b4e580 2024-02-26T06:24:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.411+0000 7f579b400640 10 client.25897 handle_command_reply: tid=0 2024-02-26T06:24:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.411+0000 7f57bf476740 1 client.25897 shutdown 2024-02-26T06:24:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f57bf476740 1 -- 172.21.15.49:0/1093351405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 msgr2=0x558cc0af2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f57bf476740 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0af2840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0be990 tx=0x7f57ac0be230 comp rx=0 tx=0).stop 2024-02-26T06:24:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f57bf476740 1 -- 172.21.15.49:0/1093351405 shutdown_connections 2024-02-26T06:24:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f57bf476740 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558cc0b4e580 0x558cc0b4e990 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f57bf476740 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc0adc300 0x558cc0af2840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f57bf476740 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cc0ad3af0 0x558cc0af2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f57bf476740 1 --2- 172.21.15.49:0/1093351405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558cc0ad2d80 0x558cc0af3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f57bf476740 1 -- 172.21.15.49:0/1093351405 >> 172.21.15.49:0/1093351405 conn(0x558cc0a9a480 msgr2=0x558cc0ad78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.412+0000 7f579b400640 0 client.25897 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.413+0000 7f57bf476740 1 -- 172.21.15.49:0/1093351405 shutdown_connections 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.413+0000 7f57bf476740 1 -- 172.21.15.49:0/1093351405 wait complete. 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.413+0000 7f57bf476740 20 client.25897 trim_cache size 0 max 16384 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:24:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:24:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:24:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 -- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f579c0a2350 msgr2=0x7f579c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f579c0a2350 0x7f579c0a4810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57b4078690 tx=0x7f57a000b390 comp rx=0 tx=0).stop 2024-02-26T06:24:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 -- 172.21.15.49:0/963996712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b4111ff0 msgr2=0x7f57b41a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b4111ff0 0x7f57b41a2620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009be0 tx=0x7f57ac033980 comp rx=0 tx=0).stop 2024-02-26T06:24:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 -- 172.21.15.49:0/963996712 shutdown_connections 2024-02-26T06:24:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f579c0a2350 0x7f579c0a4810 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b4111ff0 0x7f57b41a2620 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57b4075fd0 0x7f57b419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 --2- 172.21.15.49:0/963996712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57b40756b0 0x7f57b419dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 -- 172.21.15.49:0/963996712 >> 172.21.15.49:0/963996712 conn(0x7f57b40fee40 msgr2=0x7f57b4101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 -- 172.21.15.49:0/963996712 shutdown_connections 2024-02-26T06:24:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:24:59.414+0000 7f57bc600640 1 -- 172.21.15.49:0/963996712 wait complete. 2024-02-26T06:25:00.881 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:00 smithi080 ceph-mon[36206]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.9 MiB/s wr, 842 op/s 2024-02-26T06:25:00.882 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:00 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:25:00.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:00 smithi049 ceph-mon[31696]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.9 MiB/s wr, 842 op/s 2024-02-26T06:25:00.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:00 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:25:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:00 smithi115 ceph-mon[36426]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.9 MiB/s wr, 842 op/s 2024-02-26T06:25:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:00 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:25:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 70.0 s, 2329.0 tps, lat 41.664 ms stddev 68.394 2024-02-26T06:25:01.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:01 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:25:01.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:01 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:25:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:01 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:25:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:01 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:25:02.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:01 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:25:02.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:01 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:25:02.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:02 smithi049 ceph-mon[31696]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-02-26T06:25:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:02 smithi080 ceph-mon[36206]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-02-26T06:25:03.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:02 smithi115 ceph-mon[36426]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-02-26T06:25:04.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:04 smithi049 ceph-mon[31696]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-02-26T06:25:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:04 smithi080 ceph-mon[36206]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-02-26T06:25:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:04 smithi115 ceph-mon[36426]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-02-26T06:25:06.306 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 75.0 s, 214.2 tps, lat 457.779 ms stddev 358.263 2024-02-26T06:25:06.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:06 smithi049 ceph-mon[31696]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.4 MiB/s wr, 929 op/s 2024-02-26T06:25:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:06 smithi080 ceph-mon[36206]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.4 MiB/s wr, 929 op/s 2024-02-26T06:25:07.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:06 smithi115 ceph-mon[36426]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.4 MiB/s wr, 929 op/s 2024-02-26T06:25:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:08 smithi049 ceph-mon[31696]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.5 MiB/s wr, 624 op/s 2024-02-26T06:25:09.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:08 smithi080 ceph-mon[36206]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.5 MiB/s wr, 624 op/s 2024-02-26T06:25:09.259 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:08 smithi115 ceph-mon[36426]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.5 MiB/s wr, 624 op/s 2024-02-26T06:25:10.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:10 smithi049 ceph-mon[31696]: pgmap v56: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 28 MiB/s wr, 1.38k op/s 2024-02-26T06:25:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:10 smithi080 ceph-mon[36206]: pgmap v56: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 28 MiB/s wr, 1.38k op/s 2024-02-26T06:25:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:10 smithi115 ceph-mon[36426]: pgmap v56: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 28 MiB/s wr, 1.38k op/s 2024-02-26T06:25:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 80.0 s, 2657.8 tps, lat 39.512 ms stddev 58.084 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.588+0000 7f048c400640 1 -- 172.21.15.49:0/3049011001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 msgr2=0x7f0484073990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.588+0000 7f048c400640 1 --2- 172.21.15.49:0/3049011001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 0x7f0484073990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0478009900 tx=0x7f047802bcd0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.589+0000 7f048c400640 1 -- 172.21.15.49:0/3049011001 shutdown_connections 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.589+0000 7f048c400640 1 --2- 172.21.15.49:0/3049011001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0484073ed0 0x7f048410d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.589+0000 7f048c400640 1 --2- 172.21.15.49:0/3049011001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f048410d990 0x7f048410fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.589+0000 7f048c400640 1 --2- 172.21.15.49:0/3049011001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 0x7f0484073990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.589+0000 7f048c400640 1 -- 172.21.15.49:0/3049011001 >> 172.21.15.49:0/3049011001 conn(0x7f04840fcc40 msgr2=0x7f04840ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.589+0000 7f048c400640 1 -- 172.21.15.49:0/3049011001 shutdown_connections 2024-02-26T06:25:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.589+0000 7f048c400640 1 -- 172.21.15.49:0/3049011001 wait complete. 2024-02-26T06:25:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.590+0000 7f048c400640 1 Processor -- start 2024-02-26T06:25:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.591+0000 7f048c400640 1 -- start start 2024-02-26T06:25:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.592+0000 7f048c400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 0x7f048410d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.592+0000 7f048c400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0484073ed0 0x7f048410a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f048c400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f048410d990 0x7f048410a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f048c400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048410acf0 con 0x7f0484073ed0 2024-02-26T06:25:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f048c400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048410ae60 con 0x7f048410d990 2024-02-26T06:25:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f048c400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048410b020 con 0x7f04840735b0 2024-02-26T06:25:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 0x7f048410d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 0x7f048410d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41984/0 (socket says 172.21.15.49:41984) 2024-02-26T06:25:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489a00640 1 -- 172.21.15.49:0/2389530973 learned_addr learned my addr 172.21.15.49:0/2389530973 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489000640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0484073ed0 0x7f048410a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f048a400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f048410d990 0x7f048410a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489a00640 1 -- 172.21.15.49:0/2389530973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f048410d990 msgr2=0x7f048410a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489a00640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f048410d990 0x7f048410a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489a00640 1 -- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0484073ed0 msgr2=0x7f048410a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489a00640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0484073ed0 0x7f048410a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.593+0000 7f0489a00640 1 -- 172.21.15.49:0/2389530973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470009490 con 0x7f04840735b0 2024-02-26T06:25:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.594+0000 7f0489a00640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 0x7f048410d110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0478000c00 tx=0x7f047802c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.594+0000 7f0482000640 1 -- 172.21.15.49:0/2389530973 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047802dbe0 con 0x7f04840735b0 2024-02-26T06:25:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.594+0000 7f0482000640 1 -- 172.21.15.49:0/2389530973 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f047802dd40 con 0x7f04840735b0 2024-02-26T06:25:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.594+0000 7f0482000640 1 -- 172.21.15.49:0/2389530973 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478003b20 con 0x7f04840735b0 2024-02-26T06:25:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.595+0000 7f048c400640 1 -- 172.21.15.49:0/2389530973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0478009560 con 0x7f04840735b0 2024-02-26T06:25:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.595+0000 7f048c400640 1 -- 172.21.15.49:0/2389530973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04841aa760 con 0x7f04840735b0 2024-02-26T06:25:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.599+0000 7f048f3bf740 1 Processor -- start 2024-02-26T06:25:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.599+0000 7f048f3bf740 1 -- start start 2024-02-26T06:25:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c207aaf0 0x5647c2079840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c207a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f048f3bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c20355f0 con 0x5647c2079d80 2024-02-26T06:25:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f048f3bf740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1fe7890 con 0x5647c2083300 2024-02-26T06:25:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f048f3bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1fe4dc0 con 0x5647c207aaf0 2024-02-26T06:25:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f0482000640 1 -- 172.21.15.49:0/2389530973 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0478003c80 con 0x7f04840735b0 2024-02-26T06:25:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f0489a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c207a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f048a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c207aaf0 0x5647c2079840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f0489a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c207a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53020/0 (socket says 172.21.15.49:53020) 2024-02-26T06:25:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.600+0000 7f0489a00640 1 -- 172.21.15.49:0/2875565679 learned_addr learned my addr 172.21.15.49:0/2875565679 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.601+0000 7f0489000640 1 --2- 172.21.15.49:0/2875565679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.601+0000 7f0489a00640 1 -- 172.21.15.49:0/2875565679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c207aaf0 msgr2=0x5647c2079840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.601+0000 7f0489a00640 1 --2- 172.21.15.49:0/2875565679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c207aaf0 0x5647c2079840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.601+0000 7f0489a00640 1 -- 172.21.15.49:0/2875565679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 msgr2=0x5647c20856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.601+0000 7f0489a00640 1 --2- 172.21.15.49:0/2875565679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.601+0000 7f0489a00640 1 -- 172.21.15.49:0/2875565679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c1f61ad0 con 0x5647c2079d80 2024-02-26T06:25:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.603+0000 7f0482000640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f046c0a2350 0x7f046c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.603+0000 7f0482000640 1 -- 172.21.15.49:0/2389530973 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f04780f0700 con 0x7f04840735b0 2024-02-26T06:25:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.604+0000 7f048a400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f046c0a2350 0x7f046c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.606+0000 7f0489a00640 1 --2- 172.21.15.49:0/2875565679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c207a200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04780ede50 tx=0x7f04780f6b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96f26735ad1f978 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.606+0000 7f0476a00640 1 -- 172.21.15.49:0/2875565679 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04780b94e0 con 0x5647c2079d80 2024-02-26T06:25:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.607+0000 7f0476a00640 1 -- 172.21.15.49:0/2875565679 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04780f66d0 con 0x5647c2079d80 2024-02-26T06:25:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.607+0000 7f0476a00640 1 -- 172.21.15.49:0/2875565679 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04780bcd80 con 0x5647c2079d80 2024-02-26T06:25:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.608+0000 7f048f3bf740 1 -- 172.21.15.49:0/2875565679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 msgr2=0x5647c207a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.608+0000 7f048f3bf740 1 --2- 172.21.15.49:0/2875565679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c207a200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04780ede50 tx=0x7f04780f6b20 comp rx=0 tx=0).stop 2024-02-26T06:25:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.608+0000 7f048a400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f046c0a2350 0x7f046c0a4810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f047c005fd0 tx=0x7f047c009500 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.609+0000 7f048f3bf740 1 -- 172.21.15.49:0/2875565679 shutdown_connections 2024-02-26T06:25:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.609+0000 7f048f3bf740 1 --2- 172.21.15.49:0/2875565679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.609+0000 7f048f3bf740 1 --2- 172.21.15.49:0/2875565679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c207a200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.609+0000 7f048f3bf740 1 --2- 172.21.15.49:0/2875565679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c207aaf0 0x5647c2079840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.609+0000 7f048f3bf740 1 -- 172.21.15.49:0/2875565679 >> 172.21.15.49:0/2875565679 conn(0x5647c2041480 msgr2=0x5647c2074a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.610+0000 7f048f3bf740 1 -- 172.21.15.49:0/2875565679 shutdown_connections 2024-02-26T06:25:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.610+0000 7f048f3bf740 1 -- 172.21.15.49:0/2875565679 wait complete. 2024-02-26T06:25:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.611+0000 7f048f3bf740 1 -- start start 2024-02-26T06:25:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.611+0000 7f048f3bf740 1 Processor -- start 2024-02-26T06:25:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2080330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c207aaf0 0x5647c2080870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2083300 0x5647c2080db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f048f3bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1fe05a0 con 0x5647c2083300 2024-02-26T06:25:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f048f3bf740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1f85b20 con 0x5647c207aaf0 2024-02-26T06:25:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f048f3bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1f857c0 con 0x5647c2079d80 2024-02-26T06:25:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f048f3bf740 10 client.0 _ll_register_callbacks cb 0x5647c1f99520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2080330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2080330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42000/0 (socket says 172.21.15.49:42000) 2024-02-26T06:25:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489a00640 1 -- 172.21.15.49:0/85243646 learned_addr learned my addr 172.21.15.49:0/85243646 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489000640 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c207aaf0 0x5647c2080870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489000640 1 -- 172.21.15.49:0/85243646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 msgr2=0x5647c2080330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489000640 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2080330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f048a400640 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2083300 0x5647c2080db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489000640 1 -- 172.21.15.49:0/85243646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2083300 msgr2=0x5647c2080db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489000640 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2083300 0x5647c2080db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.612+0000 7f0489000640 1 -- 172.21.15.49:0/85243646 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04841aa760 con 0x5647c207aaf0 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.613+0000 7f0489000640 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c207aaf0 0x5647c2080870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5647c207a820 tx=0x7f0470013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3666a0f83e0a6c6f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.613+0000 7f046b400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.613+0000 7f046b400640 1 -- 172.21.15.49:0/85243646 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470011da0 con 0x5647c207aaf0 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.613+0000 7f046b400640 1 -- 172.21.15.49:0/85243646 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f0470004020 con 0x5647c207aaf0 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.613+0000 7f048f3bf740 1 -- 172.21.15.49:0/85243646 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647c1fb2ca0 con 0x5647c207aaf0 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.614+0000 7f046b400640 1 -- 172.21.15.49:0/85243646 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04700041e0 con 0x5647c207aaf0 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.614+0000 7f048f3bf740 10 client.25865 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.614+0000 7f048f3bf740 1 -- 172.21.15.49:0/85243646 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5647c1fdec20 con 0x5647c207aaf0 2024-02-26T06:25:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.615+0000 7f046b400640 1 -- 172.21.15.49:0/85243646 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5647c1fb2ca0 con 0x5647c207aaf0 2024-02-26T06:25:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.615+0000 7f046b400640 1 -- 172.21.15.49:0/85243646 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f047000f520 con 0x5647c207aaf0 2024-02-26T06:25:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.615+0000 7f048f3bf740 10 client.25865 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.615+0000 7f048f3bf740 10 client.25865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.615+0000 7f048f3bf740 20 client.25865 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.615+0000 7f048f3bf740 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647c2093230 0x5647c2095670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.615+0000 7f048f3bf740 4 client.25865 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.615+0000 7f048f3bf740 1 -- 172.21.15.49:0/85243646 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647c1fd6cb0 con 0x5647c2093230 2024-02-26T06:25:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.617+0000 7f0489a00640 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647c2093230 0x5647c2095670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.618+0000 7f0489a00640 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647c2093230 0x5647c2095670 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df23d8e1d861710 server_cookie=ed95cfc9d6a6343b in_seq=0 out_seq=0 2024-02-26T06:25:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.618+0000 7f046b400640 10 client.25865 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.620+0000 7f046b400640 1 -- 172.21.15.49:0/85243646 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647c1fd6cb0 con 0x5647c2093230 2024-02-26T06:25:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.620+0000 7f046b400640 10 client.25865 handle_command_reply: tid=0 2024-02-26T06:25:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.620+0000 7f048f3bf740 1 client.25865 shutdown 2024-02-26T06:25:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.620+0000 7f048f3bf740 1 -- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c207aaf0 msgr2=0x5647c2080870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.620+0000 7f048f3bf740 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c207aaf0 0x5647c2080870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5647c207a820 tx=0x7f0470013b70 comp rx=0 tx=0).stop 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 1 -- 172.21.15.49:0/85243646 shutdown_connections 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647c2093230 0x5647c2095670 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c207aaf0 0x5647c2080870 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5647c207a820 tx=0x7f0470013b70 comp rx=0 tx=0).stop 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2083300 0x5647c2080db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 1 --2- 172.21.15.49:0/85243646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2080330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 1 -- 172.21.15.49:0/85243646 >> 172.21.15.49:0/85243646 conn(0x5647c2041480 msgr2=0x5647c20767c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f046b400640 0 client.25865 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 1 -- 172.21.15.49:0/85243646 shutdown_connections 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 1 -- 172.21.15.49:0/85243646 wait complete. 2024-02-26T06:25:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.621+0000 7f048f3bf740 20 client.25865 trim_cache size 0 max 16384 2024-02-26T06:25:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.645+0000 7f048f3bf740 1 Processor -- start 2024-02-26T06:25:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.646+0000 7f048f3bf740 1 -- start start 2024-02-26T06:25:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.646+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c2082a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.646+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c207ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.646+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2074cb0 0x5647c207f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.646+0000 7f048f3bf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1d28e60 con 0x5647c2079d80 2024-02-26T06:25:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.646+0000 7f048f3bf740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1fb2ca0 con 0x5647c2083300 2024-02-26T06:25:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.646+0000 7f048f3bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c2091950 con 0x5647c2074cb0 2024-02-26T06:25:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.647+0000 7f0489a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c2082a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.648+0000 7f0489a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c2082a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53040/0 (socket says 172.21.15.49:53040) 2024-02-26T06:25:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.648+0000 7f0489a00640 1 -- 172.21.15.49:0/703557713 learned_addr learned my addr 172.21.15.49:0/703557713 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.649+0000 7f0489a00640 1 -- 172.21.15.49:0/703557713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2074cb0 msgr2=0x5647c207f160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:25:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.649+0000 7f0489a00640 1 --2- 172.21.15.49:0/703557713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2074cb0 0x5647c207f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.649+0000 7f0489a00640 1 -- 172.21.15.49:0/703557713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 msgr2=0x5647c207ec20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:25:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.649+0000 7f0489a00640 1 --2- 172.21.15.49:0/703557713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c207ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.649+0000 7f0489a00640 1 -- 172.21.15.49:0/703557713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c1f7b500 con 0x5647c2079d80 2024-02-26T06:25:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.650+0000 7f0489a00640 1 --2- 172.21.15.49:0/703557713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c2082a60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f047802dfd0 tx=0x7f047802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6c5499e49da12d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.651+0000 7f0476000640 1 -- 172.21.15.49:0/703557713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04780bd010 con 0x5647c2079d80 2024-02-26T06:25:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.652+0000 7f0476000640 1 -- 172.21.15.49:0/703557713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04780bd010 con 0x5647c2079d80 2024-02-26T06:25:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.652+0000 7f0476000640 1 -- 172.21.15.49:0/703557713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04780bd240 con 0x5647c2079d80 2024-02-26T06:25:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.653+0000 7f048f3bf740 1 -- 172.21.15.49:0/703557713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 msgr2=0x5647c2082a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.653+0000 7f048f3bf740 1 --2- 172.21.15.49:0/703557713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c2082a60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f047802dfd0 tx=0x7f047802bcd0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.654+0000 7f048f3bf740 1 -- 172.21.15.49:0/703557713 shutdown_connections 2024-02-26T06:25:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.654+0000 7f048f3bf740 1 --2- 172.21.15.49:0/703557713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c207ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.654+0000 7f048f3bf740 1 --2- 172.21.15.49:0/703557713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2074cb0 0x5647c207f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.654+0000 7f048f3bf740 1 --2- 172.21.15.49:0/703557713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2079d80 0x5647c2082a60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.655+0000 7f048f3bf740 1 -- 172.21.15.49:0/703557713 >> 172.21.15.49:0/703557713 conn(0x5647c2041480 msgr2=0x5647c20745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.655+0000 7f048f3bf740 1 -- 172.21.15.49:0/703557713 shutdown_connections 2024-02-26T06:25:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.655+0000 7f048f3bf740 1 -- 172.21.15.49:0/703557713 wait complete. 2024-02-26T06:25:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.656+0000 7f048f3bf740 1 -- start start 2024-02-26T06:25:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.656+0000 7f048f3bf740 1 Processor -- start 2024-02-26T06:25:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.659+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2074cb0 0x5647c2088a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.660+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2088fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.660+0000 7f048f3bf740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20894e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.661+0000 7f0489000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2088fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.661+0000 7f0489000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2088fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42014/0 (socket says 172.21.15.49:42014) 2024-02-26T06:25:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.661+0000 7f048a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20894e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.662+0000 7f048a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20894e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49510/0 (socket says 172.21.15.49:49510) 2024-02-26T06:25:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.662+0000 7f0489a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2074cb0 0x5647c2088a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.662+0000 7f0489a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2074cb0 0x5647c2088a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53046/0 (socket says 172.21.15.49:53046) 2024-02-26T06:25:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.662+0000 7f0489000640 1 -- 172.21.15.49:0/220132487 learned_addr learned my addr 172.21.15.49:0/220132487 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.662+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c20e3730 con 0x5647c2074cb0 2024-02-26T06:25:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.662+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c20e38a0 con 0x5647c2083300 2024-02-26T06:25:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.662+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c20e3a60 con 0x5647c2079d80 2024-02-26T06:25:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.662+0000 7f048f3bf740 10 client.0 _ll_register_callbacks cb 0x7f04700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.663+0000 7f048a400640 1 -- 172.21.15.49:0/220132487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 msgr2=0x5647c2088fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.663+0000 7f048a400640 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2088fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.663+0000 7f048a400640 1 -- 172.21.15.49:0/220132487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2074cb0 msgr2=0x5647c2088a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.663+0000 7f048a400640 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2074cb0 0x5647c2088a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.663+0000 7f048a400640 1 -- 172.21.15.49:0/220132487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647c1f61ad0 con 0x5647c2083300 2024-02-26T06:25:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.665+0000 7f048a400640 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20894e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f047c004410 tx=0x7f047c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfed674f368101aa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.665+0000 7f046b400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:25:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.666+0000 7f046b400640 1 -- 172.21.15.49:0/220132487 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c023070 con 0x5647c2083300 2024-02-26T06:25:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.666+0000 7f046b400640 1 -- 172.21.15.49:0/220132487 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f047c011500 con 0x5647c2083300 2024-02-26T06:25:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.666+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647c2005d00 con 0x5647c2083300 2024-02-26T06:25:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.667+0000 7f046b400640 1 -- 172.21.15.49:0/220132487 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f047c016070 con 0x5647c2083300 2024-02-26T06:25:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.667+0000 7f048f3bf740 10 client.25868 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.667+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5647c1fa1290 con 0x5647c2083300 2024-02-26T06:25:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.667+0000 7f046b400640 1 -- 172.21.15.49:0/220132487 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5647c2005d00 con 0x5647c2083300 2024-02-26T06:25:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.668+0000 7f046b400640 1 -- 172.21.15.49:0/220132487 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f047c02f070 con 0x5647c2083300 2024-02-26T06:25:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.668+0000 7f048f3bf740 10 client.25868 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.668+0000 7f048f3bf740 10 client.25868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.668+0000 7f048f3bf740 20 client.25868 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.668+0000 7f048f3bf740 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647c20e52f0 0x5647c20fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.668+0000 7f048f3bf740 4 client.25868 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:25:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.669+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647c1ce0600 con 0x5647c20e52f0 2024-02-26T06:25:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.669+0000 7f0489a00640 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647c20e52f0 0x5647c20fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.670+0000 7f0489a00640 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647c20e52f0 0x5647c20fe770 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c67a89fc17862 server_cookie=dc7659bcca251a43 in_seq=0 out_seq=0 2024-02-26T06:25:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.670+0000 7f046b400640 10 client.25868 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.681+0000 7f046b400640 1 -- 172.21.15.49:0/220132487 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647c1ce0600 con 0x5647c20e52f0 2024-02-26T06:25:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.681+0000 7f046b400640 10 client.25868 handle_command_reply: tid=0 2024-02-26T06:25:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.681+0000 7f048f3bf740 1 client.25868 shutdown 2024-02-26T06:25:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.682+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 msgr2=0x5647c20894e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.682+0000 7f048f3bf740 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20894e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f047c004410 tx=0x7f047c004ab0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 shutdown_connections 2024-02-26T06:25:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f048f3bf740 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647c20e52f0 0x5647c20fe770 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f048f3bf740 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647c2083300 0x5647c20894e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f048f3bf740 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647c2079d80 0x5647c2088fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f048f3bf740 1 --2- 172.21.15.49:0/220132487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5647c2074cb0 0x5647c2088a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 >> 172.21.15.49:0/220132487 conn(0x5647c2041480 msgr2=0x5647c2073480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f046b400640 0 client.25868 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 shutdown_connections 2024-02-26T06:25:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.683+0000 7f048f3bf740 1 -- 172.21.15.49:0/220132487 wait complete. 2024-02-26T06:25:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.684+0000 7f048f3bf740 20 client.25868 trim_cache size 0 max 16384 2024-02-26T06:25:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:25:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:25:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:25:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:25:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 -- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f046c0a2350 msgr2=0x7f046c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f046c0a2350 0x7f046c0a4810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f047c005fd0 tx=0x7f047c009500 comp rx=0 tx=0).stop 2024-02-26T06:25:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 -- 172.21.15.49:0/2389530973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 msgr2=0x7f048410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 0x7f048410d110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0478000c00 tx=0x7f047802c6d0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 -- 172.21.15.49:0/2389530973 shutdown_connections 2024-02-26T06:25:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f046c0a2350 0x7f046c0a4810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f048410d990 0x7f048410a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0484073ed0 0x7f048410a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 --2- 172.21.15.49:0/2389530973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04840735b0 0x7f048410d110 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0478000c00 tx=0x7f047802c6d0 comp rx=0 tx=0).stop 2024-02-26T06:25:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.685+0000 7f048c400640 1 -- 172.21.15.49:0/2389530973 >> 172.21.15.49:0/2389530973 conn(0x7f04840fcc40 msgr2=0x7f04840fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.686+0000 7f048c400640 1 -- 172.21.15.49:0/2389530973 shutdown_connections 2024-02-26T06:25:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:12.686+0000 7f048c400640 1 -- 172.21.15.49:0/2389530973 wait complete. 2024-02-26T06:25:12.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:12 smithi049 ceph-mon[31696]: pgmap v57: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-02-26T06:25:13.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:12 smithi080 ceph-mon[36206]: pgmap v57: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-02-26T06:25:13.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:12 smithi115 ceph-mon[36426]: pgmap v57: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-02-26T06:25:14.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:14 smithi049 ceph-mon[31696]: pgmap v58: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 MiB/s wr, 1.35k op/s 2024-02-26T06:25:15.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:14 smithi080 ceph-mon[36206]: pgmap v58: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 MiB/s wr, 1.35k op/s 2024-02-26T06:25:15.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:14 smithi115 ceph-mon[36426]: pgmap v58: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 MiB/s wr, 1.35k op/s 2024-02-26T06:25:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 85.0 s, 2630.6 tps, lat 37.836 ms stddev 43.744 2024-02-26T06:25:16.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:16 smithi049 ceph-mon[31696]: pgmap v59: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.65k op/s 2024-02-26T06:25:17.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:16 smithi080 ceph-mon[36206]: pgmap v59: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.65k op/s 2024-02-26T06:25:17.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:16 smithi115 ceph-mon[36426]: pgmap v59: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.65k op/s 2024-02-26T06:25:18.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:18 smithi049 ceph-mon[31696]: pgmap v60: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 1.42k op/s 2024-02-26T06:25:19.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:18 smithi080 ceph-mon[36206]: pgmap v60: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 1.42k op/s 2024-02-26T06:25:19.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:18 smithi115 ceph-mon[36426]: pgmap v60: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 1.42k op/s 2024-02-26T06:25:20.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:20 smithi049 ceph-mon[31696]: pgmap v61: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.66k op/s 2024-02-26T06:25:21.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:20 smithi080 ceph-mon[36206]: pgmap v61: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.66k op/s 2024-02-26T06:25:21.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:20 smithi115 ceph-mon[36426]: pgmap v61: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.66k op/s 2024-02-26T06:25:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 90.0 s, 2685.6 tps, lat 37.274 ms stddev 42.289 2024-02-26T06:25:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:22 smithi080 ceph-mon[36206]: pgmap v62: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 1.04k op/s 2024-02-26T06:25:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:22 smithi115 ceph-mon[36426]: pgmap v62: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 1.04k op/s 2024-02-26T06:25:23.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:22 smithi049 ceph-mon[31696]: pgmap v62: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 1.04k op/s 2024-02-26T06:25:24.971 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:24 smithi049 ceph-mon[31696]: pgmap v63: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.9 MiB/s wr, 676 op/s 2024-02-26T06:25:25.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:24 smithi080 ceph-mon[36206]: pgmap v63: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.9 MiB/s wr, 676 op/s 2024-02-26T06:25:25.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:24 smithi115 ceph-mon[36426]: pgmap v63: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.9 MiB/s wr, 676 op/s 2024-02-26T06:25:25.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:25:25.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:25:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.880+0000 7f14b4800640 1 -- 172.21.15.49:0/79045483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac106500 msgr2=0x7f14ac10e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.880+0000 7f14b4800640 1 --2- 172.21.15.49:0/79045483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac106500 0x7f14ac10e080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009cd0 tx=0x7f14a802c0e0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.881+0000 7f14b4800640 1 -- 172.21.15.49:0/79045483 shutdown_connections 2024-02-26T06:25:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.881+0000 7f14b4800640 1 --2- 172.21.15.49:0/79045483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac106500 0x7f14ac10e080 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.881+0000 7f14b4800640 1 --2- 172.21.15.49:0/79045483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14ac100240 0x7f14ac105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.881+0000 7f14b4800640 1 --2- 172.21.15.49:0/79045483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14ac0ff890 0x7f14ac0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.881+0000 7f14b4800640 1 -- 172.21.15.49:0/79045483 >> 172.21.15.49:0/79045483 conn(0x7f14ac0765a0 msgr2=0x7f14ac0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.881+0000 7f14b4800640 1 -- 172.21.15.49:0/79045483 shutdown_connections 2024-02-26T06:25:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.882+0000 7f14b4800640 1 -- 172.21.15.49:0/79045483 wait complete. 2024-02-26T06:25:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.883+0000 7f14b4800640 1 Processor -- start 2024-02-26T06:25:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.884+0000 7f14b4800640 1 -- start start 2024-02-26T06:25:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.884+0000 7f14b4800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14ac0ff890 0x7f14ac19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.884+0000 7f14b4800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14ac106500 0x7f14ac19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.885+0000 7f14b4800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac1997a0 0x7f14ac199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.885+0000 7f14b4800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac19a250 con 0x7f14ac106500 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.885+0000 7f14b4800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac19a3c0 con 0x7f14ac1997a0 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.885+0000 7f14b4800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac19a530 con 0x7f14ac0ff890 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.885+0000 7f14b1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14ac106500 0x7f14ac19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.885+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14ac0ff890 0x7f14ac19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.885+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14ac0ff890 0x7f14ac19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39580/0 (socket says 172.21.15.49:39580) 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.885+0000 7f14b1e00640 1 -- 172.21.15.49:0/2349827722 learned_addr learned my addr 172.21.15.49:0/2349827722 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.886+0000 7f14b2800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac1997a0 0x7f14ac199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.886+0000 7f14b2800640 1 -- 172.21.15.49:0/2349827722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14ac0ff890 msgr2=0x7f14ac19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.886+0000 7f14b2800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14ac0ff890 0x7f14ac19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.886+0000 7f14b2800640 1 -- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14ac106500 msgr2=0x7f14ac19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.886+0000 7f14b2800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14ac106500 0x7f14ac19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.886+0000 7f14b2800640 1 -- 172.21.15.49:0/2349827722 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a8009900 con 0x7f14ac1997a0 2024-02-26T06:25:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.886+0000 7f14b1400640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14ac106500 0x7f14ac19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.887+0000 7f14b2800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac1997a0 0x7f14ac199c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009bf0 tx=0x7f14a8032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.887+0000 7f14a6a00640 1 -- 172.21.15.49:0/2349827722 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8003b80 con 0x7f14ac1997a0 2024-02-26T06:25:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.887+0000 7f14a6a00640 1 -- 172.21.15.49:0/2349827722 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a8003ce0 con 0x7f14ac1997a0 2024-02-26T06:25:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.887+0000 7f14a6a00640 1 -- 172.21.15.49:0/2349827722 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8034970 con 0x7f14ac1997a0 2024-02-26T06:25:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.887+0000 7f14b4800640 1 -- 172.21.15.49:0/2349827722 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac1a6480 con 0x7f14ac1997a0 2024-02-26T06:25:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.888+0000 7f14b4800640 1 -- 172.21.15.49:0/2349827722 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac1a68b0 con 0x7f14ac1997a0 2024-02-26T06:25:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.892+0000 7f14a6a00640 1 -- 172.21.15.49:0/2349827722 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f14a803b040 con 0x7f14ac1997a0 2024-02-26T06:25:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.894+0000 7f14a6a00640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14940a2430 0x7f14940a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.895+0000 7f14a6a00640 1 -- 172.21.15.49:0/2349827722 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f14a80eff60 con 0x7f14ac1997a0 2024-02-26T06:25:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.897+0000 7f14b1e00640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14940a2430 0x7f14940a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.898+0000 7f14b1e00640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14940a2430 0x7f14940a48f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009c90 tx=0x7f14a0005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.899+0000 7f14b7684740 1 Processor -- start 2024-02-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.899+0000 7f14b7684740 1 -- start start 2024-02-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b7684740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f63063d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b7684740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063dd80 0x55f63063e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b7684740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 0x55f6306496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b7684740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6305f95f0 con 0x55f63063dd80 2024-02-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b7684740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6305ab890 con 0x55f630647300 2024-02-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b7684740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6305a8dc0 con 0x55f63063eaf0 2024-02-26T06:25:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f63063d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f63063d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39594/0 (socket says 172.21.15.49:39594) 2024-02-26T06:25:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.900+0000 7f14b1e00640 1 -- 172.21.15.49:0/2217915564 learned_addr learned my addr 172.21.15.49:0/2217915564 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.901+0000 7f14b1400640 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063dd80 0x55f63063e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.901+0000 7f14b2800640 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 0x55f6306496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.901+0000 7f14b1e00640 1 -- 172.21.15.49:0/2217915564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 msgr2=0x55f6306496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.901+0000 7f14b1e00640 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 0x55f6306496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.901+0000 7f14b1e00640 1 -- 172.21.15.49:0/2217915564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063dd80 msgr2=0x55f63063e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.901+0000 7f14b1e00640 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063dd80 0x55f63063e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.901+0000 7f14b1e00640 1 -- 172.21.15.49:0/2217915564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f630525ad0 con 0x55f63063eaf0 2024-02-26T06:25:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.902+0000 7f14b1400640 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063dd80 0x55f63063e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:25:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.902+0000 7f14b1e00640 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f63063d840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009fc0 tx=0x7f14a0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfc78d58cd06dcc2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.903+0000 7f149b000640 1 -- 172.21.15.49:0/2217915564 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0026620 con 0x55f63063eaf0 2024-02-26T06:25:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.903+0000 7f149b000640 1 -- 172.21.15.49:0/2217915564 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a0026c60 con 0x55f63063eaf0 2024-02-26T06:25:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.903+0000 7f149b000640 1 -- 172.21.15.49:0/2217915564 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000edb0 con 0x55f63063eaf0 2024-02-26T06:25:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.904+0000 7f14b7684740 1 -- 172.21.15.49:0/2217915564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 msgr2=0x55f63063d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.904+0000 7f14b7684740 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f63063d840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009fc0 tx=0x7f14a0014000 comp rx=0 tx=0).stop 2024-02-26T06:25:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.905+0000 7f14b7684740 1 -- 172.21.15.49:0/2217915564 shutdown_connections 2024-02-26T06:25:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.905+0000 7f14b7684740 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 0x55f6306496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.905+0000 7f14b7684740 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063dd80 0x55f63063e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.905+0000 7f14b7684740 1 --2- 172.21.15.49:0/2217915564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f63063d840 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.905+0000 7f14b7684740 1 -- 172.21.15.49:0/2217915564 >> 172.21.15.49:0/2217915564 conn(0x55f630605480 msgr2=0x55f630638a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.906+0000 7f14b7684740 1 -- 172.21.15.49:0/2217915564 shutdown_connections 2024-02-26T06:25:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.906+0000 7f14b7684740 1 -- 172.21.15.49:0/2217915564 wait complete. 2024-02-26T06:25:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.906+0000 7f14b7684740 1 -- start start 2024-02-26T06:25:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.907+0000 7f14b7684740 1 Processor -- start 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b7684740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f63063dd80 0x55f6306443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b7684740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630644900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b7684740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630647300 0x55f630644e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b7684740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6305a45a0 con 0x55f63063eaf0 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b7684740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f630549b20 con 0x55f63063dd80 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b7684740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6305497c0 con 0x55f630647300 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b7684740 10 client.0 _ll_register_callbacks cb 0x55f63055d520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f63063dd80 0x55f6306443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f63063dd80 0x55f6306443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44770/0 (socket says 172.21.15.49:44770) 2024-02-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b1e00640 1 -- 172.21.15.49:0/249886653 learned_addr learned my addr 172.21.15.49:0/249886653 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b1400640 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630644900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.908+0000 7f14b2800640 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630647300 0x55f630644e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.909+0000 7f14b1e00640 1 -- 172.21.15.49:0/249886653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630647300 msgr2=0x55f630644e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.909+0000 7f14b1e00640 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630647300 0x55f630644e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.909+0000 7f14b1e00640 1 -- 172.21.15.49:0/249886653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 msgr2=0x55f630644900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.909+0000 7f14b1e00640 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630644900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.909+0000 7f14b1e00640 1 -- 172.21.15.49:0/249886653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a0017050 con 0x55f63063dd80 2024-02-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.909+0000 7f14b1400640 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630644900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.909+0000 7f14b1e00640 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f63063dd80 0x55f6306443c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009760 tx=0x7f14a0009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7c2034565c7a1c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.909+0000 7f148f400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.910+0000 7f148f400640 1 -- 172.21.15.49:0/249886653 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000ecb0 con 0x55f63063dd80 2024-02-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.910+0000 7f148f400640 1 -- 172.21.15.49:0/249886653 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f14a000ee10 con 0x55f63063dd80 2024-02-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.910+0000 7f14b7684740 1 -- 172.21.15.49:0/249886653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f630576ca0 con 0x55f63063dd80 2024-02-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.911+0000 7f148f400640 1 -- 172.21.15.49:0/249886653 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a0031450 con 0x55f63063dd80 2024-02-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.911+0000 7f14b7684740 10 client.25880 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.911+0000 7f14b7684740 1 -- 172.21.15.49:0/249886653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f6305a2c20 con 0x55f63063dd80 2024-02-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f148f400640 1 -- 172.21.15.49:0/249886653 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a003e030 con 0x55f63063dd80 2024-02-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f148f400640 1 -- 172.21.15.49:0/249886653 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f14a001a880 con 0x55f63063dd80 2024-02-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f14b7684740 10 client.25880 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f14b7684740 10 client.25880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f14b7684740 20 client.25880 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f14b7684740 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f63065f920 0x55f630661d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f14b7684740 4 client.25880 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f14b7684740 1 -- 172.21.15.49:0/249886653 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f630576ca0 con 0x55f63065f920 2024-02-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.912+0000 7f14b1400640 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f63065f920 0x55f630661d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.913+0000 7f148f400640 10 client.25880 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.914+0000 7f14b1400640 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f63065f920 0x55f630661d60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f50130406e33bc3 server_cookie=b4e38ba83a405ef2 in_seq=0 out_seq=0 2024-02-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.914+0000 7f148f400640 1 -- 172.21.15.49:0/249886653 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f630576ca0 con 0x55f63065f920 2024-02-26T06:25:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.914+0000 7f148f400640 10 client.25880 handle_command_reply: tid=0 2024-02-26T06:25:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 client.25880 shutdown 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 -- 172.21.15.49:0/249886653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f63063dd80 msgr2=0x55f6306443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f63063dd80 0x55f6306443c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009760 tx=0x7f14a0009ab0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 -- 172.21.15.49:0/249886653 shutdown_connections 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f63065f920 0x55f630661d60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630647300 0x55f630644e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630644900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 --2- 172.21.15.49:0/249886653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f63063dd80 0x55f6306443c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f14b7684740 1 -- 172.21.15.49:0/249886653 >> 172.21.15.49:0/249886653 conn(0x55f630605480 msgr2=0x55f63063a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.915+0000 7f148f400640 0 client.25880 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.916+0000 7f14b7684740 1 -- 172.21.15.49:0/249886653 shutdown_connections 2024-02-26T06:25:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.917+0000 7f14b7684740 1 -- 172.21.15.49:0/249886653 wait complete. 2024-02-26T06:25:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.917+0000 7f14b7684740 20 client.25880 trim_cache size 0 max 16384 2024-02-26T06:25:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.945+0000 7f14b7684740 1 Processor -- start 2024-02-26T06:25:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.945+0000 7f14b7684740 1 -- start start 2024-02-26T06:25:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b7684740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630646af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b7684740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 0x55f630644c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b7684740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630638cb0 0x55f630642c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b7684740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6302ece60 con 0x55f63063eaf0 2024-02-26T06:25:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b7684740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63063fd30 con 0x55f630647300 2024-02-26T06:25:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b7684740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63063f0e0 con 0x55f630638cb0 2024-02-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b2800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630638cb0 0x55f630642c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b2800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630638cb0 0x55f630642c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39612/0 (socket says 172.21.15.49:39612) 2024-02-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.946+0000 7f14b2800640 1 -- 172.21.15.49:0/3523258156 learned_addr learned my addr 172.21.15.49:0/3523258156 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b1400640 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 0x55f630644c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b1e00640 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630646af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b2800640 1 -- 172.21.15.49:0/3523258156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 msgr2=0x55f630644c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b2800640 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 0x55f630644c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b2800640 1 -- 172.21.15.49:0/3523258156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 msgr2=0x55f630646af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b2800640 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630646af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b2800640 1 -- 172.21.15.49:0/3523258156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f63053f500 con 0x55f630638cb0 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b1e00640 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630646af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.947+0000 7f14b2800640 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630638cb0 0x55f630642c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14a8032490 tx=0x7f14a80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a75c9101bef867cb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.948+0000 7f149a600640 1 -- 172.21.15.49:0/3523258156 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80c0ba0 con 0x55f630638cb0 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.948+0000 7f149a600640 1 -- 172.21.15.49:0/3523258156 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a80c51f0 con 0x55f630638cb0 2024-02-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.948+0000 7f149a600640 1 -- 172.21.15.49:0/3523258156 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80b9280 con 0x55f630638cb0 2024-02-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 -- 172.21.15.49:0/3523258156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630638cb0 msgr2=0x55f630642c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630638cb0 0x55f630642c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14a8032490 tx=0x7f14a80095c0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 -- 172.21.15.49:0/3523258156 shutdown_connections 2024-02-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630647300 0x55f630644c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f630638cb0 0x55f630642c90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 --2- 172.21.15.49:0/3523258156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f63063eaf0 0x55f630646af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 -- 172.21.15.49:0/3523258156 >> 172.21.15.49:0/3523258156 conn(0x55f630605480 msgr2=0x55f6306385a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 -- 172.21.15.49:0/3523258156 shutdown_connections 2024-02-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.949+0000 7f14b7684740 1 -- 172.21.15.49:0/3523258156 wait complete. 2024-02-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.950+0000 7f14b7684740 1 -- start start 2024-02-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.950+0000 7f14b7684740 1 Processor -- start 2024-02-26T06:25:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.951+0000 7f14b7684740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630638cb0 0x55f630656ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.951+0000 7f14b7684740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f630654f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b7684740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f630647300 0x55f6306554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630638cb0 0x55f630656ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b1e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630638cb0 0x55f630656ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44784/0 (socket says 172.21.15.49:44784) 2024-02-26T06:25:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f630654f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b7684740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6306559e0 con 0x55f630647300 2024-02-26T06:25:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b7684740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f630655b50 con 0x55f630638cb0 2024-02-26T06:25:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b7684740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f630655d10 con 0x55f63063eaf0 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f630654f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39622/0 (socket says 172.21.15.49:39622) 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b1400640 1 -- 172.21.15.49:0/3879598448 learned_addr learned my addr 172.21.15.49:0/3879598448 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b7684740 10 client.0 _ll_register_callbacks cb 0x7f14a0029070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.952+0000 7f14b2800640 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f630647300 0x55f6306554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f14b1400640 1 -- 172.21.15.49:0/3879598448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630638cb0 msgr2=0x55f630656ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f14b1400640 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630638cb0 0x55f630656ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f14b1400640 1 -- 172.21.15.49:0/3879598448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f630647300 msgr2=0x55f6306554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f14b1400640 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f630647300 0x55f6306554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f14b1400640 1 -- 172.21.15.49:0/3879598448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac1a68b0 con 0x55f63063eaf0 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f14b1e00640 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630638cb0 0x55f630656ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f14b2800640 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f630647300 0x55f6306554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f14b1400640 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f630654f60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f14ac19aaf0 tx=0x7f149c013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa0d3b25ed4e96f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.953+0000 7f148f400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.954+0000 7f148f400640 1 -- 172.21.15.49:0/3879598448 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c017040 con 0x55f63063eaf0 2024-02-26T06:25:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.954+0000 7f148f400640 1 -- 172.21.15.49:0/3879598448 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f149c004250 con 0x55f63063eaf0 2024-02-26T06:25:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.954+0000 7f14b7684740 1 -- 172.21.15.49:0/3879598448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6305c9d00 con 0x55f63063eaf0 2024-02-26T06:25:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.956+0000 7f148f400640 1 -- 172.21.15.49:0/3879598448 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f149c0132e0 con 0x55f63063eaf0 2024-02-26T06:25:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.958+0000 7f14b7684740 10 client.25930 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.958+0000 7f14b7684740 1 -- 172.21.15.49:0/3879598448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f630565290 con 0x55f63063eaf0 2024-02-26T06:25:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.961+0000 7f148f400640 1 -- 172.21.15.49:0/3879598448 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f6305c9d00 con 0x55f63063eaf0 2024-02-26T06:25:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.961+0000 7f148f400640 1 -- 172.21.15.49:0/3879598448 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f149c003a70 con 0x55f63063eaf0 2024-02-26T06:25:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.961+0000 7f14b7684740 10 client.25930 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.961+0000 7f14b7684740 10 client.25930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.961+0000 7f14b7684740 20 client.25930 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.962+0000 7f14b7684740 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6306b1310 0x55f6306b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.962+0000 7f14b7684740 4 client.25930 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:25:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.962+0000 7f14b7684740 1 -- 172.21.15.49:0/3879598448 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f63059acb0 con 0x55f6306b1310 2024-02-26T06:25:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.962+0000 7f14b1e00640 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6306b1310 0x55f6306b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.966+0000 7f14b1e00640 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6306b1310 0x55f6306b1720 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c280fe2e3b78837 server_cookie=32c8e1bc79956d61 in_seq=0 out_seq=0 2024-02-26T06:25:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:25.966+0000 7f148f400640 10 client.25930 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.006+0000 7f148f400640 1 -- 172.21.15.49:0/3879598448 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f63059acb0 con 0x55f6306b1310 2024-02-26T06:25:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.006+0000 7f148f400640 10 client.25930 handle_command_reply: tid=0 2024-02-26T06:25:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.006+0000 7f14b7684740 1 client.25930 shutdown 2024-02-26T06:25:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 -- 172.21.15.49:0/3879598448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 msgr2=0x55f630654f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f630654f60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f14ac19aaf0 tx=0x7f149c013830 comp rx=0 tx=0).stop 2024-02-26T06:25:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 -- 172.21.15.49:0/3879598448 shutdown_connections 2024-02-26T06:25:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6306b1310 0x55f6306b1720 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f630647300 0x55f6306554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f63063eaf0 0x55f630654f60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 --2- 172.21.15.49:0/3879598448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f630638cb0 0x55f630656ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f148f400640 0 client.25930 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 -- 172.21.15.49:0/3879598448 >> 172.21.15.49:0/3879598448 conn(0x55f630605480 msgr2=0x55f630642c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 -- 172.21.15.49:0/3879598448 shutdown_connections 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.007+0000 7f14b7684740 1 -- 172.21.15.49:0/3879598448 wait complete. 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.008+0000 7f14b7684740 20 client.25930 trim_cache size 0 max 16384 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:25:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:25:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:25:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:25:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.008+0000 7f14b4800640 1 -- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14940a2430 msgr2=0x7f14940a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.008+0000 7f14b4800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14940a2430 0x7f14940a48f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009c90 tx=0x7f14a0005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.008+0000 7f14b4800640 1 -- 172.21.15.49:0/2349827722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac1997a0 msgr2=0x7f14ac199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.008+0000 7f14b4800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac1997a0 0x7f14ac199c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009bf0 tx=0x7f14a8032ec0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.009+0000 7f14b4800640 1 -- 172.21.15.49:0/2349827722 shutdown_connections 2024-02-26T06:25:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.009+0000 7f14b4800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f14940a2430 0x7f14940a48f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.009+0000 7f14b4800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14ac1997a0 0x7f14ac199c50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.009+0000 7f14b4800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14ac106500 0x7f14ac19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.009+0000 7f14b4800640 1 --2- 172.21.15.49:0/2349827722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14ac0ff890 0x7f14ac19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.009+0000 7f14b4800640 1 -- 172.21.15.49:0/2349827722 >> 172.21.15.49:0/2349827722 conn(0x7f14ac0765a0 msgr2=0x7f14ac1047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.009+0000 7f14b4800640 1 -- 172.21.15.49:0/2349827722 shutdown_connections 2024-02-26T06:25:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:26.009+0000 7f14b4800640 1 -- 172.21.15.49:0/2349827722 wait complete. 2024-02-26T06:25:26.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2407560843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 msgr2=0x7f2ad410e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2407560843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 0x7f2ad410e010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009ca0 tx=0x7f2acc02c120 comp rx=0 tx=0).stop 2024-02-26T06:25:26.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2407560843 shutdown_connections 2024-02-26T06:25:26.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2407560843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 0x7f2ad410e010 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2407560843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ad4101d20 0x7f2ad4105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2407560843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad4101370 0x7f2ad4101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2407560843 >> 172.21.15.49:0/2407560843 conn(0x7f2ad40fcbf0 msgr2=0x7f2ad40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2407560843 shutdown_connections 2024-02-26T06:25:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.124+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2407560843 wait complete. 2024-02-26T06:25:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.125+0000 7f2ad9a00640 1 Processor -- start 2024-02-26T06:25:26.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.125+0000 7f2ad9a00640 1 -- start start 2024-02-26T06:25:26.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad9a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ad4101370 0x7f2ad419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad9a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 0x7f2ad419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad9a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad41976c0 0x7f2ad4197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad9a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad4198190 con 0x7f2ad4101370 2024-02-26T06:25:26.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad9a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad4198300 con 0x7f2ad41976c0 2024-02-26T06:25:26.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad9a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41984c0 con 0x7f2ad4106440 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad2000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 0x7f2ad419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad2000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 0x7f2ad419daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39640/0 (socket says 172.21.15.49:39640) 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad2000640 1 -- 172.21.15.49:0/2507192028 learned_addr learned my addr 172.21.15.49:0/2507192028 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ad4101370 0x7f2ad419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.126+0000 7f2ad3400640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad41976c0 0x7f2ad4197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.127+0000 7f2ad2000640 1 -- 172.21.15.49:0/2507192028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad41976c0 msgr2=0x7f2ad4197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.127+0000 7f2ad2000640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad41976c0 0x7f2ad4197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.127+0000 7f2ad2000640 1 -- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ad4101370 msgr2=0x7f2ad419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.127+0000 7f2ad2000640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ad4101370 0x7f2ad419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.127+0000 7f2ad2000640 1 -- 172.21.15.49:0/2507192028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc009900 con 0x7f2ad4106440 2024-02-26T06:25:26.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.127+0000 7f2ad3400640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad41976c0 0x7f2ad4197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:26.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.127+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ad4101370 0x7f2ad419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:26.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.129+0000 7f2ad2000640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 0x7f2ad419daf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000b450 tx=0x7f2ac000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.129+0000 7f2acb400640 1 -- 172.21.15.49:0/2507192028 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0011890 con 0x7f2ad4106440 2024-02-26T06:25:26.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.129+0000 7f2acb400640 1 -- 172.21.15.49:0/2507192028 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ac00103e0 con 0x7f2ad4106440 2024-02-26T06:25:26.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.129+0000 7f2acb400640 1 -- 172.21.15.49:0/2507192028 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000f580 con 0x7f2ad4106440 2024-02-26T06:25:26.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.129+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2507192028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad41a4410 con 0x7f2ad4106440 2024-02-26T06:25:26.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.129+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2507192028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad41a48f0 con 0x7f2ad4106440 2024-02-26T06:25:26.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.135+0000 7f2acb400640 1 -- 172.21.15.49:0/2507192028 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2ac00108e0 con 0x7f2ad4106440 2024-02-26T06:25:26.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.135+0000 7f2adc99d740 1 Processor -- start 2024-02-26T06:25:26.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.136+0000 7f2adc99d740 1 -- start start 2024-02-26T06:25:26.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.137+0000 7f2acb400640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ab80a2400 0x7f2ab80a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.137+0000 7f2acb400640 1 -- 172.21.15.49:0/2507192028 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2ac00cdc90 con 0x7f2ad4106440 2024-02-26T06:25:26.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.138+0000 7f2ad3400640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ab80a2400 0x7f2ab80a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.138+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 0x555cf8f60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.138+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51230 0x555cf8f516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.138+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51bf0 0x555cf8f68630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.138+0000 7f2adc99d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8ec3120 con 0x555cf8f609c0 2024-02-26T06:25:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.138+0000 7f2adc99d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8ec0650 con 0x555cf8f51230 2024-02-26T06:25:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.138+0000 7f2adc99d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8ebbe30 con 0x555cf8f51bf0 2024-02-26T06:25:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.139+0000 7f2ad2a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 0x555cf8f60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.139+0000 7f2ad3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51bf0 0x555cf8f68630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.139+0000 7f2ad3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51bf0 0x555cf8f68630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39650/0 (socket says 172.21.15.49:39650) 2024-02-26T06:25:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.139+0000 7f2ad3400640 1 -- 172.21.15.49:0/41906632 learned_addr learned my addr 172.21.15.49:0/41906632 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.139+0000 7f2ad2000640 1 --2- 172.21.15.49:0/41906632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51230 0x555cf8f516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.140+0000 7f2ad2000640 1 -- 172.21.15.49:0/41906632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51bf0 msgr2=0x555cf8f68630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.140+0000 7f2ad2000640 1 --2- 172.21.15.49:0/41906632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51bf0 0x555cf8f68630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.140+0000 7f2ad2000640 1 -- 172.21.15.49:0/41906632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 msgr2=0x555cf8f60da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.140+0000 7f2ad2000640 1 --2- 172.21.15.49:0/41906632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 0x555cf8f60da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.140+0000 7f2ad2000640 1 -- 172.21.15.49:0/41906632 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf8e75030 con 0x555cf8f51230 2024-02-26T06:25:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.140+0000 7f2ad3400640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ab80a2400 0x7f2ab80a48c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2acc004af0 tx=0x7f2acc032490 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.141+0000 7f2ad2000640 1 --2- 172.21.15.49:0/41906632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51230 0x555cf8f516b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00cb4a0 tx=0x7f2ac0010ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10641ea4f89f9d29 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.141+0000 7f2abfe00640 1 -- 172.21.15.49:0/41906632 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00a2ee0 con 0x555cf8f51230 2024-02-26T06:25:26.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.141+0000 7f2abfe00640 1 -- 172.21.15.49:0/41906632 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ac0096db0 con 0x555cf8f51230 2024-02-26T06:25:26.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.142+0000 7f2adc99d740 1 -- 172.21.15.49:0/41906632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51230 msgr2=0x555cf8f516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.142+0000 7f2adc99d740 1 --2- 172.21.15.49:0/41906632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51230 0x555cf8f516b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00cb4a0 tx=0x7f2ac0010ce0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.142+0000 7f2adc99d740 1 -- 172.21.15.49:0/41906632 shutdown_connections 2024-02-26T06:25:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.143+0000 7f2adc99d740 1 --2- 172.21.15.49:0/41906632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51230 0x555cf8f516b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.143+0000 7f2adc99d740 1 --2- 172.21.15.49:0/41906632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51bf0 0x555cf8f68630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.143+0000 7f2adc99d740 1 --2- 172.21.15.49:0/41906632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 0x555cf8f60da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.143+0000 7f2adc99d740 1 -- 172.21.15.49:0/41906632 >> 172.21.15.49:0/41906632 conn(0x555cf8f1b570 msgr2=0x555cf8f50120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.143+0000 7f2adc99d740 1 -- 172.21.15.49:0/41906632 shutdown_connections 2024-02-26T06:25:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.143+0000 7f2adc99d740 1 -- 172.21.15.49:0/41906632 wait complete. 2024-02-26T06:25:26.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.144+0000 7f2adc99d740 1 -- start start 2024-02-26T06:25:26.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.144+0000 7f2adc99d740 1 Processor -- start 2024-02-26T06:25:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.145+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f65a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f51bf0 0x555cf8f65f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f609c0 0x555cf8f66490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2adc99d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8e60e90 con 0x555cf8f51bf0 2024-02-26T06:25:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2adc99d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8e60b30 con 0x555cf8f609c0 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2adc99d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8c2a1c0 con 0x555cf8f51230 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2adc99d740 10 client.0 _ll_register_callbacks cb 0x555cf8e74930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2ad2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f65a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2ad2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f65a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39664/0 (socket says 172.21.15.49:39664) 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.146+0000 7f2ad2a00640 1 -- 172.21.15.49:0/3523670106 learned_addr learned my addr 172.21.15.49:0/3523670106 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.147+0000 7f2ad2a00640 1 -- 172.21.15.49:0/3523670106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f609c0 msgr2=0x555cf8f66490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.147+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f609c0 0x555cf8f66490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.147+0000 7f2ad2a00640 1 -- 172.21.15.49:0/3523670106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f51bf0 msgr2=0x555cf8f65f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.147+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f51bf0 0x555cf8f65f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.147+0000 7f2ad2a00640 1 -- 172.21.15.49:0/3523670106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad41a48f0 con 0x555cf8f51230 2024-02-26T06:25:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.147+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f65a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4004440 tx=0x7f2ac40039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96041ea3f8e12cd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.147+0000 7f2abcc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.148+0000 7f2abcc00640 1 -- 172.21.15.49:0/3523670106 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4013930 con 0x555cf8f51230 2024-02-26T06:25:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.148+0000 7f2abcc00640 1 -- 172.21.15.49:0/3523670106 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2ac4013a90 con 0x555cf8f51230 2024-02-26T06:25:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.148+0000 7f2adc99d740 1 -- 172.21.15.49:0/3523670106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cf8bbb7e0 con 0x555cf8f51230 2024-02-26T06:25:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.148+0000 7f2abcc00640 1 -- 172.21.15.49:0/3523670106 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac4013c50 con 0x555cf8f51230 2024-02-26T06:25:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.148+0000 7f2adc99d740 10 client.25942 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.148+0000 7f2adc99d740 1 -- 172.21.15.49:0/3523670106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555cf8eba4b0 con 0x555cf8f51230 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2abcc00640 1 -- 172.21.15.49:0/3523670106 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ac4028080 con 0x555cf8f51230 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2abcc00640 1 -- 172.21.15.49:0/3523670106 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2ac4011540 con 0x555cf8f51230 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2adc99d740 10 client.25942 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2adc99d740 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2adc99d740 20 client.25942 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2adc99d740 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555cf8f76000 0x555cf8f78440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2adc99d740 4 client.25942 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2adc99d740 1 -- 172.21.15.49:0/3523670106 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cf8bf8720 con 0x555cf8f76000 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.149+0000 7f2ad2000640 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555cf8f76000 0x555cf8f78440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.151+0000 7f2ad2000640 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555cf8f76000 0x555cf8f78440 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e2648886e308a30 server_cookie=98c8b4b1de7db9b7 in_seq=0 out_seq=0 2024-02-26T06:25:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.151+0000 7f2abcc00640 10 client.25942 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:26.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.152+0000 7f2abcc00640 1 -- 172.21.15.49:0/3523670106 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555cf8bf8720 con 0x555cf8f76000 2024-02-26T06:25:26.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.152+0000 7f2abcc00640 10 client.25942 handle_command_reply: tid=0 2024-02-26T06:25:26.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.152+0000 7f2adc99d740 1 client.25942 shutdown 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 -- 172.21.15.49:0/3523670106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 msgr2=0x555cf8f65a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f65a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4004440 tx=0x7f2ac40039c0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 -- 172.21.15.49:0/3523670106 shutdown_connections 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555cf8f76000 0x555cf8f78440 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f609c0 0x555cf8f66490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f51bf0 0x555cf8f65f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2abcc00640 0 client.25942 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 --2- 172.21.15.49:0/3523670106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f65a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 -- 172.21.15.49:0/3523670106 >> 172.21.15.49:0/3523670106 conn(0x555cf8f1b570 msgr2=0x555cf8f4f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.153+0000 7f2adc99d740 1 -- 172.21.15.49:0/3523670106 shutdown_connections 2024-02-26T06:25:26.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.154+0000 7f2adc99d740 1 -- 172.21.15.49:0/3523670106 wait complete. 2024-02-26T06:25:26.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.154+0000 7f2adc99d740 20 client.25942 trim_cache size 0 max 16384 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.188+0000 7f2adc99d740 1 Processor -- start 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.188+0000 7f2adc99d740 1 -- start start 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f51230 0x555cf8f7cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f62c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f609c0 0x555cf8f631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2adc99d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8f52570 con 0x555cf8f51230 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2adc99d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8f689c0 con 0x555cf8f51bf0 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2adc99d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8f68c00 con 0x555cf8f609c0 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2ad3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f609c0 0x555cf8f631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2ad3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f609c0 0x555cf8f631b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39668/0 (socket says 172.21.15.49:39668) 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2ad3400640 1 -- 172.21.15.49:0/1569199802 learned_addr learned my addr 172.21.15.49:0/1569199802 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.189+0000 7f2ad2000640 1 --2- 172.21.15.49:0/1569199802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f62c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.190+0000 7f2ad3400640 1 -- 172.21.15.49:0/1569199802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 msgr2=0x555cf8f62c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.190+0000 7f2ad3400640 1 --2- 172.21.15.49:0/1569199802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f62c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.190+0000 7f2ad3400640 1 -- 172.21.15.49:0/1569199802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f51230 msgr2=0x555cf8f7cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.190+0000 7f2ad3400640 1 --2- 172.21.15.49:0/1569199802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f51230 0x555cf8f7cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.190+0000 7f2ad3400640 1 -- 172.21.15.49:0/1569199802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf8e56920 con 0x555cf8f609c0 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.190+0000 7f2ad3400640 1 --2- 172.21.15.49:0/1569199802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f609c0 0x555cf8f631b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009bf0 tx=0x7f2acc046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee8aeb340aae9d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.191+0000 7f2abf400640 1 -- 172.21.15.49:0/1569199802 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc003b70 con 0x555cf8f609c0 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.191+0000 7f2abf400640 1 -- 172.21.15.49:0/1569199802 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2acc003cd0 con 0x555cf8f609c0 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.191+0000 7f2abf400640 1 -- 172.21.15.49:0/1569199802 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0707c0 con 0x555cf8f609c0 2024-02-26T06:25:26.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.192+0000 7f2adc99d740 1 -- 172.21.15.49:0/1569199802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f609c0 msgr2=0x555cf8f631b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.192+0000 7f2adc99d740 1 --2- 172.21.15.49:0/1569199802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f609c0 0x555cf8f631b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009bf0 tx=0x7f2acc046000 comp rx=0 tx=0).stop 2024-02-26T06:25:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.192+0000 7f2adc99d740 1 -- 172.21.15.49:0/1569199802 shutdown_connections 2024-02-26T06:25:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.192+0000 7f2adc99d740 1 --2- 172.21.15.49:0/1569199802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f62c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.192+0000 7f2adc99d740 1 --2- 172.21.15.49:0/1569199802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f609c0 0x555cf8f631b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.192+0000 7f2adc99d740 1 --2- 172.21.15.49:0/1569199802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f51230 0x555cf8f7cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.192+0000 7f2adc99d740 1 -- 172.21.15.49:0/1569199802 >> 172.21.15.49:0/1569199802 conn(0x555cf8f1b570 msgr2=0x555cf8f5f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.193+0000 7f2adc99d740 1 -- 172.21.15.49:0/1569199802 shutdown_connections 2024-02-26T06:25:26.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.195+0000 7f2adc99d740 1 -- 172.21.15.49:0/1569199802 wait complete. 2024-02-26T06:25:26.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.196+0000 7f2adc99d740 1 -- start start 2024-02-26T06:25:26.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.196+0000 7f2adc99d740 1 Processor -- start 2024-02-26T06:25:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f6d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f5a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2adc99d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 0x555cf8f6b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2adc99d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8f6baf0 con 0x555cf8f609c0 2024-02-26T06:25:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2adc99d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8f6bc60 con 0x555cf8f51bf0 2024-02-26T06:25:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2adc99d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8f6be20 con 0x555cf8f51230 2024-02-26T06:25:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2adc99d740 10 client.0 _ll_register_callbacks cb 0x7f2ac40041e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2ad2000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f5a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2ad2000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f5a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44844/0 (socket says 172.21.15.49:44844) 2024-02-26T06:25:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.197+0000 7f2ad2000640 1 -- 172.21.15.49:0/907395871 learned_addr learned my addr 172.21.15.49:0/907395871 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.198+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f6d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.198+0000 7f2ad2000640 1 -- 172.21.15.49:0/907395871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 msgr2=0x555cf8f6d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.198+0000 7f2ad2000640 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f6d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.198+0000 7f2ad2000640 1 -- 172.21.15.49:0/907395871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 msgr2=0x555cf8f6b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.198+0000 7f2ad2000640 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 0x555cf8f6b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.198+0000 7f2ad2000640 1 -- 172.21.15.49:0/907395871 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad410e820 con 0x555cf8f51bf0 2024-02-26T06:25:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.198+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f6d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:25:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.199+0000 7f2ad2000640 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f5a9b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000be10 tx=0x7f2ac009a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=341a7f771ee37624 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.199+0000 7f2abcc00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:25:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.199+0000 7f2abcc00640 1 -- 172.21.15.49:0/907395871 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00a4040 con 0x555cf8f51bf0 2024-02-26T06:25:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.199+0000 7f2abcc00640 1 -- 172.21.15.49:0/907395871 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2ac00b3230 con 0x555cf8f51bf0 2024-02-26T06:25:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.199+0000 7f2adc99d740 1 -- 172.21.15.49:0/907395871 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ac0096db0 con 0x555cf8f51bf0 2024-02-26T06:25:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2abcc00640 1 -- 172.21.15.49:0/907395871 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac00a4040 con 0x555cf8f51bf0 2024-02-26T06:25:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2adc99d740 10 client.25901 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2adc99d740 1 -- 172.21.15.49:0/907395871 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555cf8e7c400 con 0x555cf8f51bf0 2024-02-26T06:25:26.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2abcc00640 1 -- 172.21.15.49:0/907395871 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ac0096db0 con 0x555cf8f51bf0 2024-02-26T06:25:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2abcc00640 1 -- 172.21.15.49:0/907395871 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2ac0096040 con 0x555cf8f51bf0 2024-02-26T06:25:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2adc99d740 10 client.25901 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2adc99d740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2adc99d740 20 client.25901 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2adc99d740 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555cf8fc51b0 0x555cf8fc75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2adc99d740 4 client.25901 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:25:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.200+0000 7f2adc99d740 1 -- 172.21.15.49:0/907395871 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555cf8c47890 con 0x555cf8fc51b0 2024-02-26T06:25:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.203+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555cf8fc51b0 0x555cf8fc75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.203+0000 7f2ad2a00640 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555cf8fc51b0 0x555cf8fc75d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592d616efffb5928 server_cookie=cbb64a512abd568 in_seq=0 out_seq=0 2024-02-26T06:25:26.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.204+0000 7f2abcc00640 10 client.25901 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.204+0000 7f2abcc00640 1 -- 172.21.15.49:0/907395871 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555cf8c47890 con 0x555cf8fc51b0 2024-02-26T06:25:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.204+0000 7f2abcc00640 10 client.25901 handle_command_reply: tid=0 2024-02-26T06:25:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.204+0000 7f2adc99d740 1 client.25901 shutdown 2024-02-26T06:25:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 -- 172.21.15.49:0/907395871 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 msgr2=0x555cf8f5a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f5a9b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000be10 tx=0x7f2ac009a640 comp rx=0 tx=0).stop 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 -- 172.21.15.49:0/907395871 shutdown_connections 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555cf8fc51b0 0x555cf8fc75d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cf8f51bf0 0x555cf8f5a9b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555cf8f609c0 0x555cf8f6b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 --2- 172.21.15.49:0/907395871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555cf8f51230 0x555cf8f6d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 -- 172.21.15.49:0/907395871 >> 172.21.15.49:0/907395871 conn(0x555cf8f1b570 msgr2=0x555cf8f7cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2abcc00640 0 client.25901 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 -- 172.21.15.49:0/907395871 shutdown_connections 2024-02-26T06:25:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.205+0000 7f2adc99d740 1 -- 172.21.15.49:0/907395871 wait complete. 2024-02-26T06:25:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.206+0000 7f2adc99d740 20 client.25901 trim_cache size 0 max 16384 2024-02-26T06:25:26.207 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:25:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:25:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:25:26.207 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ab80a2400 msgr2=0x7f2ab80a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ab80a2400 0x7f2ab80a48c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2acc004af0 tx=0x7f2acc032490 comp rx=0 tx=0).stop 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2507192028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 msgr2=0x7f2ad419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 0x7f2ad419daf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000b450 tx=0x7f2ac000b910 comp rx=0 tx=0).stop 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2507192028 shutdown_connections 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ab80a2400 0x7f2ab80a48c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad41976c0 0x7f2ad4197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad4106440 0x7f2ad419daf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 --2- 172.21.15.49:0/2507192028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ad4101370 0x7f2ad419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.207+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2507192028 >> 172.21.15.49:0/2507192028 conn(0x7f2ad40fcbf0 msgr2=0x7f2ad40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.208+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2507192028 shutdown_connections 2024-02-26T06:25:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.209+0000 7f2ad9a00640 1 -- 172.21.15.49:0/2507192028 wait complete. 2024-02-26T06:25:26.233 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:25:26.233 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:25:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 95.0 s, 2677.4 tps, lat 37.373 ms stddev 41.132 2024-02-26T06:25:26.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 -- 172.21.15.49:0/225467942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 msgr2=0x7f4ef0108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 --2- 172.21.15.49:0/225467942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 0x7f4ef0108750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009cd0 tx=0x7f4ee4030580 comp rx=0 tx=0).stop 2024-02-26T06:25:26.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 -- 172.21.15.49:0/225467942 shutdown_connections 2024-02-26T06:25:26.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 --2- 172.21.15.49:0/225467942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef0108c90 0x7f4ef0113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 --2- 172.21.15.49:0/225467942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ef01142d0 0x7f4ef01166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 --2- 172.21.15.49:0/225467942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 0x7f4ef0108750 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 -- 172.21.15.49:0/225467942 >> 172.21.15.49:0/225467942 conn(0x7f4ef006c270 msgr2=0x7f4ef006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 -- 172.21.15.49:0/225467942 shutdown_connections 2024-02-26T06:25:26.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.513+0000 7f4ef8200640 1 -- 172.21.15.49:0/225467942 wait complete. 2024-02-26T06:25:26.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.514+0000 7f4ef8200640 1 Processor -- start 2024-02-26T06:25:26.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.514+0000 7f4ef8200640 1 -- start start 2024-02-26T06:25:26.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.515+0000 7f4ef8200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 0x7f4ef019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.515+0000 7f4ef8200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ef0108c90 0x7f4ef019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.515+0000 7f4ef8200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01142d0 0x7f4ef01a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.515+0000 7f4ef8200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01a2c50 con 0x7f4ef0108c90 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.515+0000 7f4ef8200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01a2dc0 con 0x7f4ef01142d0 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.515+0000 7f4ef8200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01a2f80 con 0x7f4ef0108370 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.515+0000 7f4ef5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 0x7f4ef019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.516+0000 7f4ef5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 0x7f4ef019dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39690/0 (socket says 172.21.15.49:39690) 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.516+0000 7f4ef5800640 1 -- 172.21.15.49:0/2965860803 learned_addr learned my addr 172.21.15.49:0/2965860803 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.516+0000 7f4ef4e00640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ef0108c90 0x7f4ef019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.516+0000 7f4ef6200640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01142d0 0x7f4ef01a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.516+0000 7f4ef4e00640 1 -- 172.21.15.49:0/2965860803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 msgr2=0x7f4ef019dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.516+0000 7f4ef4e00640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 0x7f4ef019dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.516+0000 7f4ef4e00640 1 -- 172.21.15.49:0/2965860803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01142d0 msgr2=0x7f4ef01a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.516+0000 7f4ef4e00640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01142d0 0x7f4ef01a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.517+0000 7f4ef4e00640 1 -- 172.21.15.49:0/2965860803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4009900 con 0x7f4ef0108c90 2024-02-26T06:25:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.517+0000 7f4ef6200640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01142d0 0x7f4ef01a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.517+0000 7f4ef4e00640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ef0108c90 0x7f4ef019e320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00d6d0 tx=0x7f4edc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.517+0000 7f4eee000640 1 -- 172.21.15.49:0/2965860803 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc005220 con 0x7f4ef0108c90 2024-02-26T06:25:26.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.518+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef01a3260 con 0x7f4ef0108c90 2024-02-26T06:25:26.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.518+0000 7f4eee000640 1 -- 172.21.15.49:0/2965860803 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4edc009d00 con 0x7f4ef0108c90 2024-02-26T06:25:26.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.518+0000 7f4eee000640 1 -- 172.21.15.49:0/2965860803 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc004dd0 con 0x7f4ef0108c90 2024-02-26T06:25:26.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.518+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef01aaae0 con 0x7f4ef0108c90 2024-02-26T06:25:26.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.519+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef0067f70 con 0x7f4ef0108c90 2024-02-26T06:25:26.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.526+0000 7f4eee000640 1 -- 172.21.15.49:0/2965860803 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4edc009a30 con 0x7f4ef0108c90 2024-02-26T06:25:26.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.529+0000 7f4eee000640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ed80a22b0 0x7f4ed80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.529+0000 7f4eee000640 1 -- 172.21.15.49:0/2965860803 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4edc0ce150 con 0x7f4ef0108c90 2024-02-26T06:25:26.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.530+0000 7f4ef5800640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ed80a22b0 0x7f4ed80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.532+0000 7f4ef5800640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ed80a22b0 0x7f4ed80a4770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005e30 tx=0x7f4ee4005d80 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.533+0000 7f4eee000640 1 -- 172.21.15.49:0/2965860803 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4edc096d90 con 0x7f4ef0108c90 2024-02-26T06:25:26.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.967+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ef019f0e0 con 0x7f4ef0108c90 2024-02-26T06:25:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.969+0000 7f4eee000640 1 -- 172.21.15.49:0/2965860803 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11338 (secure 0 0 0) 0x7f4edc020020 con 0x7f4ef0108c90 2024-02-26T06:25:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:25:26.970 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:24:43.229679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:25:26.970 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 29 2024-02-26T06:25:26.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ed80a22b0 msgr2=0x7f4ed80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ed80a22b0 0x7f4ed80a4770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005e30 tx=0x7f4ee4005d80 comp rx=0 tx=0).stop 2024-02-26T06:25:26.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ef0108c90 msgr2=0x7f4ef019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ef0108c90 0x7f4ef019e320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00d6d0 tx=0x7f4edc00db90 comp rx=0 tx=0).stop 2024-02-26T06:25:26.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 shutdown_connections 2024-02-26T06:25:26.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ed80a22b0 0x7f4ed80a4770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01142d0 0x7f4ef01a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ef0108c90 0x7f4ef019e320 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 --2- 172.21.15.49:0/2965860803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef0108370 0x7f4ef019dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.973+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 >> 172.21.15.49:0/2965860803 conn(0x7f4ef006c270 msgr2=0x7f4ef006fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.974+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 shutdown_connections 2024-02-26T06:25:26.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:26.974+0000 7f4ef8200640 1 -- 172.21.15.49:0/2965860803 wait complete. 2024-02-26T06:25:27.001 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:26 smithi049 ceph-mon[31696]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 965 op/s 2024-02-26T06:25:27.005 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:25:27.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:25:27.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:26 smithi080 ceph-mon[36206]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 965 op/s 2024-02-26T06:25:27.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:26 smithi115 ceph-mon[36426]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 965 op/s 2024-02-26T06:25:27.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.310+0000 7f283e400640 1 -- 172.21.15.49:0/1782538655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103f40 msgr2=0x7f2838107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.310+0000 7f283e400640 1 --2- 172.21.15.49:0/1782538655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103f40 0x7f2838107f80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2830009be0 tx=0x7f283002bef0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.310+0000 7f283e400640 1 -- 172.21.15.49:0/1782538655 shutdown_connections 2024-02-26T06:25:27.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.310+0000 7f283e400640 1 --2- 172.21.15.49:0/1782538655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2838108660 0x7f2838110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.310+0000 7f283e400640 1 --2- 172.21.15.49:0/1782538655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103f40 0x7f2838107f80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.310+0000 7f283e400640 1 --2- 172.21.15.49:0/1782538655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2838103590 0x7f2838103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.310+0000 7f283e400640 1 -- 172.21.15.49:0/1782538655 >> 172.21.15.49:0/1782538655 conn(0x7f28380fedf0 msgr2=0x7f2838101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.310+0000 7f283e400640 1 -- 172.21.15.49:0/1782538655 shutdown_connections 2024-02-26T06:25:27.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.311+0000 7f283e400640 1 -- 172.21.15.49:0/1782538655 wait complete. 2024-02-26T06:25:27.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f283e400640 1 Processor -- start 2024-02-26T06:25:27.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f283e400640 1 -- start start 2024-02-26T06:25:27.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f283e400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103590 0x7f283819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f283e400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2838108660 0x7f283819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f283e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2838199830 0x7f2838199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f283e400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819a2e0 con 0x7f2838108660 2024-02-26T06:25:27.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f283e400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819a450 con 0x7f2838199830 2024-02-26T06:25:27.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f283e400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819a5c0 con 0x7f2838103590 2024-02-26T06:25:27.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.312+0000 7f2836a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2838108660 0x7f283819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103590 0x7f283819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103590 0x7f283819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39708/0 (socket says 172.21.15.49:39708) 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837400640 1 -- 172.21.15.49:0/2582873609 learned_addr learned my addr 172.21.15.49:0/2582873609 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837e00640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2838199830 0x7f2838199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837400640 1 -- 172.21.15.49:0/2582873609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2838199830 msgr2=0x7f2838199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2838199830 0x7f2838199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837400640 1 -- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2838108660 msgr2=0x7f283819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2838108660 0x7f283819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837400640 1 -- 172.21.15.49:0/2582873609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2830009840 con 0x7f2838103590 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.313+0000 7f2837e00640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2838199830 0x7f2838199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:27.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.314+0000 7f2837400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103590 0x7f283819f6c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f282800e9c0 tx=0x7f282800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.314+0000 7f282fe00640 1 -- 172.21.15.49:0/2582873609 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800ccc0 con 0x7f2838103590 2024-02-26T06:25:27.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.315+0000 7f282fe00640 1 -- 172.21.15.49:0/2582873609 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f282800ce20 con 0x7f2838103590 2024-02-26T06:25:27.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.315+0000 7f282fe00640 1 -- 172.21.15.49:0/2582873609 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828010430 con 0x7f2838103590 2024-02-26T06:25:27.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.315+0000 7f283e400640 1 -- 172.21.15.49:0/2582873609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28381a6570 con 0x7f2838103590 2024-02-26T06:25:27.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.315+0000 7f283e400640 1 -- 172.21.15.49:0/2582873609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28381a6a00 con 0x7f2838103590 2024-02-26T06:25:27.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.320+0000 7f284139c740 1 Processor -- start 2024-02-26T06:25:27.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f284139c740 1 -- start start 2024-02-26T06:25:27.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f284139c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f282fe00640 1 -- 172.21.15.49:0/2582873609 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f28280046c0 con 0x7f2838103590 2024-02-26T06:25:27.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f2837e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f2837e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39716/0 (socket says 172.21.15.49:39716) 2024-02-26T06:25:27.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f284139c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f2837e00640 1 -- 172.21.15.49:0/586426717 learned_addr learned my addr 172.21.15.49:0/586426717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f284139c740 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea78780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.321+0000 7f284139c740 1 -- 172.21.15.49:0/586426717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e9ca1e0 con 0x56318ea649f0 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f284139c740 1 -- 172.21.15.49:0/586426717 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e9c7710 con 0x56318ea76390 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f284139c740 1 -- 172.21.15.49:0/586426717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e9c2ef0 con 0x56318ea640d0 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f2836a00640 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea78780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f2837400640 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f2837e00640 1 -- 172.21.15.49:0/586426717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 msgr2=0x56318ea78780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f2837e00640 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea78780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f2837e00640 1 -- 172.21.15.49:0/586426717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 msgr2=0x56318ea5dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f2837e00640 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f2837e00640 1 -- 172.21.15.49:0/586426717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318e984dc0 con 0x56318ea640d0 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.322+0000 7f2837400640 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.323+0000 7f2837e00640 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea644b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2824011440 tx=0x7f2824011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a26095a65f2e16 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.323+0000 7f282cc00640 1 -- 172.21.15.49:0/586426717 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824004020 con 0x56318ea640d0 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.323+0000 7f282cc00640 1 -- 172.21.15.49:0/586426717 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2824004180 con 0x56318ea640d0 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.323+0000 7f282cc00640 1 -- 172.21.15.49:0/586426717 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282400f8b0 con 0x56318ea640d0 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.324+0000 7f282fe00640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f281c0a2440 0x7f281c0a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.324+0000 7f282fe00640 1 -- 172.21.15.49:0/2582873609 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2828014070 con 0x7f2838103590 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.324+0000 7f284139c740 1 -- 172.21.15.49:0/586426717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 msgr2=0x56318ea644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.324+0000 7f284139c740 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea644b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2824011440 tx=0x7f2824011900 comp rx=0 tx=0).stop 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.324+0000 7f2837400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f281c0a2440 0x7f281c0a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.325+0000 7f284139c740 1 -- 172.21.15.49:0/586426717 shutdown_connections 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.325+0000 7f284139c740 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea78780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.325+0000 7f284139c740 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.325+0000 7f284139c740 1 --2- 172.21.15.49:0/586426717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea644b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.325+0000 7f284139c740 1 -- 172.21.15.49:0/586426717 >> 172.21.15.49:0/586426717 conn(0x56318ea5eb50 msgr2=0x56318ea5ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.325+0000 7f284139c740 1 -- 172.21.15.49:0/586426717 shutdown_connections 2024-02-26T06:25:27.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.325+0000 7f2837400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f281c0a2440 0x7f281c0a4900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2828097000 tx=0x7f2828005880 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.325+0000 7f284139c740 1 -- 172.21.15.49:0/586426717 wait complete. 2024-02-26T06:25:27.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.326+0000 7f284139c740 1 -- start start 2024-02-26T06:25:27.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.326+0000 7f284139c740 1 Processor -- start 2024-02-26T06:25:27.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.327+0000 7f284139c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea5b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.327+0000 7f284139c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.328+0000 7f2837e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea5b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.328+0000 7f2837e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea5b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39728/0 (socket says 172.21.15.49:39728) 2024-02-26T06:25:27.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.328+0000 7f284139c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea5bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.328+0000 7f2837e00640 1 -- 172.21.15.49:0/1903760042 learned_addr learned my addr 172.21.15.49:0/1903760042 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.328+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e968120 con 0x56318ea649f0 2024-02-26T06:25:27.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.328+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e967dc0 con 0x56318ea76390 2024-02-26T06:25:27.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.328+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318e7311c0 con 0x56318ea640d0 2024-02-26T06:25:27.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.329+0000 7f284139c740 10 client.0 _ll_register_callbacks cb 0x56318e9b7280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:27.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.329+0000 7f2836a00640 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.329+0000 7f2837e00640 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea5bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.330+0000 7f2837e00640 1 -- 172.21.15.49:0/1903760042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 msgr2=0x56318ea5bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.330+0000 7f2837e00640 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea5bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.330+0000 7f2837e00640 1 -- 172.21.15.49:0/1903760042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 msgr2=0x56318ea5b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.330+0000 7f2837e00640 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.330+0000 7f2837e00640 1 -- 172.21.15.49:0/1903760042 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2824009cc0 con 0x56318ea640d0 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.330+0000 7f2836a00640 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.330+0000 7f2837e00640 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea5b190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2824005890 tx=0x7f2824003d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f964298b583a36c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.330+0000 7f2821600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.331+0000 7f2821600640 1 -- 172.21.15.49:0/1903760042 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824005250 con 0x56318ea640d0 2024-02-26T06:25:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.331+0000 7f2821600640 1 -- 172.21.15.49:0/1903760042 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f28240053b0 con 0x56318ea640d0 2024-02-26T06:25:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.331+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56318e6c27e0 con 0x56318ea640d0 2024-02-26T06:25:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.331+0000 7f2821600640 1 -- 172.21.15.49:0/1903760042 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2824005570 con 0x56318ea640d0 2024-02-26T06:25:27.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.333+0000 7f284139c740 10 client.25963 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:27.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.333+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56318e9c1570 con 0x56318ea640d0 2024-02-26T06:25:27.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.333+0000 7f2821600640 1 -- 172.21.15.49:0/1903760042 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56318e6c27e0 con 0x56318ea640d0 2024-02-26T06:25:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.334+0000 7f2821600640 1 -- 172.21.15.49:0/1903760042 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f282402e070 con 0x56318ea640d0 2024-02-26T06:25:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.334+0000 7f284139c740 10 client.25963 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.335+0000 7f284139c740 10 client.25963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:25:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.335+0000 7f284139c740 20 client.25963 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.335+0000 7f284139c740 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56318ea860d0 0x56318ea884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.335+0000 7f284139c740 4 client.25963 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.335+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56318e6ff720 con 0x56318ea860d0 2024-02-26T06:25:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.336+0000 7f2836a00640 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56318ea860d0 0x56318ea884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.337+0000 7f2836a00640 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56318ea860d0 0x56318ea884f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=188e97109085771c server_cookie=59c84b89728e94f0 in_seq=0 out_seq=0 2024-02-26T06:25:27.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.337+0000 7f2821600640 10 client.25963 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:25:27.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.337+0000 7f2821600640 1 -- 172.21.15.49:0/1903760042 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56318e6ff720 con 0x56318ea860d0 2024-02-26T06:25:27.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.337+0000 7f2821600640 10 client.25963 handle_command_reply: tid=0 2024-02-26T06:25:27.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.338+0000 7f284139c740 1 client.25963 shutdown 2024-02-26T06:25:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.339+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 msgr2=0x56318ea5b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.339+0000 7f284139c740 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea5b190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2824005890 tx=0x7f2824003d90 comp rx=0 tx=0).stop 2024-02-26T06:25:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 shutdown_connections 2024-02-26T06:25:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f284139c740 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56318ea860d0 0x56318ea884f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f284139c740 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea5bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f284139c740 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea649f0 0x56318ea5b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f284139c740 1 --2- 172.21.15.49:0/1903760042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea640d0 0x56318ea5b190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 >> 172.21.15.49:0/1903760042 conn(0x56318ea5eb50 msgr2=0x56318ea608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f2821600640 0 client.25963 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:25:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 shutdown_connections 2024-02-26T06:25:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.340+0000 7f284139c740 1 -- 172.21.15.49:0/1903760042 wait complete. 2024-02-26T06:25:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.341+0000 7f284139c740 20 client.25963 trim_cache size 0 max 16384 2024-02-26T06:25:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.363+0000 7f284139c740 1 Processor -- start 2024-02-26T06:25:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.363+0000 7f284139c740 1 -- start start 2024-02-26T06:25:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.363+0000 7f284139c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea5d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.363+0000 7f284139c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea649f0 0x56318ea8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.363+0000 7f284139c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea76390 0x56318ea7d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.363+0000 7f284139c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318ea81fc0 con 0x56318ea640d0 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.363+0000 7f284139c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318ea50850 con 0x56318ea649f0 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.363+0000 7f284139c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318ea596e0 con 0x56318ea76390 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea649f0 0x56318ea8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea649f0 0x56318ea8ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44908/0 (socket says 172.21.15.49:44908) 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 -- 172.21.15.49:0/3448389400 learned_addr learned my addr 172.21.15.49:0/3448389400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2837e00640 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea5d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2837e00640 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea76390 0x56318ea7d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 -- 172.21.15.49:0/3448389400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea76390 msgr2=0x56318ea7d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea76390 0x56318ea7d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 -- 172.21.15.49:0/3448389400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 msgr2=0x56318ea5d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea5d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 -- 172.21.15.49:0/3448389400 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318e95da80 con 0x56318ea649f0 2024-02-26T06:25:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2837e00640 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea76390 0x56318ea7d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:25:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.364+0000 7f2836a00640 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea649f0 0x56318ea8ce30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f283002cf80 tx=0x7f2830009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d100f7b5eb44e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.365+0000 7f2823e00640 1 -- 172.21.15.49:0/3448389400 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830043040 con 0x56318ea649f0 2024-02-26T06:25:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.365+0000 7f2823e00640 1 -- 172.21.15.49:0/3448389400 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2830033e40 con 0x56318ea649f0 2024-02-26T06:25:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.365+0000 7f2823e00640 1 -- 172.21.15.49:0/3448389400 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830004460 con 0x56318ea649f0 2024-02-26T06:25:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 -- 172.21.15.49:0/3448389400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea649f0 msgr2=0x56318ea8ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea649f0 0x56318ea8ce30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f283002cf80 tx=0x7f2830009500 comp rx=0 tx=0).stop 2024-02-26T06:25:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 -- 172.21.15.49:0/3448389400 shutdown_connections 2024-02-26T06:25:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea649f0 0x56318ea8ce30 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f283002cf80 tx=0x7f2830009500 comp rx=0 tx=0).stop 2024-02-26T06:25:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea76390 0x56318ea7d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 --2- 172.21.15.49:0/3448389400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea5d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 -- 172.21.15.49:0/3448389400 >> 172.21.15.49:0/3448389400 conn(0x56318ea5eb50 msgr2=0x56318ea558e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 -- 172.21.15.49:0/3448389400 shutdown_connections 2024-02-26T06:25:27.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.366+0000 7f284139c740 1 -- 172.21.15.49:0/3448389400 wait complete. 2024-02-26T06:25:27.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.367+0000 7f284139c740 1 -- start start 2024-02-26T06:25:27.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.367+0000 7f284139c740 1 Processor -- start 2024-02-26T06:25:27.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f284139c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea7d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f284139c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea7b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f284139c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea7ba40 0x56318ea7bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f284139c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318ea7d8a0 con 0x56318ea640d0 2024-02-26T06:25:27.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f284139c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318eadc5c0 con 0x56318ea76390 2024-02-26T06:25:27.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f284139c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318eadc700 con 0x56318ea7ba40 2024-02-26T06:25:27.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f284139c740 10 client.0 _ll_register_callbacks cb 0x7f2824009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:27.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f2837e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea7d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f2837e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea7d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39366/0 (socket says 172.21.15.49:39366) 2024-02-26T06:25:27.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f2837e00640 1 -- 172.21.15.49:0/3070981309 learned_addr learned my addr 172.21.15.49:0/3070981309 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f2837e00640 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea7ba40 0x56318ea7bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.368+0000 7f2836a00640 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea7b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2837e00640 1 -- 172.21.15.49:0/3070981309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 msgr2=0x56318ea7b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2837e00640 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea7b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2837e00640 1 -- 172.21.15.49:0/3070981309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 msgr2=0x56318ea7d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2837e00640 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea7d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2837e00640 1 -- 172.21.15.49:0/3070981309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2838110a40 con 0x56318ea7ba40 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2836a00640 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea7b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2837e00640 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea7ba40 0x56318ea7bec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2824000c00 tx=0x7f2824003d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bd179279b8d9049 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2821600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:27.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2821600640 1 -- 172.21.15.49:0/3070981309 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824009ee0 con 0x56318ea7ba40 2024-02-26T06:25:27.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f2821600640 1 -- 172.21.15.49:0/3070981309 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f282400f540 con 0x56318ea7ba40 2024-02-26T06:25:27.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.369+0000 7f284139c740 1 -- 172.21.15.49:0/3070981309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2824004180 con 0x56318ea7ba40 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.370+0000 7f2821600640 1 -- 172.21.15.49:0/3070981309 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f282400f700 con 0x56318ea7ba40 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.370+0000 7f284139c740 10 client.25966 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.370+0000 7f284139c740 1 -- 172.21.15.49:0/3070981309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56318e983620 con 0x56318ea7ba40 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.370+0000 7f2821600640 1 -- 172.21.15.49:0/3070981309 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2824048080 con 0x56318ea7ba40 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.370+0000 7f2821600640 1 -- 172.21.15.49:0/3070981309 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2824039040 con 0x56318ea7ba40 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.371+0000 7f284139c740 10 client.25966 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.371+0000 7f284139c740 10 client.25966 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.371+0000 7f284139c740 20 client.25966 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.371+0000 7f284139c740 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56318eaddae0 0x56318eadff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.371+0000 7f284139c740 4 client.25966 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.371+0000 7f284139c740 1 -- 172.21.15.49:0/3070981309 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56318e74e890 con 0x56318eaddae0 2024-02-26T06:25:27.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.371+0000 7f2836a00640 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56318eaddae0 0x56318eadff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.374+0000 7f2836a00640 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56318eaddae0 0x56318eadff00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dbd803e31677e23 server_cookie=4219517f900717c1 in_seq=0 out_seq=0 2024-02-26T06:25:27.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.374+0000 7f2821600640 10 client.25966 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:25:27.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.374+0000 7f2821600640 1 -- 172.21.15.49:0/3070981309 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56318e74e890 con 0x56318eaddae0 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.374+0000 7f2821600640 10 client.25966 handle_command_reply: tid=0 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.374+0000 7f284139c740 1 client.25966 shutdown 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.374+0000 7f284139c740 1 -- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea7ba40 msgr2=0x56318ea7bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.374+0000 7f284139c740 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea7ba40 0x56318ea7bec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2824000c00 tx=0x7f2824003d90 comp rx=0 tx=0).stop 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 1 -- 172.21.15.49:0/3070981309 shutdown_connections 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56318eaddae0 0x56318eadff00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56318ea76390 0x56318ea7b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56318ea7ba40 0x56318ea7bec0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 1 --2- 172.21.15.49:0/3070981309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318ea640d0 0x56318ea7d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 1 -- 172.21.15.49:0/3070981309 >> 172.21.15.49:0/3070981309 conn(0x56318ea5eb50 msgr2=0x56318ea56450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f2821600640 0 client.25966 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 1 -- 172.21.15.49:0/3070981309 shutdown_connections 2024-02-26T06:25:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 1 -- 172.21.15.49:0/3070981309 wait complete. 2024-02-26T06:25:27.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.375+0000 7f284139c740 20 client.25966 trim_cache size 0 max 16384 2024-02-26T06:25:27.377 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.376+0000 7f283e400640 1 -- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f281c0a2440 msgr2=0x7f281c0a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.376+0000 7f283e400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f281c0a2440 0x7f281c0a4900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2828097000 tx=0x7f2828005880 comp rx=0 tx=0).stop 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.376+0000 7f283e400640 1 -- 172.21.15.49:0/2582873609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103590 msgr2=0x7f283819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.376+0000 7f283e400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103590 0x7f283819f6c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f282800e9c0 tx=0x7f282800ee80 comp rx=0 tx=0).stop 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.377+0000 7f283e400640 1 -- 172.21.15.49:0/2582873609 shutdown_connections 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.377+0000 7f283e400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f281c0a2440 0x7f281c0a4900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.377+0000 7f283e400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2838199830 0x7f2838199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.377+0000 7f283e400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2838108660 0x7f283819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.377+0000 7f283e400640 1 --2- 172.21.15.49:0/2582873609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2838103590 0x7f283819f6c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.377+0000 7f283e400640 1 -- 172.21.15.49:0/2582873609 >> 172.21.15.49:0/2582873609 conn(0x7f28380fedf0 msgr2=0x7f2838101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.377+0000 7f283e400640 1 -- 172.21.15.49:0/2582873609 shutdown_connections 2024-02-26T06:25:27.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.378+0000 7f283e400640 1 -- 172.21.15.49:0/2582873609 wait complete. 2024-02-26T06:25:27.401 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:25:27.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:25:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 -- 172.21.15.49:0/2073121698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5468068380 msgr2=0x7f5468068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 --2- 172.21.15.49:0/2073121698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5468068380 0x7f5468068760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5458009860 tx=0x7f545802bc10 comp rx=0 tx=0).stop 2024-02-26T06:25:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 -- 172.21.15.49:0/2073121698 shutdown_connections 2024-02-26T06:25:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 --2- 172.21.15.49:0/2073121698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f546810db20 0x7f546810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 --2- 172.21.15.49:0/2073121698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5468068ca0 0x7f546810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 --2- 172.21.15.49:0/2073121698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5468068380 0x7f5468068760 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 -- 172.21.15.49:0/2073121698 >> 172.21.15.49:0/2073121698 conn(0x7f5468077180 msgr2=0x7f5468077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 -- 172.21.15.49:0/2073121698 shutdown_connections 2024-02-26T06:25:27.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.702+0000 7f546ee00640 1 -- 172.21.15.49:0/2073121698 wait complete. 2024-02-26T06:25:27.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.704+0000 7f546ee00640 1 Processor -- start 2024-02-26T06:25:27.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.704+0000 7f546ee00640 1 -- start start 2024-02-26T06:25:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f546ee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5468068380 0x7f546819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f546ee00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5468068ca0 0x7f546819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f546ee00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810db20 0x7f5468199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f546ee00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468199ec0 con 0x7f5468068ca0 2024-02-26T06:25:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f546ee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546819a030 con 0x7f5468068380 2024-02-26T06:25:27.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f546ee00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546819a1f0 con 0x7f546810db20 2024-02-26T06:25:27.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f5467e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5468068380 0x7f546819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f5467e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5468068380 0x7f546819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44940/0 (socket says 172.21.15.49:44940) 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f5467e00640 1 -- 172.21.15.49:0/2170571574 learned_addr learned my addr 172.21.15.49:0/2170571574 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.705+0000 7f546ce00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810db20 0x7f5468199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.706+0000 7f5467400640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5468068ca0 0x7f546819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.706+0000 7f5467e00640 1 -- 172.21.15.49:0/2170571574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810db20 msgr2=0x7f5468199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.706+0000 7f5467e00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810db20 0x7f5468199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.706+0000 7f5467e00640 1 -- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5468068ca0 msgr2=0x7f546819fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.706+0000 7f5467e00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5468068ca0 0x7f546819fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.706+0000 7f5467e00640 1 -- 172.21.15.49:0/2170571574 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5458009490 con 0x7f5468068380 2024-02-26T06:25:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.707+0000 7f546ce00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810db20 0x7f5468199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.707+0000 7f5467e00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5468068380 0x7f546819f670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f545802c150 tx=0x7f54580344c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.707+0000 7f5464c00640 1 -- 172.21.15.49:0/2170571574 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545802c5b0 con 0x7f5468068380 2024-02-26T06:25:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.707+0000 7f5464c00640 1 -- 172.21.15.49:0/2170571574 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f545802cbf0 con 0x7f5468068380 2024-02-26T06:25:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.707+0000 7f5464c00640 1 -- 172.21.15.49:0/2170571574 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580406b0 con 0x7f5468068380 2024-02-26T06:25:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.707+0000 7f546ee00640 1 -- 172.21.15.49:0/2170571574 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546819a470 con 0x7f5468068380 2024-02-26T06:25:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.707+0000 7f546ee00640 1 -- 172.21.15.49:0/2170571574 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5468102360 con 0x7f5468068380 2024-02-26T06:25:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.713+0000 7f5471bd6740 1 Processor -- start 2024-02-26T06:25:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.713+0000 7f5471bd6740 1 -- start start 2024-02-26T06:25:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.713+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fecdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.713+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fece530 0x56338fed6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.713+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fed7110 0x56338fed9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.713+0000 7f5471bd6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fe3c1e0 con 0x56338fece530 2024-02-26T06:25:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.713+0000 7f5471bd6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fe39710 con 0x56338fecdc10 2024-02-26T06:25:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.713+0000 7f5471bd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fe34ef0 con 0x56338fed7110 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f546ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fecdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f5467400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fed7110 0x56338fed9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f5467400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fed7110 0x56338fed9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39768/0 (socket says 172.21.15.49:39768) 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f5467400640 1 -- 172.21.15.49:0/3265323285 learned_addr learned my addr 172.21.15.49:0/3265323285 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f546ce00640 1 -- 172.21.15.49:0/3265323285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fed7110 msgr2=0x56338fed9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f546ce00640 1 --2- 172.21.15.49:0/3265323285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fed7110 0x56338fed9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f546ce00640 1 -- 172.21.15.49:0/3265323285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fece530 msgr2=0x56338fed6bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f546ce00640 1 --2- 172.21.15.49:0/3265323285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fece530 0x56338fed6bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.714+0000 7f546ce00640 1 -- 172.21.15.49:0/3265323285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338fdf6dc0 con 0x56338fecdc10 2024-02-26T06:25:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.715+0000 7f546ce00640 1 --2- 172.21.15.49:0/3265323285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fecdff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5460009f90 tx=0x7f5460010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f93fd96d38bbb3f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.715+0000 7f545d600640 1 -- 172.21.15.49:0/3265323285 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460011700 con 0x56338fecdc10 2024-02-26T06:25:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.715+0000 7f545d600640 1 -- 172.21.15.49:0/3265323285 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5460015070 con 0x56338fecdc10 2024-02-26T06:25:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.716+0000 7f5464c00640 1 -- 172.21.15.49:0/2170571574 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f54580037f0 con 0x7f5468068380 2024-02-26T06:25:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.716+0000 7f5471bd6740 1 -- 172.21.15.49:0/3265323285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 msgr2=0x56338fecdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.716+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3265323285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fecdff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5460009f90 tx=0x7f5460010860 comp rx=0 tx=0).stop 2024-02-26T06:25:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.717+0000 7f5471bd6740 1 -- 172.21.15.49:0/3265323285 shutdown_connections 2024-02-26T06:25:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.717+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3265323285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fed7110 0x56338fed9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.717+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3265323285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fece530 0x56338fed6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.717+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3265323285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fecdff0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.717+0000 7f5471bd6740 1 -- 172.21.15.49:0/3265323285 >> 172.21.15.49:0/3265323285 conn(0x56338fec8340 msgr2=0x56338fec8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.718+0000 7f5464c00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f544c0a2330 0x7f544c0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.718+0000 7f5471bd6740 1 -- 172.21.15.49:0/3265323285 shutdown_connections 2024-02-26T06:25:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.719+0000 7f5464c00640 1 -- 172.21.15.49:0/2170571574 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f54580efb20 con 0x7f5468068380 2024-02-26T06:25:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.719+0000 7f5467400640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f544c0a2330 0x7f544c0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.719+0000 7f5471bd6740 1 -- 172.21.15.49:0/3265323285 wait complete. 2024-02-26T06:25:27.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.720+0000 7f5471bd6740 1 -- start start 2024-02-26T06:25:27.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.720+0000 7f5471bd6740 1 Processor -- start 2024-02-26T06:25:27.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.720+0000 7f5467400640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f544c0a2330 0x7f544c0a47f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5454005fd0 tx=0x7f5454009260 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.721+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fed4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 0x56338fed4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5471bd6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fdda120 con 0x56338fed7110 2024-02-26T06:25:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5471bd6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fdd9dc0 con 0x56338fecdc10 2024-02-26T06:25:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5471bd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fba31c0 con 0x56338fece530 2024-02-26T06:25:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5471bd6740 10 client.0 _ll_register_callbacks cb 0x56338fe29280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5467400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5467400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39784/0 (socket says 172.21.15.49:39784) 2024-02-26T06:25:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5467400640 1 -- 172.21.15.49:0/3840831910 learned_addr learned my addr 172.21.15.49:0/3840831910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.722+0000 7f5467e00640 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fed4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.723+0000 7f5467e00640 1 -- 172.21.15.49:0/3840831910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 msgr2=0x56338fed4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.723+0000 7f5467e00640 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.723+0000 7f5467e00640 1 -- 172.21.15.49:0/3840831910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 msgr2=0x56338fed4b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:25:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.724+0000 7f546ce00640 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 0x56338fed4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.724+0000 7f5467e00640 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 0x56338fed4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.724+0000 7f5467e00640 1 -- 172.21.15.49:0/3840831910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5460009bf0 con 0x56338fecdc10 2024-02-26T06:25:27.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.724+0000 7f5467e00640 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fed4100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54580ed330 tx=0x7f54580bc260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269dec45f98ec925 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.725+0000 7f5452000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:25:27.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.725+0000 7f5452000640 1 -- 172.21.15.49:0/3840831910 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580bcd60 con 0x56338fecdc10 2024-02-26T06:25:27.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.725+0000 7f5452000640 1 -- 172.21.15.49:0/3840831910 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f54580035f0 con 0x56338fecdc10 2024-02-26T06:25:27.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.725+0000 7f5471bd6740 1 -- 172.21.15.49:0/3840831910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56338fb347e0 con 0x56338fecdc10 2024-02-26T06:25:27.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.725+0000 7f5452000640 1 -- 172.21.15.49:0/3840831910 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54580bc660 con 0x56338fecdc10 2024-02-26T06:25:27.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.725+0000 7f5471bd6740 10 client.25937 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:27.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.725+0000 7f5471bd6740 1 -- 172.21.15.49:0/3840831910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56338fe33570 con 0x56338fecdc10 2024-02-26T06:25:27.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.726+0000 7f5452000640 1 -- 172.21.15.49:0/3840831910 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54580d4070 con 0x56338fecdc10 2024-02-26T06:25:27.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.726+0000 7f5452000640 1 -- 172.21.15.49:0/3840831910 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f54580bc8c0 con 0x56338fecdc10 2024-02-26T06:25:27.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.726+0000 7f5471bd6740 10 client.25937 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:27.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.726+0000 7f5471bd6740 10 client.25937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:25:27.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.726+0000 7f5471bd6740 20 client.25937 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:27.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.726+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56338fee6f30 0x56338fee9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.726+0000 7f5471bd6740 4 client.25937 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:27.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.726+0000 7f5471bd6740 1 -- 172.21.15.49:0/3840831910 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56338fb71720 con 0x56338fee6f30 2024-02-26T06:25:27.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.727+0000 7f5467400640 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56338fee6f30 0x56338fee9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.729+0000 7f5467400640 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56338fee6f30 0x56338fee9350 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ca3077933f3b4a6 server_cookie=1a64640d80383120 in_seq=0 out_seq=0 2024-02-26T06:25:27.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.729+0000 7f5452000640 10 client.25937 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:25:27.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.729+0000 7f5452000640 1 -- 172.21.15.49:0/3840831910 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56338fb71720 con 0x56338fee6f30 2024-02-26T06:25:27.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.729+0000 7f5452000640 10 client.25937 handle_command_reply: tid=0 2024-02-26T06:25:27.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.730+0000 7f5471bd6740 1 client.25937 shutdown 2024-02-26T06:25:27.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.730+0000 7f5471bd6740 1 -- 172.21.15.49:0/3840831910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 msgr2=0x56338fed4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.730+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fed4100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54580ed330 tx=0x7f54580bc260 comp rx=0 tx=0).stop 2024-02-26T06:25:27.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.731+0000 7f5471bd6740 1 -- 172.21.15.49:0/3840831910 shutdown_connections 2024-02-26T06:25:27.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.731+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56338fee6f30 0x56338fee9350 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.731+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 0x56338fed4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.731+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.731+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3840831910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fecdc10 0x56338fed4100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.731+0000 7f5471bd6740 1 -- 172.21.15.49:0/3840831910 >> 172.21.15.49:0/3840831910 conn(0x56338fec8340 msgr2=0x56338feca090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.731+0000 7f5452000640 0 client.25937 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:25:27.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.731+0000 7f5471bd6740 1 -- 172.21.15.49:0/3840831910 shutdown_connections 2024-02-26T06:25:27.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.732+0000 7f5471bd6740 1 -- 172.21.15.49:0/3840831910 wait complete. 2024-02-26T06:25:27.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.732+0000 7f5471bd6740 20 client.25937 trim_cache size 0 max 16384 2024-02-26T06:25:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.760+0000 7f5471bd6740 1 Processor -- start 2024-02-26T06:25:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.760+0000 7f5471bd6740 1 -- start start 2024-02-26T06:25:27.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fecdc10 0x56338fed6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fed7110 0x56338fed18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5471bd6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fed9500 con 0x56338fecdc10 2024-02-26T06:25:27.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5471bd6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fee6ce0 con 0x56338fed7110 2024-02-26T06:25:27.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5471bd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338fecd3b0 con 0x56338fece530 2024-02-26T06:25:27.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5467400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5467400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39788/0 (socket says 172.21.15.49:39788) 2024-02-26T06:25:27.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f5467400640 1 -- 172.21.15.49:0/1782612381 learned_addr learned my addr 172.21.15.49:0/1782612381 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.761+0000 7f546ce00640 1 --2- 172.21.15.49:0/1782612381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fed7110 0x56338fed18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.762+0000 7f5467400640 1 -- 172.21.15.49:0/1782612381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fed7110 msgr2=0x56338fed18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.762+0000 7f5467400640 1 --2- 172.21.15.49:0/1782612381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fed7110 0x56338fed18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.762+0000 7f5467400640 1 -- 172.21.15.49:0/1782612381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fecdc10 msgr2=0x56338fed6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.762+0000 7f5467400640 1 --2- 172.21.15.49:0/1782612381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fecdc10 0x56338fed6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.762+0000 7f5467400640 1 -- 172.21.15.49:0/1782612381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338fdcfa80 con 0x56338fece530 2024-02-26T06:25:27.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.762+0000 7f5467400640 1 --2- 172.21.15.49:0/1782612381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed1380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x56338fed3960 tx=0x7f5454015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4977bf4983aa832 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.762+0000 7f545cc00640 1 -- 172.21.15.49:0/1782612381 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454022040 con 0x56338fece530 2024-02-26T06:25:27.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.763+0000 7f545cc00640 1 -- 172.21.15.49:0/1782612381 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f545401ddb0 con 0x56338fece530 2024-02-26T06:25:27.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 -- 172.21.15.49:0/1782612381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 msgr2=0x56338fed1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f545cc00640 1 -- 172.21.15.49:0/1782612381 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454022040 con 0x56338fece530 2024-02-26T06:25:27.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 --2- 172.21.15.49:0/1782612381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed1380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x56338fed3960 tx=0x7f5454015000 comp rx=0 tx=0).stop 2024-02-26T06:25:27.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 -- 172.21.15.49:0/1782612381 shutdown_connections 2024-02-26T06:25:27.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 --2- 172.21.15.49:0/1782612381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fed7110 0x56338fed18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 --2- 172.21.15.49:0/1782612381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fece530 0x56338fed1380 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 --2- 172.21.15.49:0/1782612381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fecdc10 0x56338fed6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 -- 172.21.15.49:0/1782612381 >> 172.21.15.49:0/1782612381 conn(0x56338fec8340 msgr2=0x56338fecf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 -- 172.21.15.49:0/1782612381 shutdown_connections 2024-02-26T06:25:27.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.764+0000 7f5471bd6740 1 -- 172.21.15.49:0/1782612381 wait complete. 2024-02-26T06:25:27.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.765+0000 7f5471bd6740 1 -- start start 2024-02-26T06:25:27.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.766+0000 7f5471bd6740 1 Processor -- start 2024-02-26T06:25:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fecdc10 0x56338ff333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fece530 0x56338ff338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5471bd6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 0x56338ff33e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5467e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fecdc10 0x56338ff333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5467e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fecdc10 0x56338ff333b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39796/0 (socket says 172.21.15.49:39796) 2024-02-26T06:25:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5471bd6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338ff3e430 con 0x56338fed7110 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338ff3e570 con 0x56338fece530 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5467e00640 1 -- 172.21.15.49:0/3018279314 learned_addr learned my addr 172.21.15.49:0/3018279314 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5467400640 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fece530 0x56338ff338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338ff3e6e0 con 0x56338fecdc10 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.768+0000 7f5471bd6740 10 client.0 _ll_register_callbacks cb 0x7f54600099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5467e00640 1 -- 172.21.15.49:0/3018279314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fece530 msgr2=0x56338ff338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5467e00640 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fece530 0x56338ff338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5467e00640 1 -- 172.21.15.49:0/3018279314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 msgr2=0x56338ff33e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5467e00640 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 0x56338ff33e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5467e00640 1 -- 172.21.15.49:0/3018279314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545400f740 con 0x56338fecdc10 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5467e00640 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fecdc10 0x56338ff333b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f54580b9030 tx=0x7f5458005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb3aa887c36fa4d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5452000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:27.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5452000640 1 -- 172.21.15.49:0/3018279314 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580b8900 con 0x56338fecdc10 2024-02-26T06:25:27.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5452000640 1 -- 172.21.15.49:0/3018279314 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f54580035f0 con 0x56338fecdc10 2024-02-26T06:25:27.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5460015070 con 0x56338fecdc10 2024-02-26T06:25:27.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5452000640 1 -- 172.21.15.49:0/3018279314 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54580bcd60 con 0x56338fecdc10 2024-02-26T06:25:27.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5471bd6740 10 client.25984 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.769+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56338fdf5620 con 0x56338fecdc10 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.770+0000 7f5452000640 1 -- 172.21.15.49:0/3018279314 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5460015070 con 0x56338fecdc10 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.770+0000 7f5452000640 1 -- 172.21.15.49:0/3018279314 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f54580bc8c0 con 0x56338fecdc10 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.770+0000 7f5471bd6740 10 client.25984 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.770+0000 7f5471bd6740 10 client.25984 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.770+0000 7f5471bd6740 20 client.25984 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.770+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56338ff3fa50 0x56338ff41e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.770+0000 7f5471bd6740 4 client.25984 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.770+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56338fbc0890 con 0x56338ff3fa50 2024-02-26T06:25:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.771+0000 7f5467400640 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56338ff3fa50 0x56338ff41e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.771+0000 7f5467400640 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56338ff3fa50 0x56338ff41e40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfc76aa91b3cb578 server_cookie=ac45620e4d3e3ff1 in_seq=0 out_seq=0 2024-02-26T06:25:27.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.771+0000 7f5452000640 10 client.25984 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:25:27.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.772+0000 7f5452000640 1 -- 172.21.15.49:0/3018279314 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56338fbc0890 con 0x56338ff3fa50 2024-02-26T06:25:27.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.772+0000 7f5452000640 10 client.25984 handle_command_reply: tid=0 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.772+0000 7f5471bd6740 1 client.25984 shutdown 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.772+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fecdc10 msgr2=0x56338ff333b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.772+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fecdc10 0x56338ff333b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f54580b9030 tx=0x7f5458005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 shutdown_connections 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56338ff3fa50 0x56338ff41e40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56338fece530 0x56338ff338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56338fed7110 0x56338ff33e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 1 --2- 172.21.15.49:0/3018279314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56338fecdc10 0x56338ff333b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5452000640 0 client.25984 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 >> 172.21.15.49:0/3018279314 conn(0x56338fec8340 msgr2=0x56338fed40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 shutdown_connections 2024-02-26T06:25:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 1 -- 172.21.15.49:0/3018279314 wait complete. 2024-02-26T06:25:27.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.773+0000 7f5471bd6740 20 client.25984 trim_cache size 0 max 16384 2024-02-26T06:25:27.775 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:25:27.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 -- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f544c0a2330 msgr2=0x7f544c0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f544c0a2330 0x7f544c0a47f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5454005fd0 tx=0x7f5454009260 comp rx=0 tx=0).stop 2024-02-26T06:25:27.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 -- 172.21.15.49:0/2170571574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5468068380 msgr2=0x7f546819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5468068380 0x7f546819f670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f545802c150 tx=0x7f54580344c0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 -- 172.21.15.49:0/2170571574 shutdown_connections 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f544c0a2330 0x7f544c0a47f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810db20 0x7f5468199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5468068ca0 0x7f546819fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 --2- 172.21.15.49:0/2170571574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5468068380 0x7f546819f670 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 -- 172.21.15.49:0/2170571574 >> 172.21.15.49:0/2170571574 conn(0x7f5468077180 msgr2=0x7f546810f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.776+0000 7f546ee00640 1 -- 172.21.15.49:0/2170571574 shutdown_connections 2024-02-26T06:25:27.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:27.777+0000 7f546ee00640 1 -- 172.21.15.49:0/2170571574 wait complete. 2024-02-26T06:25:27.802 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:25:27.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:25:28.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:27 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2965860803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:28.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:27 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2965860803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.099+0000 7f577be00640 1 -- 172.21.15.49:0/1819573420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5774103f40 msgr2=0x7f5774107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.099+0000 7f577be00640 1 --2- 172.21.15.49:0/1819573420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5774103f40 0x7f5774107f80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.101+0000 7f577be00640 1 -- 172.21.15.49:0/1819573420 shutdown_connections 2024-02-26T06:25:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.101+0000 7f577be00640 1 --2- 172.21.15.49:0/1819573420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5774108660 0x7f5774110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.101+0000 7f577be00640 1 --2- 172.21.15.49:0/1819573420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5774103f40 0x7f5774107f80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.101+0000 7f577be00640 1 --2- 172.21.15.49:0/1819573420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774103590 0x7f5774103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.101+0000 7f577be00640 1 -- 172.21.15.49:0/1819573420 >> 172.21.15.49:0/1819573420 conn(0x7f57740fedf0 msgr2=0x7f5774101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.101+0000 7f577be00640 1 -- 172.21.15.49:0/1819573420 shutdown_connections 2024-02-26T06:25:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.102+0000 7f577be00640 1 -- 172.21.15.49:0/1819573420 wait complete. 2024-02-26T06:25:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.103+0000 7f577be00640 1 Processor -- start 2024-02-26T06:25:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.103+0000 7f577be00640 1 -- start start 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f577be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5774103590 0x7f5774075690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f577be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774108660 0x7f5774075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f577be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5774076110 0x7f5774071820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f577be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774079870 con 0x7f5774108660 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f577be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774071d60 con 0x7f5774076110 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f577be00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774071f20 con 0x7f5774103590 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f5779e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5774076110 0x7f5774071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f5779e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5774076110 0x7f5774071820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44996/0 (socket says 172.21.15.49:44996) 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f5779e00640 1 -- 172.21.15.49:0/3248374052 learned_addr learned my addr 172.21.15.49:0/3248374052 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f5779400640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5774103590 0x7f5774075690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.104+0000 7f5778a00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774108660 0x7f5774075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.105+0000 7f5779e00640 1 -- 172.21.15.49:0/3248374052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5774103590 msgr2=0x7f5774075690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.105+0000 7f5779e00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5774103590 0x7f5774075690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.105+0000 7f5779e00640 1 -- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774108660 msgr2=0x7f5774075bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.105+0000 7f5779e00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774108660 0x7f5774075bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.105+0000 7f5779e00640 1 -- 172.21.15.49:0/3248374052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760009560 con 0x7f5774076110 2024-02-26T06:25:28.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.106+0000 7f5779e00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5774076110 0x7f5774071820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f576c00eb40 tx=0x7f576c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:28.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.106+0000 7f5772000640 1 -- 172.21.15.49:0/3248374052 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c004b20 con 0x7f5774076110 2024-02-26T06:25:28.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.106+0000 7f5772000640 1 -- 172.21.15.49:0/3248374052 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f576c004c80 con 0x7f5774076110 2024-02-26T06:25:28.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.106+0000 7f5772000640 1 -- 172.21.15.49:0/3248374052 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c010670 con 0x7f5774076110 2024-02-26T06:25:28.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.106+0000 7f577be00640 1 -- 172.21.15.49:0/3248374052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5774072200 con 0x7f5774076110 2024-02-26T06:25:28.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.107+0000 7f577be00640 1 -- 172.21.15.49:0/3248374052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57740726c0 con 0x7f5774076110 2024-02-26T06:25:28.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.112+0000 7f5772000640 1 -- 172.21.15.49:0/3248374052 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f576c005020 con 0x7f5774076110 2024-02-26T06:25:28.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.112+0000 7f577ed8a740 1 Processor -- start 2024-02-26T06:25:28.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.112+0000 7f577ed8a740 1 -- start start 2024-02-26T06:25:28.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.114+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 0x557af410cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.114+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410d530 0x557af4115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.114+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af4118500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.114+0000 7f577ed8a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af407b1e0 con 0x557af410cc10 2024-02-26T06:25:28.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.114+0000 7f577ed8a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af4078710 con 0x557af410d530 2024-02-26T06:25:28.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.114+0000 7f577ed8a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af4073ef0 con 0x557af4116110 2024-02-26T06:25:28.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410d530 0x557af4115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410d530 0x557af4115bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45004/0 (socket says 172.21.15.49:45004) 2024-02-26T06:25:28.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779400640 1 -- 172.21.15.49:0/2123033371 learned_addr learned my addr 172.21.15.49:0/2123033371 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:28.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5772000640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f575c0a22b0 0x7f575c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5772000640 1 -- 172.21.15.49:0/3248374052 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f576c014070 con 0x7f5774076110 2024-02-26T06:25:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779400640 1 -- 172.21.15.49:0/2123033371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 msgr2=0x557af4118500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:25:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779400640 1 --2- 172.21.15.49:0/2123033371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af4118500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779400640 1 -- 172.21.15.49:0/2123033371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 msgr2=0x557af410cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779400640 1 --2- 172.21.15.49:0/2123033371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 0x557af410cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779400640 1 -- 172.21.15.49:0/2123033371 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af4035dc0 con 0x557af410d530 2024-02-26T06:25:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.115+0000 7f5779e00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f575c0a22b0 0x7f575c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.116+0000 7f5779400640 1 --2- 172.21.15.49:0/2123033371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410d530 0x557af4115bd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f576800bcd0 tx=0x7f57680107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f0c9a656cdfe15 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.117+0000 7f5766600640 1 -- 172.21.15.49:0/2123033371 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768005220 con 0x557af410d530 2024-02-26T06:25:28.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.117+0000 7f5766600640 1 -- 172.21.15.49:0/2123033371 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5768015070 con 0x557af410d530 2024-02-26T06:25:28.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.117+0000 7f5766600640 1 -- 172.21.15.49:0/2123033371 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680116a0 con 0x557af410d530 2024-02-26T06:25:28.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.117+0000 7f5779e00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f575c0a22b0 0x7f575c0a4770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f576c0975a0 tx=0x7f576c00cfa0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:28.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.119+0000 7f577ed8a740 1 -- 172.21.15.49:0/2123033371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410d530 msgr2=0x557af4115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.119+0000 7f577ed8a740 1 --2- 172.21.15.49:0/2123033371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410d530 0x557af4115bd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f576800bcd0 tx=0x7f57680107a0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.119+0000 7f577ed8a740 1 -- 172.21.15.49:0/2123033371 shutdown_connections 2024-02-26T06:25:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.119+0000 7f577ed8a740 1 --2- 172.21.15.49:0/2123033371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410d530 0x557af4115bd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.119+0000 7f577ed8a740 1 --2- 172.21.15.49:0/2123033371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af4118500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.119+0000 7f577ed8a740 1 --2- 172.21.15.49:0/2123033371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 0x557af410cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.119+0000 7f577ed8a740 1 -- 172.21.15.49:0/2123033371 >> 172.21.15.49:0/2123033371 conn(0x557af4107340 msgr2=0x557af4107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:28.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.120+0000 7f577ed8a740 1 -- 172.21.15.49:0/2123033371 shutdown_connections 2024-02-26T06:25:28.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.120+0000 7f577ed8a740 1 -- 172.21.15.49:0/2123033371 wait complete. 2024-02-26T06:25:28.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.120+0000 7f577ed8a740 1 -- start start 2024-02-26T06:25:28.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.121+0000 7f577ed8a740 1 Processor -- start 2024-02-26T06:25:28.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.121+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410cc10 0x557af41131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.122+0000 7f5778a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410cc10 0x557af41131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.122+0000 7f5778a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410cc10 0x557af41131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45016/0 (socket says 172.21.15.49:45016) 2024-02-26T06:25:28.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.122+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410d530 0x557af41136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.122+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af4113c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.123+0000 7f5779400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410d530 0x557af41136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.123+0000 7f5779400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410d530 0x557af41136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39448/0 (socket says 172.21.15.49:39448) 2024-02-26T06:25:28.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.123+0000 7f5778a00640 1 -- 172.21.15.49:0/3895646734 learned_addr learned my addr 172.21.15.49:0/3895646734 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:28.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.123+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af4019120 con 0x557af410d530 2024-02-26T06:25:28.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.123+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af4018dc0 con 0x557af410cc10 2024-02-26T06:25:28.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.123+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af3de21c0 con 0x557af4116110 2024-02-26T06:25:28.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.123+0000 7f577ed8a740 10 client.0 _ll_register_callbacks cb 0x557af4068280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:28.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.125+0000 7f5779400640 1 -- 172.21.15.49:0/3895646734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 msgr2=0x557af4113c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.125+0000 7f5778a00640 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af4113c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.125+0000 7f5779400640 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af4113c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.125+0000 7f5779400640 1 -- 172.21.15.49:0/3895646734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410cc10 msgr2=0x557af41131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.125+0000 7f5779400640 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410cc10 0x557af41131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.125+0000 7f5779400640 1 -- 172.21.15.49:0/3895646734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576800b900 con 0x557af410d530 2024-02-26T06:25:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.126+0000 7f5778a00640 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af4113c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:25:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.127+0000 7f5779400640 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410d530 0x557af41136f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5768010ca0 tx=0x7f5768005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659af6702ae83cd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.127+0000 7f575aa00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:25:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.127+0000 7f575aa00640 1 -- 172.21.15.49:0/3895646734 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576801f070 con 0x557af410d530 2024-02-26T06:25:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.127+0000 7f575aa00640 1 -- 172.21.15.49:0/3895646734 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5768017070 con 0x557af410d530 2024-02-26T06:25:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.128+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557af3d737e0 con 0x557af410d530 2024-02-26T06:25:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.128+0000 7f575aa00640 1 -- 172.21.15.49:0/3895646734 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5768004b50 con 0x557af410d530 2024-02-26T06:25:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.128+0000 7f577ed8a740 10 client.15747 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.128+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557af4072570 con 0x557af410d530 2024-02-26T06:25:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.129+0000 7f575aa00640 1 -- 172.21.15.49:0/3895646734 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557af3d737e0 con 0x557af410d530 2024-02-26T06:25:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.131+0000 7f575aa00640 1 -- 172.21.15.49:0/3895646734 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f5768003ca0 con 0x557af410d530 2024-02-26T06:25:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.131+0000 7f577ed8a740 10 client.15747 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.131+0000 7f577ed8a740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.131+0000 7f577ed8a740 20 client.15747 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.131+0000 7f577ed8a740 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557af4126000 0x557af41283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.131+0000 7f577ed8a740 4 client.15747 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.131+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557af3db0720 con 0x557af4126000 2024-02-26T06:25:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.131+0000 7f5778a00640 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557af4126000 0x557af41283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.132+0000 7f575aa00640 10 client.15747 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.132+0000 7f5778a00640 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557af4126000 0x557af41283f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f0d34d1dbe36e9 server_cookie=fe64d2d9210e1674 in_seq=0 out_seq=0 2024-02-26T06:25:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f575aa00640 1 -- 172.21.15.49:0/3895646734 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557af3db0720 con 0x557af4126000 2024-02-26T06:25:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f575aa00640 10 client.15747 handle_command_reply: tid=0 2024-02-26T06:25:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 client.15747 shutdown 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410d530 msgr2=0x557af41136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410d530 0x557af41136f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5768010ca0 tx=0x7f5768005300 comp rx=0 tx=0).stop 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 shutdown_connections 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557af4126000 0x557af41283f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af4113c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410d530 0x557af41136f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 --2- 172.21.15.49:0/3895646734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af410cc10 0x557af41131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 >> 172.21.15.49:0/3895646734 conn(0x557af4107340 msgr2=0x557af4109090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.133+0000 7f575aa00640 0 client.15747 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.134+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 shutdown_connections 2024-02-26T06:25:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.134+0000 7f577ed8a740 1 -- 172.21.15.49:0/3895646734 wait complete. 2024-02-26T06:25:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.134+0000 7f577ed8a740 20 client.15747 trim_cache size 0 max 16384 2024-02-26T06:25:28.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.162+0000 7f577ed8a740 1 Processor -- start 2024-02-26T06:25:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f577ed8a740 1 -- start start 2024-02-26T06:25:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af410cc10 0x557af41158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af4116110 0x557af4113a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af4110450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f577ed8a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af4108140 con 0x557af4116110 2024-02-26T06:25:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f577ed8a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af4102c80 con 0x557af41121b0 2024-02-26T06:25:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f577ed8a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af410d390 con 0x557af410cc10 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f5778a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af410cc10 0x557af41158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f5778a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af410cc10 0x557af41158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39838/0 (socket says 172.21.15.49:39838) 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f5778a00640 1 -- 172.21.15.49:0/2198216653 learned_addr learned my addr 172.21.15.49:0/2198216653 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.163+0000 7f5778a00640 1 --2- 172.21.15.49:0/2198216653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af4110450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.164+0000 7f5778a00640 1 -- 172.21.15.49:0/2198216653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 msgr2=0x557af4110450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.164+0000 7f5778a00640 1 --2- 172.21.15.49:0/2198216653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af4110450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.164+0000 7f5778a00640 1 -- 172.21.15.49:0/2198216653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af4116110 msgr2=0x557af4113a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.164+0000 7f5778a00640 1 --2- 172.21.15.49:0/2198216653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af4116110 0x557af4113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.164+0000 7f5778a00640 1 -- 172.21.15.49:0/2198216653 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af400ea80 con 0x557af410cc10 2024-02-26T06:25:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.165+0000 7f5778a00640 1 --2- 172.21.15.49:0/2198216653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af410cc10 0x557af41158e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5760009870 tx=0x7f5760000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=442ecedaa962e732 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:28.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.166+0000 7f5765c00640 1 -- 172.21.15.49:0/2198216653 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576002c980 con 0x557af410cc10 2024-02-26T06:25:28.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.166+0000 7f5765c00640 1 -- 172.21.15.49:0/2198216653 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f576002ce90 con 0x557af410cc10 2024-02-26T06:25:28.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.166+0000 7f5765c00640 1 -- 172.21.15.49:0/2198216653 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760005350 con 0x557af410cc10 2024-02-26T06:25:28.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.167+0000 7f577ed8a740 1 -- 172.21.15.49:0/2198216653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af410cc10 msgr2=0x557af41158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.167+0000 7f577ed8a740 1 --2- 172.21.15.49:0/2198216653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af410cc10 0x557af41158e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5760009870 tx=0x7f5760000bc0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.167+0000 7f577ed8a740 1 -- 172.21.15.49:0/2198216653 shutdown_connections 2024-02-26T06:25:28.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.167+0000 7f577ed8a740 1 --2- 172.21.15.49:0/2198216653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af4110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.167+0000 7f577ed8a740 1 --2- 172.21.15.49:0/2198216653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af4116110 0x557af4113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.167+0000 7f577ed8a740 1 --2- 172.21.15.49:0/2198216653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af410cc10 0x557af41158e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.167+0000 7f577ed8a740 1 -- 172.21.15.49:0/2198216653 >> 172.21.15.49:0/2198216653 conn(0x557af4107340 msgr2=0x557af4111aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:28.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.168+0000 7f577ed8a740 1 -- 172.21.15.49:0/2198216653 shutdown_connections 2024-02-26T06:25:28.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.168+0000 7f577ed8a740 1 -- 172.21.15.49:0/2198216653 wait complete. 2024-02-26T06:25:28.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:27 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2965860803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:28.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.168+0000 7f577ed8a740 1 -- start start 2024-02-26T06:25:28.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.169+0000 7f577ed8a740 1 Processor -- start 2024-02-26T06:25:28.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.169+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 0x557af4177bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.170+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af411c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.170+0000 7f577ed8a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af411c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.170+0000 7f577ed8a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af411cec0 con 0x557af410cc10 2024-02-26T06:25:28.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.170+0000 7f577ed8a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af411d030 con 0x557af41121b0 2024-02-26T06:25:28.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.170+0000 7f577ed8a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af411d1f0 con 0x557af4116110 2024-02-26T06:25:28.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.170+0000 7f577ed8a740 10 client.0 _ll_register_callbacks cb 0x7f576800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:28.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.171+0000 7f5779400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af411c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.171+0000 7f5778a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 0x557af4177bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.171+0000 7f5778a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 0x557af4177bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39466/0 (socket says 172.21.15.49:39466) 2024-02-26T06:25:28.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.171+0000 7f5778a00640 1 -- 172.21.15.49:0/70639349 learned_addr learned my addr 172.21.15.49:0/70639349 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:28.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.173+0000 7f5778a00640 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af411c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.173+0000 7f5779400640 1 -- 172.21.15.49:0/70639349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 msgr2=0x557af411c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:28.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.173+0000 7f5779400640 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af411c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.173+0000 7f5779400640 1 -- 172.21.15.49:0/70639349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 msgr2=0x557af4177bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:28.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.173+0000 7f5779400640 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 0x557af4177bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.173+0000 7f5779400640 1 -- 172.21.15.49:0/70639349 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576004c040 con 0x557af41121b0 2024-02-26T06:25:28.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.174+0000 7f5779400640 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af411c440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x557af4114240 tx=0x7f576800bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67732f17d86655a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:28.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.174+0000 7f575aa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:25:28.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.175+0000 7f575aa00640 1 -- 172.21.15.49:0/70639349 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576801f040 con 0x557af41121b0 2024-02-26T06:25:28.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.175+0000 7f575aa00640 1 -- 172.21.15.49:0/70639349 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5768017070 con 0x557af41121b0 2024-02-26T06:25:28.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.176+0000 7f577ed8a740 1 -- 172.21.15.49:0/70639349 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5768015070 con 0x557af41121b0 2024-02-26T06:25:28.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.176+0000 7f575aa00640 1 -- 172.21.15.49:0/70639349 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5768004490 con 0x557af41121b0 2024-02-26T06:25:28.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.177+0000 7f577ed8a740 10 client.25961 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:28.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.177+0000 7f577ed8a740 1 -- 172.21.15.49:0/70639349 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557af4034620 con 0x557af41121b0 2024-02-26T06:25:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f575aa00640 1 -- 172.21.15.49:0/70639349 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5768015070 con 0x557af41121b0 2024-02-26T06:25:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f575aa00640 1 -- 172.21.15.49:0/70639349 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f5768003c70 con 0x557af41121b0 2024-02-26T06:25:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f577ed8a740 10 client.25961 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f577ed8a740 10 client.25961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f577ed8a740 20 client.25961 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:28.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f577ed8a740 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557af418f740 0x557af4191b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:28.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f577ed8a740 4 client.25961 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:25:28.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f577ed8a740 1 -- 172.21.15.49:0/70639349 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557af3dff890 con 0x557af418f740 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.178+0000 7f5778a00640 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557af418f740 0x557af4191b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f5778a00640 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557af418f740 0x557af4191b60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d2453b5237ec5c server_cookie=fec63cd416722a66 in_seq=0 out_seq=0 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f575aa00640 10 client.25961 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f575aa00640 1 -- 172.21.15.49:0/70639349 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557af3dff890 con 0x557af418f740 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f575aa00640 10 client.25961 handle_command_reply: tid=0 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f577ed8a740 1 client.25961 shutdown 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f577ed8a740 1 -- 172.21.15.49:0/70639349 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 msgr2=0x557af411c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f577ed8a740 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af411c440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x557af4114240 tx=0x7f576800bb20 comp rx=0 tx=0).stop 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f577ed8a740 1 -- 172.21.15.49:0/70639349 shutdown_connections 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.179+0000 7f577ed8a740 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557af418f740 0x557af4191b60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.180+0000 7f577ed8a740 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557af41121b0 0x557af411c440 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.180+0000 7f577ed8a740 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557af4116110 0x557af411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.180+0000 7f577ed8a740 1 --2- 172.21.15.49:0/70639349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557af410cc10 0x557af4177bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.180+0000 7f577ed8a740 1 -- 172.21.15.49:0/70639349 >> 172.21.15.49:0/70639349 conn(0x557af4107340 msgr2=0x557af4108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.180+0000 7f575aa00640 0 client.25961 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.180+0000 7f577ed8a740 1 -- 172.21.15.49:0/70639349 shutdown_connections 2024-02-26T06:25:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.180+0000 7f577ed8a740 1 -- 172.21.15.49:0/70639349 wait complete. 2024-02-26T06:25:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.180+0000 7f577ed8a740 20 client.25961 trim_cache size 0 max 16384 2024-02-26T06:25:28.182 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.181+0000 7f577be00640 1 -- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f575c0a22b0 msgr2=0x7f575c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.181+0000 7f577be00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f575c0a22b0 0x7f575c0a4770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f576c0975a0 tx=0x7f576c00cfa0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.181+0000 7f577be00640 1 -- 172.21.15.49:0/3248374052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5774076110 msgr2=0x7f5774071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.181+0000 7f577be00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5774076110 0x7f5774071820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f576c00eb40 tx=0x7f576c00c2d0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.182+0000 7f577be00640 1 -- 172.21.15.49:0/3248374052 shutdown_connections 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.182+0000 7f577be00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f575c0a22b0 0x7f575c0a4770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.182+0000 7f577be00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5774076110 0x7f5774071820 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.182+0000 7f577be00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5774108660 0x7f5774075bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.182+0000 7f577be00640 1 --2- 172.21.15.49:0/3248374052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5774103590 0x7f5774075690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.182+0000 7f577be00640 1 -- 172.21.15.49:0/3248374052 >> 172.21.15.49:0/3248374052 conn(0x7f57740fedf0 msgr2=0x7f5774101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:28.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.183+0000 7f577be00640 1 -- 172.21.15.49:0/3248374052 shutdown_connections 2024-02-26T06:25:28.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:28.183+0000 7f577be00640 1 -- 172.21.15.49:0/3248374052 wait complete. 2024-02-26T06:25:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:28 smithi080 ceph-mon[36206]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 671 op/s 2024-02-26T06:25:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:28 smithi115 ceph-mon[36426]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 671 op/s 2024-02-26T06:25:29.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:28 smithi049 ceph-mon[31696]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 671 op/s 2024-02-26T06:25:29.211 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:25:29.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:25:29.212 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:25:29.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.500+0000 7f9d7e400640 1 -- 172.21.15.49:0/746237838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 msgr2=0x7f9d78110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.500+0000 7f9d7e400640 1 --2- 172.21.15.49:0/746237838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 0x7f9d78110230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009ca0 tx=0x7f9d7002c100 comp rx=0 tx=0).stop 2024-02-26T06:25:29.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.500+0000 7f9d7e400640 1 -- 172.21.15.49:0/746237838 shutdown_connections 2024-02-26T06:25:29.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.500+0000 7f9d7e400640 1 --2- 172.21.15.49:0/746237838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78103f40 0x7f9d78107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.500+0000 7f9d7e400640 1 --2- 172.21.15.49:0/746237838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 0x7f9d78110230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.500+0000 7f9d7e400640 1 --2- 172.21.15.49:0/746237838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d78103590 0x7f9d78103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.501+0000 7f9d7e400640 1 -- 172.21.15.49:0/746237838 >> 172.21.15.49:0/746237838 conn(0x7f9d780fedf0 msgr2=0x7f9d78101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.501+0000 7f9d7e400640 1 -- 172.21.15.49:0/746237838 shutdown_connections 2024-02-26T06:25:29.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.501+0000 7f9d7e400640 1 -- 172.21.15.49:0/746237838 wait complete. 2024-02-26T06:25:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.502+0000 7f9d7e400640 1 Processor -- start 2024-02-26T06:25:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.503+0000 7f9d7e400640 1 -- start start 2024-02-26T06:25:29.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.504+0000 7f9d7e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78103590 0x7f9d78110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.504+0000 7f9d7e400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 0x7f9d7810d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.504+0000 7f9d7e400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d7810d730 0x7f9d7810dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d7e400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d781aa3a0 con 0x7f9d7810d730 2024-02-26T06:25:29.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d7e400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d781aa510 con 0x7f9d78103590 2024-02-26T06:25:29.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d7e400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d781aa6d0 con 0x7f9d78108660 2024-02-26T06:25:29.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78103590 0x7f9d78110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 0x7f9d7810d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d7810d730 0x7f9d7810dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78103590 0x7f9d78110030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45050/0 (socket says 172.21.15.49:45050) 2024-02-26T06:25:29.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77400640 1 -- 172.21.15.49:0/559440304 learned_addr learned my addr 172.21.15.49:0/559440304 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 0x7f9d7810d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39876/0 (socket says 172.21.15.49:39876) 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77e00640 1 -- 172.21.15.49:0/559440304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 msgr2=0x7f9d7810d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77e00640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 0x7f9d7810d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77e00640 1 -- 172.21.15.49:0/559440304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78103590 msgr2=0x7f9d78110030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77e00640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78103590 0x7f9d78110030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.505+0000 7f9d77e00640 1 -- 172.21.15.49:0/559440304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70009900 con 0x7f9d7810d730 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.506+0000 7f9d77400640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78103590 0x7f9d78110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.506+0000 7f9d77e00640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d7810d730 0x7f9d7810dbe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d70037040 tx=0x7f9d70032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.506+0000 7f9d6fe00640 1 -- 172.21.15.49:0/559440304 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70034640 con 0x7f9d7810d730 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.506+0000 7f9d6fe00640 1 -- 172.21.15.49:0/559440304 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d70034e00 con 0x7f9d7810d730 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.506+0000 7f9d7e400640 1 -- 172.21.15.49:0/559440304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d781aa950 con 0x7f9d7810d730 2024-02-26T06:25:29.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.506+0000 7f9d6fe00640 1 -- 172.21.15.49:0/559440304 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d700046f0 con 0x7f9d7810d730 2024-02-26T06:25:29.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.507+0000 7f9d7e400640 1 -- 172.21.15.49:0/559440304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d781aae50 con 0x7f9d7810d730 2024-02-26T06:25:29.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.512+0000 7f9d6fe00640 1 -- 172.21.15.49:0/559440304 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9d7000a040 con 0x7f9d7810d730 2024-02-26T06:25:29.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.512+0000 7f9d81247740 1 Processor -- start 2024-02-26T06:25:29.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.513+0000 7f9d81247740 1 -- start start 2024-02-26T06:25:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.513+0000 7f9d81247740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c3983ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.514+0000 7f9d81247740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.514+0000 7f9d81247740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983c3410 0x55c3983c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.514+0000 7f9d81247740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3983752d0 con 0x55c3983b9f10 2024-02-26T06:25:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.514+0000 7f9d81247740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c398327e20 con 0x55c3983ba830 2024-02-26T06:25:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.514+0000 7f9d81247740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c398325350 con 0x55c3983c3410 2024-02-26T06:25:29.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.515+0000 7f9d6fe00640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9d5c0a22f0 0x7f9d5c0a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.515+0000 7f9d6fe00640 1 -- 172.21.15.49:0/559440304 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9d700f07d0 con 0x7f9d7810d730 2024-02-26T06:25:29.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.515+0000 7f9d76a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c3983ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.515+0000 7f9d77e00640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9d5c0a22f0 0x7f9d5c0a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.516+0000 7f9d76a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c3983ba2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39492/0 (socket says 172.21.15.49:39492) 2024-02-26T06:25:29.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.516+0000 7f9d76a00640 1 -- 172.21.15.49:0/488524320 learned_addr learned my addr 172.21.15.49:0/488524320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.516+0000 7f9d77400640 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.516+0000 7f9d77e00640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9d5c0a22f0 0x7f9d5c0a47b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d700b9740 tx=0x7f9d700095c0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.517+0000 7f9d76a00640 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983c3410 0x55c3983c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.517+0000 7f9d77400640 1 -- 172.21.15.49:0/488524320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983c3410 msgr2=0x55c3983c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.517+0000 7f9d77400640 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983c3410 0x55c3983c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.517+0000 7f9d77400640 1 -- 172.21.15.49:0/488524320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 msgr2=0x55c3983ba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.517+0000 7f9d77400640 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c3983ba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.517+0000 7f9d77400640 1 -- 172.21.15.49:0/488524320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3982e9c50 con 0x55c3983ba830 2024-02-26T06:25:29.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.518+0000 7f9d76a00640 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983c3410 0x55c3983c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:25:29.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.518+0000 7f9d77400640 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c2ed0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9d68008c30 tx=0x7f9d6800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d88fec843faafd1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.518+0000 7f9d6cc00640 1 -- 172.21.15.49:0/488524320 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68004220 con 0x55c3983ba830 2024-02-26T06:25:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.519+0000 7f9d6cc00640 1 -- 172.21.15.49:0/488524320 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d68004380 con 0x55c3983ba830 2024-02-26T06:25:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.519+0000 7f9d6cc00640 1 -- 172.21.15.49:0/488524320 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68004e20 con 0x55c3983ba830 2024-02-26T06:25:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.521+0000 7f9d81247740 1 -- 172.21.15.49:0/488524320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 msgr2=0x55c3983c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.521+0000 7f9d81247740 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c2ed0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9d68008c30 tx=0x7f9d6800e720 comp rx=0 tx=0).stop 2024-02-26T06:25:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.522+0000 7f9d81247740 1 -- 172.21.15.49:0/488524320 shutdown_connections 2024-02-26T06:25:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.522+0000 7f9d81247740 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c2ed0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.522+0000 7f9d81247740 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983c3410 0x55c3983c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.522+0000 7f9d81247740 1 --2- 172.21.15.49:0/488524320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c3983ba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.522+0000 7f9d81247740 1 -- 172.21.15.49:0/488524320 >> 172.21.15.49:0/488524320 conn(0x55c3983b4640 msgr2=0x55c3983b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.522+0000 7f9d81247740 1 -- 172.21.15.49:0/488524320 shutdown_connections 2024-02-26T06:25:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.522+0000 7f9d81247740 1 -- 172.21.15.49:0/488524320 wait complete. 2024-02-26T06:25:29.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.523+0000 7f9d81247740 1 -- start start 2024-02-26T06:25:29.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.524+0000 7f9d81247740 1 Processor -- start 2024-02-26T06:25:29.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d81247740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d81247740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d81247740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983c3410 0x55c3983c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d81247740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c398320b30 con 0x55c3983c3410 2024-02-26T06:25:29.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d81247740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3982c5fa0 con 0x55c3983ba830 2024-02-26T06:25:29.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d81247740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3982c5c40 con 0x55c3983b9f10 2024-02-26T06:25:29.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d81247740 10 client.0 _ll_register_callbacks cb 0x55c398031c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d77400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d77400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45074/0 (socket says 172.21.15.49:45074) 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d77400640 1 -- 172.21.15.49:0/1412306148 learned_addr learned my addr 172.21.15.49:0/1412306148 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d76a00640 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.525+0000 7f9d76a00640 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983c3410 0x55c3983c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d76a00640 1 -- 172.21.15.49:0/1412306148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 msgr2=0x55c3983c0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d76a00640 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d76a00640 1 -- 172.21.15.49:0/1412306148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983c3410 msgr2=0x55c3983c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d76a00640 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983c3410 0x55c3983c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d76a00640 1 -- 172.21.15.49:0/1412306148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d68008890 con 0x55c3983b9f10 2024-02-26T06:25:29.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d77400640 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:29.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d76a00640 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c03d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c3983b57d0 tx=0x7f9d64015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46c6e8a43ebebdc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d61600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:29.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.526+0000 7f9d61600640 1 -- 172.21.15.49:0/1412306148 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64018070 con 0x55c3983b9f10 2024-02-26T06:25:29.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d61600640 1 -- 172.21.15.49:0/1412306148 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9d64010cd0 con 0x55c3983b9f10 2024-02-26T06:25:29.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 1 -- 172.21.15.49:0/1412306148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c398020860 con 0x55c3983b9f10 2024-02-26T06:25:29.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d61600640 1 -- 172.21.15.49:0/1412306148 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d64010e90 con 0x55c3983b9f10 2024-02-26T06:25:29.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 10 client.26005 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:29.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 1 -- 172.21.15.49:0/1412306148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c39831f1b0 con 0x55c3983b9f10 2024-02-26T06:25:29.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d61600640 1 -- 172.21.15.49:0/1412306148 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c398020860 con 0x55c3983b9f10 2024-02-26T06:25:29.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d61600640 1 -- 172.21.15.49:0/1412306148 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9d64029540 con 0x55c3983b9f10 2024-02-26T06:25:29.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 10 client.26005 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:29.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:29.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 20 client.26005 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c3983d38d0 0x55c3983d5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 4 client.26005 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.527+0000 7f9d81247740 1 -- 172.21.15.49:0/1412306148 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c398071410 con 0x55c3983d38d0 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.528+0000 7f9d77400640 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c3983d38d0 0x55c3983d5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.528+0000 7f9d77400640 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c3983d38d0 0x55c3983d5cf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6e88203b9f1d7c server_cookie=a74451df4fa78039 in_seq=0 out_seq=0 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.528+0000 7f9d61600640 10 client.26005 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.529+0000 7f9d61600640 1 -- 172.21.15.49:0/1412306148 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c398071410 con 0x55c3983d38d0 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.529+0000 7f9d61600640 10 client.26005 handle_command_reply: tid=0 2024-02-26T06:25:29.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.529+0000 7f9d81247740 1 client.26005 shutdown 2024-02-26T06:25:29.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d81247740 1 -- 172.21.15.49:0/1412306148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 msgr2=0x55c3983c03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d81247740 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c03d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c3983b57d0 tx=0x7f9d64015040 comp rx=0 tx=0).stop 2024-02-26T06:25:29.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d81247740 1 -- 172.21.15.49:0/1412306148 shutdown_connections 2024-02-26T06:25:29.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d81247740 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c3983d38d0 0x55c3983d5cf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d81247740 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983ba830 0x55c3983c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d81247740 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983c3410 0x55c3983c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d81247740 1 --2- 172.21.15.49:0/1412306148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c03d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d61600640 0 client.26005 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:29.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.530+0000 7f9d81247740 1 -- 172.21.15.49:0/1412306148 >> 172.21.15.49:0/1412306148 conn(0x55c3983b4640 msgr2=0x55c3983b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.531+0000 7f9d81247740 1 -- 172.21.15.49:0/1412306148 shutdown_connections 2024-02-26T06:25:29.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.531+0000 7f9d81247740 1 -- 172.21.15.49:0/1412306148 wait complete. 2024-02-26T06:25:29.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.531+0000 7f9d81247740 20 client.26005 trim_cache size 0 max 16384 2024-02-26T06:25:29.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.561+0000 7f9d81247740 1 Processor -- start 2024-02-26T06:25:29.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.561+0000 7f9d81247740 1 -- start start 2024-02-26T06:25:29.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.561+0000 7f9d81247740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.561+0000 7f9d81247740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983ba830 0x55c3983b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.561+0000 7f9d81247740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 0x55c3983ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.561+0000 7f9d81247740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3980690c0 con 0x55c3983ba830 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.561+0000 7f9d81247740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3983d1fa0 con 0x55c3983c3410 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.561+0000 7f9d81247740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3983aff80 con 0x55c3983b9f10 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39902/0 (socket says 172.21.15.49:39902) 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 -- 172.21.15.49:0/899992736 learned_addr learned my addr 172.21.15.49:0/899992736 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d77400640 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983ba830 0x55c3983b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 0x55c3983ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 -- 172.21.15.49:0/899992736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 msgr2=0x55c3983ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 0x55c3983ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 -- 172.21.15.49:0/899992736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983ba830 msgr2=0x55c3983b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983ba830 0x55c3983b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.562+0000 7f9d76a00640 1 -- 172.21.15.49:0/899992736 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3982bba60 con 0x55c3983b9f10 2024-02-26T06:25:29.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.563+0000 7f9d76a00640 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c0c70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9d64010010 tx=0x7f9d64009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=713862b514d0e3b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.563+0000 7f9d63e00640 1 -- 172.21.15.49:0/899992736 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64018070 con 0x55c3983b9f10 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.563+0000 7f9d63e00640 1 -- 172.21.15.49:0/899992736 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d64010dd0 con 0x55c3983b9f10 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.563+0000 7f9d63e00640 1 -- 172.21.15.49:0/899992736 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64020430 con 0x55c3983b9f10 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.564+0000 7f9d81247740 1 -- 172.21.15.49:0/899992736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 msgr2=0x55c3983c0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.564+0000 7f9d81247740 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c0c70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9d64010010 tx=0x7f9d64009620 comp rx=0 tx=0).stop 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.564+0000 7f9d81247740 1 -- 172.21.15.49:0/899992736 shutdown_connections 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.564+0000 7f9d81247740 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 0x55c3983ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.564+0000 7f9d81247740 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983ba830 0x55c3983b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.564+0000 7f9d81247740 1 --2- 172.21.15.49:0/899992736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983b9f10 0x55c3983c0c70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.564+0000 7f9d81247740 1 -- 172.21.15.49:0/899992736 >> 172.21.15.49:0/899992736 conn(0x55c3983b4640 msgr2=0x55c3983b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.565+0000 7f9d81247740 1 -- 172.21.15.49:0/899992736 shutdown_connections 2024-02-26T06:25:29.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.565+0000 7f9d81247740 1 -- 172.21.15.49:0/899992736 wait complete. 2024-02-26T06:25:29.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.565+0000 7f9d81247740 1 -- start start 2024-02-26T06:25:29.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.566+0000 7f9d81247740 1 Processor -- start 2024-02-26T06:25:29.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.566+0000 7f9d81247740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c398424120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.566+0000 7f9d81247740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983ba830 0x55c398424660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.566+0000 7f9d81247740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 0x55c3983c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.566+0000 7f9d81247740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3983c9ac0 con 0x55c3983b9f10 2024-02-26T06:25:29.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.566+0000 7f9d81247740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3983c9c30 con 0x55c3983c3410 2024-02-26T06:25:29.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.566+0000 7f9d81247740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3983c9df0 con 0x55c3983ba830 2024-02-26T06:25:29.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.566+0000 7f9d81247740 10 client.0 _ll_register_callbacks cb 0x7f9d68008410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:29.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d77400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983ba830 0x55c398424660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d76a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c398424120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d77400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983ba830 0x55c398424660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39904/0 (socket says 172.21.15.49:39904) 2024-02-26T06:25:29.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d77400640 1 -- 172.21.15.49:0/2033261004 learned_addr learned my addr 172.21.15.49:0/2033261004 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d77400640 1 -- 172.21.15.49:0/2033261004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 msgr2=0x55c3983c94b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:25:29.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d77400640 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 0x55c3983c94b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d77400640 1 -- 172.21.15.49:0/2033261004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 msgr2=0x55c398424120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d77400640 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c398424120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d77400640 1 -- 172.21.15.49:0/2033261004 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c39831f1b0 con 0x55c3983ba830 2024-02-26T06:25:29.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.567+0000 7f9d76a00640 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c398424120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:29.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.568+0000 7f9d77400640 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983ba830 0x55c398424660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800ec20 tx=0x7f9d68008fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05b1aaed900a9cc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.568+0000 7f9d61600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:29.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.568+0000 7f9d61600640 1 -- 172.21.15.49:0/2033261004 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68028040 con 0x55c3983ba830 2024-02-26T06:25:29.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.568+0000 7f9d81247740 1 -- 172.21.15.49:0/2033261004 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d68004380 con 0x55c3983ba830 2024-02-26T06:25:29.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.568+0000 7f9d61600640 1 -- 172.21.15.49:0/2033261004 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9d680281a0 con 0x55c3983ba830 2024-02-26T06:25:29.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.568+0000 7f9d61600640 1 -- 172.21.15.49:0/2033261004 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d68028380 con 0x55c3983ba830 2024-02-26T06:25:29.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.568+0000 7f9d81247740 10 client.26011 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:29.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.568+0000 7f9d81247740 1 -- 172.21.15.49:0/2033261004 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c3982e1670 con 0x55c3983ba830 2024-02-26T06:25:29.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.569+0000 7f9d61600640 1 -- 172.21.15.49:0/2033261004 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d6803d080 con 0x55c3983ba830 2024-02-26T06:25:29.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.569+0000 7f9d61600640 1 -- 172.21.15.49:0/2033261004 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9d68023040 con 0x55c3983ba830 2024-02-26T06:25:29.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.570+0000 7f9d81247740 10 client.26011 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:29.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.570+0000 7f9d81247740 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:29.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.570+0000 7f9d81247740 20 client.26011 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:29.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.570+0000 7f9d81247740 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c39842c110 0x55c39842e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.570+0000 7f9d81247740 4 client.26011 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:25:29.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.570+0000 7f9d81247740 1 -- 172.21.15.49:0/2033261004 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c39805d7b0 con 0x55c39842c110 2024-02-26T06:25:29.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.571+0000 7f9d76a00640 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c39842c110 0x55c39842e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.571+0000 7f9d76a00640 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c39842c110 0x55c39842e530 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f334c8164fa9259c server_cookie=d796305bac2c7c39 in_seq=0 out_seq=0 2024-02-26T06:25:29.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.571+0000 7f9d61600640 10 client.26011 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:29.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d61600640 1 -- 172.21.15.49:0/2033261004 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c39805d7b0 con 0x55c39842c110 2024-02-26T06:25:29.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d61600640 10 client.26011 handle_command_reply: tid=0 2024-02-26T06:25:29.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 client.26011 shutdown 2024-02-26T06:25:29.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 -- 172.21.15.49:0/2033261004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983ba830 msgr2=0x55c398424660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983ba830 0x55c398424660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800ec20 tx=0x7f9d68008fc0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 -- 172.21.15.49:0/2033261004 shutdown_connections 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c39842c110 0x55c39842e530 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c3983c3410 0x55c3983c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3983ba830 0x55c398424660 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 --2- 172.21.15.49:0/2033261004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c3983b9f10 0x55c398424120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d61600640 0 client.26011 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.573+0000 7f9d81247740 1 -- 172.21.15.49:0/2033261004 >> 172.21.15.49:0/2033261004 conn(0x55c3983b4640 msgr2=0x55c3983b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.574+0000 7f9d81247740 1 -- 172.21.15.49:0/2033261004 shutdown_connections 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.574+0000 7f9d81247740 1 -- 172.21.15.49:0/2033261004 wait complete. 2024-02-26T06:25:29.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.574+0000 7f9d81247740 20 client.26011 trim_cache size 0 max 16384 2024-02-26T06:25:29.576 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:25:29.576 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:25:29.576 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "e8df09d6-fff3-47db-8dd5-5ba8a6c034e8", 2024-02-26T06:25:29.576 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:25:29.576 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:25:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.575+0000 7f9d7e400640 1 -- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9d5c0a22f0 msgr2=0x7f9d5c0a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.575+0000 7f9d7e400640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9d5c0a22f0 0x7f9d5c0a47b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d700b9740 tx=0x7f9d700095c0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.575+0000 7f9d7e400640 1 -- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d7810d730 msgr2=0x7f9d7810dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.575+0000 7f9d7e400640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d7810d730 0x7f9d7810dbe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d70037040 tx=0x7f9d70032570 comp rx=0 tx=0).stop 2024-02-26T06:25:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.576+0000 7f9d7e400640 1 -- 172.21.15.49:0/559440304 shutdown_connections 2024-02-26T06:25:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.576+0000 7f9d7e400640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9d5c0a22f0 0x7f9d5c0a47b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.576+0000 7f9d7e400640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d7810d730 0x7f9d7810dbe0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.576+0000 7f9d7e400640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d78108660 0x7f9d7810d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.576+0000 7f9d7e400640 1 --2- 172.21.15.49:0/559440304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d78103590 0x7f9d78110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.576+0000 7f9d7e400640 1 -- 172.21.15.49:0/559440304 >> 172.21.15.49:0/559440304 conn(0x7f9d780fedf0 msgr2=0x7f9d78101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.577+0000 7f9d7e400640 1 -- 172.21.15.49:0/559440304 shutdown_connections 2024-02-26T06:25:29.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.577+0000 7f9d7e400640 1 -- 172.21.15.49:0/559440304 wait complete. 2024-02-26T06:25:29.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:25:29.606 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:25:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.898+0000 7f5d7f200640 1 -- 172.21.15.49:0/3060704072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d80103e20 msgr2=0x7f5d80107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.898+0000 7f5d7f200640 1 --2- 172.21.15.49:0/3060704072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d80103e20 0x7f5d80107e60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009930 tx=0x7f5d6c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.898+0000 7f5d7f200640 1 -- 172.21.15.49:0/3060704072 shutdown_connections 2024-02-26T06:25:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.898+0000 7f5d7f200640 1 --2- 172.21.15.49:0/3060704072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d80108540 0x7f5d80110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.898+0000 7f5d7f200640 1 --2- 172.21.15.49:0/3060704072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d80103e20 0x7f5d80107e60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.898+0000 7f5d7f200640 1 --2- 172.21.15.49:0/3060704072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d80103470 0x7f5d80103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.898+0000 7f5d7f200640 1 -- 172.21.15.49:0/3060704072 >> 172.21.15.49:0/3060704072 conn(0x7f5d800fec90 msgr2=0x7f5d801010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.898+0000 7f5d7f200640 1 -- 172.21.15.49:0/3060704072 shutdown_connections 2024-02-26T06:25:29.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.899+0000 7f5d7f200640 1 -- 172.21.15.49:0/3060704072 wait complete. 2024-02-26T06:25:29.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.900+0000 7f5d7f200640 1 Processor -- start 2024-02-26T06:25:29.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7f200640 1 -- start start 2024-02-26T06:25:29.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d80103470 0x7f5d80072b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d80108540 0x7f5d8006bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d8006c0c0 0x7f5d8006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80070620 con 0x7f5d8006c0c0 2024-02-26T06:25:29.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80070790 con 0x7f5d80108540 2024-02-26T06:25:29.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80070950 con 0x7f5d80103470 2024-02-26T06:25:29.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d80103470 0x7f5d80072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d80103470 0x7f5d80072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39922/0 (socket says 172.21.15.49:39922) 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.902+0000 7f5d7de00640 1 -- 172.21.15.49:0/810278587 learned_addr learned my addr 172.21.15.49:0/810278587 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.903+0000 7f5d7e800640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d8006c0c0 0x7f5d8006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.903+0000 7f5d7d400640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d80108540 0x7f5d8006bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.903+0000 7f5d7de00640 1 -- 172.21.15.49:0/810278587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d80108540 msgr2=0x7f5d8006bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.903+0000 7f5d7de00640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d80108540 0x7f5d8006bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.903+0000 7f5d7de00640 1 -- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d8006c0c0 msgr2=0x7f5d8006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.903+0000 7f5d7de00640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d8006c0c0 0x7f5d8006c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.903+0000 7f5d7de00640 1 -- 172.21.15.49:0/810278587 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70009490 con 0x7f5d80103470 2024-02-26T06:25:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.903+0000 7f5d7de00640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d80103470 0x7f5d80072b00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000ea60 tx=0x7f5d7000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.904+0000 7f5d76a00640 1 -- 172.21.15.49:0/810278587 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000cde0 con 0x7f5d80103470 2024-02-26T06:25:29.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.904+0000 7f5d76a00640 1 -- 172.21.15.49:0/810278587 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d700035f0 con 0x7f5d80103470 2024-02-26T06:25:29.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.904+0000 7f5d76a00640 1 -- 172.21.15.49:0/810278587 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70010690 con 0x7f5d80103470 2024-02-26T06:25:29.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.904+0000 7f5d7f200640 1 -- 172.21.15.49:0/810278587 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c009560 con 0x7f5d80103470 2024-02-26T06:25:29.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.904+0000 7f5d7f200640 1 -- 172.21.15.49:0/810278587 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80070f30 con 0x7f5d80103470 2024-02-26T06:25:29.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.910+0000 7f5d76a00640 1 -- 172.21.15.49:0/810278587 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5d700107f0 con 0x7f5d80103470 2024-02-26T06:25:29.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.911+0000 7f5d87a67740 1 Processor -- start 2024-02-26T06:25:29.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.912+0000 7f5d87a67740 1 -- start start 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.912+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 0x5583e7b9de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 0x5583e7b9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7ba77d0 0x5583e7ba9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d87a67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7b0d120 con 0x5583e7b9f140 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d87a67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7b0a650 con 0x5583e7ba77d0 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d87a67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7b05e30 con 0x5583e7b9e460 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 0x5583e7b9de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 0x5583e7b9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 0x5583e7b9e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39938/0 (socket says 172.21.15.49:39938) 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d7de00640 1 -- 172.21.15.49:0/1522465438 learned_addr learned my addr 172.21.15.49:0/1522465438 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.913+0000 7f5d7d400640 1 --2- 172.21.15.49:0/1522465438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7ba77d0 0x5583e7ba9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.914+0000 7f5d7d400640 1 -- 172.21.15.49:0/1522465438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 msgr2=0x5583e7b9e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.914+0000 7f5d7d400640 1 --2- 172.21.15.49:0/1522465438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 0x5583e7b9e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.914+0000 7f5d7d400640 1 -- 172.21.15.49:0/1522465438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 msgr2=0x5583e7b9de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.914+0000 7f5d7d400640 1 --2- 172.21.15.49:0/1522465438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 0x5583e7b9de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.914+0000 7f5d7d400640 1 -- 172.21.15.49:0/1522465438 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583e7abf030 con 0x5583e7ba77d0 2024-02-26T06:25:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.914+0000 7f5d7d400640 1 --2- 172.21.15.49:0/1522465438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7ba77d0 0x5583e7ba9bc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009230 tx=0x7f5d6c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3742a34892d20870 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.914+0000 7f5d6b000640 1 -- 172.21.15.49:0/1522465438 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0049c0 con 0x5583e7ba77d0 2024-02-26T06:25:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.915+0000 7f5d6b000640 1 -- 172.21.15.49:0/1522465438 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d6c004040 con 0x5583e7ba77d0 2024-02-26T06:25:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.915+0000 7f5d6b000640 1 -- 172.21.15.49:0/1522465438 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c005210 con 0x5583e7ba77d0 2024-02-26T06:25:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.915+0000 7f5d76a00640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5d640a2330 0x7f5d640a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.915+0000 7f5d76a00640 1 -- 172.21.15.49:0/810278587 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5d70014070 con 0x7f5d80103470 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.915+0000 7f5d87a67740 1 -- 172.21.15.49:0/1522465438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7ba77d0 msgr2=0x5583e7ba9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.915+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1522465438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7ba77d0 0x5583e7ba9bc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009230 tx=0x7f5d6c02c650 comp rx=0 tx=0).stop 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.915+0000 7f5d7e800640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5d640a2330 0x7f5d640a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.916+0000 7f5d87a67740 1 -- 172.21.15.49:0/1522465438 shutdown_connections 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.916+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1522465438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7ba77d0 0x5583e7ba9bc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.916+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1522465438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 0x5583e7b9e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.916+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1522465438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 0x5583e7b9de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.916+0000 7f5d87a67740 1 -- 172.21.15.49:0/1522465438 >> 172.21.15.49:0/1522465438 conn(0x5583e7b65570 msgr2=0x5583e7b98eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.916+0000 7f5d87a67740 1 -- 172.21.15.49:0/1522465438 shutdown_connections 2024-02-26T06:25:29.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.916+0000 7f5d87a67740 1 -- 172.21.15.49:0/1522465438 wait complete. 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.917+0000 7f5d87a67740 1 -- start start 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.917+0000 7f5d7e800640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5d640a2330 0x7f5d640a47f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d78005fd0 tx=0x7f5d78009500 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.917+0000 7f5d87a67740 1 Processor -- start 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.918+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7ba4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.918+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 0x5583e7ba4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.918+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7ba77d0 0x5583e7ba51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.918+0000 7f5d87a67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7aaae90 con 0x5583e7b9f140 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.918+0000 7f5d87a67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7aaab30 con 0x5583e7b9e460 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.918+0000 7f5d87a67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e78741c0 con 0x5583e7ba77d0 2024-02-26T06:25:29.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.918+0000 7f5d87a67740 10 client.0 _ll_register_callbacks cb 0x5583e7abe930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7ba77d0 0x5583e7ba51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7ba77d0 0x5583e7ba51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39946/0 (socket says 172.21.15.49:39946) 2024-02-26T06:25:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7e800640 1 -- 172.21.15.49:0/1023732421 learned_addr learned my addr 172.21.15.49:0/1023732421 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7de00640 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7ba4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7d400640 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 0x5583e7ba4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7e800640 1 -- 172.21.15.49:0/1023732421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 msgr2=0x5583e7ba4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7e800640 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7ba4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7e800640 1 -- 172.21.15.49:0/1023732421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 msgr2=0x5583e7ba4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7e800640 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 0x5583e7ba4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.919+0000 7f5d7e800640 1 -- 172.21.15.49:0/1023732421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80110920 con 0x5583e7ba77d0 2024-02-26T06:25:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.920+0000 7f5d7e800640 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7ba77d0 0x5583e7ba51c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d7801f650 tx=0x7f5d78004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=439bb18eb6de8aa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.920+0000 7f5d5f400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.920+0000 7f5d5f400640 1 -- 172.21.15.49:0/1023732421 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78016070 con 0x5583e7ba77d0 2024-02-26T06:25:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.920+0000 7f5d5f400640 1 -- 172.21.15.49:0/1023732421 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5d78011810 con 0x5583e7ba77d0 2024-02-26T06:25:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.920+0000 7f5d87a67740 1 -- 172.21.15.49:0/1023732421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583e78057e0 con 0x5583e7ba77d0 2024-02-26T06:25:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.921+0000 7f5d5f400640 1 -- 172.21.15.49:0/1023732421 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d780119d0 con 0x5583e7ba77d0 2024-02-26T06:25:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.921+0000 7f5d87a67740 10 client.26023 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.921+0000 7f5d87a67740 1 -- 172.21.15.49:0/1023732421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5583e7b044b0 con 0x5583e7ba77d0 2024-02-26T06:25:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.921+0000 7f5d5f400640 1 -- 172.21.15.49:0/1023732421 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583e78057e0 con 0x5583e7ba77d0 2024-02-26T06:25:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.922+0000 7f5d5f400640 1 -- 172.21.15.49:0/1023732421 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f5d7802f070 con 0x5583e7ba77d0 2024-02-26T06:25:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.922+0000 7f5d87a67740 10 client.26023 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.922+0000 7f5d87a67740 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.922+0000 7f5d87a67740 20 client.26023 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.922+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583e7bb7940 0x5583e7bb9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.922+0000 7f5d87a67740 4 client.26023 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.922+0000 7f5d87a67740 1 -- 172.21.15.49:0/1023732421 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583e7842720 con 0x5583e7bb7940 2024-02-26T06:25:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.923+0000 7f5d7de00640 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583e7bb7940 0x5583e7bb9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.924+0000 7f5d7de00640 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583e7bb7940 0x5583e7bb9d60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce81e1a528060d02 server_cookie=7f0d2e57f8883801 in_seq=0 out_seq=0 2024-02-26T06:25:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.924+0000 7f5d5f400640 10 client.26023 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:29.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.924+0000 7f5d5f400640 1 -- 172.21.15.49:0/1023732421 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f5d7007ae90 con 0x5583e7bb7940 2024-02-26T06:25:29.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.924+0000 7f5d5f400640 10 client.26023 handle_command_reply: tid=0 2024-02-26T06:25:29.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.925+0000 7f5d87a67740 1 client.26023 shutdown 2024-02-26T06:25:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 -- 172.21.15.49:0/1023732421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7ba77d0 msgr2=0x5583e7ba51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7ba77d0 0x5583e7ba51c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d7801f650 tx=0x7f5d78004ab0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 -- 172.21.15.49:0/1023732421 shutdown_connections 2024-02-26T06:25:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583e7bb7940 0x5583e7bb9d60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7ba77d0 0x5583e7ba51c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7b9f140 0x5583e7ba4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1023732421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7ba4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 -- 172.21.15.49:0/1023732421 >> 172.21.15.49:0/1023732421 conn(0x5583e7b65570 msgr2=0x5583e7b9ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d5f400640 0 client.26023 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:29.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 -- 172.21.15.49:0/1023732421 shutdown_connections 2024-02-26T06:25:29.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 1 -- 172.21.15.49:0/1023732421 wait complete. 2024-02-26T06:25:29.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.926+0000 7f5d87a67740 20 client.26023 trim_cache size 0 max 16384 2024-02-26T06:25:29.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.953+0000 7f5d87a67740 1 Processor -- start 2024-02-26T06:25:29.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.953+0000 7f5d87a67740 1 -- start start 2024-02-26T06:25:29.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 0x5583e7ba4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9f140 0x5583e7ba51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7ba77d0 0x5583e7ba3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d87a67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7b9d6e0 con 0x5583e7ba77d0 2024-02-26T06:25:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d87a67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7b9eda0 con 0x5583e7b9f140 2024-02-26T06:25:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d87a67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7b9d490 con 0x5583e7b9e460 2024-02-26T06:25:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d7d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9f140 0x5583e7ba51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d7d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9f140 0x5583e7ba51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50990/0 (socket says 172.21.15.49:50990) 2024-02-26T06:25:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d7d400640 1 -- 172.21.15.49:0/2647271576 learned_addr learned my addr 172.21.15.49:0/2647271576 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d7d400640 1 -- 172.21.15.49:0/2647271576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 msgr2=0x5583e7ba4c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:25:29.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d7d400640 1 --2- 172.21.15.49:0/2647271576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 0x5583e7ba4c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.954+0000 7f5d7d400640 1 -- 172.21.15.49:0/2647271576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7ba77d0 msgr2=0x5583e7ba3010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:25:29.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.955+0000 7f5d7d400640 1 --2- 172.21.15.49:0/2647271576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7ba77d0 0x5583e7ba3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.955+0000 7f5d7d400640 1 -- 172.21.15.49:0/2647271576 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583e7aa0920 con 0x5583e7b9f140 2024-02-26T06:25:29.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.957+0000 7f5d7d400640 1 --2- 172.21.15.49:0/2647271576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9f140 0x5583e7ba51d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009230 tx=0x7f5d6c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131ebfaa42f7fc0d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.958+0000 7f5d6a600640 1 -- 172.21.15.49:0/2647271576 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c004840 con 0x5583e7b9f140 2024-02-26T06:25:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.958+0000 7f5d6a600640 1 -- 172.21.15.49:0/2647271576 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d6c0049a0 con 0x5583e7b9f140 2024-02-26T06:25:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.958+0000 7f5d6a600640 1 -- 172.21.15.49:0/2647271576 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c005350 con 0x5583e7b9f140 2024-02-26T06:25:29.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.959+0000 7f5d87a67740 1 -- 172.21.15.49:0/2647271576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9f140 msgr2=0x5583e7ba51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.959+0000 7f5d87a67740 1 --2- 172.21.15.49:0/2647271576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9f140 0x5583e7ba51d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009230 tx=0x7f5d6c005fd0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.962+0000 7f5d87a67740 1 -- 172.21.15.49:0/2647271576 shutdown_connections 2024-02-26T06:25:29.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.962+0000 7f5d87a67740 1 --2- 172.21.15.49:0/2647271576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9f140 0x5583e7ba51d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.962+0000 7f5d87a67740 1 --2- 172.21.15.49:0/2647271576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7ba77d0 0x5583e7ba3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.962+0000 7f5d87a67740 1 --2- 172.21.15.49:0/2647271576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9e460 0x5583e7ba4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.962+0000 7f5d87a67740 1 -- 172.21.15.49:0/2647271576 >> 172.21.15.49:0/2647271576 conn(0x5583e7b65570 msgr2=0x5583e7b989e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.962+0000 7f5d87a67740 1 -- 172.21.15.49:0/2647271576 shutdown_connections 2024-02-26T06:25:29.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.962+0000 7f5d87a67740 1 -- 172.21.15.49:0/2647271576 wait complete. 2024-02-26T06:25:29.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.963+0000 7f5d87a67740 1 -- start start 2024-02-26T06:25:29.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.964+0000 7f5d87a67740 1 Processor -- start 2024-02-26T06:25:29.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.964+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7bac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.964+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9f140 0x5583e7bace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.964+0000 7f5d87a67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7ba77d0 0x5583e7c0ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.964+0000 7f5d87a67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7c0d380 con 0x5583e7ba77d0 2024-02-26T06:25:29.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.964+0000 7f5d87a67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7c0d4c0 con 0x5583e7b9e460 2024-02-26T06:25:29.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.964+0000 7f5d87a67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e7c0d630 con 0x5583e7b9f140 2024-02-26T06:25:29.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.964+0000 7f5d87a67740 10 client.0 _ll_register_callbacks cb 0x7f5d78009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:29.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.965+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7bac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.965+0000 7f5d7d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9f140 0x5583e7bace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.965+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7bac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50996/0 (socket says 172.21.15.49:50996) 2024-02-26T06:25:29.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.965+0000 7f5d7de00640 1 -- 172.21.15.49:0/1834847448 learned_addr learned my addr 172.21.15.49:0/1834847448 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:29.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.966+0000 7f5d7de00640 1 -- 172.21.15.49:0/1834847448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9f140 msgr2=0x5583e7bace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.966+0000 7f5d7de00640 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9f140 0x5583e7bace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.966+0000 7f5d7de00640 1 -- 172.21.15.49:0/1834847448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7ba77d0 msgr2=0x5583e7c0ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:29.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.966+0000 7f5d7de00640 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7ba77d0 0x5583e7c0ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.966+0000 7f5d7de00640 1 -- 172.21.15.49:0/1834847448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583e7abf030 con 0x5583e7b9e460 2024-02-26T06:25:29.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.966+0000 7f5d7d400640 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9f140 0x5583e7bace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:29.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.967+0000 7f5d7de00640 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7bac8c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000ea30 tx=0x7f5d700d1e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c84e5477fbe9e8b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:29.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.967+0000 7f5d5f400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:25:29.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.967+0000 7f5d5f400640 1 -- 172.21.15.49:0/1834847448 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7001e2e0 con 0x5583e7b9e460 2024-02-26T06:25:29.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.967+0000 7f5d5f400640 1 -- 172.21.15.49:0/1834847448 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5d7009b730 con 0x5583e7b9e460 2024-02-26T06:25:29.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.967+0000 7f5d87a67740 1 -- 172.21.15.49:0/1834847448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d6c004040 con 0x5583e7b9e460 2024-02-26T06:25:29.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.968+0000 7f5d5f400640 1 -- 172.21.15.49:0/1834847448 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7009b030 con 0x5583e7b9e460 2024-02-26T06:25:29.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.968+0000 7f5d87a67740 10 client.25994 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:29.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.968+0000 7f5d87a67740 1 -- 172.21.15.49:0/1834847448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5583e7ac6400 con 0x5583e7b9e460 2024-02-26T06:25:29.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.969+0000 7f5d5f400640 1 -- 172.21.15.49:0/1834847448 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d6c004040 con 0x5583e7b9e460 2024-02-26T06:25:29.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.969+0000 7f5d5f400640 1 -- 172.21.15.49:0/1834847448 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f5d700976a0 con 0x5583e7b9e460 2024-02-26T06:25:29.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.969+0000 7f5d87a67740 10 client.25994 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:29.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.969+0000 7f5d87a67740 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:29.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.969+0000 7f5d87a67740 20 client.25994 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:29.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.969+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583e7c0ef90 0x5583e7c113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:29.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.969+0000 7f5d87a67740 4 client.25994 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:25:29.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.969+0000 7f5d87a67740 1 -- 172.21.15.49:0/1834847448 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5583e7891890 con 0x5583e7c0ef90 2024-02-26T06:25:29.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.970+0000 7f5d7d400640 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583e7c0ef90 0x5583e7c113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:29.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d7d400640 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583e7c0ef90 0x5583e7c113b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556f1ba20d43410f server_cookie=4002cbe26f24dd95 in_seq=0 out_seq=0 2024-02-26T06:25:29.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d5f400640 10 client.25994 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d5f400640 1 -- 172.21.15.49:0/1834847448 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5583e7891890 con 0x5583e7c0ef90 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d5f400640 10 client.25994 handle_command_reply: tid=0 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 client.25994 shutdown 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 -- 172.21.15.49:0/1834847448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 msgr2=0x5583e7bac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7bac8c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000ea30 tx=0x7f5d700d1e40 comp rx=0 tx=0).stop 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 -- 172.21.15.49:0/1834847448 shutdown_connections 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583e7c0ef90 0x5583e7c113b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583e7ba77d0 0x5583e7c0ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583e7b9f140 0x5583e7bace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 --2- 172.21.15.49:0/1834847448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583e7b9e460 0x5583e7bac8c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d5f400640 0 client.25994 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:29.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.971+0000 7f5d87a67740 1 -- 172.21.15.49:0/1834847448 >> 172.21.15.49:0/1834847448 conn(0x5583e7b65570 msgr2=0x5583e7ba2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.972+0000 7f5d87a67740 1 -- 172.21.15.49:0/1834847448 shutdown_connections 2024-02-26T06:25:29.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.972+0000 7f5d87a67740 1 -- 172.21.15.49:0/1834847448 wait complete. 2024-02-26T06:25:29.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.972+0000 7f5d87a67740 20 client.25994 trim_cache size 0 max 16384 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "e8df09d6-fff3-47db-8dd5-5ba8a6c034e8": { 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "e8df09d6-fff3-47db-8dd5-5ba8a6c034e8", 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:25:29.974 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:25:29.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 -- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5d640a2330 msgr2=0x7f5d640a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5d640a2330 0x7f5d640a47f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d78005fd0 tx=0x7f5d78009500 comp rx=0 tx=0).stop 2024-02-26T06:25:29.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 -- 172.21.15.49:0/810278587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d80103470 msgr2=0x7f5d80072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:29.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d80103470 0x7f5d80072b00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000ea60 tx=0x7f5d7000ef20 comp rx=0 tx=0).stop 2024-02-26T06:25:29.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 -- 172.21.15.49:0/810278587 shutdown_connections 2024-02-26T06:25:29.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5d640a2330 0x7f5d640a47f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d80108540 0x7f5d8006bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d8006c0c0 0x7f5d8006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 --2- 172.21.15.49:0/810278587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d80103470 0x7f5d80072b00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:29.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.973+0000 7f5d7f200640 1 -- 172.21.15.49:0/810278587 >> 172.21.15.49:0/810278587 conn(0x7f5d800fec90 msgr2=0x7f5d80100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:29.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.974+0000 7f5d7f200640 1 -- 172.21.15.49:0/810278587 shutdown_connections 2024-02-26T06:25:29.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:25:29.974+0000 7f5d7f200640 1 -- 172.21.15.49:0/810278587 wait complete. 2024-02-26T06:25:29.999 INFO:tasks.cephfs.filesystem:scrub status for tag:e8df09d6-fff3-47db-8dd5-5ba8a6c034e8 - {'path': '/', 'tag': 'e8df09d6-fff3-47db-8dd5-5ba8a6c034e8', 'options': 'recursive,force'} 2024-02-26T06:25:31.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.7 MiB/s wr, 871 op/s 2024-02-26T06:25:31.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:25:31.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:30 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.7 MiB/s wr, 871 op/s 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:25:31.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:25:31.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:30 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.7 MiB/s wr, 871 op/s 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:25:31.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:25:31.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:30 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:25:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 100.0 s, 2439.4 tps, lat 40.907 ms stddev 46.540 2024-02-26T06:25:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:32 smithi080 ceph-mon[36206]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 774 op/s 2024-02-26T06:25:33.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:32 smithi115 ceph-mon[36426]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 774 op/s 2024-02-26T06:25:33.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:32 smithi049 ceph-mon[31696]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 774 op/s 2024-02-26T06:25:35.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:34 smithi080 ceph-mon[36206]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 639 op/s 2024-02-26T06:25:35.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:34 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:35.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:34 smithi115 ceph-mon[36426]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 639 op/s 2024-02-26T06:25:35.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:34 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:35.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:34 smithi049 ceph-mon[31696]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 639 op/s 2024-02-26T06:25:35.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:34 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:36.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:35 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:25:36.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:35 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:25:36.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:35 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:25:36.305 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 105.0 s, 2341.0 tps, lat 42.677 ms stddev 48.632 2024-02-26T06:25:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:36 smithi080 ceph-mon[36206]: pgmap v69: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-02-26T06:25:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:36 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:25:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:36 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:25:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:36 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:36 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:36 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:25:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:36 smithi115 ceph-mon[36426]: pgmap v69: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-02-26T06:25:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:36 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:25:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:36 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:25:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:36 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:36 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:36 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:25:37.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:36 smithi049 ceph-mon[31696]: pgmap v69: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-02-26T06:25:37.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:36 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:25:37.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:36 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:25:37.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:36 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:37.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:36 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:37.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:36 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:25:39.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:38 smithi080 ceph-mon[36206]: pgmap v70: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.6 MiB/s wr, 709 op/s 2024-02-26T06:25:39.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:38 smithi115 ceph-mon[36426]: pgmap v70: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.6 MiB/s wr, 709 op/s 2024-02-26T06:25:39.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:38 smithi049 ceph-mon[31696]: pgmap v70: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.6 MiB/s wr, 709 op/s 2024-02-26T06:25:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.212+0000 7ff76e600640 1 -- 172.21.15.49:0/3050392415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff760099bc0 msgr2=0x7ff760099fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.212+0000 7ff76e600640 1 --2- 172.21.15.49:0/3050392415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff760099bc0 0x7ff760099fa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009930 tx=0x7ff75c02bc50 comp rx=0 tx=0).stop 2024-02-26T06:25:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.212+0000 7ff76e600640 1 -- 172.21.15.49:0/3050392415 shutdown_connections 2024-02-26T06:25:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.212+0000 7ff76e600640 1 --2- 172.21.15.49:0/3050392415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7600957f0 0x7ff7600a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.212+0000 7ff76e600640 1 --2- 172.21.15.49:0/3050392415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff760094e50 0x7ff7600952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.212+0000 7ff76e600640 1 --2- 172.21.15.49:0/3050392415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff760099bc0 0x7ff760099fa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.212+0000 7ff76e600640 1 -- 172.21.15.49:0/3050392415 >> 172.21.15.49:0/3050392415 conn(0x7ff76000a740 msgr2=0x7ff76000ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.213+0000 7ff76e600640 1 -- 172.21.15.49:0/3050392415 shutdown_connections 2024-02-26T06:25:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.213+0000 7ff76e600640 1 -- 172.21.15.49:0/3050392415 wait complete. 2024-02-26T06:25:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.214+0000 7ff76e600640 1 Processor -- start 2024-02-26T06:25:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.214+0000 7ff76e600640 1 -- start start 2024-02-26T06:25:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.214+0000 7ff76e600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff760094e50 0x7ff76012a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.214+0000 7ff76e600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7600957f0 0x7ff76012ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.214+0000 7ff76e600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff760099bc0 0x7ff76012eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.214+0000 7ff76e600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760136e60 con 0x7ff760099bc0 2024-02-26T06:25:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.214+0000 7ff76e600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76012eff0 con 0x7ff7600957f0 2024-02-26T06:25:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.214+0000 7ff76e600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76012f1b0 con 0x7ff760094e50 2024-02-26T06:25:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.215+0000 7ff767e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7600957f0 0x7ff76012ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.215+0000 7ff767e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7600957f0 0x7ff76012ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51014/0 (socket says 172.21.15.49:51014) 2024-02-26T06:25:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.215+0000 7ff767e00640 1 -- 172.21.15.49:0/2929776529 learned_addr learned my addr 172.21.15.49:0/2929776529 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.216+0000 7ff76d200640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff760094e50 0x7ff76012a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.216+0000 7ff76dc00640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff760099bc0 0x7ff76012eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.216+0000 7ff767e00640 1 -- 172.21.15.49:0/2929776529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff760094e50 msgr2=0x7ff76012a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.216+0000 7ff767e00640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff760094e50 0x7ff76012a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.216+0000 7ff767e00640 1 -- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff760099bc0 msgr2=0x7ff76012eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.216+0000 7ff767e00640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff760099bc0 0x7ff76012eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.216+0000 7ff767e00640 1 -- 172.21.15.49:0/2929776529 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c009560 con 0x7ff7600957f0 2024-02-26T06:25:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.217+0000 7ff767e00640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7600957f0 0x7ff76012ab40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff75800b4a0 tx=0x7ff75800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.217+0000 7ff765600640 1 -- 172.21.15.49:0/2929776529 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758010610 con 0x7ff7600957f0 2024-02-26T06:25:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.218+0000 7ff765600640 1 -- 172.21.15.49:0/2929776529 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff758010c50 con 0x7ff7600957f0 2024-02-26T06:25:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.218+0000 7ff765600640 1 -- 172.21.15.49:0/2929776529 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758017e20 con 0x7ff7600957f0 2024-02-26T06:25:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.218+0000 7ff76e600640 1 -- 172.21.15.49:0/2929776529 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76012f490 con 0x7ff7600957f0 2024-02-26T06:25:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.218+0000 7ff76e600640 1 -- 172.21.15.49:0/2929776529 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff760148050 con 0x7ff7600957f0 2024-02-26T06:25:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.222+0000 7ff765600640 1 -- 172.21.15.49:0/2929776529 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff758017480 con 0x7ff7600957f0 2024-02-26T06:25:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.223+0000 7ff772c21740 1 Processor -- start 2024-02-26T06:25:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.223+0000 7ff772c21740 1 -- start start 2024-02-26T06:25:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff772c21740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff772c21740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d69440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff772c21740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 0x55bdd7d74930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff772c21740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7d245f0 con 0x55bdd7d72540 2024-02-26T06:25:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff772c21740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7cd6890 con 0x55bdd7d69d30 2024-02-26T06:25:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff772c21740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7cd3dc0 con 0x55bdd7d68fc0 2024-02-26T06:25:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d68a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51026/0 (socket says 172.21.15.49:51026) 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff76dc00640 1 -- 172.21.15.49:0/2203469153 learned_addr learned my addr 172.21.15.49:0/2203469153 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.224+0000 7ff76d200640 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d69440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.225+0000 7ff76dc00640 1 -- 172.21.15.49:0/2203469153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 msgr2=0x55bdd7d69440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.225+0000 7ff76dc00640 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d69440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.225+0000 7ff76dc00640 1 -- 172.21.15.49:0/2203469153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 msgr2=0x55bdd7d74930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.225+0000 7ff767e00640 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 0x55bdd7d74930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.225+0000 7ff76dc00640 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 0x55bdd7d74930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.225+0000 7ff76dc00640 1 -- 172.21.15.49:0/2203469153 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdd7c50ad0 con 0x55bdd7d69d30 2024-02-26T06:25:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.225+0000 7ff76d200640 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d69440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:25:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.226+0000 7ff765600640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff7500a2380 0x7ff7500a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.226+0000 7ff765600640 1 -- 172.21.15.49:0/2929776529 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7ff7580cd920 con 0x7ff7600957f0 2024-02-26T06:25:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.227+0000 7ff76dc00640 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d68a80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff7680506c0 tx=0x7ff768068f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1533e5632f6290aa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.228+0000 7ff76d200640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff7500a2380 0x7ff7500a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.228+0000 7ff756000640 1 -- 172.21.15.49:0/2203469153 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768085070 con 0x55bdd7d69d30 2024-02-26T06:25:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.228+0000 7ff756000640 1 -- 172.21.15.49:0/2203469153 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff76807c050 con 0x55bdd7d69d30 2024-02-26T06:25:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.228+0000 7ff756000640 1 -- 172.21.15.49:0/2203469153 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768081840 con 0x55bdd7d69d30 2024-02-26T06:25:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.229+0000 7ff772c21740 1 -- 172.21.15.49:0/2203469153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 msgr2=0x55bdd7d68a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.229+0000 7ff772c21740 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d68a80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff7680506c0 tx=0x7ff768068f00 comp rx=0 tx=0).stop 2024-02-26T06:25:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.229+0000 7ff772c21740 1 -- 172.21.15.49:0/2203469153 shutdown_connections 2024-02-26T06:25:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.229+0000 7ff772c21740 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 0x55bdd7d74930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.229+0000 7ff772c21740 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d69440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.229+0000 7ff772c21740 1 --2- 172.21.15.49:0/2203469153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d68a80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.229+0000 7ff772c21740 1 -- 172.21.15.49:0/2203469153 >> 172.21.15.49:0/2203469153 conn(0x55bdd7d30480 msgr2=0x55bdd7d63cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.230+0000 7ff772c21740 1 -- 172.21.15.49:0/2203469153 shutdown_connections 2024-02-26T06:25:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.230+0000 7ff772c21740 1 -- 172.21.15.49:0/2203469153 wait complete. 2024-02-26T06:25:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.230+0000 7ff76d200640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff7500a2380 0x7ff7500a4840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009900 tx=0x7ff75c005b70 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.231+0000 7ff772c21740 1 -- start start 2024-02-26T06:25:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.231+0000 7ff772c21740 1 Processor -- start 2024-02-26T06:25:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.232+0000 7ff772c21740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d6f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.232+0000 7ff772c21740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.232+0000 7ff772c21740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.232+0000 7ff772c21740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7ccf5a0 con 0x55bdd7d69d30 2024-02-26T06:25:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.232+0000 7ff772c21740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7c74b20 con 0x55bdd7d68fc0 2024-02-26T06:25:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.232+0000 7ff772c21740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7c747c0 con 0x55bdd7d72540 2024-02-26T06:25:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.232+0000 7ff772c21740 10 client.0 _ll_register_callbacks cb 0x55bdd7c88520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58992/0 (socket says 172.21.15.49:58992) 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76dc00640 1 -- 172.21.15.49:0/3840068674 learned_addr learned my addr 172.21.15.49:0/3840068674 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff767e00640 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76d200640 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d6f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76dc00640 1 -- 172.21.15.49:0/3840068674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 msgr2=0x55bdd7d6f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76dc00640 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d6f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76dc00640 1 -- 172.21.15.49:0/3840068674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 msgr2=0x55bdd7d6faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76dc00640 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.233+0000 7ff76dc00640 1 -- 172.21.15.49:0/3840068674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff768067880 con 0x55bdd7d72540 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.234+0000 7ff76dc00640 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6ffe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff768050150 tx=0x7ff7680827c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3566ebd61a6f67ef server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.234+0000 7ff74aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.234+0000 7ff74aa00640 1 -- 172.21.15.49:0/3840068674 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76808e070 con 0x55bdd7d72540 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.234+0000 7ff772c21740 1 -- 172.21.15.49:0/3840068674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdd7ca1ca0 con 0x55bdd7d72540 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.234+0000 7ff74aa00640 1 -- 172.21.15.49:0/3840068674 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7ff768082c80 con 0x55bdd7d72540 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.234+0000 7ff74aa00640 1 -- 172.21.15.49:0/3840068674 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff768085040 con 0x55bdd7d72540 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.234+0000 7ff772c21740 10 client.26035 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.234+0000 7ff772c21740 1 -- 172.21.15.49:0/3840068674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bdd7ccdc20 con 0x55bdd7d72540 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.235+0000 7ff74aa00640 1 -- 172.21.15.49:0/3840068674 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bdd7ca1ca0 con 0x55bdd7d72540 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.235+0000 7ff74aa00640 1 -- 172.21.15.49:0/3840068674 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff768088030 con 0x55bdd7d72540 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.235+0000 7ff772c21740 10 client.26035 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.235+0000 7ff772c21740 10 client.26035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.235+0000 7ff772c21740 20 client.26035 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.235+0000 7ff772c21740 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55bdd7d82370 0x55bdd7d84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.235+0000 7ff772c21740 4 client.26035 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.235+0000 7ff772c21740 1 -- 172.21.15.49:0/3840068674 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdd7cc5cb0 con 0x55bdd7d82370 2024-02-26T06:25:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.236+0000 7ff76d200640 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55bdd7d82370 0x55bdd7d84790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.238+0000 7ff76d200640 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55bdd7d82370 0x55bdd7d84790 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a22fed776b789d server_cookie=93b5f79a50500538 in_seq=0 out_seq=0 2024-02-26T06:25:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.238+0000 7ff74aa00640 10 client.26035 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.239+0000 7ff74aa00640 1 -- 172.21.15.49:0/3840068674 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdd7cc5cb0 con 0x55bdd7d82370 2024-02-26T06:25:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.239+0000 7ff74aa00640 10 client.26035 handle_command_reply: tid=0 2024-02-26T06:25:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.239+0000 7ff772c21740 1 client.26035 shutdown 2024-02-26T06:25:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.239+0000 7ff772c21740 1 -- 172.21.15.49:0/3840068674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 msgr2=0x55bdd7d6ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.239+0000 7ff772c21740 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6ffe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff768050150 tx=0x7ff7680827c0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.239+0000 7ff772c21740 1 -- 172.21.15.49:0/3840068674 shutdown_connections 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.239+0000 7ff772c21740 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55bdd7d82370 0x55bdd7d84790 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.240+0000 7ff772c21740 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6ffe0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.240+0000 7ff772c21740 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.240+0000 7ff772c21740 1 --2- 172.21.15.49:0/3840068674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d6f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.240+0000 7ff772c21740 1 -- 172.21.15.49:0/3840068674 >> 172.21.15.49:0/3840068674 conn(0x55bdd7d30480 msgr2=0x55bdd7d65a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.240+0000 7ff74aa00640 0 client.26035 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.240+0000 7ff772c21740 1 -- 172.21.15.49:0/3840068674 shutdown_connections 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.240+0000 7ff772c21740 1 -- 172.21.15.49:0/3840068674 wait complete. 2024-02-26T06:25:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.240+0000 7ff772c21740 20 client.26035 trim_cache size 0 max 16384 2024-02-26T06:25:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.264+0000 7ff772c21740 1 Processor -- start 2024-02-26T06:25:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.265+0000 7ff772c21740 1 -- start start 2024-02-26T06:25:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.265+0000 7ff772c21740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d71c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.265+0000 7ff772c21740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.265+0000 7ff772c21740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.265+0000 7ff772c21740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7a17e60 con 0x55bdd7d69d30 2024-02-26T06:25:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.265+0000 7ff772c21740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7d75150 con 0x55bdd7d68fc0 2024-02-26T06:25:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.265+0000 7ff772c21740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7d6af60 con 0x55bdd7d72540 2024-02-26T06:25:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.266+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.266+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58996/0 (socket says 172.21.15.49:58996) 2024-02-26T06:25:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.266+0000 7ff76dc00640 1 -- 172.21.15.49:0/690824326 learned_addr learned my addr 172.21.15.49:0/690824326 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.266+0000 7ff76d200640 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d71c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.266+0000 7ff767e00640 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.267+0000 7ff76dc00640 1 -- 172.21.15.49:0/690824326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 msgr2=0x55bdd7d71c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.267+0000 7ff76dc00640 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d71c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.267+0000 7ff76dc00640 1 -- 172.21.15.49:0/690824326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 msgr2=0x55bdd7d6fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.267+0000 7ff76dc00640 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.267+0000 7ff76dc00640 1 -- 172.21.15.49:0/690824326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdd7c6a500 con 0x55bdd7d72540 2024-02-26T06:25:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.267+0000 7ff76d200640 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d71c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.267+0000 7ff767e00640 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:25:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.267+0000 7ff76dc00640 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6de50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff768066950 tx=0x7ff768082420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ffc4310753e8f62 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.268+0000 7ff755600640 1 -- 172.21.15.49:0/690824326 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76808e070 con 0x55bdd7d72540 2024-02-26T06:25:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.268+0000 7ff755600640 1 -- 172.21.15.49:0/690824326 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff76807c050 con 0x55bdd7d72540 2024-02-26T06:25:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.269+0000 7ff755600640 1 -- 172.21.15.49:0/690824326 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768085070 con 0x55bdd7d72540 2024-02-26T06:25:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.270+0000 7ff772c21740 1 -- 172.21.15.49:0/690824326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 msgr2=0x55bdd7d6de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.270+0000 7ff772c21740 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6de50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff768066950 tx=0x7ff768082420 comp rx=0 tx=0).stop 2024-02-26T06:25:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.271+0000 7ff772c21740 1 -- 172.21.15.49:0/690824326 shutdown_connections 2024-02-26T06:25:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.271+0000 7ff772c21740 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d72540 0x55bdd7d6de50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.271+0000 7ff772c21740 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d69d30 0x55bdd7d6fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.271+0000 7ff772c21740 1 --2- 172.21.15.49:0/690824326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d71c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.271+0000 7ff772c21740 1 -- 172.21.15.49:0/690824326 >> 172.21.15.49:0/690824326 conn(0x55bdd7d30480 msgr2=0x55bdd7d637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.271+0000 7ff772c21740 1 -- 172.21.15.49:0/690824326 shutdown_connections 2024-02-26T06:25:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.271+0000 7ff772c21740 1 -- 172.21.15.49:0/690824326 wait complete. 2024-02-26T06:25:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.271+0000 7ff772c21740 1 -- start start 2024-02-26T06:25:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.272+0000 7ff772c21740 1 Processor -- start 2024-02-26T06:25:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.273+0000 7ff772c21740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff772c21740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d77a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff76d200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff772c21740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 0x55bdd7dd7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff772c21740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7dd7db0 con 0x55bdd7d72540 2024-02-26T06:25:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff76d200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d774f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58998/0 (socket says 172.21.15.49:58998) 2024-02-26T06:25:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff76d200640 1 -- 172.21.15.49:0/200396382 learned_addr learned my addr 172.21.15.49:0/200396382 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7dd7ef0 con 0x55bdd7d69d30 2024-02-26T06:25:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff767e00640 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d77a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.274+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd7dd8030 con 0x55bdd7d68fc0 2024-02-26T06:25:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.275+0000 7ff76dc00640 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 0x55bdd7dd7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.275+0000 7ff772c21740 10 client.0 _ll_register_callbacks cb 0x7ff76804f730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.275+0000 7ff76d200640 1 -- 172.21.15.49:0/200396382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 msgr2=0x55bdd7d77a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.275+0000 7ff76d200640 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d77a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.275+0000 7ff76d200640 1 -- 172.21.15.49:0/200396382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 msgr2=0x55bdd7dd7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.275+0000 7ff76d200640 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 0x55bdd7dd7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.275+0000 7ff76d200640 1 -- 172.21.15.49:0/200396382 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdd7ccdc20 con 0x55bdd7d68fc0 2024-02-26T06:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.276+0000 7ff76d200640 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d774f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005c40 tx=0x7ff75c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59b6219621fdd49 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.276+0000 7ff74aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.276+0000 7ff74aa00640 1 -- 172.21.15.49:0/200396382 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c050040 con 0x55bdd7d68fc0 2024-02-26T06:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.276+0000 7ff74aa00640 1 -- 172.21.15.49:0/200396382 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7ff75c053be0 con 0x55bdd7d68fc0 2024-02-26T06:25:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.276+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdd7cf4d00 con 0x55bdd7d68fc0 2024-02-26T06:25:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.276+0000 7ff74aa00640 1 -- 172.21.15.49:0/200396382 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75c053de0 con 0x55bdd7d68fc0 2024-02-26T06:25:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff772c21740 10 client.26041 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bdd7c90290 con 0x55bdd7d68fc0 2024-02-26T06:25:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff74aa00640 1 -- 172.21.15.49:0/200396382 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff75c068080 con 0x55bdd7d68fc0 2024-02-26T06:25:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff74aa00640 1 -- 172.21.15.49:0/200396382 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff75c046040 con 0x55bdd7d68fc0 2024-02-26T06:25:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff772c21740 10 client.26041 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff772c21740 10 client.26041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff772c21740 20 client.26041 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff772c21740 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55bdd7dd9800 0x55bdd7ddbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff772c21740 4 client.26041 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:25:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdd79cf600 con 0x55bdd7dd9800 2024-02-26T06:25:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.277+0000 7ff767e00640 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55bdd7dd9800 0x55bdd7ddbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.278+0000 7ff767e00640 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55bdd7dd9800 0x55bdd7ddbc20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc06416d8d4dc6cd server_cookie=8259bb3118bc806a in_seq=0 out_seq=0 2024-02-26T06:25:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.278+0000 7ff74aa00640 10 client.26041 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff74aa00640 1 -- 172.21.15.49:0/200396382 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdd79cf600 con 0x55bdd7dd9800 2024-02-26T06:25:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff74aa00640 10 client.26041 handle_command_reply: tid=0 2024-02-26T06:25:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff772c21740 1 client.26041 shutdown 2024-02-26T06:25:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 msgr2=0x55bdd7d774f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff772c21740 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d774f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005c40 tx=0x7ff75c009260 comp rx=0 tx=0).stop 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 shutdown_connections 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff772c21740 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55bdd7dd9800 0x55bdd7ddbc20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff772c21740 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55bdd7d69d30 0x55bdd7d77a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.291+0000 7ff772c21740 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdd7d72540 0x55bdd7dd7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.292+0000 7ff772c21740 1 --2- 172.21.15.49:0/200396382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdd7d68fc0 0x55bdd7d774f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.292+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 >> 172.21.15.49:0/200396382 conn(0x55bdd7d30480 msgr2=0x55bdd7d6de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.292+0000 7ff74aa00640 0 client.26041 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.292+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 shutdown_connections 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.292+0000 7ff772c21740 1 -- 172.21.15.49:0/200396382 wait complete. 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.292+0000 7ff772c21740 20 client.26041 trim_cache size 0 max 16384 2024-02-26T06:25:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:25:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:25:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:25:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:25:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 -- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff7500a2380 msgr2=0x7ff7500a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff7500a2380 0x7ff7500a4840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009900 tx=0x7ff75c005b70 comp rx=0 tx=0).stop 2024-02-26T06:25:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 -- 172.21.15.49:0/2929776529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7600957f0 msgr2=0x7ff76012ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7600957f0 0x7ff76012ab40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff75800b4a0 tx=0x7ff75800b960 comp rx=0 tx=0).stop 2024-02-26T06:25:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 -- 172.21.15.49:0/2929776529 shutdown_connections 2024-02-26T06:25:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff7500a2380 0x7ff7500a4840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7600957f0 0x7ff76012ab40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff760099bc0 0x7ff76012eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 --2- 172.21.15.49:0/2929776529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff760094e50 0x7ff76012a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 -- 172.21.15.49:0/2929776529 >> 172.21.15.49:0/2929776529 conn(0x7ff76000a740 msgr2=0x7ff760091f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.293+0000 7ff76e600640 1 -- 172.21.15.49:0/2929776529 shutdown_connections 2024-02-26T06:25:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:39.294+0000 7ff76e600640 1 -- 172.21.15.49:0/2929776529 wait complete. 2024-02-26T06:25:41.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:40 smithi080 ceph-mon[36206]: pgmap v71: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.6 MiB/s wr, 837 op/s 2024-02-26T06:25:41.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:40 smithi115 ceph-mon[36426]: pgmap v71: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.6 MiB/s wr, 837 op/s 2024-02-26T06:25:41.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:40 smithi049 ceph-mon[31696]: pgmap v71: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.6 MiB/s wr, 837 op/s 2024-02-26T06:25:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 110.0 s, 2539.7 tps, lat 39.313 ms stddev 44.213 2024-02-26T06:25:43.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:42 smithi080 ceph-mon[36206]: pgmap v72: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.4 MiB/s wr, 795 op/s 2024-02-26T06:25:43.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:42 smithi115 ceph-mon[36426]: pgmap v72: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.4 MiB/s wr, 795 op/s 2024-02-26T06:25:43.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:42 smithi049 ceph-mon[31696]: pgmap v72: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.4 MiB/s wr, 795 op/s 2024-02-26T06:25:45.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:44 smithi080 ceph-mon[36206]: pgmap v73: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 644 op/s 2024-02-26T06:25:45.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:44 smithi115 ceph-mon[36426]: pgmap v73: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 644 op/s 2024-02-26T06:25:45.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:44 smithi049 ceph-mon[31696]: pgmap v73: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 644 op/s 2024-02-26T06:25:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 115.0 s, 2305.0 tps, lat 43.341 ms stddev 47.093 2024-02-26T06:25:47.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:46 smithi080 ceph-mon[36206]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.2 MiB/s wr, 924 op/s 2024-02-26T06:25:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:46 smithi115 ceph-mon[36426]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.2 MiB/s wr, 924 op/s 2024-02-26T06:25:47.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:46 smithi049 ceph-mon[31696]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.2 MiB/s wr, 924 op/s 2024-02-26T06:25:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:48 smithi080 ceph-mon[36206]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 565 op/s 2024-02-26T06:25:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:48 smithi115 ceph-mon[36426]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 565 op/s 2024-02-26T06:25:49.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:48 smithi049 ceph-mon[31696]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 565 op/s 2024-02-26T06:25:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:50 smithi080 ceph-mon[36206]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 742 op/s 2024-02-26T06:25:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:50 smithi115 ceph-mon[36426]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 742 op/s 2024-02-26T06:25:51.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:50 smithi049 ceph-mon[31696]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 742 op/s 2024-02-26T06:25:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 120.0 s, 2935.5 tps, lat 34.191 ms stddev 38.209 2024-02-26T06:25:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.487+0000 7fd508c00640 1 -- 172.21.15.49:0/3506369264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 msgr2=0x7fd504068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.487+0000 7fd508c00640 1 --2- 172.21.15.49:0/3506369264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 0x7fd504068c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009900 tx=0x7fd4f402bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.488+0000 7fd508c00640 1 -- 172.21.15.49:0/3506369264 shutdown_connections 2024-02-26T06:25:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.488+0000 7fd508c00640 1 --2- 172.21.15.49:0/3506369264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 0x7fd504068c80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.488+0000 7fd508c00640 1 --2- 172.21.15.49:0/3506369264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5040691c0 0x7fd50410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.488+0000 7fd508c00640 1 --2- 172.21.15.49:0/3506369264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5041070d0 0x7fd5041074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.488+0000 7fd508c00640 1 -- 172.21.15.49:0/3506369264 >> 172.21.15.49:0/3506369264 conn(0x7fd504075a00 msgr2=0x7fd504075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.488+0000 7fd508c00640 1 -- 172.21.15.49:0/3506369264 shutdown_connections 2024-02-26T06:25:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.488+0000 7fd508c00640 1 -- 172.21.15.49:0/3506369264 wait complete. 2024-02-26T06:25:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.489+0000 7fd508c00640 1 Processor -- start 2024-02-26T06:25:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.490+0000 7fd508c00640 1 -- start start 2024-02-26T06:25:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.490+0000 7fd508c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 0x7fd5041001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.490+0000 7fd508c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5040691c0 0x7fd504100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.490+0000 7fd508c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5041070d0 0x7fd504100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.490+0000 7fd508c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a5940 con 0x7fd5041070d0 2024-02-26T06:25:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.490+0000 7fd508c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a5ab0 con 0x7fd504068800 2024-02-26T06:25:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.490+0000 7fd508c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a5c70 con 0x7fd5040691c0 2024-02-26T06:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd503400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 0x7fd5041001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd503400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 0x7fd5041001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39166/0 (socket says 172.21.15.49:39166) 2024-02-26T06:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd503400640 1 -- 172.21.15.49:0/2464931080 learned_addr learned my addr 172.21.15.49:0/2464931080 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd502a00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5040691c0 0x7fd504100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd502a00640 1 -- 172.21.15.49:0/2464931080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 msgr2=0x7fd5041001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd502a00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 0x7fd5041001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd502a00640 1 -- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5041070d0 msgr2=0x7fd504100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd502a00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5041070d0 0x7fd504100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.491+0000 7fd502a00640 1 -- 172.21.15.49:0/2464931080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f4009560 con 0x7fd5040691c0 2024-02-26T06:25:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.492+0000 7fd502a00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5040691c0 0x7fd504100720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009850 tx=0x7fd4f4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.492+0000 7fd4fbe00640 1 -- 172.21.15.49:0/2464931080 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f402dea0 con 0x7fd5040691c0 2024-02-26T06:25:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.492+0000 7fd4fbe00640 1 -- 172.21.15.49:0/2464931080 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4f4004480 con 0x7fd5040691c0 2024-02-26T06:25:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.492+0000 7fd4fbe00640 1 -- 172.21.15.49:0/2464931080 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4003bc0 con 0x7fd5040691c0 2024-02-26T06:25:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.492+0000 7fd508c00640 1 -- 172.21.15.49:0/2464931080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5041a5ef0 con 0x7fd5040691c0 2024-02-26T06:25:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.492+0000 7fd508c00640 1 -- 172.21.15.49:0/2464931080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5041a62c0 con 0x7fd5040691c0 2024-02-26T06:25:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.500+0000 7fd50d164740 1 Processor -- start 2024-02-26T06:25:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.500+0000 7fd4fbe00640 1 -- 172.21.15.49:0/2464931080 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd4f4003d20 con 0x7fd5040691c0 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.503+0000 7fd50d164740 1 -- start start 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd50d164740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd50d164740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b437b0 0x55f6f6b43c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd50d164740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd50d164740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6af65f0 con 0x55f6f6b44520 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd50d164740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6aa8890 con 0x55f6f6b437b0 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd50d164740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6aa5dc0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd503400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b437b0 0x55f6f6b43c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd503e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd503400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b437b0 0x55f6f6b43c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39178/0 (socket says 172.21.15.49:39178) 2024-02-26T06:25:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd503400640 1 -- 172.21.15.49:0/3861474333 learned_addr learned my addr 172.21.15.49:0/3861474333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd502a00640 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.504+0000 7fd4fbe00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4e80a2430 0x7fd4e80a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd503e00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4e80a2430 0x7fd4e80a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd4fbe00640 1 -- 172.21.15.49:0/2464931080 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd4f40f0580 con 0x7fd5040691c0 2024-02-26T06:25:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd503400640 1 -- 172.21.15.49:0/3861474333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 msgr2=0x55f6f6b4f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd503400640 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd503400640 1 -- 172.21.15.49:0/3861474333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 msgr2=0x55f6f6b43270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd503400640 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b43270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd503400640 1 -- 172.21.15.49:0/3861474333 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6f6a22ad0 con 0x55f6f6b437b0 2024-02-26T06:25:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd503e00640 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.505+0000 7fd503400640 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b437b0 0x55f6f6b43c30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009d40 tx=0x7fd4f0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a1e96abb3fa22f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.506+0000 7fd503e00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4e80a2430 0x7fd4e80a48f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009270 tx=0x7fd4fc009200 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.506+0000 7fd4efe00640 1 -- 172.21.15.49:0/3861474333 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0004220 con 0x55f6f6b437b0 2024-02-26T06:25:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.506+0000 7fd4efe00640 1 -- 172.21.15.49:0/3861474333 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4f0004380 con 0x55f6f6b437b0 2024-02-26T06:25:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.506+0000 7fd4efe00640 1 -- 172.21.15.49:0/3861474333 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0004e60 con 0x55f6f6b437b0 2024-02-26T06:25:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.508+0000 7fd50d164740 1 -- 172.21.15.49:0/3861474333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b437b0 msgr2=0x55f6f6b43c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.508+0000 7fd50d164740 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b437b0 0x55f6f6b43c30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009d40 tx=0x7fd4f0011810 comp rx=0 tx=0).stop 2024-02-26T06:25:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.508+0000 7fd50d164740 1 -- 172.21.15.49:0/3861474333 shutdown_connections 2024-02-26T06:25:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.508+0000 7fd50d164740 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b437b0 0x55f6f6b43c30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.508+0000 7fd50d164740 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.508+0000 7fd50d164740 1 --2- 172.21.15.49:0/3861474333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.508+0000 7fd50d164740 1 -- 172.21.15.49:0/3861474333 >> 172.21.15.49:0/3861474333 conn(0x55f6f6b02480 msgr2=0x55f6f6b3e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.508+0000 7fd50d164740 1 -- 172.21.15.49:0/3861474333 shutdown_connections 2024-02-26T06:25:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.509+0000 7fd50d164740 1 -- 172.21.15.49:0/3861474333 wait complete. 2024-02-26T06:25:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.509+0000 7fd50d164740 1 -- start start 2024-02-26T06:25:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.510+0000 7fd50d164740 1 Processor -- start 2024-02-26T06:25:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.511+0000 7fd50d164740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b49d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.511+0000 7fd50d164740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b4a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.511+0000 7fd503400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b49d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.511+0000 7fd503400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b49d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55722/0 (socket says 172.21.15.49:55722) 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.512+0000 7fd50d164740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.512+0000 7fd503400640 1 -- 172.21.15.49:0/3198619886 learned_addr learned my addr 172.21.15.49:0/3198619886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.512+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6aa15a0 con 0x55f6f6b437b0 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.512+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6a46b20 con 0x55f6f6b44520 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.512+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6a467c0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.513+0000 7fd503e00640 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.513+0000 7fd50d164740 10 client.0 _ll_register_callbacks cb 0x55f6f6a5a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.513+0000 7fd503e00640 1 -- 172.21.15.49:0/3198619886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 msgr2=0x55f6f6b4a250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.513+0000 7fd503e00640 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b4a250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.513+0000 7fd503e00640 1 -- 172.21.15.49:0/3198619886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 msgr2=0x55f6f6b49d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.513+0000 7fd503e00640 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b49d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.513+0000 7fd503e00640 1 -- 172.21.15.49:0/3198619886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f00099a0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.514+0000 7fd503e00640 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4a790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc003c20 tx=0x7fd4fc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a2d35ddd3c993b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.514+0000 7fd4ecc00640 10 client.26053 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.514+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6f6a73ca0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.514+0000 7fd503400640 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b49d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.514+0000 7fd4ecc00640 1 -- 172.21.15.49:0/3198619886 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc025070 con 0x55f6f6b4cd30 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.514+0000 7fd4ecc00640 1 -- 172.21.15.49:0/3198619886 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd4fc004b20 con 0x55f6f6b4cd30 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.514+0000 7fd4ecc00640 1 -- 172.21.15.49:0/3198619886 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4fc004ce0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd50d164740 10 client.26053 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55f6f6a9fc20 con 0x55f6f6b4cd30 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd4ecc00640 1 -- 172.21.15.49:0/3198619886 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd4fc028030 con 0x55f6f6b4cd30 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd50d164740 10 client.26053 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd50d164740 10 client.26053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd50d164740 20 client.26053 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd50d164740 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6f6b6dcc0 0x55f6f6b70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd50d164740 4 client.26053 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:25:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.515+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6f6a97cb0 con 0x55f6f6b6dcc0 2024-02-26T06:25:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.516+0000 7fd503400640 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6f6b6dcc0 0x55f6f6b70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.517+0000 7fd503400640 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6f6b6dcc0 0x55f6f6b70100 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde3a308e03d8b92 server_cookie=acdcce32755a309a in_seq=0 out_seq=0 2024-02-26T06:25:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.517+0000 7fd4ecc00640 10 client.26053 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.518+0000 7fd4ecc00640 1 -- 172.21.15.49:0/3198619886 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f6f6a97cb0 con 0x55f6f6b6dcc0 2024-02-26T06:25:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.518+0000 7fd4ecc00640 10 client.26053 handle_command_reply: tid=0 2024-02-26T06:25:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.518+0000 7fd50d164740 1 client.26053 shutdown 2024-02-26T06:25:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.518+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 msgr2=0x55f6f6b4a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.518+0000 7fd50d164740 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4a790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc003c20 tx=0x7fd4fc004a40 comp rx=0 tx=0).stop 2024-02-26T06:25:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 shutdown_connections 2024-02-26T06:25:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd50d164740 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6f6b6dcc0 0x55f6f6b70100 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd50d164740 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b4a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd50d164740 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b4a790 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd50d164740 1 --2- 172.21.15.49:0/3198619886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b49d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 >> 172.21.15.49:0/3198619886 conn(0x55f6f6b02480 msgr2=0x55f6f6b401f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd4ecc00640 0 client.26053 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 shutdown_connections 2024-02-26T06:25:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.519+0000 7fd50d164740 1 -- 172.21.15.49:0/3198619886 wait complete. 2024-02-26T06:25:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.520+0000 7fd50d164740 20 client.26053 trim_cache size 0 max 16384 2024-02-26T06:25:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.544+0000 7fd50d164740 1 Processor -- start 2024-02-26T06:25:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.544+0000 7fd50d164740 1 -- start start 2024-02-26T06:25:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.545+0000 7fd50d164740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b437b0 0x55f6f6b4c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.546+0000 7fd50d164740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.546+0000 7fd50d164740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.546+0000 7fd50d164740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f67e9e60 con 0x55f6f6b44520 2024-02-26T06:25:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.546+0000 7fd50d164740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6a97cb0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.546+0000 7fd50d164740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6b6a820 con 0x55f6f6b437b0 2024-02-26T06:25:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.546+0000 7fd503e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.546+0000 7fd503400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b437b0 0x55f6f6b4c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.547+0000 7fd502a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.547+0000 7fd503400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b437b0 0x55f6f6b4c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58024/0 (socket says 172.21.15.49:58024) 2024-02-26T06:25:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.547+0000 7fd503400640 1 -- 172.21.15.49:0/2638853486 learned_addr learned my addr 172.21.15.49:0/2638853486 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.548+0000 7fd502a00640 1 -- 172.21.15.49:0/2638853486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b437b0 msgr2=0x55f6f6b4c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.548+0000 7fd502a00640 1 --2- 172.21.15.49:0/2638853486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b437b0 0x55f6f6b4c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.548+0000 7fd502a00640 1 -- 172.21.15.49:0/2638853486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b4cd30 msgr2=0x55f6f6b650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.548+0000 7fd502a00640 1 --2- 172.21.15.49:0/2638853486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.548+0000 7fd502a00640 1 -- 172.21.15.49:0/2638853486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6f6a3c500 con 0x55f6f6b44520 2024-02-26T06:25:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.548+0000 7fd503400640 1 --2- 172.21.15.49:0/2638853486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b437b0 0x55f6f6b4c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.548+0000 7fd502a00640 1 --2- 172.21.15.49:0/2638853486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40edd90 tx=0x7fd4f402c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfca4b19b4eb798f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.549+0000 7fd4ef400640 1 -- 172.21.15.49:0/2638853486 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40bcb70 con 0x55f6f6b44520 2024-02-26T06:25:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.549+0000 7fd4ef400640 1 -- 172.21.15.49:0/2638853486 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4f40bc990 con 0x55f6f6b44520 2024-02-26T06:25:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.549+0000 7fd4ef400640 1 -- 172.21.15.49:0/2638853486 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40b9840 con 0x55f6f6b44520 2024-02-26T06:25:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 -- 172.21.15.49:0/2638853486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 msgr2=0x55f6f6b64bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 --2- 172.21.15.49:0/2638853486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40edd90 tx=0x7fd4f402c150 comp rx=0 tx=0).stop 2024-02-26T06:25:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 -- 172.21.15.49:0/2638853486 shutdown_connections 2024-02-26T06:25:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 --2- 172.21.15.49:0/2638853486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 --2- 172.21.15.49:0/2638853486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 --2- 172.21.15.49:0/2638853486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b437b0 0x55f6f6b4c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 -- 172.21.15.49:0/2638853486 >> 172.21.15.49:0/2638853486 conn(0x55f6f6b02480 msgr2=0x55f6f6b74aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 -- 172.21.15.49:0/2638853486 shutdown_connections 2024-02-26T06:25:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.551+0000 7fd50d164740 1 -- 172.21.15.49:0/2638853486 wait complete. 2024-02-26T06:25:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.552+0000 7fd50d164740 1 -- start start 2024-02-26T06:25:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.552+0000 7fd50d164740 1 Processor -- start 2024-02-26T06:25:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.552+0000 7fd50d164740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b47a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.552+0000 7fd50d164740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd50d164740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b45f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd50d164740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6b654c0 con 0x55f6f6b437b0 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd50d164740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6b464c0 con 0x55f6f6b44520 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd50d164740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f6b46680 con 0x55f6f6b4cd30 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd50d164740 10 client.0 _ll_register_callbacks cb 0x7fd4fc015420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd502a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd502a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39214/0 (socket says 172.21.15.49:39214) 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd502a00640 1 -- 172.21.15.49:0/2583118023 learned_addr learned my addr 172.21.15.49:0/2583118023 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd503e00640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b45f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.553+0000 7fd503400640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b47a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.554+0000 7fd503e00640 1 -- 172.21.15.49:0/2583118023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 msgr2=0x55f6f6b64bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.554+0000 7fd503e00640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.554+0000 7fd503e00640 1 -- 172.21.15.49:0/2583118023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 msgr2=0x55f6f6b47a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:25:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.554+0000 7fd503e00640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b47a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.555+0000 7fd503e00640 1 -- 172.21.15.49:0/2583118023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6f6a22ad0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.555+0000 7fd502a00640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.555+0000 7fd503400640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b47a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.555+0000 7fd503e00640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b45f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0092f0 tx=0x7fd4fc00e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4c4fbfcee1c203 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.555+0000 7fd4ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:25:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.556+0000 7fd4ecc00640 1 -- 172.21.15.49:0/2583118023 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc025040 con 0x55f6f6b4cd30 2024-02-26T06:25:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.556+0000 7fd50d164740 1 -- 172.21.15.49:0/2583118023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6f6ac6d00 con 0x55f6f6b4cd30 2024-02-26T06:25:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.556+0000 7fd4ecc00640 1 -- 172.21.15.49:0/2583118023 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd4fc004b20 con 0x55f6f6b4cd30 2024-02-26T06:25:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.557+0000 7fd4ecc00640 1 -- 172.21.15.49:0/2583118023 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4fc004ce0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.557+0000 7fd50d164740 10 client.26059 fetch_fsmap learned FSMap version 29 2024-02-26T06:25:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.557+0000 7fd50d164740 1 -- 172.21.15.49:0/2583118023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fd5041a62c0 con 0x55f6f6b4cd30 2024-02-26T06:25:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.557+0000 7fd4ecc00640 1 -- 172.21.15.49:0/2583118023 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f6f6ac6d00 con 0x55f6f6b4cd30 2024-02-26T06:25:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.557+0000 7fd4ecc00640 1 -- 172.21.15.49:0/2583118023 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd4fc028070 con 0x55f6f6b4cd30 2024-02-26T06:25:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.558+0000 7fd50d164740 10 client.26059 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:25:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.558+0000 7fd50d164740 10 client.26059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:25:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.558+0000 7fd50d164740 20 client.26059 populate_metadata read hostname 'smithi049' 2024-02-26T06:25:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.558+0000 7fd50d164740 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6f6bc5100 0x55f6f6bc7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.558+0000 7fd50d164740 4 client.26059 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:25:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.558+0000 7fd50d164740 1 -- 172.21.15.49:0/2583118023 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f6f67a1600 con 0x55f6f6bc5100 2024-02-26T06:25:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.558+0000 7fd503400640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6f6bc5100 0x55f6f6bc7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.559+0000 7fd503400640 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6f6bc5100 0x55f6f6bc7520 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2220f5873b2bb454 server_cookie=bd4a91f360409a3e in_seq=0 out_seq=0 2024-02-26T06:25:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.559+0000 7fd4ecc00640 10 client.26059 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.570+0000 7fd4ecc00640 1 -- 172.21.15.49:0/2583118023 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f6f67a1600 con 0x55f6f6bc5100 2024-02-26T06:25:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.570+0000 7fd4ecc00640 10 client.26059 handle_command_reply: tid=0 2024-02-26T06:25:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.570+0000 7fd50d164740 1 client.26059 shutdown 2024-02-26T06:25:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.571+0000 7fd50d164740 1 -- 172.21.15.49:0/2583118023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 msgr2=0x55f6f6b45f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.571+0000 7fd50d164740 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b45f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0092f0 tx=0x7fd4fc00e000 comp rx=0 tx=0).stop 2024-02-26T06:25:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd50d164740 1 -- 172.21.15.49:0/2583118023 shutdown_connections 2024-02-26T06:25:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd50d164740 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f6f6bc5100 0x55f6f6bc7520 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd50d164740 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f6f6b44520 0x55f6f6b64bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd50d164740 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6f6b4cd30 0x55f6f6b45f80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd50d164740 1 --2- 172.21.15.49:0/2583118023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f6f6b437b0 0x55f6f6b47a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd50d164740 1 -- 172.21.15.49:0/2583118023 >> 172.21.15.49:0/2583118023 conn(0x55f6f6b02480 msgr2=0x55f6f6b45010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd4ecc00640 0 client.26059 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:25:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd50d164740 1 -- 172.21.15.49:0/2583118023 shutdown_connections 2024-02-26T06:25:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.572+0000 7fd50d164740 1 -- 172.21.15.49:0/2583118023 wait complete. 2024-02-26T06:25:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.573+0000 7fd50d164740 20 client.26059 trim_cache size 0 max 16384 2024-02-26T06:25:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:25:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:25:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:25:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:25:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.573+0000 7fd508c00640 1 -- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4e80a2430 msgr2=0x7fd4e80a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.573+0000 7fd508c00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4e80a2430 0x7fd4e80a48f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009270 tx=0x7fd4fc009200 comp rx=0 tx=0).stop 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.573+0000 7fd508c00640 1 -- 172.21.15.49:0/2464931080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5040691c0 msgr2=0x7fd504100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.573+0000 7fd508c00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5040691c0 0x7fd504100720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009850 tx=0x7fd4f4004040 comp rx=0 tx=0).stop 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.574+0000 7fd508c00640 1 -- 172.21.15.49:0/2464931080 shutdown_connections 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.574+0000 7fd508c00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4e80a2430 0x7fd4e80a48f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.574+0000 7fd508c00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5041070d0 0x7fd504100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.574+0000 7fd508c00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5040691c0 0x7fd504100720 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.574+0000 7fd508c00640 1 --2- 172.21.15.49:0/2464931080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd504068800 0x7fd5041001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.574+0000 7fd508c00640 1 -- 172.21.15.49:0/2464931080 >> 172.21.15.49:0/2464931080 conn(0x7fd504075a00 msgr2=0x7fd5040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.575+0000 7fd508c00640 1 -- 172.21.15.49:0/2464931080 shutdown_connections 2024-02-26T06:25:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:25:52.575+0000 7fd508c00640 1 -- 172.21.15.49:0/2464931080 wait complete. 2024-02-26T06:25:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:52 smithi080 ceph-mon[36206]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 750 op/s 2024-02-26T06:25:53.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:52 smithi115 ceph-mon[36426]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 750 op/s 2024-02-26T06:25:53.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:52 smithi049 ceph-mon[31696]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 750 op/s 2024-02-26T06:25:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:54 smithi080 ceph-mon[36206]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.1 MiB/s wr, 592 op/s 2024-02-26T06:25:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:54 smithi115 ceph-mon[36426]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.1 MiB/s wr, 592 op/s 2024-02-26T06:25:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:54 smithi049 ceph-mon[31696]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.1 MiB/s wr, 592 op/s 2024-02-26T06:25:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 125.0 s, 2560.0 tps, lat 39.030 ms stddev 42.178 2024-02-26T06:25:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:56 smithi080 ceph-mon[36206]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 893 op/s 2024-02-26T06:25:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:56 smithi115 ceph-mon[36426]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 893 op/s 2024-02-26T06:25:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:56 smithi049 ceph-mon[31696]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 893 op/s 2024-02-26T06:25:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:25:58 smithi080 ceph-mon[36206]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 613 op/s 2024-02-26T06:25:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:25:58 smithi115 ceph-mon[36426]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 613 op/s 2024-02-26T06:25:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:25:58 smithi049 ceph-mon[31696]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 613 op/s 2024-02-26T06:25:59.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:26:00.000 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:26:00.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.288+0000 7f6636800640 1 -- 172.21.15.49:0/1246526101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6630108660 msgr2=0x7f6630110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.288+0000 7f6636800640 1 --2- 172.21.15.49:0/1246526101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6630108660 0x7f6630110230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6628009cd0 tx=0x7f662802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.289+0000 7f6636800640 1 -- 172.21.15.49:0/1246526101 shutdown_connections 2024-02-26T06:26:00.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.289+0000 7f6636800640 1 --2- 172.21.15.49:0/1246526101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6630108660 0x7f6630110230 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.289+0000 7f6636800640 1 --2- 172.21.15.49:0/1246526101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6630103f40 0x7f6630107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.289+0000 7f6636800640 1 --2- 172.21.15.49:0/1246526101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6630103590 0x7f6630103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.289+0000 7f6636800640 1 -- 172.21.15.49:0/1246526101 >> 172.21.15.49:0/1246526101 conn(0x7f66300fee30 msgr2=0x7f6630101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:00.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.289+0000 7f6636800640 1 -- 172.21.15.49:0/1246526101 shutdown_connections 2024-02-26T06:26:00.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.290+0000 7f6636800640 1 -- 172.21.15.49:0/1246526101 wait complete. 2024-02-26T06:26:00.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.291+0000 7f6636800640 1 Processor -- start 2024-02-26T06:26:00.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.292+0000 7f6636800640 1 -- start start 2024-02-26T06:26:00.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f6636800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6630103590 0x7f663010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f6636800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6630108660 0x7f663010d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f6636800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663010d590 0x7f663010da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f6636800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301aa370 con 0x7f6630108660 2024-02-26T06:26:00.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f6636800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301aa4b0 con 0x7f6630103590 2024-02-26T06:26:00.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f6636800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301aa670 con 0x7f663010d590 2024-02-26T06:26:00.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f662ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6630108660 0x7f663010d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f662f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6630103590 0x7f663010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f662f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6630103590 0x7f663010ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46782/0 (socket says 172.21.15.49:46782) 2024-02-26T06:26:00.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.293+0000 7f662f400640 1 -- 172.21.15.49:0/794946127 learned_addr learned my addr 172.21.15.49:0/794946127 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:00.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.294+0000 7f662fe00640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663010d590 0x7f663010da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.294+0000 7f662f400640 1 -- 172.21.15.49:0/794946127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663010d590 msgr2=0x7f663010da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.294+0000 7f662f400640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663010d590 0x7f663010da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.294+0000 7f662f400640 1 -- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6630108660 msgr2=0x7f663010d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.294+0000 7f662f400640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6630108660 0x7f663010d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.294+0000 7f662f400640 1 -- 172.21.15.49:0/794946127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620009490 con 0x7f6630103590 2024-02-26T06:26:00.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.294+0000 7f662fe00640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663010d590 0x7f663010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:00.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.296+0000 7f662f400640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6630103590 0x7f663010ffb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f662000ea90 tx=0x7f662000ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:00.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.297+0000 7f6627e00640 1 -- 172.21.15.49:0/794946127 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662000ce40 con 0x7f6630103590 2024-02-26T06:26:00.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.297+0000 7f6627e00640 1 -- 172.21.15.49:0/794946127 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66200035f0 con 0x7f6630103590 2024-02-26T06:26:00.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.297+0000 7f6627e00640 1 -- 172.21.15.49:0/794946127 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620010690 con 0x7f6630103590 2024-02-26T06:26:00.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.297+0000 7f6636800640 1 -- 172.21.15.49:0/794946127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6628009900 con 0x7f6630103590 2024-02-26T06:26:00.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.299+0000 7f6636800640 1 -- 172.21.15.49:0/794946127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66301aac80 con 0x7f6630103590 2024-02-26T06:26:00.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.308+0000 7f6627e00640 1 -- 172.21.15.49:0/794946127 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f66200107f0 con 0x7f6630103590 2024-02-26T06:26:00.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.311+0000 7f6627e00640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66140a23b0 0x7f66140a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.311+0000 7f6627e00640 1 -- 172.21.15.49:0/794946127 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6620014070 con 0x7f6630103590 2024-02-26T06:26:00.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.313+0000 7f662ea00640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66140a23b0 0x7f66140a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.315+0000 7f6639814740 1 Processor -- start 2024-02-26T06:26:00.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.315+0000 7f662ea00640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66140a23b0 0x7f66140a4870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f663010e810 tx=0x7f661c005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:00.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.318+0000 7f6639814740 1 -- start start 2024-02-26T06:26:00.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.319+0000 7f6639814740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a860fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.319+0000 7f6639814740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8610710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.319+0000 7f6639814740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a861b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.319+0000 7f6639814740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a857f120 con 0x55b2a8610290 2024-02-26T06:26:00.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.319+0000 7f6639814740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a857c650 con 0x55b2a8610f70 2024-02-26T06:26:00.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.319+0000 7f6639814740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a8577e30 con 0x55b2a8619600 2024-02-26T06:26:00.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.321+0000 7f662fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a861b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.321+0000 7f662fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a861b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53480/0 (socket says 172.21.15.49:53480) 2024-02-26T06:26:00.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.321+0000 7f662fe00640 1 -- 172.21.15.49:0/2110157397 learned_addr learned my addr 172.21.15.49:0/2110157397 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:00.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.321+0000 7f662ea00640 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8610710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.322+0000 7f662fe00640 1 -- 172.21.15.49:0/2110157397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 msgr2=0x55b2a860fcc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:26:00.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.322+0000 7f662fe00640 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a860fcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.322+0000 7f662fe00640 1 -- 172.21.15.49:0/2110157397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 msgr2=0x55b2a8610710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:00.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.322+0000 7f662fe00640 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8610710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.322+0000 7f662fe00640 1 -- 172.21.15.49:0/2110157397 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a8531030 con 0x55b2a8619600 2024-02-26T06:26:00.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.323+0000 7f662ea00640 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8610710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:00.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.325+0000 7f662fe00640 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a861b9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6628009ca0 tx=0x7f6628034880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52204ecd45ab4abe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:00.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.325+0000 7f6624c00640 1 -- 172.21.15.49:0/2110157397 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628032340 con 0x55b2a8619600 2024-02-26T06:26:00.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.325+0000 7f6624c00640 1 -- 172.21.15.49:0/2110157397 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66280324a0 con 0x55b2a8619600 2024-02-26T06:26:00.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.325+0000 7f6624c00640 1 -- 172.21.15.49:0/2110157397 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628003d90 con 0x55b2a8619600 2024-02-26T06:26:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.326+0000 7f6639814740 1 -- 172.21.15.49:0/2110157397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 msgr2=0x55b2a861b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.326+0000 7f6639814740 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a861b9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6628009ca0 tx=0x7f6628034880 comp rx=0 tx=0).stop 2024-02-26T06:26:00.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.330+0000 7f6639814740 1 -- 172.21.15.49:0/2110157397 shutdown_connections 2024-02-26T06:26:00.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.330+0000 7f6639814740 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a861b9f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.330+0000 7f6639814740 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8610710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.330+0000 7f6639814740 1 --2- 172.21.15.49:0/2110157397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a860fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.330+0000 7f6639814740 1 -- 172.21.15.49:0/2110157397 >> 172.21.15.49:0/2110157397 conn(0x55b2a85d7570 msgr2=0x55b2a860ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:00.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.330+0000 7f6639814740 1 -- 172.21.15.49:0/2110157397 shutdown_connections 2024-02-26T06:26:00.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.331+0000 7f6639814740 1 -- 172.21.15.49:0/2110157397 wait complete. 2024-02-26T06:26:00.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.331+0000 7f6639814740 1 -- start start 2024-02-26T06:26:00.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.332+0000 7f6639814740 1 Processor -- start 2024-02-26T06:26:00.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.335+0000 7f6639814740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8616540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.335+0000 7f6639814740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a8616a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.335+0000 7f6639814740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a8616fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.336+0000 7f662fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a8616fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.336+0000 7f662ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a8616a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.336+0000 7f662fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a8616fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53484/0 (socket says 172.21.15.49:53484) 2024-02-26T06:26:00.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.336+0000 7f662fe00640 1 -- 172.21.15.49:0/663112269 learned_addr learned my addr 172.21.15.49:0/663112269 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:00.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.336+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a851ce90 con 0x55b2a8610290 2024-02-26T06:26:00.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.336+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a851cb30 con 0x55b2a8610f70 2024-02-26T06:26:00.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.336+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a82e61c0 con 0x55b2a8619600 2024-02-26T06:26:00.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.336+0000 7f662f400640 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8616540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.337+0000 7f6639814740 10 client.0 _ll_register_callbacks cb 0x55b2a8530930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:00.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.337+0000 7f662fe00640 1 -- 172.21.15.49:0/663112269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 msgr2=0x55b2a8616a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:00.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.337+0000 7f662fe00640 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a8616a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.337+0000 7f662fe00640 1 -- 172.21.15.49:0/663112269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 msgr2=0x55b2a8616540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:00.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.337+0000 7f662fe00640 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8616540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.337+0000 7f662fe00640 1 -- 172.21.15.49:0/663112269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6630110a40 con 0x55b2a8619600 2024-02-26T06:26:00.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.337+0000 7f662f400640 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8616540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:00.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.337+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2a82777e0 con 0x55b2a8619600 2024-02-26T06:26:00.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.338+0000 7f662fe00640 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a8616fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6628037040 tx=0x7f6628009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c932bc361efd687a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:00.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.338+0000 7f6619600640 10 client.26071 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:00.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.338+0000 7f6619600640 1 -- 172.21.15.49:0/663112269 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628003d80 con 0x55b2a8619600 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.338+0000 7f6619600640 1 -- 172.21.15.49:0/663112269 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6628004040 con 0x55b2a8619600 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.338+0000 7f6619600640 1 -- 172.21.15.49:0/663112269 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6628004240 con 0x55b2a8619600 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.339+0000 7f6639814740 10 client.26071 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.339+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55b2a85764b0 con 0x55b2a8619600 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.339+0000 7f6619600640 1 -- 172.21.15.49:0/663112269 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f66280043a0 con 0x55b2a8619600 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.339+0000 7f6639814740 10 client.26071 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.339+0000 7f6639814740 10 client.26071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.339+0000 7f6639814740 20 client.26071 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.339+0000 7f6639814740 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2a8628fd0 0x55b2a862b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.340+0000 7f6639814740 4 client.26071 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.340+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2a82b4720 con 0x55b2a8628fd0 2024-02-26T06:26:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.340+0000 7f662f400640 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2a8628fd0 0x55b2a862b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.341+0000 7f662f400640 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2a8628fd0 0x55b2a862b3f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b7bb94f61cd465 server_cookie=c53c1279c0d4de6c in_seq=0 out_seq=0 2024-02-26T06:26:00.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.341+0000 7f6619600640 10 client.26071 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:00.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.341+0000 7f6619600640 1 -- 172.21.15.49:0/663112269 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2a82b4720 con 0x55b2a8628fd0 2024-02-26T06:26:00.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.341+0000 7f6619600640 10 client.26071 handle_command_reply: tid=0 2024-02-26T06:26:00.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.341+0000 7f6639814740 1 client.26071 shutdown 2024-02-26T06:26:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.342+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 msgr2=0x55b2a8616fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.342+0000 7f6639814740 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a8616fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6628037040 tx=0x7f6628009580 comp rx=0 tx=0).stop 2024-02-26T06:26:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.342+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 shutdown_connections 2024-02-26T06:26:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.343+0000 7f6639814740 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2a8628fd0 0x55b2a862b3f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.343+0000 7f6639814740 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a8616a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.343+0000 7f6639814740 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a8616fc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.343+0000 7f6619600640 0 client.26071 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.343+0000 7f6639814740 1 --2- 172.21.15.49:0/663112269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a8616540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.343+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 >> 172.21.15.49:0/663112269 conn(0x55b2a85d7570 msgr2=0x55b2a860ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:00.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.344+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 shutdown_connections 2024-02-26T06:26:00.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.345+0000 7f6639814740 1 -- 172.21.15.49:0/663112269 wait complete. 2024-02-26T06:26:00.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.345+0000 7f6639814740 20 client.26071 trim_cache size 0 max 16384 2024-02-26T06:26:00.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.372+0000 7f6639814740 1 Processor -- start 2024-02-26T06:26:00.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f6639814740 1 -- start start 2024-02-26T06:26:00.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f6639814740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610290 0x55b2a8618c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f6639814740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8610f70 0x55b2a8616de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f6639814740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8619600 0x55b2a8615520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f6639814740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a8626240 con 0x55b2a8619600 2024-02-26T06:26:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f6639814740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a86305c0 con 0x55b2a8610290 2024-02-26T06:26:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f6639814740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a861c350 con 0x55b2a8610f70 2024-02-26T06:26:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f662ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8610f70 0x55b2a8616de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f662ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8610f70 0x55b2a8616de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53492/0 (socket says 172.21.15.49:53492) 2024-02-26T06:26:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f662ea00640 1 -- 172.21.15.49:0/2915350787 learned_addr learned my addr 172.21.15.49:0/2915350787 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.373+0000 7f662f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610290 0x55b2a8618c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.374+0000 7f662fe00640 1 --2- 172.21.15.49:0/2915350787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8619600 0x55b2a8615520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.374+0000 7f662ea00640 1 -- 172.21.15.49:0/2915350787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610290 msgr2=0x55b2a8618c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.374+0000 7f662ea00640 1 --2- 172.21.15.49:0/2915350787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610290 0x55b2a8618c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.374+0000 7f662ea00640 1 -- 172.21.15.49:0/2915350787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8619600 msgr2=0x55b2a8615520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.374+0000 7f662ea00640 1 --2- 172.21.15.49:0/2915350787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8619600 0x55b2a8615520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.374+0000 7f662ea00640 1 -- 172.21.15.49:0/2915350787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a8512920 con 0x55b2a8610f70 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.374+0000 7f662ea00640 1 --2- 172.21.15.49:0/2915350787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8610f70 0x55b2a8616de0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f663006a8a0 tx=0x7f661c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efa7f0be36c098e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.374+0000 7f661be00640 1 -- 172.21.15.49:0/2915350787 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c023070 con 0x55b2a8610f70 2024-02-26T06:26:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.375+0000 7f661be00640 1 -- 172.21.15.49:0/2915350787 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f661c003b90 con 0x55b2a8610f70 2024-02-26T06:26:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.375+0000 7f661be00640 1 -- 172.21.15.49:0/2915350787 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c01eb70 con 0x55b2a8610f70 2024-02-26T06:26:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.375+0000 7f6639814740 1 -- 172.21.15.49:0/2915350787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8610f70 msgr2=0x55b2a8616de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.375+0000 7f6639814740 1 --2- 172.21.15.49:0/2915350787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8610f70 0x55b2a8616de0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f663006a8a0 tx=0x7f661c014000 comp rx=0 tx=0).stop 2024-02-26T06:26:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.375+0000 7f6639814740 1 -- 172.21.15.49:0/2915350787 shutdown_connections 2024-02-26T06:26:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.376+0000 7f6639814740 1 --2- 172.21.15.49:0/2915350787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8619600 0x55b2a8615520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.376+0000 7f6639814740 1 --2- 172.21.15.49:0/2915350787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8610f70 0x55b2a8616de0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.376+0000 7f6639814740 1 --2- 172.21.15.49:0/2915350787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610290 0x55b2a8618c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.376+0000 7f6639814740 1 -- 172.21.15.49:0/2915350787 >> 172.21.15.49:0/2915350787 conn(0x55b2a85d7570 msgr2=0x55b2a8614e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.376+0000 7f6639814740 1 -- 172.21.15.49:0/2915350787 shutdown_connections 2024-02-26T06:26:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.376+0000 7f6639814740 1 -- 172.21.15.49:0/2915350787 wait complete. 2024-02-26T06:26:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.376+0000 7f6639814740 1 -- start start 2024-02-26T06:26:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.377+0000 7f6639814740 1 Processor -- start 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.377+0000 7f6639814740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a861f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.377+0000 7f6639814740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a861f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.377+0000 7f6639814740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a867f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.377+0000 7f6639814740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a86200c0 con 0x55b2a8610290 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.377+0000 7f6639814740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a8620230 con 0x55b2a8610f70 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.377+0000 7f6639814740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a867fa30 con 0x55b2a8619600 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.378+0000 7f6639814740 10 client.0 _ll_register_callbacks cb 0x7f661c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.378+0000 7f662f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a861f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.378+0000 7f662f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a861f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41978/0 (socket says 172.21.15.49:41978) 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.378+0000 7f662f400640 1 -- 172.21.15.49:0/3126441389 learned_addr learned my addr 172.21.15.49:0/3126441389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.378+0000 7f662ea00640 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a861f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.378+0000 7f662fe00640 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a867f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.379+0000 7f662f400640 1 -- 172.21.15.49:0/3126441389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 msgr2=0x55b2a867f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.379+0000 7f662f400640 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a867f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.379+0000 7f662f400640 1 -- 172.21.15.49:0/3126441389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 msgr2=0x55b2a861f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.379+0000 7f662f400640 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a861f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.379+0000 7f662f400640 1 -- 172.21.15.49:0/3126441389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c017070 con 0x55b2a8610290 2024-02-26T06:26:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.379+0000 7f662f400640 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a861f370 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f662000d010 tx=0x7f662009ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ec3db621e10628 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.379+0000 7f6619600640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:26:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.380+0000 7f6619600640 1 -- 172.21.15.49:0/3126441389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662009a1f0 con 0x55b2a8610290 2024-02-26T06:26:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.380+0000 7f6619600640 1 -- 172.21.15.49:0/3126441389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f66200ba2f0 con 0x55b2a8610290 2024-02-26T06:26:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.380+0000 7f6639814740 1 -- 172.21.15.49:0/3126441389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66280324a0 con 0x55b2a8610290 2024-02-26T06:26:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.380+0000 7f6619600640 1 -- 172.21.15.49:0/3126441389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66200cce10 con 0x55b2a8610290 2024-02-26T06:26:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6639814740 10 client.15810 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6639814740 1 -- 172.21.15.49:0/3126441389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b2a8538400 con 0x55b2a8610290 2024-02-26T06:26:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6619600640 1 -- 172.21.15.49:0/3126441389 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66280324a0 con 0x55b2a8610290 2024-02-26T06:26:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6619600640 1 -- 172.21.15.49:0/3126441389 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f662009b310 con 0x55b2a8610290 2024-02-26T06:26:00.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6639814740 10 client.15810 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:00.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6639814740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:00.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6639814740 20 client.15810 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:00.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6639814740 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2a86811f0 0x55b2a8683610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6639814740 4 client.15810 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:26:00.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.381+0000 7f6639814740 1 -- 172.21.15.49:0/3126441389 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b2a8303890 con 0x55b2a86811f0 2024-02-26T06:26:00.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.382+0000 7f662ea00640 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2a86811f0 0x55b2a8683610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.382+0000 7f662ea00640 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2a86811f0 0x55b2a8683610 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=394925e57d8a22e6 server_cookie=9ce3f5360065459 in_seq=0 out_seq=0 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.382+0000 7f6619600640 10 client.15810 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6619600640 1 -- 172.21.15.49:0/3126441389 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b2a8303890 con 0x55b2a86811f0 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6619600640 10 client.15810 handle_command_reply: tid=0 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 client.15810 shutdown 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 -- 172.21.15.49:0/3126441389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 msgr2=0x55b2a861f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a861f370 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f662000d010 tx=0x7f662009ba90 comp rx=0 tx=0).stop 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 -- 172.21.15.49:0/3126441389 shutdown_connections 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2a86811f0 0x55b2a8683610 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2a8610f70 0x55b2a861f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2a8619600 0x55b2a867f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 --2- 172.21.15.49:0/3126441389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2a8610290 0x55b2a861f370 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6639814740 1 -- 172.21.15.49:0/3126441389 >> 172.21.15.49:0/3126441389 conn(0x55b2a85d7570 msgr2=0x55b2a860c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.383+0000 7f6619600640 0 client.15810 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:00.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.384+0000 7f6639814740 1 -- 172.21.15.49:0/3126441389 shutdown_connections 2024-02-26T06:26:00.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.384+0000 7f6639814740 1 -- 172.21.15.49:0/3126441389 wait complete. 2024-02-26T06:26:00.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.384+0000 7f6639814740 20 client.15810 trim_cache size 0 max 16384 2024-02-26T06:26:00.386 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:26:00.386 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:26:00.386 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:26:00.386 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:26:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.385+0000 7f6636800640 1 -- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66140a23b0 msgr2=0x7f66140a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.385+0000 7f6636800640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66140a23b0 0x7f66140a4870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f663010e810 tx=0x7f661c005f90 comp rx=0 tx=0).stop 2024-02-26T06:26:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.385+0000 7f6636800640 1 -- 172.21.15.49:0/794946127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6630103590 msgr2=0x7f663010ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.385+0000 7f6636800640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6630103590 0x7f663010ffb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f662000ea90 tx=0x7f662000ef50 comp rx=0 tx=0).stop 2024-02-26T06:26:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.386+0000 7f6636800640 1 -- 172.21.15.49:0/794946127 shutdown_connections 2024-02-26T06:26:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.386+0000 7f6636800640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66140a23b0 0x7f66140a4870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.386+0000 7f6636800640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663010d590 0x7f663010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.386+0000 7f6636800640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6630108660 0x7f663010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.386+0000 7f6636800640 1 --2- 172.21.15.49:0/794946127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6630103590 0x7f663010ffb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.386+0000 7f6636800640 1 -- 172.21.15.49:0/794946127 >> 172.21.15.49:0/794946127 conn(0x7f66300fee30 msgr2=0x7f66301010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:00.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.386+0000 7f6636800640 1 -- 172.21.15.49:0/794946127 shutdown_connections 2024-02-26T06:26:00.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.387+0000 7f6636800640 1 -- 172.21.15.49:0/794946127 wait complete. 2024-02-26T06:26:00.412 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:26:00.413 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 -- 172.21.15.49:0/1762036537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac4103f60 msgr2=0x7feac4107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 --2- 172.21.15.49:0/1762036537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac4103f60 0x7feac4107fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feab0009900 tx=0x7feab002bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 -- 172.21.15.49:0/1762036537 shutdown_connections 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 --2- 172.21.15.49:0/1762036537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4108680 0x7feac4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 --2- 172.21.15.49:0/1762036537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac4103f60 0x7feac4107fa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 --2- 172.21.15.49:0/1762036537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac41035b0 0x7feac4103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 -- 172.21.15.49:0/1762036537 >> 172.21.15.49:0/1762036537 conn(0x7feac40fee30 msgr2=0x7feac4101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 -- 172.21.15.49:0/1762036537 shutdown_connections 2024-02-26T06:26:00.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.647+0000 7feacb000640 1 -- 172.21.15.49:0/1762036537 wait complete. 2024-02-26T06:26:00.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.648+0000 7feacb000640 1 Processor -- start 2024-02-26T06:26:00.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.648+0000 7feacb000640 1 -- start start 2024-02-26T06:26:00.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.648+0000 7feacb000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac41035b0 0x7feac419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.648+0000 7feacb000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac4108680 0x7feac419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.648+0000 7feacb000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4199840 0x7feac4199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.648+0000 7feacb000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac419a2f0 con 0x7feac4108680 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.648+0000 7feacb000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac419a460 con 0x7feac4199840 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.648+0000 7feacb000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac419a5d0 con 0x7feac41035b0 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac41035b0 0x7feac419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac41035b0 0x7feac419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53524/0 (socket says 172.21.15.49:53524) 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 -- 172.21.15.49:0/1522830313 learned_addr learned my addr 172.21.15.49:0/1522830313 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac9000640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4199840 0x7feac4199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3400640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac4108680 0x7feac419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 -- 172.21.15.49:0/1522830313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4199840 msgr2=0x7feac4199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4199840 0x7feac4199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 -- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac4108680 msgr2=0x7feac419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac4108680 0x7feac419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 -- 172.21.15.49:0/1522830313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab0009560 con 0x7feac41035b0 2024-02-26T06:26:00.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac3e00640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac41035b0 0x7feac419f6d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feab400e9f0 tx=0x7feab400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:00.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.649+0000 7feac0c00640 1 -- 172.21.15.49:0/1522830313 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4009700 con 0x7feac41035b0 2024-02-26T06:26:00.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.650+0000 7feac0c00640 1 -- 172.21.15.49:0/1522830313 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feab400cd80 con 0x7feac41035b0 2024-02-26T06:26:00.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.650+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac41a6580 con 0x7feac41035b0 2024-02-26T06:26:00.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.650+0000 7feac0c00640 1 -- 172.21.15.49:0/1522830313 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4018920 con 0x7feac41035b0 2024-02-26T06:26:00.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.650+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac41a6a10 con 0x7feac41035b0 2024-02-26T06:26:00.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.651+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac4104a80 con 0x7feac41035b0 2024-02-26T06:26:00.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.653+0000 7feac0c00640 1 -- 172.21.15.49:0/1522830313 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feab4018a80 con 0x7feac41035b0 2024-02-26T06:26:00.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.654+0000 7feac0c00640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feaa80a2230 0x7feaa80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:00.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.654+0000 7feac0c00640 1 -- 172.21.15.49:0/1522830313 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7feab4014070 con 0x7feac41035b0 2024-02-26T06:26:00.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.654+0000 7feac3400640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feaa80a2230 0x7feaa80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:00.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.655+0000 7feac3400640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feaa80a2230 0x7feaa80a46f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feab0009230 tx=0x7feab0005fd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:00.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.658+0000 7feac0c00640 1 -- 172.21.15.49:0/1522830313 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feab4052d20 con 0x7feac41035b0 2024-02-26T06:26:00.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.949+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feac410b950 con 0x7feac41035b0 2024-02-26T06:26:00.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.950+0000 7feac0c00640 1 -- 172.21.15.49:0/1522830313 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11338 (secure 0 0 0) 0x7feab4097170 con 0x7feac41035b0 2024-02-26T06:26:00.951 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:26:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:24:43.229679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:26:00.951 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 29 2024-02-26T06:26:00.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feaa80a2230 msgr2=0x7feaa80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feaa80a2230 0x7feaa80a46f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feab0009230 tx=0x7feab0005fd0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac41035b0 msgr2=0x7feac419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac41035b0 0x7feac419f6d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feab400e9f0 tx=0x7feab400eeb0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 shutdown_connections 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feaa80a2230 0x7feaa80a46f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4199840 0x7feac4199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac4108680 0x7feac419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 --2- 172.21.15.49:0/1522830313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac41035b0 0x7feac419f6d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 >> 172.21.15.49:0/1522830313 conn(0x7feac40fee30 msgr2=0x7feac41011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.953+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 shutdown_connections 2024-02-26T06:26:00.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:00.954+0000 7feacb000640 1 -- 172.21.15.49:0/1522830313 wait complete. 2024-02-26T06:26:00.981 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:26:00.981 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:26:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:00 smithi080 ceph-mon[36206]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 MiB/s wr, 757 op/s 2024-02-26T06:26:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:00 smithi115 ceph-mon[36426]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 MiB/s wr, 757 op/s 2024-02-26T06:26:01.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:00 smithi049 ceph-mon[31696]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 MiB/s wr, 757 op/s 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 -- 172.21.15.49:0/4069250494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924810f920 msgr2=0x7f924810fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 --2- 172.21.15.49:0/4069250494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924810f920 0x7f924810fd00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9234009890 tx=0x7f923402bb70 comp rx=0 tx=0).stop 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 -- 172.21.15.49:0/4069250494 shutdown_connections 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 --2- 172.21.15.49:0/4069250494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9248109910 0x7f9248109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 --2- 172.21.15.49:0/4069250494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f924810a2d0 0x7f92481167e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 --2- 172.21.15.49:0/4069250494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924810f920 0x7f924810fd00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 -- 172.21.15.49:0/4069250494 >> 172.21.15.49:0/4069250494 conn(0x7f9248072750 msgr2=0x7f9248072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 -- 172.21.15.49:0/4069250494 shutdown_connections 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.170+0000 7f9250000640 1 -- 172.21.15.49:0/4069250494 wait complete. 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.171+0000 7f9250000640 1 Processor -- start 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.171+0000 7f9250000640 1 -- start start 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.171+0000 7f9250000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9248109910 0x7f92481a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.171+0000 7f9250000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f924810a2d0 0x7f92481a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.171+0000 7f9250000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924810f920 0x7f92481a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.171+0000 7f9250000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92481a7160 con 0x7f924810a2d0 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.171+0000 7f9250000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92481a72d0 con 0x7f9248109910 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.171+0000 7f9250000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92481a7490 con 0x7f924810f920 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f924810a2d0 0x7f92481a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f924810a2d0 0x7f92481a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41998/0 (socket says 172.21.15.49:41998) 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 -- 172.21.15.49:0/2790675586 learned_addr learned my addr 172.21.15.49:0/2790675586 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924e000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924810f920 0x7f92481a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924d600640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9248109910 0x7f92481a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 -- 172.21.15.49:0/2790675586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924810f920 msgr2=0x7f92481a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924810f920 0x7f92481a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 -- 172.21.15.49:0/2790675586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9248109910 msgr2=0x7f92481a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9248109910 0x7f92481a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 -- 172.21.15.49:0/2790675586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9240009900 con 0x7f924810a2d0 2024-02-26T06:26:01.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.172+0000 7f924cc00640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f924810a2d0 0x7f92481a27e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f923c00e9f0 tx=0x7f923c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.173+0000 7f9246000640 1 -- 172.21.15.49:0/2790675586 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c009720 con 0x7f924810a2d0 2024-02-26T06:26:01.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.173+0000 7f9250000640 1 -- 172.21.15.49:0/2790675586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9234009490 con 0x7f924810a2d0 2024-02-26T06:26:01.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.173+0000 7f9246000640 1 -- 172.21.15.49:0/2790675586 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f923c00cd80 con 0x7f924810a2d0 2024-02-26T06:26:01.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.173+0000 7f9246000640 1 -- 172.21.15.49:0/2790675586 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c010690 con 0x7f924810a2d0 2024-02-26T06:26:01.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.173+0000 7f9250000640 1 -- 172.21.15.49:0/2790675586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92481aee10 con 0x7f924810a2d0 2024-02-26T06:26:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.177+0000 7f9252f75740 1 Processor -- start 2024-02-26T06:26:01.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.178+0000 7f9252f75740 1 -- start start 2024-02-26T06:26:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.178+0000 7f9246000640 1 -- 172.21.15.49:0/2790675586 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f923c0046c0 con 0x7f924810a2d0 2024-02-26T06:26:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.178+0000 7f9252f75740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0cc10 0x55746ef0cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.178+0000 7f9252f75740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef15bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.178+0000 7f9252f75740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef18500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.178+0000 7f9252f75740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ee7b1e0 con 0x55746ef16110 2024-02-26T06:26:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.178+0000 7f9252f75740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ee78710 con 0x55746ef0d530 2024-02-26T06:26:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.178+0000 7f9252f75740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ee73ef0 con 0x55746ef0cc10 2024-02-26T06:26:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef18500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef18500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42000/0 (socket says 172.21.15.49:42000) 2024-02-26T06:26:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924cc00640 1 -- 172.21.15.49:0/91734757 learned_addr learned my addr 172.21.15.49:0/91734757 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924d600640 1 --2- 172.21.15.49:0/91734757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef15bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924cc00640 1 -- 172.21.15.49:0/91734757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0cc10 msgr2=0x55746ef0cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924cc00640 1 --2- 172.21.15.49:0/91734757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0cc10 0x55746ef0cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924cc00640 1 -- 172.21.15.49:0/91734757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 msgr2=0x55746ef15bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924cc00640 1 --2- 172.21.15.49:0/91734757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef15bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.179+0000 7f924cc00640 1 -- 172.21.15.49:0/91734757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55746ee35dc0 con 0x55746ef16110 2024-02-26T06:26:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.180+0000 7f924cc00640 1 --2- 172.21.15.49:0/91734757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef18500 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f923c0cba60 tx=0x7f923c0d6320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ead6b8d9c2c28c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.180+0000 7f9246000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f92300a22b0 0x7f92300a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.181+0000 7f9246000640 1 -- 172.21.15.49:0/2790675586 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f923c014070 con 0x7f924810a2d0 2024-02-26T06:26:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.181+0000 7f923a600640 1 -- 172.21.15.49:0/91734757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c097030 con 0x55746ef16110 2024-02-26T06:26:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.181+0000 7f923a600640 1 -- 172.21.15.49:0/91734757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f923c09ab60 con 0x55746ef16110 2024-02-26T06:26:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.181+0000 7f923a600640 1 -- 172.21.15.49:0/91734757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c09b500 con 0x55746ef16110 2024-02-26T06:26:01.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.181+0000 7f924e000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f92300a22b0 0x7f92300a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.183+0000 7f9252f75740 1 -- 172.21.15.49:0/91734757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 msgr2=0x55746ef18500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.183+0000 7f924e000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f92300a22b0 0x7f92300a4770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9240005fd0 tx=0x7f9240010040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.183+0000 7f9252f75740 1 --2- 172.21.15.49:0/91734757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef18500 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f923c0cba60 tx=0x7f923c0d6320 comp rx=0 tx=0).stop 2024-02-26T06:26:01.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.183+0000 7f9252f75740 1 -- 172.21.15.49:0/91734757 shutdown_connections 2024-02-26T06:26:01.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.184+0000 7f9252f75740 1 --2- 172.21.15.49:0/91734757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef15bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.184+0000 7f9252f75740 1 --2- 172.21.15.49:0/91734757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef18500 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.184+0000 7f9252f75740 1 --2- 172.21.15.49:0/91734757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0cc10 0x55746ef0cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.184+0000 7f9252f75740 1 -- 172.21.15.49:0/91734757 >> 172.21.15.49:0/91734757 conn(0x55746ef07340 msgr2=0x55746ef07720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.184+0000 7f9252f75740 1 -- 172.21.15.49:0/91734757 shutdown_connections 2024-02-26T06:26:01.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.184+0000 7f9252f75740 1 -- 172.21.15.49:0/91734757 wait complete. 2024-02-26T06:26:01.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.184+0000 7f9252f75740 1 -- start start 2024-02-26T06:26:01.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.185+0000 7f9252f75740 1 Processor -- start 2024-02-26T06:26:01.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.185+0000 7f9252f75740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.185+0000 7f9252f75740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.185+0000 7f9252f75740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef16110 0x55746ef13c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.185+0000 7f9252f75740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ee19120 con 0x55746ef0cc10 2024-02-26T06:26:01.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.185+0000 7f9252f75740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ee18dc0 con 0x55746ef16110 2024-02-26T06:26:01.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.185+0000 7f9252f75740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ebe21c0 con 0x55746ef0d530 2024-02-26T06:26:01.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f9252f75740 10 client.0 _ll_register_callbacks cb 0x55746ee68280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:01.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53542/0 (socket says 172.21.15.49:53542) 2024-02-26T06:26:01.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924cc00640 1 -- 172.21.15.49:0/136182372 learned_addr learned my addr 172.21.15.49:0/136182372 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924cc00640 1 -- 172.21.15.49:0/136182372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef16110 msgr2=0x55746ef13c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924cc00640 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef16110 0x55746ef13c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924cc00640 1 -- 172.21.15.49:0/136182372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 msgr2=0x55746ef131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924d600640 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924cc00640 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924cc00640 1 -- 172.21.15.49:0/136182372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92481aee10 con 0x55746ef0d530 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.186+0000 7f924d600640 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.187+0000 7f924cc00640 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f923c0cc0f0 tx=0x7f923c0cba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2fa9f827c3160a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.187+0000 7f922ea00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.187+0000 7f922ea00640 1 -- 172.21.15.49:0/136182372 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0d1e10 con 0x55746ef0d530 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.187+0000 7f922ea00640 1 -- 172.21.15.49:0/136182372 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f923c09ad90 con 0x55746ef0d530 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.187+0000 7f9252f75740 1 -- 172.21.15.49:0/136182372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55746eb737e0 con 0x55746ef0d530 2024-02-26T06:26:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.187+0000 7f922ea00640 1 -- 172.21.15.49:0/136182372 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f923c09b500 con 0x55746ef0d530 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.187+0000 7f9252f75740 10 client.26092 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.187+0000 7f9252f75740 1 -- 172.21.15.49:0/136182372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55746ee72570 con 0x55746ef0d530 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f922ea00640 1 -- 172.21.15.49:0/136182372 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f923c0b9080 con 0x55746ef0d530 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f922ea00640 1 -- 172.21.15.49:0/136182372 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f923c01f060 con 0x55746ef0d530 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f9252f75740 10 client.26092 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f9252f75740 10 client.26092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f9252f75740 20 client.26092 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f9252f75740 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55746ef2e790 0x55746ef30bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f9252f75740 4 client.26092 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:01.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f9252f75740 1 -- 172.21.15.49:0/136182372 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55746ebb0720 con 0x55746ef2e790 2024-02-26T06:26:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.188+0000 7f924d600640 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55746ef2e790 0x55746ef30bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.189+0000 7f924d600640 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55746ef2e790 0x55746ef30bb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95c5d47c86ed86a9 server_cookie=8d1f95fff4e5b84d in_seq=0 out_seq=0 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.189+0000 7f922ea00640 10 client.26092 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f922ea00640 1 -- 172.21.15.49:0/136182372 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55746ebb0720 con 0x55746ef2e790 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f922ea00640 10 client.26092 handle_command_reply: tid=0 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 client.26092 shutdown 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 -- 172.21.15.49:0/136182372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 msgr2=0x55746ef13710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f923c0cc0f0 tx=0x7f923c0cba90 comp rx=0 tx=0).stop 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 -- 172.21.15.49:0/136182372 shutdown_connections 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55746ef2e790 0x55746ef30bb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef16110 0x55746ef13c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13710 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 --2- 172.21.15.49:0/136182372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f922ea00640 0 client.26092 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:26:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.190+0000 7f9252f75740 1 -- 172.21.15.49:0/136182372 >> 172.21.15.49:0/136182372 conn(0x55746ef07340 msgr2=0x55746ef09090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.191+0000 7f9252f75740 1 -- 172.21.15.49:0/136182372 shutdown_connections 2024-02-26T06:26:01.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.192+0000 7f9252f75740 1 -- 172.21.15.49:0/136182372 wait complete. 2024-02-26T06:26:01.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.192+0000 7f9252f75740 20 client.26092 trim_cache size 0 max 16384 2024-02-26T06:26:01.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.206+0000 7f9252f75740 1 Processor -- start 2024-02-26T06:26:01.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.206+0000 7f9252f75740 1 -- start start 2024-02-26T06:26:01.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f9252f75740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0cc10 0x55746ef15900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f9252f75740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f9252f75740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef0d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f9252f75740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ef11720 con 0x55746ef16110 2024-02-26T06:26:01.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f9252f75740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ef13fb0 con 0x55746ef0cc10 2024-02-26T06:26:01.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f9252f75740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ef14170 con 0x55746ef0d530 2024-02-26T06:26:01.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924d600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0cc10 0x55746ef15900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924d600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0cc10 0x55746ef15900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46872/0 (socket says 172.21.15.49:46872) 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53550/0 (socket says 172.21.15.49:53550) 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924d600640 1 -- 172.21.15.49:0/1205024435 learned_addr learned my addr 172.21.15.49:0/1205024435 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924d600640 1 -- 172.21.15.49:0/1205024435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 msgr2=0x55746ef13a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924d600640 1 --2- 172.21.15.49:0/1205024435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924d600640 1 -- 172.21.15.49:0/1205024435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 msgr2=0x55746ef0d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924d600640 1 --2- 172.21.15.49:0/1205024435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef0d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.207+0000 7f924d600640 1 -- 172.21.15.49:0/1205024435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55746ee0ea80 con 0x55746ef0cc10 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.208+0000 7f924d600640 1 --2- 172.21.15.49:0/1205024435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0cc10 0x55746ef15900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f92340097b0 tx=0x7f9234005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee4165945624b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.208+0000 7f9239c00640 1 -- 172.21.15.49:0/1205024435 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234044070 con 0x55746ef0cc10 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.208+0000 7f9239c00640 1 -- 172.21.15.49:0/1205024435 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9234034440 con 0x55746ef0cc10 2024-02-26T06:26:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.208+0000 7f9239c00640 1 -- 172.21.15.49:0/1205024435 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923402c890 con 0x55746ef0cc10 2024-02-26T06:26:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 -- 172.21.15.49:0/1205024435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0cc10 msgr2=0x55746ef15900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 --2- 172.21.15.49:0/1205024435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0cc10 0x55746ef15900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f92340097b0 tx=0x7f9234005d20 comp rx=0 tx=0).stop 2024-02-26T06:26:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 -- 172.21.15.49:0/1205024435 shutdown_connections 2024-02-26T06:26:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 --2- 172.21.15.49:0/1205024435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef16110 0x55746ef0d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 --2- 172.21.15.49:0/1205024435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef0d530 0x55746ef13a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 --2- 172.21.15.49:0/1205024435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0cc10 0x55746ef15900 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 -- 172.21.15.49:0/1205024435 >> 172.21.15.49:0/1205024435 conn(0x55746ef07340 msgr2=0x55746ef07720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 -- 172.21.15.49:0/1205024435 shutdown_connections 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.209+0000 7f9252f75740 1 -- 172.21.15.49:0/1205024435 wait complete. 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 1 -- start start 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 1 Processor -- start 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef1d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef1b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef16110 0x55746ef1bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ef1c190 con 0x55746ef0cc10 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ef1c300 con 0x55746ef0d530 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746ef1c4c0 con 0x55746ef16110 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.210+0000 7f9252f75740 10 client.0 _ll_register_callbacks cb 0x7f923c0d1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924d600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef1d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924d600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef1d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42012/0 (socket says 172.21.15.49:42012) 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef16110 0x55746ef1bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924d600640 1 -- 172.21.15.49:0/1991642790 learned_addr learned my addr 172.21.15.49:0/1991642790 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924cc00640 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef1b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924cc00640 1 -- 172.21.15.49:0/1991642790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef16110 msgr2=0x55746ef1bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924cc00640 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef16110 0x55746ef1bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924cc00640 1 -- 172.21.15.49:0/1991642790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 msgr2=0x55746ef1d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924cc00640 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef1d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924cc00640 1 -- 172.21.15.49:0/1991642790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92481170e0 con 0x55746ef0d530 2024-02-26T06:26:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924d600640 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef1d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f924cc00640 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef1b610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f923c0971f0 tx=0x7f923c0b92e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b553b566401be60 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.211+0000 7f922ea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.212+0000 7f922ea00640 1 -- 172.21.15.49:0/1991642790 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0d1e10 con 0x55746ef0d530 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.212+0000 7f922ea00640 1 -- 172.21.15.49:0/1991642790 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f923c09b490 con 0x55746ef0d530 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.212+0000 7f9252f75740 1 -- 172.21.15.49:0/1991642790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f923c09ab60 con 0x55746ef0d530 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.212+0000 7f922ea00640 1 -- 172.21.15.49:0/1991642790 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f923c0a3530 con 0x55746ef0d530 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.212+0000 7f9252f75740 10 client.26063 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.212+0000 7f9252f75740 1 -- 172.21.15.49:0/1991642790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55746ee34620 con 0x55746ef0d530 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.212+0000 7f922ea00640 1 -- 172.21.15.49:0/1991642790 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f923c0a7080 con 0x55746ef0d530 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.212+0000 7f922ea00640 1 -- 172.21.15.49:0/1991642790 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f923c01f040 con 0x55746ef0d530 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f9252f75740 10 client.26063 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f9252f75740 10 client.26063 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f9252f75740 20 client.26063 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f9252f75740 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55746ef85f10 0x55746ef88300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f9252f75740 4 client.26063 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f9252f75740 1 -- 172.21.15.49:0/1991642790 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55746ebff890 con 0x55746ef85f10 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f924d600640 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55746ef85f10 0x55746ef88300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f922ea00640 10 client.26063 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.213+0000 7f924d600640 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55746ef85f10 0x55746ef88300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=209ca7d7172c2de6 server_cookie=e1919fae38d1ebcf in_seq=0 out_seq=0 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f922ea00640 1 -- 172.21.15.49:0/1991642790 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55746ebff890 con 0x55746ef85f10 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f922ea00640 10 client.26063 handle_command_reply: tid=0 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 client.26063 shutdown 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 -- 172.21.15.49:0/1991642790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 msgr2=0x55746ef1b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef1b610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f923c0971f0 tx=0x7f923c0b92e0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 -- 172.21.15.49:0/1991642790 shutdown_connections 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55746ef85f10 0x55746ef88300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55746ef0d530 0x55746ef1b610 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55746ef16110 0x55746ef1bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f922ea00640 0 client.26063 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:26:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 --2- 172.21.15.49:0/1991642790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55746ef0cc10 0x55746ef1d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 -- 172.21.15.49:0/1991642790 >> 172.21.15.49:0/1991642790 conn(0x55746ef07340 msgr2=0x55746ef0e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 -- 172.21.15.49:0/1991642790 shutdown_connections 2024-02-26T06:26:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 1 -- 172.21.15.49:0/1991642790 wait complete. 2024-02-26T06:26:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.214+0000 7f9252f75740 20 client.26063 trim_cache size 0 max 16384 2024-02-26T06:26:01.218 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 -- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f92300a22b0 msgr2=0x7f92300a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f92300a22b0 0x7f92300a4770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9240005fd0 tx=0x7f9240010040 comp rx=0 tx=0).stop 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 -- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f924810a2d0 msgr2=0x7f92481a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f924810a2d0 0x7f92481a27e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f923c00e9f0 tx=0x7f923c00edf0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 -- 172.21.15.49:0/2790675586 shutdown_connections 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f92300a22b0 0x7f92300a4770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924810f920 0x7f92481a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f924810a2d0 0x7f92481a27e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 --2- 172.21.15.49:0/2790675586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9248109910 0x7f92481a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 -- 172.21.15.49:0/2790675586 >> 172.21.15.49:0/2790675586 conn(0x7f9248072750 msgr2=0x7f924806dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.215+0000 7f9250000640 1 -- 172.21.15.49:0/2790675586 shutdown_connections 2024-02-26T06:26:01.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.216+0000 7f9250000640 1 -- 172.21.15.49:0/2790675586 wait complete. 2024-02-26T06:26:01.231 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:26:01.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:26:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 130.0 s, 2123.2 tps, lat 43.311 ms stddev 57.652 2024-02-26T06:26:01.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 -- 172.21.15.49:0/2333881169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f94103e20 msgr2=0x7f4f94107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2333881169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f94103e20 0x7f4f94107e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009930 tx=0x7f4f8002bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 -- 172.21.15.49:0/2333881169 shutdown_connections 2024-02-26T06:26:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2333881169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f94103e20 0x7f4f94107e60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2333881169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94108540 0x7f4f94110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2333881169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f94103470 0x7f4f94103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 -- 172.21.15.49:0/2333881169 >> 172.21.15.49:0/2333881169 conn(0x7f4f940fec90 msgr2=0x7f4f941010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 -- 172.21.15.49:0/2333881169 shutdown_connections 2024-02-26T06:26:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.449+0000 7f4f9b000640 1 -- 172.21.15.49:0/2333881169 wait complete. 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.450+0000 7f4f9b000640 1 Processor -- start 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.450+0000 7f4f9b000640 1 -- start start 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.450+0000 7f4f9b000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94103470 0x7f4f9419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.450+0000 7f4f9b000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f94108540 0x7f4f9419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94103470 0x7f4f9419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94103470 0x7f4f9419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53580/0 (socket says 172.21.15.49:53580) 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f94108540 0x7f4f9419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f94108540 0x7f4f9419fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42028/0 (socket says 172.21.15.49:42028) 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99c00640 1 -- 172.21.15.49:0/2482394797 learned_addr learned my addr 172.21.15.49:0/2482394797 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f94199720 0x7f4f94199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9419a1f0 con 0x7f4f94108540 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9419a360 con 0x7f4f94199720 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9419a520 con 0x7f4f94103470 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f9a600640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f94199720 0x7f4f94199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99c00640 1 -- 172.21.15.49:0/2482394797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f94199720 msgr2=0x7f4f94199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99c00640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f94199720 0x7f4f94199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99c00640 1 -- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f94108540 msgr2=0x7f4f9419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99c00640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f94108540 0x7f4f9419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.451+0000 7f4f99c00640 1 -- 172.21.15.49:0/2482394797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80009560 con 0x7f4f94103470 2024-02-26T06:26:01.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.452+0000 7f4f99c00640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94103470 0x7f4f9419f660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e9f0 tx=0x7f4f8800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.452+0000 7f4f92000640 1 -- 172.21.15.49:0/2482394797 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88009700 con 0x7f4f94103470 2024-02-26T06:26:01.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.452+0000 7f4f92000640 1 -- 172.21.15.49:0/2482394797 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f8800cd80 con 0x7f4f94103470 2024-02-26T06:26:01.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.452+0000 7f4f92000640 1 -- 172.21.15.49:0/2482394797 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88018920 con 0x7f4f94103470 2024-02-26T06:26:01.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.452+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f941a64c0 con 0x7f4f94103470 2024-02-26T06:26:01.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.452+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f941a6950 con 0x7f4f94103470 2024-02-26T06:26:01.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f92000640 1 -- 172.21.15.49:0/2482394797 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4f88018a80 con 0x7f4f94103470 2024-02-26T06:26:01.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f9f5db740 1 Processor -- start 2024-02-26T06:26:01.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f9f5db740 1 -- start start 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364bdc0 0x55be6364c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364c6e0 0x55be63654d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be636576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be635ba1e0 con 0x55be6364c6e0 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be635b7710 con 0x55be636552c0 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.456+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be635b2ef0 con 0x55be6364bdc0 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364bdc0 0x55be6364c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364bdc0 0x55be6364c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53582/0 (socket says 172.21.15.49:53582) 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 -- 172.21.15.49:0/497826424 learned_addr learned my addr 172.21.15.49:0/497826424 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f99c00640 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364c6e0 0x55be63654d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f99200640 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be636576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 -- 172.21.15.49:0/497826424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 msgr2=0x55be636576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be636576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 -- 172.21.15.49:0/497826424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364c6e0 msgr2=0x55be63654d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364c6e0 0x55be63654d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 -- 172.21.15.49:0/497826424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be63574dc0 con 0x55be6364bdc0 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.457+0000 7f4f9a600640 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364bdc0 0x55be6364c1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009f40 tx=0x7f4f8c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e674356914efe9e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.458+0000 7f4f86a00640 1 -- 172.21.15.49:0/497826424 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c009a90 con 0x55be6364bdc0 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.458+0000 7f4f92000640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f7c0a22b0 0x7f4f7c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.458+0000 7f4f86a00640 1 -- 172.21.15.49:0/497826424 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c015070 con 0x55be6364bdc0 2024-02-26T06:26:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.458+0000 7f4f86a00640 1 -- 172.21.15.49:0/497826424 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011780 con 0x55be6364bdc0 2024-02-26T06:26:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.458+0000 7f4f99c00640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f7c0a22b0 0x7f4f7c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.458+0000 7f4f92000640 1 -- 172.21.15.49:0/2482394797 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4f88014070 con 0x7f4f94103470 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 -- 172.21.15.49:0/497826424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364bdc0 msgr2=0x55be6364c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364bdc0 0x55be6364c1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009f40 tx=0x7f4f8c010860 comp rx=0 tx=0).stop 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f99c00640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f7c0a22b0 0x7f4f7c0a4770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f880cbc40 tx=0x7f4f880d1d70 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 -- 172.21.15.49:0/497826424 shutdown_connections 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be636576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364c6e0 0x55be63654d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/497826424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364bdc0 0x55be6364c1a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 -- 172.21.15.49:0/497826424 >> 172.21.15.49:0/497826424 conn(0x55be636464f0 msgr2=0x55be636468d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 -- 172.21.15.49:0/497826424 shutdown_connections 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 -- 172.21.15.49:0/497826424 wait complete. 2024-02-26T06:26:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 -- start start 2024-02-26T06:26:01.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.459+0000 7f4f9f5db740 1 Processor -- start 2024-02-26T06:26:01.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be636522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be63652d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be63558120 con 0x55be6364bdc0 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be63557dc0 con 0x55be636552c0 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be633211c0 con 0x55be6364c6e0 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f9f5db740 10 client.0 _ll_register_callbacks cb 0x55be635a7280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f99200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f9a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be636522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f99200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53598/0 (socket says 172.21.15.49:53598) 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.460+0000 7f4f99200640 1 -- 172.21.15.49:0/4278883890 learned_addr learned my addr 172.21.15.49:0/4278883890 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f9a600640 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be63652d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f99200640 1 -- 172.21.15.49:0/4278883890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 msgr2=0x55be63652d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f99200640 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be63652d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f99200640 1 -- 172.21.15.49:0/4278883890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 msgr2=0x55be636522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f99200640 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be636522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f99200640 1 -- 172.21.15.49:0/4278883890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c009bf0 con 0x55be6364c6e0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f9a600640 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be636522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f99200640 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f80005ab0 tx=0x7f4f8002ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ca0622ad8d1ae7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f7b400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f7b400640 1 -- 172.21.15.49:0/4278883890 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80004040 con 0x55be6364c6e0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f7b400640 1 -- 172.21.15.49:0/4278883890 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4f800041a0 con 0x55be6364c6e0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.461+0000 7f4f9f5db740 1 -- 172.21.15.49:0/4278883890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be632b27e0 con 0x55be6364c6e0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f7b400640 1 -- 172.21.15.49:0/4278883890 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f80004360 con 0x55be6364c6e0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f9f5db740 10 client.26110 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f9f5db740 1 -- 172.21.15.49:0/4278883890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55be635b1570 con 0x55be6364c6e0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f7b400640 1 -- 172.21.15.49:0/4278883890 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55be632b27e0 con 0x55be6364c6e0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f7b400640 1 -- 172.21.15.49:0/4278883890 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4f8003c040 con 0x55be6364c6e0 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f9f5db740 10 client.26110 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f9f5db740 10 client.26110 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:26:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f9f5db740 20 client.26110 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55be63665320 0x55be63667740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f9f5db740 4 client.26110 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.462+0000 7f4f9f5db740 1 -- 172.21.15.49:0/4278883890 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be632ef720 con 0x55be63665320 2024-02-26T06:26:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.463+0000 7f4f9a600640 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55be63665320 0x55be63667740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.464+0000 7f4f7b400640 10 client.26110 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:26:01.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.464+0000 7f4f9a600640 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55be63665320 0x55be63667740 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14b0547436305e0d server_cookie=41bd76a1f47a3936 in_seq=0 out_seq=0 2024-02-26T06:26:01.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.464+0000 7f4f7b400640 1 -- 172.21.15.49:0/4278883890 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be632ef720 con 0x55be63665320 2024-02-26T06:26:01.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.464+0000 7f4f7b400640 10 client.26110 handle_command_reply: tid=0 2024-02-26T06:26:01.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 client.26110 shutdown 2024-02-26T06:26:01.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 -- 172.21.15.49:0/4278883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 msgr2=0x55be63652800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f80005ab0 tx=0x7f4f8002ca10 comp rx=0 tx=0).stop 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 -- 172.21.15.49:0/4278883890 shutdown_connections 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55be63665320 0x55be63667740 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be63652d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652800 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f7b400640 0 client.26110 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/4278883890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be636522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.465+0000 7f4f9f5db740 1 -- 172.21.15.49:0/4278883890 >> 172.21.15.49:0/4278883890 conn(0x55be636464f0 msgr2=0x55be63648240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.466+0000 7f4f9f5db740 1 -- 172.21.15.49:0/4278883890 shutdown_connections 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.466+0000 7f4f9f5db740 1 -- 172.21.15.49:0/4278883890 wait complete. 2024-02-26T06:26:01.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.466+0000 7f4f9f5db740 20 client.26110 trim_cache size 0 max 16384 2024-02-26T06:26:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.481+0000 7f4f9f5db740 1 Processor -- start 2024-02-26T06:26:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.481+0000 7f4f9f5db740 1 -- start start 2024-02-26T06:26:01.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.481+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be6364bdc0 0x55be636549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.481+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.481+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be636552c0 0x55be6364f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.481+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6364bc50 con 0x55be636552c0 2024-02-26T06:26:01.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.481+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6364aeb0 con 0x55be6364bdc0 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.481+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be63661e50 con 0x55be6364c6e0 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f9a600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be6364bdc0 0x55be636549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f99200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f9a600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be6364bdc0 0x55be636549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46916/0 (socket says 172.21.15.49:46916) 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f9a600640 1 -- 172.21.15.49:0/2778295251 learned_addr learned my addr 172.21.15.49:0/2778295251 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f9a600640 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be636552c0 0x55be6364f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f99200640 1 -- 172.21.15.49:0/2778295251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be6364bdc0 msgr2=0x55be636549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f99200640 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be6364bdc0 0x55be636549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f99200640 1 -- 172.21.15.49:0/2778295251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be636552c0 msgr2=0x55be6364f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f99200640 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be636552c0 0x55be6364f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f99200640 1 -- 172.21.15.49:0/2778295251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6354da80 con 0x55be6364c6e0 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.482+0000 7f4f9a600640 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be636552c0 0x55be6364f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:01.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.483+0000 7f4f99200640 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652b60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4f80003850 tx=0x7f4f8002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b93cbf1da168a5d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.483+0000 7f4f86000640 1 -- 172.21.15.49:0/2778295251 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80003bc0 con 0x55be6364c6e0 2024-02-26T06:26:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.483+0000 7f4f86000640 1 -- 172.21.15.49:0/2778295251 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f800035f0 con 0x55be6364c6e0 2024-02-26T06:26:01.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.483+0000 7f4f86000640 1 -- 172.21.15.49:0/2778295251 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80005210 con 0x55be6364c6e0 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 -- 172.21.15.49:0/2778295251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 msgr2=0x55be63652b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652b60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4f80003850 tx=0x7f4f8002bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 -- 172.21.15.49:0/2778295251 shutdown_connections 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be636552c0 0x55be6364f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be63652b60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/2778295251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be6364bdc0 0x55be636549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 -- 172.21.15.49:0/2778295251 >> 172.21.15.49:0/2778295251 conn(0x55be636464f0 msgr2=0x55be6364d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 -- 172.21.15.49:0/2778295251 shutdown_connections 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.484+0000 7f4f9f5db740 1 -- 172.21.15.49:0/2778295251 wait complete. 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 1 -- start start 2024-02-26T06:26:01.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 1 Processor -- start 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be6365c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be6365a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be6365afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6365b680 con 0x55be6364bdc0 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6365b7f0 con 0x55be636552c0 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be636bb870 con 0x55be6364c6e0 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.485+0000 7f4f9f5db740 10 client.0 _ll_register_callbacks cb 0x7f4f8c0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f9a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be6365c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f99200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be6365a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f9a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be6365c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42060/0 (socket says 172.21.15.49:42060) 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f9a600640 1 -- 172.21.15.49:0/418975254 learned_addr learned my addr 172.21.15.49:0/418975254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f99200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be6365a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53626/0 (socket says 172.21.15.49:53626) 2024-02-26T06:26:01.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f9a600640 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be6365afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f99200640 1 -- 172.21.15.49:0/418975254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 msgr2=0x55be6365afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f99200640 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be6365afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f99200640 1 -- 172.21.15.49:0/418975254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 msgr2=0x55be6365c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f99200640 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be6365c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f99200640 1 -- 172.21.15.49:0/418975254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be635b1570 con 0x55be6364c6e0 2024-02-26T06:26:01.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f99200640 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be6365a9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55be63642160 tx=0x7f4f80005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe1c933213ba0df server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.486+0000 7f4f7b400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:01.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.487+0000 7f4f7b400640 1 -- 172.21.15.49:0/418975254 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80053880 con 0x55be6364c6e0 2024-02-26T06:26:01.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.487+0000 7f4f7b400640 1 -- 172.21.15.49:0/418975254 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4f80000e40 con 0x55be6364c6e0 2024-02-26T06:26:01.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.487+0000 7f4f9f5db740 1 -- 172.21.15.49:0/418975254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f8c015070 con 0x55be6364c6e0 2024-02-26T06:26:01.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.487+0000 7f4f7b400640 1 -- 172.21.15.49:0/418975254 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f8002c9f0 con 0x55be6364c6e0 2024-02-26T06:26:01.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.487+0000 7f4f9f5db740 10 client.26116 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.487+0000 7f4f9f5db740 1 -- 172.21.15.49:0/418975254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55be63573620 con 0x55be6364c6e0 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.487+0000 7f4f7b400640 1 -- 172.21.15.49:0/418975254 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c015070 con 0x55be6364c6e0 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.488+0000 7f4f7b400640 1 -- 172.21.15.49:0/418975254 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4f8003c040 con 0x55be6364c6e0 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.488+0000 7f4f9f5db740 10 client.26116 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.488+0000 7f4f9f5db740 10 client.26116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.488+0000 7f4f9f5db740 20 client.26116 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.488+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55be636bcc40 0x55be636bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.488+0000 7f4f9f5db740 4 client.26116 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.488+0000 7f4f9f5db740 1 -- 172.21.15.49:0/418975254 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be6333e890 con 0x55be636bcc40 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.488+0000 7f4f9a600640 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55be636bcc40 0x55be636bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9a600640 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55be636bcc40 0x55be636bf060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfc276f111171a43 server_cookie=f8f4073fd643aa00 in_seq=0 out_seq=0 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f7b400640 10 client.26116 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f7b400640 1 -- 172.21.15.49:0/418975254 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be6333e890 con 0x55be636bcc40 2024-02-26T06:26:01.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f7b400640 10 client.26116 handle_command_reply: tid=0 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 client.26116 shutdown 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 -- 172.21.15.49:0/418975254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 msgr2=0x55be6365a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be6365a9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55be63642160 tx=0x7f4f80005640 comp rx=0 tx=0).stop 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 -- 172.21.15.49:0/418975254 shutdown_connections 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55be636bcc40 0x55be636bf060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be636552c0 0x55be6365afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be6364c6e0 0x55be6365a9b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f7b400640 0 client.26116 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 --2- 172.21.15.49:0/418975254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be6364bdc0 0x55be6365c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 -- 172.21.15.49:0/418975254 >> 172.21.15.49:0/418975254 conn(0x55be636464f0 msgr2=0x55be63644e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 -- 172.21.15.49:0/418975254 shutdown_connections 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 1 -- 172.21.15.49:0/418975254 wait complete. 2024-02-26T06:26:01.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.489+0000 7f4f9f5db740 20 client.26116 trim_cache size 0 max 16384 2024-02-26T06:26:01.494 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:26:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f7c0a22b0 msgr2=0x7f4f7c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f7c0a22b0 0x7f4f7c0a4770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f880cbc40 tx=0x7f4f880d1d70 comp rx=0 tx=0).stop 2024-02-26T06:26:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94103470 msgr2=0x7f4f9419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94103470 0x7f4f9419f660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e9f0 tx=0x7f4f8800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 shutdown_connections 2024-02-26T06:26:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f7c0a22b0 0x7f4f7c0a4770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f94199720 0x7f4f94199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f94108540 0x7f4f9419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 --2- 172.21.15.49:0/2482394797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94103470 0x7f4f9419f660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 >> 172.21.15.49:0/2482394797 conn(0x7f4f940fec90 msgr2=0x7f4f94100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 shutdown_connections 2024-02-26T06:26:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.490+0000 7f4f9b000640 1 -- 172.21.15.49:0/2482394797 wait complete. 2024-02-26T06:26:01.510 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:26:01.511 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:26:01.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.745+0000 7f2277200640 1 -- 172.21.15.49:0/1046968652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 msgr2=0x7f2270111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.745+0000 7f2277200640 1 --2- 172.21.15.49:0/1046968652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 0x7f2270111a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f225c009890 tx=0x7f225c02bb70 comp rx=0 tx=0).stop 2024-02-26T06:26:01.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.746+0000 7f2277200640 1 -- 172.21.15.49:0/1046968652 shutdown_connections 2024-02-26T06:26:01.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.746+0000 7f2277200640 1 --2- 172.21.15.49:0/1046968652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2270111fc0 0x7f22701143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.746+0000 7f2277200640 1 --2- 172.21.15.49:0/1046968652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 0x7f2270111a80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.746+0000 7f2277200640 1 --2- 172.21.15.49:0/1046968652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22700756b0 0x7f2270075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.746+0000 7f2277200640 1 -- 172.21.15.49:0/1046968652 >> 172.21.15.49:0/1046968652 conn(0x7f22700fedf0 msgr2=0x7f2270101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.746+0000 7f2277200640 1 -- 172.21.15.49:0/1046968652 shutdown_connections 2024-02-26T06:26:01.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.746+0000 7f2277200640 1 -- 172.21.15.49:0/1046968652 wait complete. 2024-02-26T06:26:01.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.747+0000 7f2277200640 1 Processor -- start 2024-02-26T06:26:01.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.748+0000 7f2277200640 1 -- start start 2024-02-26T06:26:01.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.748+0000 7f2277200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22700756b0 0x7f227019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.748+0000 7f2277200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 0x7f227019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.748+0000 7f2277200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2270111fc0 0x7f22701a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.748+0000 7f2277200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22701a2cd0 con 0x7f2270111fc0 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.748+0000 7f2277200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22701a2e40 con 0x7f22700756b0 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.748+0000 7f2277200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22701a3000 con 0x7f2270075fd0 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22700756b0 0x7f227019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22700756b0 0x7f227019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46932/0 (socket says 172.21.15.49:46932) 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 -- 172.21.15.49:0/239120104 learned_addr learned my addr 172.21.15.49:0/239120104 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226f400640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 0x7f227019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 -- 172.21.15.49:0/239120104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 msgr2=0x7f227019e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 0x7f227019e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 -- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2270111fc0 msgr2=0x7f22701a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2270111fc0 0x7f22701a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 -- 172.21.15.49:0/239120104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2260009560 con 0x7f22700756b0 2024-02-26T06:26:01.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226f400640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 0x7f227019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:01.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.749+0000 7f226fe00640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22700756b0 0x7f227019de60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f226800ebc0 tx=0x7f226800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.750+0000 7f226cc00640 1 -- 172.21.15.49:0/239120104 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680099b0 con 0x7f22700756b0 2024-02-26T06:26:01.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.750+0000 7f226cc00640 1 -- 172.21.15.49:0/239120104 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f226800eec0 con 0x7f22700756b0 2024-02-26T06:26:01.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.750+0000 7f226cc00640 1 -- 172.21.15.49:0/239120104 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268018b60 con 0x7f22700756b0 2024-02-26T06:26:01.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.750+0000 7f2277200640 1 -- 172.21.15.49:0/239120104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c009490 con 0x7f22700756b0 2024-02-26T06:26:01.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.750+0000 7f2277200640 1 -- 172.21.15.49:0/239120104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22701aa980 con 0x7f22700756b0 2024-02-26T06:26:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.755+0000 7f226cc00640 1 -- 172.21.15.49:0/239120104 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2268018cc0 con 0x7f22700756b0 2024-02-26T06:26:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.756+0000 7f2279fe2740 1 Processor -- start 2024-02-26T06:26:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.756+0000 7f2279fe2740 1 -- start start 2024-02-26T06:26:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.756+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c6aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.756+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c73bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.756+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c76500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.756+0000 7f2279fe2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65bd91e0 con 0x560f65c6ac10 2024-02-26T06:26:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.756+0000 7f2279fe2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65bd6710 con 0x560f65c6b530 2024-02-26T06:26:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.756+0000 7f2279fe2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65bd1ef0 con 0x560f65c74110 2024-02-26T06:26:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f2275200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c6aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c76500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c73bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c76500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53660/0 (socket says 172.21.15.49:53660) 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f2275200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c6aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42096/0 (socket says 172.21.15.49:42096) 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226f400640 1 -- 172.21.15.49:0/1191549328 learned_addr learned my addr 172.21.15.49:0/1191549328 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226cc00640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22540a2230 0x7f22540a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226f400640 1 -- 172.21.15.49:0/1191549328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 msgr2=0x560f65c73bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226f400640 1 --2- 172.21.15.49:0/1191549328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c73bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226f400640 1 -- 172.21.15.49:0/1191549328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 msgr2=0x560f65c6aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226f400640 1 --2- 172.21.15.49:0/1191549328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c6aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226f400640 1 -- 172.21.15.49:0/1191549328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f65b93dc0 con 0x560f65c74110 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.757+0000 7f226fe00640 1 --2- 172.21.15.49:0/1191549328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c73bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.758+0000 7f226cc00640 1 -- 172.21.15.49:0/239120104 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2268014070 con 0x7f22700756b0 2024-02-26T06:26:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.758+0000 7f2275200640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22540a2230 0x7f22540a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.758+0000 7f226f400640 1 --2- 172.21.15.49:0/1191549328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c76500 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f225c005b60 tx=0x7f225c035bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ee3848bdaafed8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.758+0000 7f2265600640 1 -- 172.21.15.49:0/1191549328 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c004040 con 0x560f65c74110 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.758+0000 7f2265600640 1 -- 172.21.15.49:0/1191549328 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f225c034440 con 0x560f65c74110 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.758+0000 7f2265600640 1 -- 172.21.15.49:0/1191549328 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c005570 con 0x560f65c74110 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2275200640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22540a2230 0x7f22540a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2260009530 tx=0x7f22600046c0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2279fe2740 1 -- 172.21.15.49:0/1191549328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 msgr2=0x560f65c76500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2279fe2740 1 --2- 172.21.15.49:0/1191549328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c76500 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f225c005b60 tx=0x7f225c035bf0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2279fe2740 1 -- 172.21.15.49:0/1191549328 shutdown_connections 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2279fe2740 1 --2- 172.21.15.49:0/1191549328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c73bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2279fe2740 1 --2- 172.21.15.49:0/1191549328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c76500 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2279fe2740 1 --2- 172.21.15.49:0/1191549328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c6aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2279fe2740 1 -- 172.21.15.49:0/1191549328 >> 172.21.15.49:0/1191549328 conn(0x560f65c65340 msgr2=0x560f65c65720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.759+0000 7f2279fe2740 1 -- 172.21.15.49:0/1191549328 shutdown_connections 2024-02-26T06:26:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.760+0000 7f2279fe2740 1 -- 172.21.15.49:0/1191549328 wait complete. 2024-02-26T06:26:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.760+0000 7f2279fe2740 1 -- start start 2024-02-26T06:26:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.760+0000 7f2279fe2740 1 Processor -- start 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c71110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c71650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c71b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f2279fe2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65b77120 con 0x560f65c6ac10 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f2279fe2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65b76dc0 con 0x560f65c6b530 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f2279fe2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f659401c0 con 0x560f65c74110 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f2279fe2740 10 client.0 _ll_register_callbacks cb 0x560f65bc6280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f226fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c71110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f226f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c71650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f226fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c71110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42104/0 (socket says 172.21.15.49:42104) 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f226fe00640 1 -- 172.21.15.49:0/2971053688 learned_addr learned my addr 172.21.15.49:0/2971053688 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.761+0000 7f2275200640 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c71b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f226fe00640 1 -- 172.21.15.49:0/2971053688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 msgr2=0x560f65c71b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f226fe00640 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c71b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f226fe00640 1 -- 172.21.15.49:0/2971053688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 msgr2=0x560f65c71650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f226fe00640 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c71650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f226fe00640 1 -- 172.21.15.49:0/2971053688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2270114e40 con 0x560f65c6ac10 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f2275200640 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c71b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f226f400640 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c71650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f226fe00640 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c71110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f22680cc020 tx=0x7f2268009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128cf0187b9a4543 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.762+0000 7f225a000640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.763+0000 7f225a000640 1 -- 172.21.15.49:0/2971053688 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226809b890 con 0x560f65c6ac10 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.763+0000 7f225a000640 1 -- 172.21.15.49:0/2971053688 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2268010dc0 con 0x560f65c6ac10 2024-02-26T06:26:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.763+0000 7f2279fe2740 1 -- 172.21.15.49:0/2971053688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f658d17e0 con 0x560f65c6ac10 2024-02-26T06:26:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.763+0000 7f225a000640 1 -- 172.21.15.49:0/2971053688 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f226809b9f0 con 0x560f65c6ac10 2024-02-26T06:26:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.763+0000 7f2279fe2740 10 client.15855 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.763+0000 7f2279fe2740 1 -- 172.21.15.49:0/2971053688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560f65bd0570 con 0x560f65c6ac10 2024-02-26T06:26:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.764+0000 7f225a000640 1 -- 172.21.15.49:0/2971053688 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560f658d17e0 con 0x560f65c6ac10 2024-02-26T06:26:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.764+0000 7f225a000640 1 -- 172.21.15.49:0/2971053688 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f22680d6070 con 0x560f65c6ac10 2024-02-26T06:26:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.764+0000 7f2279fe2740 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.764+0000 7f2279fe2740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.764+0000 7f2279fe2740 20 client.15855 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.764+0000 7f2279fe2740 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560f65c83ef0 0x560f65c86330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.764+0000 7f2279fe2740 4 client.15855 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.764+0000 7f2279fe2740 1 -- 172.21.15.49:0/2971053688 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f6590e720 con 0x560f65c83ef0 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.765+0000 7f226f400640 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560f65c83ef0 0x560f65c86330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.765+0000 7f226f400640 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560f65c83ef0 0x560f65c86330 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34b1463df262812 server_cookie=eed4dfc7f80fceee in_seq=0 out_seq=0 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.765+0000 7f225a000640 10 client.15855 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.766+0000 7f225a000640 1 -- 172.21.15.49:0/2971053688 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f6590e720 con 0x560f65c83ef0 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.766+0000 7f225a000640 10 client.15855 handle_command_reply: tid=0 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.766+0000 7f2279fe2740 1 client.15855 shutdown 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.766+0000 7f2279fe2740 1 -- 172.21.15.49:0/2971053688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 msgr2=0x560f65c71110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.766+0000 7f2279fe2740 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c71110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f22680cc020 tx=0x7f2268009200 comp rx=0 tx=0).stop 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.766+0000 7f2279fe2740 1 -- 172.21.15.49:0/2971053688 shutdown_connections 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f2279fe2740 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560f65c83ef0 0x560f65c86330 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f2279fe2740 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6b530 0x560f65c71650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f2279fe2740 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c71b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f2279fe2740 1 --2- 172.21.15.49:0/2971053688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6ac10 0x560f65c71110 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f2279fe2740 1 -- 172.21.15.49:0/2971053688 >> 172.21.15.49:0/2971053688 conn(0x560f65c65340 msgr2=0x560f65c67090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f225a000640 0 client.15855 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f2279fe2740 1 -- 172.21.15.49:0/2971053688 shutdown_connections 2024-02-26T06:26:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f2279fe2740 1 -- 172.21.15.49:0/2971053688 wait complete. 2024-02-26T06:26:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.767+0000 7f2279fe2740 20 client.15855 trim_cache size 0 max 16384 2024-02-26T06:26:01.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.783+0000 7f2279fe2740 1 Processor -- start 2024-02-26T06:26:01.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.783+0000 7f2279fe2740 1 -- start start 2024-02-26T06:26:01.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.783+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c6ac10 0x560f65c642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.783+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c7ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.783+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c74110 0x560f65c7b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.783+0000 7f2279fe2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65c66370 con 0x560f65c6b530 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.783+0000 7f2279fe2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65c60cb0 con 0x560f65c74110 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.783+0000 7f2279fe2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65c6a000 con 0x560f65c6ac10 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c6ac10 0x560f65c642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c6ac10 0x560f65c642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53680/0 (socket says 172.21.15.49:53680) 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 -- 172.21.15.49:0/1912859197 learned_addr learned my addr 172.21.15.49:0/1912859197 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226f400640 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c7ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f2275200640 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c74110 0x560f65c7b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 -- 172.21.15.49:0/1912859197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c74110 msgr2=0x560f65c7b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c74110 0x560f65c7b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 -- 172.21.15.49:0/1912859197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 msgr2=0x560f65c7ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c7ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 -- 172.21.15.49:0/1912859197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f65b6ca80 con 0x560f65c6ac10 2024-02-26T06:26:01.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f2275200640 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c74110 0x560f65c7b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:01.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.784+0000 7f226fe00640 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c6ac10 0x560f65c642c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f226800cba0 tx=0x7f2268000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6336b05aaa0ef7ac server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2264c00640 1 -- 172.21.15.49:0/1912859197 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226809b890 con 0x560f65c6ac10 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2264c00640 1 -- 172.21.15.49:0/1912859197 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f226809b9f0 con 0x560f65c6ac10 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2264c00640 1 -- 172.21.15.49:0/1912859197 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226809b360 con 0x560f65c6ac10 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2279fe2740 1 -- 172.21.15.49:0/1912859197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c6ac10 msgr2=0x560f65c642c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2279fe2740 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c6ac10 0x560f65c642c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f226800cba0 tx=0x7f2268000f40 comp rx=0 tx=0).stop 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2279fe2740 1 -- 172.21.15.49:0/1912859197 shutdown_connections 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2279fe2740 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c74110 0x560f65c7b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2279fe2740 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c7ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2279fe2740 1 --2- 172.21.15.49:0/1912859197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c6ac10 0x560f65c642c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.785+0000 7f2279fe2740 1 -- 172.21.15.49:0/1912859197 >> 172.21.15.49:0/1912859197 conn(0x560f65c65340 msgr2=0x560f65c63bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.786+0000 7f2279fe2740 1 -- 172.21.15.49:0/1912859197 shutdown_connections 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.786+0000 7f2279fe2740 1 -- 172.21.15.49:0/1912859197 wait complete. 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.786+0000 7f2279fe2740 1 -- start start 2024-02-26T06:26:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.786+0000 7f2279fe2740 1 Processor -- start 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6ac10 0x560f65c799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c79ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f2279fe2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f2279fe2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65cda460 con 0x560f65c6b530 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f2279fe2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65cda5d0 con 0x560f65c6ac10 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f2279fe2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f65cda790 con 0x560f65c74110 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f2279fe2740 10 client.0 _ll_register_callbacks cb 0x7f225c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c79ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6ac10 0x560f65c799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c79ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42124/0 (socket says 172.21.15.49:42124) 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226f400640 1 -- 172.21.15.49:0/3283153969 learned_addr learned my addr 172.21.15.49:0/3283153969 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f2275200640 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226f400640 1 -- 172.21.15.49:0/3283153969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 msgr2=0x560f65c7a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226f400640 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c7a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226f400640 1 -- 172.21.15.49:0/3283153969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6ac10 msgr2=0x560f65c799a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226f400640 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6ac10 0x560f65c799a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.787+0000 7f226f400640 1 -- 172.21.15.49:0/3283153969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f65bd0570 con 0x560f65c6b530 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f226f400640 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c79ee0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f225c02c0b0 tx=0x7f225c005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24408133fcc39d84 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f225a000640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f225a000640 1 -- 172.21.15.49:0/3283153969 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0347b0 con 0x560f65c6b530 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f2279fe2740 1 -- 172.21.15.49:0/3283153969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f225c034440 con 0x560f65c6b530 2024-02-26T06:26:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f225a000640 1 -- 172.21.15.49:0/3283153969 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f225c04c040 con 0x560f65c6b530 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f225a000640 1 -- 172.21.15.49:0/3283153969 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f225c005680 con 0x560f65c6b530 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f2279fe2740 10 client.15861 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f2279fe2740 1 -- 172.21.15.49:0/3283153969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560f65b92620 con 0x560f65c6b530 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.788+0000 7f225a000640 1 -- 172.21.15.49:0/3283153969 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f225c034440 con 0x560f65c6b530 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.789+0000 7f225a000640 1 -- 172.21.15.49:0/3283153969 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f225c02c950 con 0x560f65c6b530 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.789+0000 7f2279fe2740 10 client.15861 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.789+0000 7f2279fe2740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.789+0000 7f2279fe2740 20 client.15861 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.789+0000 7f2279fe2740 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560f65cdbce0 0x560f65cde100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.789+0000 7f2279fe2740 4 client.15861 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:26:01.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.789+0000 7f2279fe2740 1 -- 172.21.15.49:0/3283153969 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f6595d890 con 0x560f65cdbce0 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.790+0000 7f226fe00640 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560f65cdbce0 0x560f65cde100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.790+0000 7f226fe00640 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560f65cdbce0 0x560f65cde100 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d0415b7da781b6 server_cookie=b92ca5e25fcda61b in_seq=0 out_seq=0 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.790+0000 7f225a000640 10 client.15861 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f225a000640 1 -- 172.21.15.49:0/3283153969 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f6595d890 con 0x560f65cdbce0 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f225a000640 10 client.15861 handle_command_reply: tid=0 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 client.15861 shutdown 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 -- 172.21.15.49:0/3283153969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 msgr2=0x560f65c79ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c79ee0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f225c02c0b0 tx=0x7f225c005150 comp rx=0 tx=0).stop 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 -- 172.21.15.49:0/3283153969 shutdown_connections 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560f65cdbce0 0x560f65cde100 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f65c74110 0x560f65c7a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f65c6b530 0x560f65c79ee0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 --2- 172.21.15.49:0/3283153969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560f65c6ac10 0x560f65c799a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f225a000640 0 client.15861 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 -- 172.21.15.49:0/3283153969 >> 172.21.15.49:0/3283153969 conn(0x560f65c65340 msgr2=0x560f65c64440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.791+0000 7f2279fe2740 1 -- 172.21.15.49:0/3283153969 shutdown_connections 2024-02-26T06:26:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.792+0000 7f2279fe2740 1 -- 172.21.15.49:0/3283153969 wait complete. 2024-02-26T06:26:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.792+0000 7f2279fe2740 20 client.15861 trim_cache size 0 max 16384 2024-02-26T06:26:01.796 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:26:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.792+0000 7f2277200640 1 -- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22540a2230 msgr2=0x7f22540a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.792+0000 7f2277200640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22540a2230 0x7f22540a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2260009530 tx=0x7f22600046c0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.792+0000 7f2277200640 1 -- 172.21.15.49:0/239120104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22700756b0 msgr2=0x7f227019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.792+0000 7f2277200640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22700756b0 0x7f227019de60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f226800ebc0 tx=0x7f226800c6a0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.793+0000 7f2277200640 1 -- 172.21.15.49:0/239120104 shutdown_connections 2024-02-26T06:26:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.793+0000 7f2277200640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22540a2230 0x7f22540a46f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.793+0000 7f2277200640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2270111fc0 0x7f22701a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.793+0000 7f2277200640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2270075fd0 0x7f227019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.793+0000 7f2277200640 1 --2- 172.21.15.49:0/239120104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22700756b0 0x7f227019de60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.793+0000 7f2277200640 1 -- 172.21.15.49:0/239120104 >> 172.21.15.49:0/239120104 conn(0x7f22700fedf0 msgr2=0x7f2270113750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.793+0000 7f2277200640 1 -- 172.21.15.49:0/239120104 shutdown_connections 2024-02-26T06:26:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:01.793+0000 7f2277200640 1 -- 172.21.15.49:0/239120104 wait complete. 2024-02-26T06:26:02.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:01 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1522830313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:26:02.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:01 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1522830313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:26:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:01 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1522830313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:26:02.813 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:26:02.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:26:02.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:26:03.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.051+0000 7f2449200640 1 -- 172.21.15.49:0/2953746335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2444101250 msgr2=0x7f2444101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.051+0000 7f2449200640 1 --2- 172.21.15.49:0/2953746335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2444101250 0x7f2444101630 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2434009930 tx=0x7f243402bc20 comp rx=0 tx=0).stop 2024-02-26T06:26:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.052+0000 7f2449200640 1 -- 172.21.15.49:0/2953746335 shutdown_connections 2024-02-26T06:26:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.052+0000 7f2449200640 1 --2- 172.21.15.49:0/2953746335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2444106320 0x7f244410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.052+0000 7f2449200640 1 --2- 172.21.15.49:0/2953746335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2444101c00 0x7f2444105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.052+0000 7f2449200640 1 --2- 172.21.15.49:0/2953746335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2444101250 0x7f2444101630 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.052+0000 7f2449200640 1 -- 172.21.15.49:0/2953746335 >> 172.21.15.49:0/2953746335 conn(0x7f24440fcaf0 msgr2=0x7f24440fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.052+0000 7f2449200640 1 -- 172.21.15.49:0/2953746335 shutdown_connections 2024-02-26T06:26:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.052+0000 7f2449200640 1 -- 172.21.15.49:0/2953746335 wait complete. 2024-02-26T06:26:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.053+0000 7f2449200640 1 Processor -- start 2024-02-26T06:26:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.053+0000 7f2449200640 1 -- start start 2024-02-26T06:26:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.053+0000 7f2449200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2444101250 0x7f2444197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.053+0000 7f2449200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2444106320 0x7f2444197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.053+0000 7f2449200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f244419bf90 0x7f244419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2449200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244419c980 con 0x7f2444101250 2024-02-26T06:26:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2449200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244419caf0 con 0x7f244419bf90 2024-02-26T06:26:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2449200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244419cc60 con 0x7f2444106320 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f244419bf90 0x7f244419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2442a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2444106320 0x7f2444197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f244419bf90 0x7f244419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47000/0 (socket says 172.21.15.49:47000) 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2443e00640 1 -- 172.21.15.49:0/3926491333 learned_addr learned my addr 172.21.15.49:0/3926491333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2443400640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2444101250 0x7f2444197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2442a00640 1 -- 172.21.15.49:0/3926491333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f244419bf90 msgr2=0x7f244419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2442a00640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f244419bf90 0x7f244419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2442a00640 1 -- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2444101250 msgr2=0x7f2444197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2442a00640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2444101250 0x7f2444197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2442a00640 1 -- 172.21.15.49:0/3926491333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430009490 con 0x7f2444106320 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2443400640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2444101250 0x7f2444197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.054+0000 7f2443e00640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f244419bf90 0x7f244419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.055+0000 7f2442a00640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2444106320 0x7f2444197c60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f243000e9c0 tx=0x7f243000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.055+0000 7f243be00640 1 -- 172.21.15.49:0/3926491333 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243000cd80 con 0x7f2444106320 2024-02-26T06:26:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.055+0000 7f243be00640 1 -- 172.21.15.49:0/3926491333 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24300035f0 con 0x7f2444106320 2024-02-26T06:26:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.055+0000 7f243be00640 1 -- 172.21.15.49:0/3926491333 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430010670 con 0x7f2444106320 2024-02-26T06:26:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.055+0000 7f2449200640 1 -- 172.21.15.49:0/3926491333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2434009560 con 0x7f2444106320 2024-02-26T06:26:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.055+0000 7f2449200640 1 -- 172.21.15.49:0/3926491333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24441a4510 con 0x7f2444106320 2024-02-26T06:26:03.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.059+0000 7f244d66b740 1 Processor -- start 2024-02-26T06:26:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.059+0000 7f244d66b740 1 -- start start 2024-02-26T06:26:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f244d66b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb3bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f244d66b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3c470 0x55cdbfb494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f244d66b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f244d66b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfaf72d0 con 0x55cdbfb3c470 2024-02-26T06:26:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f244d66b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfaa9e20 con 0x55cdbfb499f0 2024-02-26T06:26:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f244d66b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfaa7350 con 0x55cdbfb3bac0 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f2443e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb3bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f2443e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb3bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53726/0 (socket says 172.21.15.49:53726) 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f2443e00640 1 -- 172.21.15.49:0/2102544118 learned_addr learned my addr 172.21.15.49:0/2102544118 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.060+0000 7f2443400640 1 --2- 172.21.15.49:0/2102544118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3c470 0x55cdbfb494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f243be00640 1 -- 172.21.15.49:0/3926491333 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2430003af0 con 0x7f2444106320 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2443e00640 1 -- 172.21.15.49:0/2102544118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 msgr2=0x55cdbfb4bde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2443e00640 1 --2- 172.21.15.49:0/2102544118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2443e00640 1 -- 172.21.15.49:0/2102544118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3c470 msgr2=0x55cdbfb494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2443e00640 1 --2- 172.21.15.49:0/2102544118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3c470 0x55cdbfb494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2443e00640 1 -- 172.21.15.49:0/2102544118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdbfa6bc50 con 0x55cdbfb3bac0 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2443e00640 1 --2- 172.21.15.49:0/2102544118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb3bea0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f243c011850 tx=0x7f243c011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46d5300f9163724 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2438c00640 1 -- 172.21.15.49:0/2102544118 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c011330 con 0x55cdbfb3bac0 2024-02-26T06:26:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2438c00640 1 -- 172.21.15.49:0/2102544118 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243c00fe80 con 0x55cdbfb3bac0 2024-02-26T06:26:03.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.061+0000 7f2438c00640 1 -- 172.21.15.49:0/2102544118 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0193e0 con 0x55cdbfb3bac0 2024-02-26T06:26:03.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.062+0000 7f244d66b740 1 -- 172.21.15.49:0/2102544118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3bac0 msgr2=0x55cdbfb3bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.062+0000 7f244d66b740 1 --2- 172.21.15.49:0/2102544118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb3bea0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f243c011850 tx=0x7f243c011d10 comp rx=0 tx=0).stop 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.062+0000 7f244d66b740 1 -- 172.21.15.49:0/2102544118 shutdown_connections 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.062+0000 7f244d66b740 1 --2- 172.21.15.49:0/2102544118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.062+0000 7f244d66b740 1 --2- 172.21.15.49:0/2102544118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3c470 0x55cdbfb494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.062+0000 7f244d66b740 1 --2- 172.21.15.49:0/2102544118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb3bea0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.062+0000 7f244d66b740 1 -- 172.21.15.49:0/2102544118 >> 172.21.15.49:0/2102544118 conn(0x55cdbfb36810 msgr2=0x55cdbfb36bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.063+0000 7f244d66b740 1 -- 172.21.15.49:0/2102544118 shutdown_connections 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.063+0000 7f244d66b740 1 -- 172.21.15.49:0/2102544118 wait complete. 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.063+0000 7f243be00640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a2240 0x7f24280a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.063+0000 7f243be00640 1 -- 172.21.15.49:0/3926491333 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2430014070 con 0x7f2444106320 2024-02-26T06:26:03.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.063+0000 7f244d66b740 1 -- start start 2024-02-26T06:26:03.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f244d66b740 1 Processor -- start 2024-02-26T06:26:03.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f2443400640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a2240 0x7f24280a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f244d66b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f244d66b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfb46f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f244d66b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb499f0 0x55cdbfb47450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f244d66b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfaa2b30 con 0x55cdbfb499f0 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f244d66b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfa47fa0 con 0x55cdbfb3bac0 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f244d66b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfa47c40 con 0x55cdbfb3c470 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.064+0000 7f244d66b740 10 client.0 _ll_register_callbacks cb 0x55cdbf7b3c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2443400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2443400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47014/0 (socket says 172.21.15.49:47014) 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2442a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfb46f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2443400640 1 -- 172.21.15.49:0/2431354041 learned_addr learned my addr 172.21.15.49:0/2431354041 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2443e00640 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb499f0 0x55cdbfb47450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2442a00640 1 -- 172.21.15.49:0/2431354041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3bac0 msgr2=0x55cdbfb469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2442a00640 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2442a00640 1 -- 172.21.15.49:0/2431354041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb499f0 msgr2=0x55cdbfb47450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2442a00640 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb499f0 0x55cdbfb47450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2442a00640 1 -- 172.21.15.49:0/2431354041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c011500 con 0x55cdbfb3c470 2024-02-26T06:26:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2443400640 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.065+0000 7f2443400640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a2240 0x7f24280a4700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2434006010 tx=0x7f24340058b0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f2442a00640 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfb46f10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f24300cbbb0 tx=0x7f243009a7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900823ba4ca522b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f242d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f242d600640 1 -- 172.21.15.49:0/2431354041 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300aeea0 con 0x55cdbfb3c470 2024-02-26T06:26:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f242d600640 1 -- 172.21.15.49:0/2431354041 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f243009efa0 con 0x55cdbfb3c470 2024-02-26T06:26:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f244d66b740 1 -- 172.21.15.49:0/2431354041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdbf7a2860 con 0x55cdbfb3c470 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f242d600640 1 -- 172.21.15.49:0/2431354041 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24300a35f0 con 0x55cdbfb3c470 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f244d66b740 10 client.26146 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f244d66b740 1 -- 172.21.15.49:0/2431354041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cdbfaa11b0 con 0x55cdbfb3c470 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.066+0000 7f242d600640 1 -- 172.21.15.49:0/2431354041 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cdbf7a2860 con 0x55cdbfb3c470 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.067+0000 7f242d600640 1 -- 172.21.15.49:0/2431354041 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f24300cbe30 con 0x55cdbfb3c470 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.067+0000 7f244d66b740 10 client.26146 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.067+0000 7f244d66b740 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.067+0000 7f244d66b740 20 client.26146 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.067+0000 7f244d66b740 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cdbfb5e1a0 0x55cdbfb605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.067+0000 7f244d66b740 4 client.26146 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.067+0000 7f244d66b740 1 -- 172.21.15.49:0/2431354041 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdbf7f3410 con 0x55cdbfb5e1a0 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.067+0000 7f2443400640 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cdbfb5e1a0 0x55cdbfb605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.068+0000 7f2443400640 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cdbfb5e1a0 0x55cdbfb605c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d50ed2d22c399d9 server_cookie=8a0032aefe008caf in_seq=0 out_seq=0 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.068+0000 7f242d600640 10 client.26146 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.068+0000 7f242d600640 1 -- 172.21.15.49:0/2431354041 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cdbf7f3410 con 0x55cdbfb5e1a0 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f242d600640 10 client.26146 handle_command_reply: tid=0 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 client.26146 shutdown 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 -- 172.21.15.49:0/2431354041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 msgr2=0x55cdbfb46f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfb46f10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f24300cbbb0 tx=0x7f243009a7b0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 -- 172.21.15.49:0/2431354041 shutdown_connections 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cdbfb5e1a0 0x55cdbfb605c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb499f0 0x55cdbfb47450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfb46f10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f242d600640 0 client.26146 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 --2- 172.21.15.49:0/2431354041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 -- 172.21.15.49:0/2431354041 >> 172.21.15.49:0/2431354041 conn(0x55cdbfb36810 msgr2=0x55cdbfb38560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 -- 172.21.15.49:0/2431354041 shutdown_connections 2024-02-26T06:26:03.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 1 -- 172.21.15.49:0/2431354041 wait complete. 2024-02-26T06:26:03.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.069+0000 7f244d66b740 20 client.26146 trim_cache size 0 max 16384 2024-02-26T06:26:03.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.088+0000 7f244d66b740 1 Processor -- start 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.088+0000 7f244d66b740 1 -- start start 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.088+0000 7f244d66b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb43c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.088+0000 7f244d66b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3c470 0x55cdbfb44190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.088+0000 7f244d66b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb499f0 0x55cdbfb452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f244d66b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf7eb0c0 con 0x55cdbfb3bac0 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f244d66b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfb479a0 con 0x55cdbfb3c470 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f244d66b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfb65800 con 0x55cdbfb499f0 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb499f0 0x55cdbfb452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb499f0 0x55cdbfb452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53740/0 (socket says 172.21.15.49:53740) 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443e00640 1 -- 172.21.15.49:0/495561439 learned_addr learned my addr 172.21.15.49:0/495561439 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2442a00640 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3c470 0x55cdbfb44190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443400640 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb43c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443e00640 1 -- 172.21.15.49:0/495561439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3c470 msgr2=0x55cdbfb44190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443e00640 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3c470 0x55cdbfb44190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443e00640 1 -- 172.21.15.49:0/495561439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 msgr2=0x55cdbfb43c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443e00640 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb43c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.089+0000 7f2443e00640 1 -- 172.21.15.49:0/495561439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdbfa3da60 con 0x55cdbfb499f0 2024-02-26T06:26:03.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:02 smithi115 ceph-mon[36426]: pgmap v82: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 739 op/s 2024-02-26T06:26:03.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.090+0000 7f2443e00640 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb499f0 0x55cdbfb452a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f243c0117f0 tx=0x7f243c005c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20b1238e9866d786 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.090+0000 7f242fe00640 1 -- 172.21.15.49:0/495561439 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00f9a0 con 0x55cdbfb499f0 2024-02-26T06:26:03.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.090+0000 7f242fe00640 1 -- 172.21.15.49:0/495561439 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243c00fb00 con 0x55cdbfb499f0 2024-02-26T06:26:03.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.090+0000 7f242fe00640 1 -- 172.21.15.49:0/495561439 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c005860 con 0x55cdbfb499f0 2024-02-26T06:26:03.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 -- 172.21.15.49:0/495561439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb499f0 msgr2=0x55cdbfb452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb499f0 0x55cdbfb452a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f243c0117f0 tx=0x7f243c005c30 comp rx=0 tx=0).stop 2024-02-26T06:26:03.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 -- 172.21.15.49:0/495561439 shutdown_connections 2024-02-26T06:26:03.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb3c470 0x55cdbfb44190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb499f0 0x55cdbfb452a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 --2- 172.21.15.49:0/495561439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfb43c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 -- 172.21.15.49:0/495561439 >> 172.21.15.49:0/495561439 conn(0x55cdbfb36810 msgr2=0x55cdbfb31380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 -- 172.21.15.49:0/495561439 shutdown_connections 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 -- 172.21.15.49:0/495561439 wait complete. 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.091+0000 7f244d66b740 1 -- start start 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f244d66b740 1 Processor -- start 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f244d66b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfbada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f244d66b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfbadf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f244d66b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f244d66b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfb50360 con 0x55cdbfb3bac0 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f244d66b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfb504d0 con 0x55cdbfb499f0 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f244d66b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbfb50690 con 0x55cdbfb3c470 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f244d66b740 10 client.0 _ll_register_callbacks cb 0x7f243009ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f2443400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfbada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.092+0000 7f2443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2442a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfbadf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47034/0 (socket says 172.21.15.49:47034) 2024-02-26T06:26:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfbada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42166/0 (socket says 172.21.15.49:42166) 2024-02-26T06:26:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443e00640 1 -- 172.21.15.49:0/4209411810 learned_addr learned my addr 172.21.15.49:0/4209411810 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443e00640 1 -- 172.21.15.49:0/4209411810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 msgr2=0x55cdbfbadf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443e00640 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfbadf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443e00640 1 -- 172.21.15.49:0/4209411810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 msgr2=0x55cdbfbada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443e00640 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfbada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443e00640 1 -- 172.21.15.49:0/4209411810 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdbfa6bc50 con 0x55cdbfb499f0 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2442a00640 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfbadf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f2443e00640 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4fc60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f243c009270 tx=0x7f243c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49057362e5d31b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.093+0000 7f242d600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.094+0000 7f242d600640 1 -- 172.21.15.49:0/4209411810 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00f9a0 con 0x55cdbfb499f0 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.094+0000 7f242d600640 1 -- 172.21.15.49:0/4209411810 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f243c0049d0 con 0x55cdbfb499f0 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.094+0000 7f244d66b740 1 -- 172.21.15.49:0/4209411810 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f243c00fe80 con 0x55cdbfb499f0 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.094+0000 7f242d600640 1 -- 172.21.15.49:0/4209411810 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f243c004b90 con 0x55cdbfb499f0 2024-02-26T06:26:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.094+0000 7f244d66b740 10 client.26114 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:03.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.094+0000 7f244d66b740 1 -- 172.21.15.49:0/4209411810 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cdbfa63670 con 0x55cdbfb499f0 2024-02-26T06:26:03.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f242d600640 1 -- 172.21.15.49:0/4209411810 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243c00fe80 con 0x55cdbfb499f0 2024-02-26T06:26:03.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f242d600640 1 -- 172.21.15.49:0/4209411810 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f243c03c020 con 0x55cdbfb499f0 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f244d66b740 10 client.26114 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f244d66b740 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f244d66b740 20 client.26114 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f244d66b740 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cdbfbb6540 0x55cdbfbb8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f244d66b740 4 client.26114 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f244d66b740 1 -- 172.21.15.49:0/4209411810 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cdbf7df7b0 con 0x55cdbfbb6540 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.095+0000 7f2443400640 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cdbfbb6540 0x55cdbfbb8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.096+0000 7f2443400640 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cdbfbb6540 0x55cdbfbb8930 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9383fa48ad3bd550 server_cookie=721a93f495669f76 in_seq=0 out_seq=0 2024-02-26T06:26:03.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.096+0000 7f242d600640 10 client.26114 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:03.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f242d600640 1 -- 172.21.15.49:0/4209411810 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cdbf7df7b0 con 0x55cdbfbb6540 2024-02-26T06:26:03.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f242d600640 10 client.26114 handle_command_reply: tid=0 2024-02-26T06:26:03.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 client.26114 shutdown 2024-02-26T06:26:03.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 -- 172.21.15.49:0/4209411810 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 msgr2=0x55cdbfb4fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4fc60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f243c009270 tx=0x7f243c0058a0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 -- 172.21.15.49:0/4209411810 shutdown_connections 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cdbfbb6540 0x55cdbfbb8930 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cdbfb499f0 0x55cdbfb4fc60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdbfb3c470 0x55cdbfbadf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f242d600640 0 client.26114 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 --2- 172.21.15.49:0/4209411810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdbfb3bac0 0x55cdbfbada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 -- 172.21.15.49:0/4209411810 >> 172.21.15.49:0/4209411810 conn(0x55cdbfb36810 msgr2=0x55cdbfb452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.097+0000 7f244d66b740 1 -- 172.21.15.49:0/4209411810 shutdown_connections 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.098+0000 7f244d66b740 1 -- 172.21.15.49:0/4209411810 wait complete. 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.098+0000 7f244d66b740 20 client.26114 trim_cache size 0 max 16384 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:26:03.102 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:26:03.103 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "973d287b-597f-41fd-b62b-e2468d8b950c", 2024-02-26T06:26:03.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:26:03.103 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:26:03.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.098+0000 7f2449200640 1 -- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a2240 msgr2=0x7f24280a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.098+0000 7f2449200640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a2240 0x7f24280a4700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2434006010 tx=0x7f24340058b0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.098+0000 7f2449200640 1 -- 172.21.15.49:0/3926491333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2444106320 msgr2=0x7f2444197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.098+0000 7f2449200640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2444106320 0x7f2444197c60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f243000e9c0 tx=0x7f243000ee80 comp rx=0 tx=0).stop 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.099+0000 7f2449200640 1 -- 172.21.15.49:0/3926491333 shutdown_connections 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.099+0000 7f2449200640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a2240 0x7f24280a4700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.099+0000 7f2449200640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f244419bf90 0x7f244419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.099+0000 7f2449200640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2444106320 0x7f2444197c60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.099+0000 7f2449200640 1 --2- 172.21.15.49:0/3926491333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2444101250 0x7f2444197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.099+0000 7f2449200640 1 -- 172.21.15.49:0/3926491333 >> 172.21.15.49:0/3926491333 conn(0x7f24440fcaf0 msgr2=0x7f24440fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.099+0000 7f2449200640 1 -- 172.21.15.49:0/3926491333 shutdown_connections 2024-02-26T06:26:03.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.099+0000 7f2449200640 1 -- 172.21.15.49:0/3926491333 wait complete. 2024-02-26T06:26:03.117 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:02 smithi080 ceph-mon[36206]: pgmap v82: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 739 op/s 2024-02-26T06:26:03.119 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:02 smithi049 ceph-mon[31696]: pgmap v82: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 739 op/s 2024-02-26T06:26:03.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:26:03.122 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:26:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 -- 172.21.15.49:0/4045374726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102d70 msgr2=0x7f9e3c10b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 --2- 172.21.15.49:0/4045374726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102d70 0x7f9e3c10b250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009900 tx=0x7f9e2802bcc0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 -- 172.21.15.49:0/4045374726 shutdown_connections 2024-02-26T06:26:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 --2- 172.21.15.49:0/4045374726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102d70 0x7f9e3c10b250 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 --2- 172.21.15.49:0/4045374726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e3c10b790 0x7f9e3c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 --2- 172.21.15.49:0/4045374726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e3c102450 0x7f9e3c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 -- 172.21.15.49:0/4045374726 >> 172.21.15.49:0/4045374726 conn(0x7f9e3c0771e0 msgr2=0x7f9e3c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 -- 172.21.15.49:0/4045374726 shutdown_connections 2024-02-26T06:26:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.365+0000 7f9e42600640 1 -- 172.21.15.49:0/4045374726 wait complete. 2024-02-26T06:26:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.366+0000 7f9e42600640 1 Processor -- start 2024-02-26T06:26:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.366+0000 7f9e42600640 1 -- start start 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e42600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102450 0x7f9e3c1000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e42600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e3c102d70 0x7f9e3c1005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e42600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e3c10b790 0x7f9e3c100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e42600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1a5c10 con 0x7f9e3c10b790 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e42600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1a5d80 con 0x7f9e3c102450 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e42600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1a5f40 con 0x7f9e3c102d70 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e3aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e3c102d70 0x7f9e3c1005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e3b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102450 0x7f9e3c1000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e3aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e3c102d70 0x7f9e3c1005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53776/0 (socket says 172.21.15.49:53776) 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e3aa00640 1 -- 172.21.15.49:0/4234962947 learned_addr learned my addr 172.21.15.49:0/4234962947 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.367+0000 7f9e3be00640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e3c10b790 0x7f9e3c100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.368+0000 7f9e3aa00640 1 -- 172.21.15.49:0/4234962947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102450 msgr2=0x7f9e3c1000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.368+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102450 0x7f9e3c1000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.368+0000 7f9e3aa00640 1 -- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e3c10b790 msgr2=0x7f9e3c100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.368+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e3c10b790 0x7f9e3c100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.368+0000 7f9e3aa00640 1 -- 172.21.15.49:0/4234962947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e28009560 con 0x7f9e3c102d70 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.368+0000 7f9e3b400640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102450 0x7f9e3c1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.368+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e3c102d70 0x7f9e3c1005e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e28005940 tx=0x7f9e2802c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.369+0000 7f9e33e00640 1 -- 172.21.15.49:0/4234962947 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28004350 con 0x7f9e3c102d70 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.369+0000 7f9e33e00640 1 -- 172.21.15.49:0/4234962947 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e280044b0 con 0x7f9e3c102d70 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.369+0000 7f9e33e00640 1 -- 172.21.15.49:0/4234962947 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28003d30 con 0x7f9e3c102d70 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.369+0000 7f9e42600640 1 -- 172.21.15.49:0/4234962947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c1a61c0 con 0x7f9e3c102d70 2024-02-26T06:26:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.369+0000 7f9e42600640 1 -- 172.21.15.49:0/4234962947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c1a6720 con 0x7f9e3c102d70 2024-02-26T06:26:03.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.372+0000 7f9e33e00640 1 -- 172.21.15.49:0/4234962947 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9e28004df0 con 0x7f9e3c102d70 2024-02-26T06:26:03.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.373+0000 7f9e45432740 1 Processor -- start 2024-02-26T06:26:03.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.373+0000 7f9e45432740 1 -- start start 2024-02-26T06:26:03.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.373+0000 7f9e45432740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e553cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.373+0000 7f9e45432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554290 0x55957e554710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e45432740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e45432740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e4c3120 con 0x55957e55d600 2024-02-26T06:26:03.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e45432740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e4c0650 con 0x55957e554f70 2024-02-26T06:26:03.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e45432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e4bbe30 con 0x55957e554290 2024-02-26T06:26:03.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e553cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e553cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47060/0 (socket says 172.21.15.49:47060) 2024-02-26T06:26:03.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554290 0x55957e554710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3be00640 1 -- 172.21.15.49:0/2423728219 learned_addr learned my addr 172.21.15.49:0/2423728219 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3b400640 1 -- 172.21.15.49:0/2423728219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 msgr2=0x55957e553cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3b400640 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e553cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3b400640 1 -- 172.21.15.49:0/2423728219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 msgr2=0x55957e55f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3b400640 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3b400640 1 -- 172.21.15.49:0/2423728219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55957e475030 con 0x55957e554290 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3be00640 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e553cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.374+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.375+0000 7f9e3b400640 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554290 0x55957e554710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009fd0 tx=0x7f9e2c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6efc08570efde4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.375+0000 7f9e33e00640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9e200a22b0 0x7f9e200a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.375+0000 7f9e30c00640 1 -- 172.21.15.49:0/2423728219 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c004310 con 0x55957e554290 2024-02-26T06:26:03.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.375+0000 7f9e30c00640 1 -- 172.21.15.49:0/2423728219 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e2c004470 con 0x55957e554290 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.375+0000 7f9e30c00640 1 -- 172.21.15.49:0/2423728219 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0035f0 con 0x55957e554290 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.375+0000 7f9e33e00640 1 -- 172.21.15.49:0/4234962947 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9e280f05f0 con 0x7f9e3c102d70 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.375+0000 7f9e3be00640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9e200a22b0 0x7f9e200a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 -- 172.21.15.49:0/2423728219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554290 msgr2=0x55957e554710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554290 0x55957e554710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009fd0 tx=0x7f9e2c0118d0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e3be00640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9e200a22b0 0x7f9e200a4770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009fd0 tx=0x7f9e34009500 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 -- 172.21.15.49:0/2423728219 shutdown_connections 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554290 0x55957e554710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 --2- 172.21.15.49:0/2423728219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e553cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 -- 172.21.15.49:0/2423728219 >> 172.21.15.49:0/2423728219 conn(0x55957e51b570 msgr2=0x55957e54ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 -- 172.21.15.49:0/2423728219 shutdown_connections 2024-02-26T06:26:03.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.376+0000 7f9e45432740 1 -- 172.21.15.49:0/2423728219 wait complete. 2024-02-26T06:26:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.377+0000 7f9e45432740 1 -- start start 2024-02-26T06:26:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.377+0000 7f9e45432740 1 Processor -- start 2024-02-26T06:26:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e45432740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554290 0x55957e55a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e45432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554f70 0x55957e55aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e45432740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e45432740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e460e90 con 0x55957e55d600 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e45432740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e460b30 con 0x55957e554290 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e45432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e22a1c0 con 0x55957e554f70 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e45432740 10 client.0 _ll_register_callbacks cb 0x55957e474930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e3b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554290 0x55957e55a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e3b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554290 0x55957e55a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47066/0 (socket says 172.21.15.49:47066) 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e3b400640 1 -- 172.21.15.49:0/1162758893 learned_addr learned my addr 172.21.15.49:0/1162758893 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.378+0000 7f9e3be00640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554f70 0x55957e55aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3b400640 1 -- 172.21.15.49:0/1162758893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554f70 msgr2=0x55957e55aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3b400640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554f70 0x55957e55aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3b400640 1 -- 172.21.15.49:0/1162758893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 msgr2=0x55957e55b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3b400640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3b400640 1 -- 172.21.15.49:0/1162758893 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2c009c00 con 0x55957e554290 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554f70 0x55957e55aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3be00640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e3b400640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554290 0x55957e55a580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c005890 tx=0x7f9e2c004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a643a920b27684 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.379+0000 7f9e25600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.380+0000 7f9e25600640 1 -- 172.21.15.49:0/1162758893 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c003b00 con 0x55957e554290 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.380+0000 7f9e25600640 1 -- 172.21.15.49:0/1162758893 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9e2c003c60 con 0x55957e554290 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.380+0000 7f9e45432740 1 -- 172.21.15.49:0/1162758893 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55957e1bb7e0 con 0x55957e554290 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.380+0000 7f9e25600640 1 -- 172.21.15.49:0/1162758893 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e2c003e20 con 0x55957e554290 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.380+0000 7f9e45432740 10 client.26126 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.380+0000 7f9e45432740 1 -- 172.21.15.49:0/1162758893 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55957e4ba4b0 con 0x55957e554290 2024-02-26T06:26:03.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.380+0000 7f9e25600640 1 -- 172.21.15.49:0/1162758893 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55957e1bb7e0 con 0x55957e554290 2024-02-26T06:26:03.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.380+0000 7f9e25600640 1 -- 172.21.15.49:0/1162758893 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9e2c021040 con 0x55957e554290 2024-02-26T06:26:03.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e45432740 10 client.26126 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:03.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e45432740 10 client.26126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:03.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e45432740 20 client.26126 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:03.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e45432740 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55957e56d620 0x55957e56fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e45432740 4 client.26126 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:03.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e45432740 1 -- 172.21.15.49:0/1162758893 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55957e1f8720 con 0x55957e56d620 2024-02-26T06:26:03.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55957e56d620 0x55957e56fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55957e56d620 0x55957e56fa60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26a1efa6089e81c server_cookie=8fba0b693fa06652 in_seq=0 out_seq=0 2024-02-26T06:26:03.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.381+0000 7f9e25600640 10 client.26126 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:03.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.382+0000 7f9e25600640 1 -- 172.21.15.49:0/1162758893 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55957e1f8720 con 0x55957e56d620 2024-02-26T06:26:03.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.382+0000 7f9e25600640 10 client.26126 handle_command_reply: tid=0 2024-02-26T06:26:03.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.382+0000 7f9e45432740 1 client.26126 shutdown 2024-02-26T06:26:03.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.382+0000 7f9e45432740 1 -- 172.21.15.49:0/1162758893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554290 msgr2=0x55957e55a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.382+0000 7f9e45432740 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554290 0x55957e55a580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c005890 tx=0x7f9e2c004a30 comp rx=0 tx=0).stop 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 1 -- 172.21.15.49:0/1162758893 shutdown_connections 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55957e56d620 0x55957e56fa60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e55d600 0x55957e55b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e554f70 0x55957e55aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 1 --2- 172.21.15.49:0/1162758893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554290 0x55957e55a580 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e25600640 0 client.26126 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 1 -- 172.21.15.49:0/1162758893 >> 172.21.15.49:0/1162758893 conn(0x55957e51b570 msgr2=0x55957e550a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 1 -- 172.21.15.49:0/1162758893 shutdown_connections 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 1 -- 172.21.15.49:0/1162758893 wait complete. 2024-02-26T06:26:03.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.383+0000 7f9e45432740 20 client.26126 trim_cache size 0 max 16384 2024-02-26T06:26:03.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.409+0000 7f9e45432740 1 Processor -- start 2024-02-26T06:26:03.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.409+0000 7f9e45432740 1 -- start start 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e45432740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e55ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e45432740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e55e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e45432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e55ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e45432740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e55b510 con 0x55957e554290 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e45432740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e54fce0 con 0x55957e554f70 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e45432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e574c70 con 0x55957e55d600 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e3be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e55ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e3be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e55ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53788/0 (socket says 172.21.15.49:53788) 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e3be00640 1 -- 172.21.15.49:0/2533938934 learned_addr learned my addr 172.21.15.49:0/2533938934 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e55e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.410+0000 7f9e3b400640 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e55ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.411+0000 7f9e3be00640 1 -- 172.21.15.49:0/2533938934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 msgr2=0x55957e55e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.411+0000 7f9e3be00640 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e55e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.411+0000 7f9e3be00640 1 -- 172.21.15.49:0/2533938934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 msgr2=0x55957e55ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.411+0000 7f9e3be00640 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e55ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.411+0000 7f9e3be00640 1 -- 172.21.15.49:0/2533938934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55957e456920 con 0x55957e55d600 2024-02-26T06:26:03.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.411+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e55e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:03.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.411+0000 7f9e3b400640 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e55ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:03.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.411+0000 7f9e3be00640 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e55ef30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400f990 tx=0x7f9e34016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd1c54fcc9b68c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.413+0000 7f9e27e00640 1 -- 172.21.15.49:0/2533938934 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34016070 con 0x55957e55d600 2024-02-26T06:26:03.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.413+0000 7f9e27e00640 1 -- 172.21.15.49:0/2533938934 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e34023040 con 0x55957e55d600 2024-02-26T06:26:03.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.413+0000 7f9e27e00640 1 -- 172.21.15.49:0/2533938934 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34004590 con 0x55957e55d600 2024-02-26T06:26:03.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.414+0000 7f9e45432740 1 -- 172.21.15.49:0/2533938934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 msgr2=0x55957e55ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.414+0000 7f9e45432740 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e55ef30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400f990 tx=0x7f9e34016000 comp rx=0 tx=0).stop 2024-02-26T06:26:03.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.414+0000 7f9e45432740 1 -- 172.21.15.49:0/2533938934 shutdown_connections 2024-02-26T06:26:03.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.414+0000 7f9e45432740 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e55e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.414+0000 7f9e45432740 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e55ef30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.414+0000 7f9e45432740 1 --2- 172.21.15.49:0/2533938934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e55ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.414+0000 7f9e45432740 1 -- 172.21.15.49:0/2533938934 >> 172.21.15.49:0/2533938934 conn(0x55957e51b570 msgr2=0x55957e5558f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.415+0000 7f9e45432740 1 -- 172.21.15.49:0/2533938934 shutdown_connections 2024-02-26T06:26:03.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.415+0000 7f9e45432740 1 -- 172.21.15.49:0/2533938934 wait complete. 2024-02-26T06:26:03.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.416+0000 7f9e45432740 1 -- start start 2024-02-26T06:26:03.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.416+0000 7f9e45432740 1 Processor -- start 2024-02-26T06:26:03.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.416+0000 7f9e45432740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e5649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.416+0000 7f9e45432740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e562db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.416+0000 7f9e45432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e563330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.416+0000 7f9e45432740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e563a30 con 0x55957e554290 2024-02-26T06:26:03.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.416+0000 7f9e45432740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e563ba0 con 0x55957e554f70 2024-02-26T06:26:03.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.416+0000 7f9e45432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957e5dd370 con 0x55957e55d600 2024-02-26T06:26:03.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.417+0000 7f9e45432740 10 client.0 _ll_register_callbacks cb 0x7f9e34009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:03.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.417+0000 7f9e3aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e562db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.417+0000 7f9e3aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e562db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47094/0 (socket says 172.21.15.49:47094) 2024-02-26T06:26:03.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.417+0000 7f9e3aa00640 1 -- 172.21.15.49:0/292913317 learned_addr learned my addr 172.21.15.49:0/292913317 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:03.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.417+0000 7f9e3b400640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e5649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.417+0000 7f9e3be00640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e563330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.417+0000 7f9e3aa00640 1 -- 172.21.15.49:0/292913317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 msgr2=0x55957e563330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.418+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e563330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.418+0000 7f9e3aa00640 1 -- 172.21.15.49:0/292913317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 msgr2=0x55957e5649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:03.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.418+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e5649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.418+0000 7f9e3aa00640 1 -- 172.21.15.49:0/292913317 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c1a6720 con 0x55957e554f70 2024-02-26T06:26:03.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.418+0000 7f9e3be00640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e563330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:03.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.418+0000 7f9e3b400640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e5649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:03.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.419+0000 7f9e3aa00640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e562db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e28006010 tx=0x7f9e280037d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13c9a3ffe76bd9d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:03.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.419+0000 7f9e25600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:03.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.419+0000 7f9e25600640 1 -- 172.21.15.49:0/292913317 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280bd0a0 con 0x55957e554f70 2024-02-26T06:26:03.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.419+0000 7f9e25600640 1 -- 172.21.15.49:0/292913317 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9e280d3030 con 0x55957e554f70 2024-02-26T06:26:03.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.420+0000 7f9e45432740 1 -- 172.21.15.49:0/292913317 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e2c004470 con 0x55957e554f70 2024-02-26T06:26:03.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.420+0000 7f9e25600640 1 -- 172.21.15.49:0/292913317 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e280c11f0 con 0x55957e554f70 2024-02-26T06:26:03.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e45432740 10 client.26132 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:03.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e45432740 1 -- 172.21.15.49:0/292913317 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f9e34009ce0 con 0x55957e554f70 2024-02-26T06:26:03.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e25600640 1 -- 172.21.15.49:0/292913317 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e2c004470 con 0x55957e554f70 2024-02-26T06:26:03.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e25600640 1 -- 172.21.15.49:0/292913317 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9e2803c070 con 0x55957e554f70 2024-02-26T06:26:03.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e45432740 10 client.26132 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:03.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e45432740 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:03.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e45432740 20 client.26132 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:03.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e45432740 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55957e5de840 0x55957e5e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:03.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e45432740 4 client.26132 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:26:03.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.421+0000 7f9e45432740 1 -- 172.21.15.49:0/292913317 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55957e247890 con 0x55957e5de840 2024-02-26T06:26:03.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.423+0000 7f9e3b400640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55957e5de840 0x55957e5e0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:03.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.423+0000 7f9e3b400640 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55957e5de840 0x55957e5e0c60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b61053e8fc1883 server_cookie=9b89db98013aad06 in_seq=0 out_seq=0 2024-02-26T06:26:03.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.423+0000 7f9e25600640 10 client.26132 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:03.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.423+0000 7f9e25600640 1 -- 172.21.15.49:0/292913317 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55957e247890 con 0x55957e5de840 2024-02-26T06:26:03.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.423+0000 7f9e25600640 10 client.26132 handle_command_reply: tid=0 2024-02-26T06:26:03.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 client.26132 shutdown 2024-02-26T06:26:03.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 -- 172.21.15.49:0/292913317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 msgr2=0x55957e562db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e562db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e28006010 tx=0x7f9e280037d0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 -- 172.21.15.49:0/292913317 shutdown_connections 2024-02-26T06:26:03.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55957e5de840 0x55957e5e0c60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55957e554f70 0x55957e562db0 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e28006010 tx=0x7f9e280037d0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55957e55d600 0x55957e563330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 --2- 172.21.15.49:0/292913317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55957e554290 0x55957e5649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 -- 172.21.15.49:0/292913317 >> 172.21.15.49:0/292913317 conn(0x55957e51b570 msgr2=0x55957e54d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e25600640 0 client.26132 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.424+0000 7f9e45432740 1 -- 172.21.15.49:0/292913317 shutdown_connections 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.425+0000 7f9e45432740 1 -- 172.21.15.49:0/292913317 wait complete. 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.425+0000 7f9e45432740 20 client.26132 trim_cache size 0 max 16384 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stdout: "973d287b-597f-41fd-b62b-e2468d8b950c": { 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "973d287b-597f-41fd-b62b-e2468d8b950c", 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:26:03.426 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:26:03.427 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:26:03.427 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:26:03.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.426+0000 7f9e42600640 1 -- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9e200a22b0 msgr2=0x7f9e200a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.426+0000 7f9e42600640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9e200a22b0 0x7f9e200a4770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009fd0 tx=0x7f9e34009500 comp rx=0 tx=0).stop 2024-02-26T06:26:03.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.427+0000 7f9e42600640 1 -- 172.21.15.49:0/4234962947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e3c102d70 msgr2=0x7f9e3c1005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:03.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.427+0000 7f9e42600640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e3c102d70 0x7f9e3c1005e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e28005940 tx=0x7f9e2802c700 comp rx=0 tx=0).stop 2024-02-26T06:26:03.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.427+0000 7f9e42600640 1 -- 172.21.15.49:0/4234962947 shutdown_connections 2024-02-26T06:26:03.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.427+0000 7f9e42600640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9e200a22b0 0x7f9e200a4770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.427+0000 7f9e42600640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e3c10b790 0x7f9e3c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.427+0000 7f9e42600640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e3c102d70 0x7f9e3c1005e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.427+0000 7f9e42600640 1 --2- 172.21.15.49:0/4234962947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c102450 0x7f9e3c1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:03.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.427+0000 7f9e42600640 1 -- 172.21.15.49:0/4234962947 >> 172.21.15.49:0/4234962947 conn(0x7f9e3c0771e0 msgr2=0x7f9e3c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:03.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.428+0000 7f9e42600640 1 -- 172.21.15.49:0/4234962947 shutdown_connections 2024-02-26T06:26:03.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:03.428+0000 7f9e42600640 1 -- 172.21.15.49:0/4234962947 wait complete. 2024-02-26T06:26:03.452 INFO:tasks.cephfs.filesystem:scrub status for tag:973d287b-597f-41fd-b62b-e2468d8b950c - {'path': '/', 'tag': '973d287b-597f-41fd-b62b-e2468d8b950c', 'options': 'recursive,force'} 2024-02-26T06:26:03.580 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: 2024-02-26T06:26:03.116+0000 7f2b87eed700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...4,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_subtrans/ auth{1=1,2=1} v1239 ap=2 f(v3 m2024-02-26T06:25:39.752300+0000 4=4+0) n(v5 rc2024-02-26T06:25:39.752300+0000 b786432 5=4+1) old_inodes=2 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25193=pAsLsXsFs/pAsLsXsFsx@12} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5582523e0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v252>,<0x10000000005/data v111>,<0x100000001fb/postgres v19>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v33>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v38>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v1239>,<0x10000000005/data v297>,<0x100000001fb/postgres v139>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v162>,<0x10000000001/sv_1 v180>,<0x10000000000/qa v169>,<0x1/volumes v155>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:25:13.916008+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:25:39.752300+0000 b786432 5=4+1)","memory_value.dirstat":"f(v3 m2024-02-26T06:25:39.752300+0000 4=4+0)","memory_value.rstat":"n(v5 rc2024-02-26T06:25:39.752300+0000 b786432 5=4+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-02-26T06:26:04.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:26:04.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:04.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:04.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:04.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:03 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.472 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.473 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:03 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:26:04.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:04.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:04.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:03 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:05.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:04 smithi049 ceph-mon[31696]: pgmap v83: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 603 op/s 2024-02-26T06:26:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:04 smithi080 ceph-mon[36206]: pgmap v83: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 603 op/s 2024-02-26T06:26:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:04 smithi115 ceph-mon[36426]: pgmap v83: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 603 op/s 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 -- 172.21.15.49:0/531194209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4efc1033e0 msgr2=0x7f4efc1037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 --2- 172.21.15.49:0/531194209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4efc1033e0 0x7f4efc1037c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009860 tx=0x7f4ef002bb70 comp rx=0 tx=0).stop 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 -- 172.21.15.49:0/531194209 shutdown_connections 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 --2- 172.21.15.49:0/531194209 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc103d90 0x7f4efc107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 --2- 172.21.15.49:0/531194209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4efc1084b0 0x7f4efc110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 --2- 172.21.15.49:0/531194209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4efc1033e0 0x7f4efc1037c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 -- 172.21.15.49:0/531194209 >> 172.21.15.49:0/531194209 conn(0x7f4efc0fec60 msgr2=0x7f4efc101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 -- 172.21.15.49:0/531194209 shutdown_connections 2024-02-26T06:26:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.742+0000 7f4f03200640 1 -- 172.21.15.49:0/531194209 wait complete. 2024-02-26T06:26:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.743+0000 7f4f03200640 1 Processor -- start 2024-02-26T06:26:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.744+0000 7f4f03200640 1 -- start start 2024-02-26T06:26:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.744+0000 7f4f03200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4efc1033e0 0x7f4efc19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.744+0000 7f4f03200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4efc1084b0 0x7f4efc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.744+0000 7f4f03200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc199630 0x7f4efc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.744+0000 7f4f03200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc19a100 con 0x7f4efc1084b0 2024-02-26T06:26:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.744+0000 7f4f03200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc19a270 con 0x7f4efc199630 2024-02-26T06:26:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.744+0000 7f4f03200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc19a430 con 0x7f4efc1033e0 2024-02-26T06:26:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc199630 0x7f4efc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc199630 0x7f4efc199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47108/0 (socket says 172.21.15.49:47108) 2024-02-26T06:26:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4efc1033e0 0x7f4efc19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 -- 172.21.15.49:0/2561385274 learned_addr learned my addr 172.21.15.49:0/2561385274 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f01400640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4efc1084b0 0x7f4efc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 -- 172.21.15.49:0/2561385274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4efc1033e0 msgr2=0x7f4efc19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4efc1033e0 0x7f4efc19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 -- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4efc1084b0 msgr2=0x7f4efc19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4efc1084b0 0x7f4efc19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 -- 172.21.15.49:0/2561385274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef0009490 con 0x7f4efc199630 2024-02-26T06:26:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f01400640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4efc1084b0 0x7f4efc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.745+0000 7f4f02800640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc199630 0x7f4efc199ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800eb10 tx=0x7f4ef800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.746+0000 7f4ef6a00640 1 -- 172.21.15.49:0/2561385274 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8004b20 con 0x7f4efc199630 2024-02-26T06:26:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.746+0000 7f4ef6a00640 1 -- 172.21.15.49:0/2561385274 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ef8004c80 con 0x7f4efc199630 2024-02-26T06:26:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.746+0000 7f4ef6a00640 1 -- 172.21.15.49:0/2561385274 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8010650 con 0x7f4efc199630 2024-02-26T06:26:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.746+0000 7f4f03200640 1 -- 172.21.15.49:0/2561385274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc1a63d0 con 0x7f4efc199630 2024-02-26T06:26:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.746+0000 7f4f03200640 1 -- 172.21.15.49:0/2561385274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc1a6860 con 0x7f4efc199630 2024-02-26T06:26:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.753+0000 7f4f07820740 1 Processor -- start 2024-02-26T06:26:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.755+0000 7f4ef6a00640 1 -- 172.21.15.49:0/2561385274 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4ef8005020 con 0x7f4efc199630 2024-02-26T06:26:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.756+0000 7f4f07820740 1 -- start start 2024-02-26T06:26:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.757+0000 7f4f07820740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d21517a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.757+0000 7f4f07820740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21518440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.757+0000 7f4f07820740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d21523930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.757+0000 7f4f07820740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d214d35f0 con 0x556d21521540 2024-02-26T06:26:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.757+0000 7f4f07820740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d21485890 con 0x556d21518d30 2024-02-26T06:26:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.757+0000 7f4f07820740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d21482dc0 con 0x556d21517fc0 2024-02-26T06:26:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.758+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21518440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.758+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21518440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53810/0 (socket says 172.21.15.49:53810) 2024-02-26T06:26:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.758+0000 7f4f01e00640 1 -- 172.21.15.49:0/470583432 learned_addr learned my addr 172.21.15.49:0/470583432 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.758+0000 7f4ef6a00640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ee40a23f0 0x7f4ee40a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.758+0000 7f4f01400640 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d21517a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.758+0000 7f4ef6a00640 1 -- 172.21.15.49:0/2561385274 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4ef8014070 con 0x7f4efc199630 2024-02-26T06:26:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.758+0000 7f4f01e00640 1 -- 172.21.15.49:0/470583432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 msgr2=0x556d21517a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.759+0000 7f4f01e00640 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d21517a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.759+0000 7f4f01e00640 1 -- 172.21.15.49:0/470583432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 msgr2=0x556d21523930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.759+0000 7f4f01e00640 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d21523930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.759+0000 7f4f01e00640 1 -- 172.21.15.49:0/470583432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d213ffad0 con 0x556d21517fc0 2024-02-26T06:26:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.759+0000 7f4f01400640 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d21517a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.759+0000 7f4f02800640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ee40a23f0 0x7f4ee40a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.760+0000 7f4f01e00640 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21518440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009780 tx=0x7f4ef00323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca704216f6483fca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.760+0000 7f4eeb000640 1 -- 172.21.15.49:0/470583432 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0004380 con 0x556d21517fc0 2024-02-26T06:26:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.760+0000 7f4eeb000640 1 -- 172.21.15.49:0/470583432 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ef00044e0 con 0x556d21517fc0 2024-02-26T06:26:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.761+0000 7f4eeb000640 1 -- 172.21.15.49:0/470583432 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0003850 con 0x556d21517fc0 2024-02-26T06:26:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.762+0000 7f4f02800640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ee40a23f0 0x7f4ee40a48b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009ae0 tx=0x7f4ef8009a70 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.762+0000 7f4f07820740 1 -- 172.21.15.49:0/470583432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 msgr2=0x556d21518440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.762+0000 7f4f07820740 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21518440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009780 tx=0x7f4ef00323c0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.762+0000 7f4f07820740 1 -- 172.21.15.49:0/470583432 shutdown_connections 2024-02-26T06:26:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.762+0000 7f4f07820740 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d21523930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.762+0000 7f4f07820740 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21518440 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.762+0000 7f4f07820740 1 --2- 172.21.15.49:0/470583432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d21517a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.762+0000 7f4f07820740 1 -- 172.21.15.49:0/470583432 >> 172.21.15.49:0/470583432 conn(0x556d214df480 msgr2=0x556d21512cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.763+0000 7f4f07820740 1 -- 172.21.15.49:0/470583432 shutdown_connections 2024-02-26T06:26:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.763+0000 7f4f07820740 1 -- 172.21.15.49:0/470583432 wait complete. 2024-02-26T06:26:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.764+0000 7f4f07820740 1 -- start start 2024-02-26T06:26:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.764+0000 7f4f07820740 1 Processor -- start 2024-02-26T06:26:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.765+0000 7f4f07820740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21517fc0 0x556d2151e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.765+0000 7f4f07820740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21518d30 0x556d2151e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.765+0000 7f4f07820740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d2151eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.765+0000 7f4f07820740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2147e5a0 con 0x556d21521540 2024-02-26T06:26:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.765+0000 7f4f07820740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d21423b20 con 0x556d21517fc0 2024-02-26T06:26:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.765+0000 7f4f07820740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d214237c0 con 0x556d21518d30 2024-02-26T06:26:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.765+0000 7f4f07820740 10 client.0 _ll_register_callbacks cb 0x556d21437520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21518d30 0x556d2151e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d2151eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21518d30 0x556d2151e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53812/0 (socket says 172.21.15.49:53812) 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01e00640 1 -- 172.21.15.49:0/857253 learned_addr learned my addr 172.21.15.49:0/857253 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01400640 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21517fc0 0x556d2151e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01e00640 1 -- 172.21.15.49:0/857253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21517fc0 msgr2=0x556d2151e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01e00640 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21517fc0 0x556d2151e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01e00640 1 -- 172.21.15.49:0/857253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 msgr2=0x556d2151eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01e00640 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d2151eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01e00640 1 -- 172.21.15.49:0/857253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc110890 con 0x556d21518d30 2024-02-26T06:26:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.766+0000 7f4f01400640 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d2151eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.767+0000 7f4f01e00640 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21518d30 0x556d2151e990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4ef002c0b0 tx=0x7f4ef002c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48fae39a7aff2b9e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.767+0000 7f4edf400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.767+0000 7f4edf400640 1 -- 172.21.15.49:0/857253 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef002c9b0 con 0x556d21518d30 2024-02-26T06:26:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.767+0000 7f4edf400640 1 -- 172.21.15.49:0/857253 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4ef002cb90 con 0x556d21518d30 2024-02-26T06:26:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.767+0000 7f4f07820740 1 -- 172.21.15.49:0/857253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d21450ca0 con 0x556d21518d30 2024-02-26T06:26:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.768+0000 7f4edf400640 1 -- 172.21.15.49:0/857253 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ef0034650 con 0x556d21518d30 2024-02-26T06:26:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.768+0000 7f4f07820740 10 client.26179 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.768+0000 7f4f07820740 1 -- 172.21.15.49:0/857253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556d2147cc20 con 0x556d21518d30 2024-02-26T06:26:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.768+0000 7f4edf400640 1 -- 172.21.15.49:0/857253 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d21450ca0 con 0x556d21518d30 2024-02-26T06:26:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.770+0000 7f4edf400640 1 -- 172.21.15.49:0/857253 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4ef00035f0 con 0x556d21518d30 2024-02-26T06:26:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.770+0000 7f4f07820740 10 client.26179 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.770+0000 7f4f07820740 10 client.26179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.770+0000 7f4f07820740 20 client.26179 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.770+0000 7f4f07820740 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556d215420a0 0x556d215444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.770+0000 7f4f07820740 4 client.26179 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.770+0000 7f4f07820740 1 -- 172.21.15.49:0/857253 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d21474cb0 con 0x556d215420a0 2024-02-26T06:26:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.770+0000 7f4f01400640 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556d215420a0 0x556d215444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.771+0000 7f4f01400640 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556d215420a0 0x556d215444c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aaa96bb2071f21e server_cookie=3916b6a42007b6e9 in_seq=0 out_seq=0 2024-02-26T06:26:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.771+0000 7f4edf400640 10 client.26179 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.771+0000 7f4edf400640 1 -- 172.21.15.49:0/857253 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d21474cb0 con 0x556d215420a0 2024-02-26T06:26:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.771+0000 7f4edf400640 10 client.26179 handle_command_reply: tid=0 2024-02-26T06:26:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.772+0000 7f4f07820740 1 client.26179 shutdown 2024-02-26T06:26:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 -- 172.21.15.49:0/857253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21518d30 msgr2=0x556d2151e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21518d30 0x556d2151e990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4ef002c0b0 tx=0x7f4ef002c7c0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 -- 172.21.15.49:0/857253 shutdown_connections 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556d215420a0 0x556d215444c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4edf400640 0 client.26179 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d2151eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21518d30 0x556d2151e990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 --2- 172.21.15.49:0/857253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21517fc0 0x556d2151e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 -- 172.21.15.49:0/857253 >> 172.21.15.49:0/857253 conn(0x556d214df480 msgr2=0x556d21514a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 -- 172.21.15.49:0/857253 shutdown_connections 2024-02-26T06:26:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.773+0000 7f4f07820740 1 -- 172.21.15.49:0/857253 wait complete. 2024-02-26T06:26:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.774+0000 7f4f07820740 20 client.26179 trim_cache size 0 max 16384 2024-02-26T06:26:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.793+0000 7f4f07820740 1 Processor -- start 2024-02-26T06:26:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f07820740 1 -- start start 2024-02-26T06:26:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f07820740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21520b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f07820740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d215392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f07820740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d21539810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f07820740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d211c6e60 con 0x556d21521540 2024-02-26T06:26:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f07820740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d215172c0 con 0x556d21518d30 2024-02-26T06:26:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f07820740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d215496a0 con 0x556d21517fc0 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f01400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21520b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d215392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f01400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21520b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53820/0 (socket says 172.21.15.49:53820) 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f01400640 1 -- 172.21.15.49:0/2472271338 learned_addr learned my addr 172.21.15.49:0/2472271338 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.794+0000 7f4f01400640 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d21539810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4f01400640 1 -- 172.21.15.49:0/2472271338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 msgr2=0x556d215392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4f01400640 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d215392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4f01400640 1 -- 172.21.15.49:0/2472271338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 msgr2=0x556d21539810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4f01400640 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d21539810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4f01400640 1 -- 172.21.15.49:0/2472271338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d21419500 con 0x556d21517fc0 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4f01e00640 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d215392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4f01400640 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21520b80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4efc19ae80 tx=0x7f4eec005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bfb8aac4eadc050 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4eea600640 1 -- 172.21.15.49:0/2472271338 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec022040 con 0x556d21517fc0 2024-02-26T06:26:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4eea600640 1 -- 172.21.15.49:0/2472271338 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4eec005510 con 0x556d21517fc0 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.795+0000 7f4eea600640 1 -- 172.21.15.49:0/2472271338 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec004db0 con 0x556d21517fc0 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 -- 172.21.15.49:0/2472271338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 msgr2=0x556d21520b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21520b80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4efc19ae80 tx=0x7f4eec005d10 comp rx=0 tx=0).stop 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 -- 172.21.15.49:0/2472271338 shutdown_connections 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21518d30 0x556d215392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21521540 0x556d21539810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 --2- 172.21.15.49:0/2472271338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d21520b80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 -- 172.21.15.49:0/2472271338 >> 172.21.15.49:0/2472271338 conn(0x556d214df480 msgr2=0x556d21517530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 -- 172.21.15.49:0/2472271338 shutdown_connections 2024-02-26T06:26:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.796+0000 7f4f07820740 1 -- 172.21.15.49:0/2472271338 wait complete. 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.797+0000 7f4f07820740 1 -- start start 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.797+0000 7f4f07820740 1 Processor -- start 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.797+0000 7f4f07820740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d2151b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.797+0000 7f4f07820740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21518d30 0x556d2151b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f07820740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21521540 0x556d2151bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f07820740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d21598680 con 0x556d21518d30 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f07820740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d215987f0 con 0x556d21521540 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f07820740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d215989b0 con 0x556d21517fc0 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f07820740 10 client.0 _ll_register_callbacks cb 0x7f4eec005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d2151b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d2151b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53826/0 (socket says 172.21.15.49:53826) 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 -- 172.21.15.49:0/696990272 learned_addr learned my addr 172.21.15.49:0/696990272 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21521540 0x556d2151bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01e00640 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21518d30 0x556d2151b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 -- 172.21.15.49:0/696990272 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21521540 msgr2=0x556d2151bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21521540 0x556d2151bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 -- 172.21.15.49:0/696990272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21518d30 msgr2=0x556d2151b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21518d30 0x556d2151b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 -- 172.21.15.49:0/696990272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d2147cc20 con 0x556d21517fc0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4f01400640 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d2151b2d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0098d0 tx=0x7f4eec015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c23cd5a947062d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.798+0000 7f4edf400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.799+0000 7f4edf400640 1 -- 172.21.15.49:0/696990272 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec01d7b0 con 0x556d21517fc0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.799+0000 7f4edf400640 1 -- 172.21.15.49:0/696990272 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4eec018040 con 0x556d21517fc0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.799+0000 7f4f07820740 1 -- 172.21.15.49:0/696990272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d214a3d00 con 0x556d21517fc0 2024-02-26T06:26:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.799+0000 7f4edf400640 1 -- 172.21.15.49:0/696990272 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eec01d910 con 0x556d21517fc0 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.799+0000 7f4f07820740 10 client.26185 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.799+0000 7f4f07820740 1 -- 172.21.15.49:0/696990272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4eec01d310 con 0x556d21517fc0 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.799+0000 7f4edf400640 1 -- 172.21.15.49:0/696990272 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d214a3d00 con 0x556d21517fc0 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4edf400640 1 -- 172.21.15.49:0/696990272 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4eec036070 con 0x556d21517fc0 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4f07820740 10 client.26185 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4f07820740 10 client.26185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4f07820740 20 client.26185 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4f07820740 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556d215420a0 0x556d2159c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4f07820740 4 client.26185 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4f07820740 1 -- 172.21.15.49:0/696990272 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556d2117e600 con 0x556d215420a0 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4f01e00640 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556d215420a0 0x556d2159c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4f01e00640 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556d215420a0 0x556d2159c170 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb94059bf7ab40b1 server_cookie=f26978cf0b52a3a0 in_seq=0 out_seq=0 2024-02-26T06:26:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.800+0000 7f4edf400640 10 client.26185 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4edf400640 1 -- 172.21.15.49:0/696990272 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556d2117e600 con 0x556d215420a0 2024-02-26T06:26:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4edf400640 10 client.26185 handle_command_reply: tid=0 2024-02-26T06:26:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 client.26185 shutdown 2024-02-26T06:26:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 -- 172.21.15.49:0/696990272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 msgr2=0x556d2151b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d2151b2d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0098d0 tx=0x7f4eec015040 comp rx=0 tx=0).stop 2024-02-26T06:26:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 -- 172.21.15.49:0/696990272 shutdown_connections 2024-02-26T06:26:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556d215420a0 0x556d2159c170 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556d21521540 0x556d2151bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d21518d30 0x556d2151b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 --2- 172.21.15.49:0/696990272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d21517fc0 0x556d2151b2d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.812+0000 7f4f07820740 1 -- 172.21.15.49:0/696990272 >> 172.21.15.49:0/696990272 conn(0x556d214df480 msgr2=0x556d2151cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.813+0000 7f4edf400640 0 client.26185 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.813+0000 7f4f07820740 1 -- 172.21.15.49:0/696990272 shutdown_connections 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.813+0000 7f4f07820740 1 -- 172.21.15.49:0/696990272 wait complete. 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.813+0000 7f4f07820740 20 client.26185 trim_cache size 0 max 16384 2024-02-26T06:26:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:26:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:26:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:26:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:26:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.814+0000 7f4f03200640 1 -- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ee40a23f0 msgr2=0x7f4ee40a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.814+0000 7f4f03200640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ee40a23f0 0x7f4ee40a48b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009ae0 tx=0x7f4ef8009a70 comp rx=0 tx=0).stop 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.814+0000 7f4f03200640 1 -- 172.21.15.49:0/2561385274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc199630 msgr2=0x7f4efc199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.814+0000 7f4f03200640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc199630 0x7f4efc199ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800eb10 tx=0x7f4ef800c2d0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.815+0000 7f4f03200640 1 -- 172.21.15.49:0/2561385274 shutdown_connections 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.815+0000 7f4f03200640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4ee40a23f0 0x7f4ee40a48b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.815+0000 7f4f03200640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc199630 0x7f4efc199ab0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.815+0000 7f4f03200640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4efc1084b0 0x7f4efc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.815+0000 7f4f03200640 1 --2- 172.21.15.49:0/2561385274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4efc1033e0 0x7f4efc19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.815+0000 7f4f03200640 1 -- 172.21.15.49:0/2561385274 >> 172.21.15.49:0/2561385274 conn(0x7f4efc0fec60 msgr2=0x7f4efc100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.815+0000 7f4f03200640 1 -- 172.21.15.49:0/2561385274 shutdown_connections 2024-02-26T06:26:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:05.815+0000 7f4f03200640 1 -- 172.21.15.49:0/2561385274 wait complete. 2024-02-26T06:26:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 135.0 s, 245.0 tps, lat 413.109 ms stddev 258.858 2024-02-26T06:26:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:06 smithi049 ceph-mon[31696]: pgmap v84: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 9.2 MiB/s wr, 896 op/s 2024-02-26T06:26:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:06 smithi080 ceph-mon[36206]: pgmap v84: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 9.2 MiB/s wr, 896 op/s 2024-02-26T06:26:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:06 smithi115 ceph-mon[36426]: pgmap v84: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 9.2 MiB/s wr, 896 op/s 2024-02-26T06:26:09.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:08 smithi049 ceph-mon[31696]: pgmap v85: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.4 MiB/s wr, 595 op/s 2024-02-26T06:26:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:08 smithi080 ceph-mon[36206]: pgmap v85: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.4 MiB/s wr, 595 op/s 2024-02-26T06:26:09.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:08 smithi115 ceph-mon[36426]: pgmap v85: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.4 MiB/s wr, 595 op/s 2024-02-26T06:26:11.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:10 smithi049 ceph-mon[31696]: pgmap v86: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.7 MiB/s wr, 668 op/s 2024-02-26T06:26:11.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:10 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:26:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 140.0 s, 188.0 tps, lat 504.599 ms stddev 305.270 2024-02-26T06:26:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:10 smithi080 ceph-mon[36206]: pgmap v86: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.7 MiB/s wr, 668 op/s 2024-02-26T06:26:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:10 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:26:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:10 smithi115 ceph-mon[36426]: pgmap v86: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.7 MiB/s wr, 668 op/s 2024-02-26T06:26:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:10 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:26:12.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:11 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:26:12.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:11 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:26:12.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:11 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:26:12.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:11 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:26:12.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:11 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:26:12.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:11 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:26:13.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:12 smithi049 ceph-mon[31696]: pgmap v87: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 4.6 MiB/s wr, 597 op/s 2024-02-26T06:26:13.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:12 smithi080 ceph-mon[36206]: pgmap v87: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 4.6 MiB/s wr, 597 op/s 2024-02-26T06:26:13.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:12 smithi115 ceph-mon[36426]: pgmap v87: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 4.6 MiB/s wr, 597 op/s 2024-02-26T06:26:15.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:14 smithi049 ceph-mon[31696]: pgmap v88: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 1.9 MiB/s wr, 439 op/s 2024-02-26T06:26:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:14 smithi080 ceph-mon[36206]: pgmap v88: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 1.9 MiB/s wr, 439 op/s 2024-02-26T06:26:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:14 smithi115 ceph-mon[36426]: pgmap v88: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 1.9 MiB/s wr, 439 op/s 2024-02-26T06:26:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 145.0 s, 1926.6 tps, lat 57.629 ms stddev 153.653 2024-02-26T06:26:17.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:16 smithi049 ceph-mon[31696]: pgmap v89: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 30 MiB/s wr, 1.67k op/s 2024-02-26T06:26:17.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:16 smithi080 ceph-mon[36206]: pgmap v89: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 30 MiB/s wr, 1.67k op/s 2024-02-26T06:26:17.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:16 smithi115 ceph-mon[36426]: pgmap v89: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 30 MiB/s wr, 1.67k op/s 2024-02-26T06:26:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.018+0000 7fb186a00640 1 -- 172.21.15.49:0/2542765477 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18002cbd0 con 0x7fb188106490 2024-02-26T06:26:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.019+0000 7fb18e800640 1 -- 172.21.15.49:0/2542765477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb188106490 msgr2=0x7fb18810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.019+0000 7fb18e800640 1 --2- 172.21.15.49:0/2542765477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb188106490 0x7fb18810e060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb180009900 tx=0x7fb18002bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.019+0000 7fb18e800640 1 -- 172.21.15.49:0/2542765477 shutdown_connections 2024-02-26T06:26:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.019+0000 7fb18e800640 1 --2- 172.21.15.49:0/2542765477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb188101d70 0x7fb188105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.019+0000 7fb18e800640 1 --2- 172.21.15.49:0/2542765477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb188106490 0x7fb18810e060 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.019+0000 7fb18e800640 1 --2- 172.21.15.49:0/2542765477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1881013c0 0x7fb1881017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.019+0000 7fb18e800640 1 -- 172.21.15.49:0/2542765477 >> 172.21.15.49:0/2542765477 conn(0x7fb1880fcc40 msgr2=0x7fb1880ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.020+0000 7fb18e800640 1 -- 172.21.15.49:0/2542765477 shutdown_connections 2024-02-26T06:26:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.020+0000 7fb18e800640 1 -- 172.21.15.49:0/2542765477 wait complete. 2024-02-26T06:26:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.022+0000 7fb18e800640 1 Processor -- start 2024-02-26T06:26:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.022+0000 7fb18e800640 1 -- start start 2024-02-26T06:26:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.022+0000 7fb18e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1881013c0 0x7fb188073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.022+0000 7fb18e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb188106490 0x7fb1880739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb187400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1881013c0 0x7fb188073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb18e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18819f4a0 0x7fb188073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb18e800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1881aa2b0 con 0x7fb188106490 2024-02-26T06:26:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb18e800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1881aa420 con 0x7fb1881013c0 2024-02-26T06:26:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb18e800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1881aa5e0 con 0x7fb18819f4a0 2024-02-26T06:26:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb187400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1881013c0 0x7fb188073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38078/0 (socket says 172.21.15.49:38078) 2024-02-26T06:26:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb187400640 1 -- 172.21.15.49:0/2584139679 learned_addr learned my addr 172.21.15.49:0/2584139679 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb187400640 1 -- 172.21.15.49:0/2584139679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18819f4a0 msgr2=0x7fb188073f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb187400640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18819f4a0 0x7fb188073f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb187400640 1 -- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb188106490 msgr2=0x7fb1880739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb187400640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb188106490 0x7fb1880739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.023+0000 7fb187400640 1 -- 172.21.15.49:0/2584139679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180009560 con 0x7fb1881013c0 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.024+0000 7fb187400640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1881013c0 0x7fb188073480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb17000d950 tx=0x7fb17000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.024+0000 7fb184c00640 1 -- 172.21.15.49:0/2584139679 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170009c30 con 0x7fb1881013c0 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.024+0000 7fb184c00640 1 -- 172.21.15.49:0/2584139679 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb170009d90 con 0x7fb1881013c0 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.024+0000 7fb184c00640 1 -- 172.21.15.49:0/2584139679 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000f7e0 con 0x7fb1881013c0 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.024+0000 7fb18e800640 1 -- 172.21.15.49:0/2584139679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1881aa8c0 con 0x7fb1881013c0 2024-02-26T06:26:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.024+0000 7fb18e800640 1 -- 172.21.15.49:0/2584139679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1881aad80 con 0x7fb1881013c0 2024-02-26T06:26:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.031+0000 7fb191614740 1 Processor -- start 2024-02-26T06:26:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.032+0000 7fb191614740 1 -- start start 2024-02-26T06:26:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.032+0000 7fb191614740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c01557840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.032+0000 7fb191614740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01557d80 0x556c01558200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.032+0000 7fb191614740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01561300 0x556c015636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.032+0000 7fb191614740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c015135f0 con 0x556c01558af0 2024-02-26T06:26:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.032+0000 7fb191614740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c014c5890 con 0x556c01557d80 2024-02-26T06:26:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.032+0000 7fb191614740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c014c2dc0 con 0x556c01561300 2024-02-26T06:26:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.033+0000 7fb17ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01561300 0x556c015636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.033+0000 7fb17ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01561300 0x556c015636f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:35748/0 (socket says 172.21.15.49:35748) 2024-02-26T06:26:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.033+0000 7fb17ea00640 1 -- 172.21.15.49:0/552373037 learned_addr learned my addr 172.21.15.49:0/552373037 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.033+0000 7fb17ea00640 1 -- 172.21.15.49:0/552373037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01557d80 msgr2=0x556c01558200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:26:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.033+0000 7fb17ea00640 1 --2- 172.21.15.49:0/552373037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01557d80 0x556c01558200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.033+0000 7fb17ea00640 1 -- 172.21.15.49:0/552373037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 msgr2=0x556c01557840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.033+0000 7fb17ea00640 1 --2- 172.21.15.49:0/552373037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c01557840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.033+0000 7fb17ea00640 1 -- 172.21.15.49:0/552373037 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c0143fad0 con 0x556c01561300 2024-02-26T06:26:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.034+0000 7fb17ea00640 1 --2- 172.21.15.49:0/552373037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01561300 0x556c015636f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb178009d20 tx=0x7fb17800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3a3689b2bf3028 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.034+0000 7fb17cc00640 1 -- 172.21.15.49:0/552373037 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178004b20 con 0x556c01561300 2024-02-26T06:26:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.034+0000 7fb17cc00640 1 -- 172.21.15.49:0/552373037 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb178004c80 con 0x556c01561300 2024-02-26T06:26:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.034+0000 7fb17cc00640 1 -- 172.21.15.49:0/552373037 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178011710 con 0x556c01561300 2024-02-26T06:26:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.035+0000 7fb191614740 1 -- 172.21.15.49:0/552373037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01561300 msgr2=0x556c015636f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.035+0000 7fb191614740 1 --2- 172.21.15.49:0/552373037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01561300 0x556c015636f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb178009d20 tx=0x7fb17800e6c0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.035+0000 7fb184c00640 1 -- 172.21.15.49:0/2584139679 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb170010ad0 con 0x7fb1881013c0 2024-02-26T06:26:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.035+0000 7fb191614740 1 -- 172.21.15.49:0/552373037 shutdown_connections 2024-02-26T06:26:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.035+0000 7fb191614740 1 --2- 172.21.15.49:0/552373037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01557d80 0x556c01558200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.035+0000 7fb191614740 1 --2- 172.21.15.49:0/552373037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01561300 0x556c015636f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.035+0000 7fb191614740 1 --2- 172.21.15.49:0/552373037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c01557840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.036+0000 7fb191614740 1 -- 172.21.15.49:0/552373037 >> 172.21.15.49:0/552373037 conn(0x556c0151f480 msgr2=0x556c01552a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.037+0000 7fb191614740 1 -- 172.21.15.49:0/552373037 shutdown_connections 2024-02-26T06:26:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.037+0000 7fb191614740 1 -- 172.21.15.49:0/552373037 wait complete. 2024-02-26T06:26:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.039+0000 7fb191614740 1 -- start start 2024-02-26T06:26:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.039+0000 7fb191614740 1 Processor -- start 2024-02-26T06:26:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.039+0000 7fb184c00640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb16c0a22e0 0x7fb16c0a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.040+0000 7fb184c00640 1 -- 172.21.15.49:0/2584139679 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb1700cddd0 con 0x7fb1881013c0 2024-02-26T06:26:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.040+0000 7fb187e00640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb16c0a22e0 0x7fb16c0a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.042+0000 7fb191614740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01557d80 0x556c0155e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.042+0000 7fb191614740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c0155e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.042+0000 7fb191614740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.042+0000 7fb191614740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c014be5a0 con 0x556c01558af0 2024-02-26T06:26:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.042+0000 7fb191614740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01463b20 con 0x556c01561300 2024-02-26T06:26:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.042+0000 7fb191614740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c014637c0 con 0x556c01557d80 2024-02-26T06:26:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.042+0000 7fb191614740 10 client.0 _ll_register_callbacks cb 0x556c01477520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.042+0000 7fb187e00640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb16c0a22e0 0x7fb16c0a47a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb180036040 tx=0x7fb18002bc30 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.043+0000 7fb187400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01557d80 0x556c0155e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.043+0000 7fb17ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c0155e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.043+0000 7fb187e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.044+0000 7fb17ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c0155e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38332/0 (socket says 172.21.15.49:38332) 2024-02-26T06:26:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.044+0000 7fb187e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38084/0 (socket says 172.21.15.49:38084) 2024-02-26T06:26:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.044+0000 7fb17ea00640 1 -- 172.21.15.49:0/2850608551 learned_addr learned my addr 172.21.15.49:0/2850608551 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.043+0000 7fb187400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01557d80 0x556c0155e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:35754/0 (socket says 172.21.15.49:35754) 2024-02-26T06:26:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.045+0000 7fb187400640 1 -- 172.21.15.49:0/2850608551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 msgr2=0x556c0155eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.045+0000 7fb187400640 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.045+0000 7fb187400640 1 -- 172.21.15.49:0/2850608551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 msgr2=0x556c0155e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.045+0000 7fb187400640 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c0155e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.045+0000 7fb187400640 1 -- 172.21.15.49:0/2850608551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb178009980 con 0x556c01557d80 2024-02-26T06:26:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.045+0000 7fb17ea00640 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c0155e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.045+0000 7fb187e00640 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.046+0000 7fb187400640 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01557d80 0x556c0155e450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb1700cb4d0 tx=0x7fb17009a3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336041cf1b80aad5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.046+0000 7fb175600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.046+0000 7fb175600640 1 -- 172.21.15.49:0/2850608551 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17009e8c0 con 0x556c01557d80 2024-02-26T06:26:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.046+0000 7fb175600640 1 -- 172.21.15.49:0/2850608551 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb1700a2f10 con 0x556c01557d80 2024-02-26T06:26:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.047+0000 7fb191614740 1 -- 172.21.15.49:0/2850608551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c01490ca0 con 0x556c01557d80 2024-02-26T06:26:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.047+0000 7fb175600640 1 -- 172.21.15.49:0/2850608551 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1700cb750 con 0x556c01557d80 2024-02-26T06:26:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.048+0000 7fb191614740 10 client.26194 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.048+0000 7fb191614740 1 -- 172.21.15.49:0/2850608551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556c014bcc20 con 0x556c01557d80 2024-02-26T06:26:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb175600640 1 -- 172.21.15.49:0/2850608551 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556c01490ca0 con 0x556c01557d80 2024-02-26T06:26:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb175600640 1 -- 172.21.15.49:0/2850608551 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb1700b3090 con 0x556c01557d80 2024-02-26T06:26:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb191614740 10 client.26194 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb191614740 10 client.26194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb191614740 20 client.26194 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb191614740 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556c01571440 0x556c01573860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb191614740 4 client.26194 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb191614740 1 -- 172.21.15.49:0/2850608551 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c014b4cb0 con 0x556c01571440 2024-02-26T06:26:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.049+0000 7fb17ea00640 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556c01571440 0x556c01573860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.050+0000 7fb17ea00640 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556c01571440 0x556c01573860 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7b0a6325e1fa1f server_cookie=1fa8a6e3f4274c08 in_seq=0 out_seq=0 2024-02-26T06:26:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.050+0000 7fb175600640 10 client.26194 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.051+0000 7fb175600640 1 -- 172.21.15.49:0/2850608551 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c014b4cb0 con 0x556c01571440 2024-02-26T06:26:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.051+0000 7fb175600640 10 client.26194 handle_command_reply: tid=0 2024-02-26T06:26:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.051+0000 7fb191614740 1 client.26194 shutdown 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb191614740 1 -- 172.21.15.49:0/2850608551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01557d80 msgr2=0x556c0155e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb191614740 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01557d80 0x556c0155e450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb1700cb4d0 tx=0x7fb17009a3a0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb191614740 1 -- 172.21.15.49:0/2850608551 shutdown_connections 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb191614740 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556c01571440 0x556c01573860 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb191614740 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb191614740 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01558af0 0x556c0155e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb175600640 0 client.26194 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb191614740 1 --2- 172.21.15.49:0/2850608551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01557d80 0x556c0155e450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.052+0000 7fb191614740 1 -- 172.21.15.49:0/2850608551 >> 172.21.15.49:0/2850608551 conn(0x556c0151f480 msgr2=0x556c015547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.053+0000 7fb191614740 1 -- 172.21.15.49:0/2850608551 shutdown_connections 2024-02-26T06:26:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.053+0000 7fb191614740 1 -- 172.21.15.49:0/2850608551 wait complete. 2024-02-26T06:26:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.054+0000 7fb191614740 20 client.26194 trim_cache size 0 max 16384 2024-02-26T06:26:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.094+0000 7fb191614740 1 Processor -- start 2024-02-26T06:26:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.094+0000 7fb191614740 1 -- start start 2024-02-26T06:26:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb191614740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01560b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb191614740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c015637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb191614740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c01568120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb191614740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01206e60 con 0x556c01557d80 2024-02-26T06:26:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb191614740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01578a80 con 0x556c01561300 2024-02-26T06:26:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb191614740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c01557370 con 0x556c01558af0 2024-02-26T06:26:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb17ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c015637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb17ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c015637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:35760/0 (socket says 172.21.15.49:35760) 2024-02-26T06:26:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.095+0000 7fb17ea00640 1 -- 172.21.15.49:0/2306866851 learned_addr learned my addr 172.21.15.49:0/2306866851 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.096+0000 7fb17ea00640 1 -- 172.21.15.49:0/2306866851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 msgr2=0x556c01568120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:26:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.096+0000 7fb187400640 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01560b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.096+0000 7fb187e00640 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c01568120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.097+0000 7fb17ea00640 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c01568120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.097+0000 7fb17ea00640 1 -- 172.21.15.49:0/2306866851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 msgr2=0x556c01560b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.097+0000 7fb17ea00640 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01560b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.097+0000 7fb17ea00640 1 -- 172.21.15.49:0/2306866851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c01459500 con 0x556c01558af0 2024-02-26T06:26:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.097+0000 7fb187e00640 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c01568120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.098+0000 7fb17ea00640 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c015637d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb178004a80 tx=0x7fb1780093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a79898498818eba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.098+0000 7fb177e00640 1 -- 172.21.15.49:0/2306866851 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17802e070 con 0x556c01558af0 2024-02-26T06:26:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.098+0000 7fb177e00640 1 -- 172.21.15.49:0/2306866851 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb178022040 con 0x556c01558af0 2024-02-26T06:26:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.098+0000 7fb177e00640 1 -- 172.21.15.49:0/2306866851 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178021040 con 0x556c01558af0 2024-02-26T06:26:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.099+0000 7fb191614740 1 -- 172.21.15.49:0/2306866851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 msgr2=0x556c015637d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.099+0000 7fb191614740 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c015637d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb178004a80 tx=0x7fb1780093c0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.100+0000 7fb191614740 1 -- 172.21.15.49:0/2306866851 shutdown_connections 2024-02-26T06:26:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.100+0000 7fb191614740 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c01568120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.100+0000 7fb191614740 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c015637d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.100+0000 7fb191614740 1 --2- 172.21.15.49:0/2306866851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01560b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.100+0000 7fb191614740 1 -- 172.21.15.49:0/2306866851 >> 172.21.15.49:0/2306866851 conn(0x556c0151f480 msgr2=0x556c015525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.100+0000 7fb191614740 1 -- 172.21.15.49:0/2306866851 shutdown_connections 2024-02-26T06:26:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.101+0000 7fb191614740 1 -- 172.21.15.49:0/2306866851 wait complete. 2024-02-26T06:26:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.101+0000 7fb191614740 1 -- start start 2024-02-26T06:26:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.102+0000 7fb191614740 1 Processor -- start 2024-02-26T06:26:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb191614740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01568130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb191614740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c01568670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb191614740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb191614740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0155b020 con 0x556c01557d80 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb191614740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0155b190 con 0x556c01561300 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb191614740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0155b350 con 0x556c01558af0 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb191614740 10 client.0 _ll_register_callbacks cb 0x7fb170010d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb187e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb187e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38100/0 (socket says 172.21.15.49:38100) 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.104+0000 7fb187e00640 1 -- 172.21.15.49:0/818078439 learned_addr learned my addr 172.21.15.49:0/818078439 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.105+0000 7fb17ea00640 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c01568670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.105+0000 7fb187e00640 1 -- 172.21.15.49:0/818078439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 msgr2=0x556c01568670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.105+0000 7fb187e00640 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c01568670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.105+0000 7fb187400640 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01568130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.105+0000 7fb187e00640 1 -- 172.21.15.49:0/818078439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 msgr2=0x556c01568130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.105+0000 7fb187e00640 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01568130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.105+0000 7fb187e00640 1 -- 172.21.15.49:0/818078439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c0143fad0 con 0x556c01561300 2024-02-26T06:26:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.106+0000 7fb187400640 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01568130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.106+0000 7fb187e00640 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155a920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb180035000 tx=0x7fb180009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a537ae29c03a4b28 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.106+0000 7fb175600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.107+0000 7fb175600640 1 -- 172.21.15.49:0/818078439 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180004df0 con 0x556c01561300 2024-02-26T06:26:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.107+0000 7fb175600640 1 -- 172.21.15.49:0/818078439 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb180004f50 con 0x556c01561300 2024-02-26T06:26:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.108+0000 7fb191614740 1 -- 172.21.15.49:0/818078439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c014e3d00 con 0x556c01561300 2024-02-26T06:26:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.108+0000 7fb175600640 1 -- 172.21.15.49:0/818078439 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb180005150 con 0x556c01561300 2024-02-26T06:26:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.108+0000 7fb191614740 10 client.26159 fetch_fsmap learned FSMap version 29 2024-02-26T06:26:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.108+0000 7fb191614740 1 -- 172.21.15.49:0/818078439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556c0147f290 con 0x556c01561300 2024-02-26T06:26:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.109+0000 7fb175600640 1 -- 172.21.15.49:0/818078439 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556c014e3d00 con 0x556c01561300 2024-02-26T06:26:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.109+0000 7fb175600640 1 -- 172.21.15.49:0/818078439 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb18003c070 con 0x556c01561300 2024-02-26T06:26:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.109+0000 7fb191614740 10 client.26159 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T06:26:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.109+0000 7fb191614740 10 client.26159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.109+0000 7fb191614740 20 client.26159 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.110+0000 7fb191614740 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556c015c8b70 0x556c015caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.110+0000 7fb191614740 4 client.26159 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:26:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.110+0000 7fb191614740 1 -- 172.21.15.49:0/818078439 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556c011be600 con 0x556c015c8b70 2024-02-26T06:26:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.110+0000 7fb187400640 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556c015c8b70 0x556c015caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.111+0000 7fb187400640 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556c015c8b70 0x556c015caf90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1c3310fa4be621 server_cookie=7b5a68b76bb9f251 in_seq=0 out_seq=0 2024-02-26T06:26:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.111+0000 7fb175600640 10 client.26159 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb175600640 1 -- 172.21.15.49:0/818078439 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556c011be600 con 0x556c015c8b70 2024-02-26T06:26:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb175600640 10 client.26159 handle_command_reply: tid=0 2024-02-26T06:26:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 client.26159 shutdown 2024-02-26T06:26:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 -- 172.21.15.49:0/818078439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 msgr2=0x556c0155a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155a920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb180035000 tx=0x7fb180009260 comp rx=0 tx=0).stop 2024-02-26T06:26:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 -- 172.21.15.49:0/818078439 shutdown_connections 2024-02-26T06:26:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556c015c8b70 0x556c015caf90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556c01561300 0x556c0155a920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c01558af0 0x556c01568670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 --2- 172.21.15.49:0/818078439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556c01557d80 0x556c01568130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb191614740 1 -- 172.21.15.49:0/818078439 >> 172.21.15.49:0/818078439 conn(0x556c0151f480 msgr2=0x556c0155cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.123+0000 7fb175600640 0 client.26159 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.124+0000 7fb191614740 1 -- 172.21.15.49:0/818078439 shutdown_connections 2024-02-26T06:26:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.124+0000 7fb191614740 1 -- 172.21.15.49:0/818078439 wait complete. 2024-02-26T06:26:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.124+0000 7fb191614740 20 client.26159 trim_cache size 0 max 16384 2024-02-26T06:26:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:26:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:26:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:26:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:26:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 -- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb16c0a22e0 msgr2=0x7fb16c0a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb16c0a22e0 0x7fb16c0a47a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb180036040 tx=0x7fb18002bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 -- 172.21.15.49:0/2584139679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1881013c0 msgr2=0x7fb188073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1881013c0 0x7fb188073480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb17000d950 tx=0x7fb17000de10 comp rx=0 tx=0).stop 2024-02-26T06:26:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 -- 172.21.15.49:0/2584139679 shutdown_connections 2024-02-26T06:26:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb16c0a22e0 0x7fb16c0a47a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18819f4a0 0x7fb188073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb188106490 0x7fb1880739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 --2- 172.21.15.49:0/2584139679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1881013c0 0x7fb188073480 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 -- 172.21.15.49:0/2584139679 >> 172.21.15.49:0/2584139679 conn(0x7fb1880fcc40 msgr2=0x7fb1880fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 -- 172.21.15.49:0/2584139679 shutdown_connections 2024-02-26T06:26:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:19.127+0000 7fb18e800640 1 -- 172.21.15.49:0/2584139679 wait complete. 2024-02-26T06:26:19.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:18 smithi049 ceph-mon[31696]: pgmap v90: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.38k op/s 2024-02-26T06:26:19.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:18 smithi080 ceph-mon[36206]: pgmap v90: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.38k op/s 2024-02-26T06:26:19.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:18 smithi115 ceph-mon[36426]: pgmap v90: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.38k op/s 2024-02-26T06:26:21.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:20 smithi049 ceph-mon[31696]: pgmap v91: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 1.67k op/s 2024-02-26T06:26:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 150.0 s, 3039.3 tps, lat 32.964 ms stddev 37.256 2024-02-26T06:26:21.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:20 smithi080 ceph-mon[36206]: pgmap v91: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 1.67k op/s 2024-02-26T06:26:21.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:20 smithi115 ceph-mon[36426]: pgmap v91: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 1.67k op/s 2024-02-26T06:26:23.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:22 smithi049 ceph-mon[31696]: pgmap v92: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 1.74k op/s 2024-02-26T06:26:23.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:22 smithi080 ceph-mon[36206]: pgmap v92: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 1.74k op/s 2024-02-26T06:26:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:22 smithi115 ceph-mon[36426]: pgmap v92: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 1.74k op/s 2024-02-26T06:26:25.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:24 smithi080 ceph-mon[36206]: pgmap v93: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 1.67k op/s 2024-02-26T06:26:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:24 smithi115 ceph-mon[36426]: pgmap v93: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 1.67k op/s 2024-02-26T06:26:25.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:24 smithi049 ceph-mon[31696]: pgmap v93: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 1.67k op/s 2024-02-26T06:26:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 155.0 s, 2683.2 tps, lat 37.404 ms stddev 43.181 2024-02-26T06:26:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:25 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:26.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:25 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:26.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:25 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:27.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:26 smithi080 ceph-mon[36206]: pgmap v94: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 1.97k op/s 2024-02-26T06:26:27.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:26 smithi049 ceph-mon[31696]: pgmap v94: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 1.97k op/s 2024-02-26T06:26:27.568 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:26 smithi115 ceph-mon[36426]: pgmap v94: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 1.97k op/s 2024-02-26T06:26:29.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:28 smithi080 ceph-mon[36206]: pgmap v95: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.2 MiB/s wr, 730 op/s 2024-02-26T06:26:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:28 smithi115 ceph-mon[36426]: pgmap v95: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.2 MiB/s wr, 730 op/s 2024-02-26T06:26:29.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:28 smithi049 ceph-mon[31696]: pgmap v95: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.2 MiB/s wr, 730 op/s 2024-02-26T06:26:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 160.0 s, 3102.2 tps, lat 32.358 ms stddev 34.671 2024-02-26T06:26:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:30 smithi080 ceph-mon[36206]: pgmap v96: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 997 op/s 2024-02-26T06:26:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:30 smithi115 ceph-mon[36426]: pgmap v96: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 997 op/s 2024-02-26T06:26:31.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:30 smithi049 ceph-mon[31696]: pgmap v96: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 997 op/s 2024-02-26T06:26:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.328+0000 7fc39d800640 1 -- 172.21.15.49:0/3273754083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc398102130 msgr2=0x7fc398102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.328+0000 7fc39d800640 1 --2- 172.21.15.49:0/3273754083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc398102130 0x7fc398102510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc388009900 tx=0x7fc38802bcc0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.329+0000 7fc39d800640 1 -- 172.21.15.49:0/3273754083 shutdown_connections 2024-02-26T06:26:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.329+0000 7fc39d800640 1 --2- 172.21.15.49:0/3273754083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc398102a50 0x7fc39810af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.329+0000 7fc39d800640 1 --2- 172.21.15.49:0/3273754083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc39810b550 0x7fc39810d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.329+0000 7fc39d800640 1 --2- 172.21.15.49:0/3273754083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc398102130 0x7fc398102510 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.329+0000 7fc39d800640 1 -- 172.21.15.49:0/3273754083 >> 172.21.15.49:0/3273754083 conn(0x7fc3980fb990 msgr2=0x7fc3980fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.329+0000 7fc39d800640 1 -- 172.21.15.49:0/3273754083 shutdown_connections 2024-02-26T06:26:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.330+0000 7fc39d800640 1 -- 172.21.15.49:0/3273754083 wait complete. 2024-02-26T06:26:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.332+0000 7fc39d800640 1 Processor -- start 2024-02-26T06:26:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.334+0000 7fc39d800640 1 -- start start 2024-02-26T06:26:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc39d800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc398102130 0x7fc39810a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc39d800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc398102a50 0x7fc39810aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc39d800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc39810b550 0x7fc398106c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc39d800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398107320 con 0x7fc39810b550 2024-02-26T06:26:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc39d800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398107490 con 0x7fc398102a50 2024-02-26T06:26:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc39d800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398107650 con 0x7fc398102130 2024-02-26T06:26:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc397400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc398102a50 0x7fc39810aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc397400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc398102a50 0x7fc39810aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34268/0 (socket says 172.21.15.49:34268) 2024-02-26T06:26:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.335+0000 7fc397400640 1 -- 172.21.15.49:0/2970578256 learned_addr learned my addr 172.21.15.49:0/2970578256 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.336+0000 7fc397e00640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc398102130 0x7fc39810a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.336+0000 7fc397400640 1 -- 172.21.15.49:0/2970578256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc398102130 msgr2=0x7fc39810a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.336+0000 7fc397400640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc398102130 0x7fc39810a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.336+0000 7fc397400640 1 -- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc39810b550 msgr2=0x7fc398106c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.336+0000 7fc397400640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc39810b550 0x7fc398106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.336+0000 7fc397400640 1 -- 172.21.15.49:0/2970578256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388009560 con 0x7fc398102a50 2024-02-26T06:26:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.338+0000 7fc397400640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc398102a50 0x7fc39810aea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc38400d750 tx=0x7fc38400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.338+0000 7fc394c00640 1 -- 172.21.15.49:0/2970578256 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384009a30 con 0x7fc398102a50 2024-02-26T06:26:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.339+0000 7fc394c00640 1 -- 172.21.15.49:0/2970578256 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc384005500 con 0x7fc398102a50 2024-02-26T06:26:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.339+0000 7fc394c00640 1 -- 172.21.15.49:0/2970578256 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384004dd0 con 0x7fc398102a50 2024-02-26T06:26:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.339+0000 7fc39d800640 1 -- 172.21.15.49:0/2970578256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3981aa520 con 0x7fc398102a50 2024-02-26T06:26:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.339+0000 7fc39d800640 1 -- 172.21.15.49:0/2970578256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3981aa890 con 0x7fc398102a50 2024-02-26T06:26:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.351+0000 7fc394c00640 1 -- 172.21.15.49:0/2970578256 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc384010460 con 0x7fc398102a50 2024-02-26T06:26:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.354+0000 7fc3a1da0740 1 Processor -- start 2024-02-26T06:26:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.355+0000 7fc3a1da0740 1 -- start start 2024-02-26T06:26:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.355+0000 7fc394c00640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc37c0a2370 0x7fc37c0a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.355+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12ddd30 0x563fc12dca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.355+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12dcfc0 0x563fc12dd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.356+0000 7fc394c00640 1 -- 172.21.15.49:0/2970578256 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc3840046c0 con 0x7fc398102a50 2024-02-26T06:26:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.357+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.357+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc37c0a2370 0x7fc37c0a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.357+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc12985f0 con 0x563fc12dcfc0 2024-02-26T06:26:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.357+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc124a890 con 0x563fc12e6540 2024-02-26T06:26:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.357+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc1247dc0 con 0x563fc12ddd30 2024-02-26T06:26:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.356+0000 7fc397e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12ddd30 0x563fc12dca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.358+0000 7fc397e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12ddd30 0x563fc12dca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58526/0 (socket says 172.21.15.49:58526) 2024-02-26T06:26:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.358+0000 7fc397400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12dcfc0 0x563fc12dd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.358+0000 7fc397e00640 1 -- 172.21.15.49:0/1965267696 learned_addr learned my addr 172.21.15.49:0/1965267696 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.358+0000 7fc39ce00640 1 --2- 172.21.15.49:0/1965267696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.359+0000 7fc397e00640 1 -- 172.21.15.49:0/1965267696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 msgr2=0x563fc12e8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.359+0000 7fc397e00640 1 --2- 172.21.15.49:0/1965267696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.359+0000 7fc397e00640 1 -- 172.21.15.49:0/1965267696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12dcfc0 msgr2=0x563fc12dd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.359+0000 7fc397e00640 1 --2- 172.21.15.49:0/1965267696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12dcfc0 0x563fc12dd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.359+0000 7fc397e00640 1 -- 172.21.15.49:0/1965267696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc11c4ad0 con 0x563fc12ddd30 2024-02-26T06:26:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.359+0000 7fc397e00640 1 --2- 172.21.15.49:0/1965267696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12ddd30 0x563fc12dca80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc388009230 tx=0x7fc388004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7101e66ea3aebdd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.360+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc37c0a2370 0x7fc37c0a4830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3900104a0 tx=0x7fc390010960 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.360+0000 7fc38d600640 1 -- 172.21.15.49:0/1965267696 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880049f0 con 0x563fc12ddd30 2024-02-26T06:26:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.360+0000 7fc38d600640 1 -- 172.21.15.49:0/1965267696 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc38802c870 con 0x563fc12ddd30 2024-02-26T06:26:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.360+0000 7fc38d600640 1 -- 172.21.15.49:0/1965267696 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880050e0 con 0x563fc12ddd30 2024-02-26T06:26:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.363+0000 7fc3a1da0740 1 -- 172.21.15.49:0/1965267696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12ddd30 msgr2=0x563fc12dca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.363+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/1965267696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12ddd30 0x563fc12dca80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc388009230 tx=0x7fc388004740 comp rx=0 tx=0).stop 2024-02-26T06:26:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.363+0000 7fc3a1da0740 1 -- 172.21.15.49:0/1965267696 shutdown_connections 2024-02-26T06:26:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.363+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/1965267696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.363+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/1965267696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12dcfc0 0x563fc12dd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.363+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/1965267696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12ddd30 0x563fc12dca80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.363+0000 7fc3a1da0740 1 -- 172.21.15.49:0/1965267696 >> 172.21.15.49:0/1965267696 conn(0x563fc12a4480 msgr2=0x563fc12d7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.365+0000 7fc3a1da0740 1 -- 172.21.15.49:0/1965267696 shutdown_connections 2024-02-26T06:26:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.365+0000 7fc3a1da0740 1 -- 172.21.15.49:0/1965267696 wait complete. 2024-02-26T06:26:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.365+0000 7fc3a1da0740 1 -- start start 2024-02-26T06:26:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.366+0000 7fc3a1da0740 1 Processor -- start 2024-02-26T06:26:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.369+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.369+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 0x563fc12e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.369+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.369+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc12435a0 con 0x563fc12ddd30 2024-02-26T06:26:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.369+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc11e8b20 con 0x563fc12e6540 2024-02-26T06:26:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.369+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc11e87c0 con 0x563fc12dcfc0 2024-02-26T06:26:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.369+0000 7fc3a1da0740 10 client.0 _ll_register_callbacks cb 0x563fc11fc520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.370+0000 7fc397e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.371+0000 7fc397e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12e35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58540/0 (socket says 172.21.15.49:58540) 2024-02-26T06:26:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.371+0000 7fc397e00640 1 -- 172.21.15.49:0/2414544649 learned_addr learned my addr 172.21.15.49:0/2414544649 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.371+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.371+0000 7fc397400640 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 0x563fc12e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.372+0000 7fc39ce00640 1 -- 172.21.15.49:0/2414544649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 msgr2=0x563fc12e35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.372+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12e35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.372+0000 7fc39ce00640 1 -- 172.21.15.49:0/2414544649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 msgr2=0x563fc12e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.372+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 0x563fc12e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.372+0000 7fc39ce00640 1 -- 172.21.15.49:0/2414544649 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39810e670 con 0x563fc12e6540 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.373+0000 7fc382000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.373+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e4050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x563fc12dda60 tx=0x7fc390016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2dea8fe44da2bcc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.373+0000 7fc382000640 1 -- 172.21.15.49:0/2414544649 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39000ed90 con 0x563fc12e6540 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.373+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2414544649 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fc1215ca0 con 0x563fc12e6540 2024-02-26T06:26:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.373+0000 7fc382000640 1 -- 172.21.15.49:0/2414544649 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc390026410 con 0x563fc12e6540 2024-02-26T06:26:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.374+0000 7fc382000640 1 -- 172.21.15.49:0/2414544649 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc39000ed90 con 0x563fc12e6540 2024-02-26T06:26:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.374+0000 7fc3a1da0740 10 client.26171 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.374+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2414544649 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563fc1241c20 con 0x563fc12e6540 2024-02-26T06:26:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc382000640 1 -- 172.21.15.49:0/2414544649 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563fc1215ca0 con 0x563fc12e6540 2024-02-26T06:26:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc382000640 1 -- 172.21.15.49:0/2414544649 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc39002d070 con 0x563fc12e6540 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc3a1da0740 10 client.26171 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc3a1da0740 10 client.26171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc3a1da0740 20 client.26171 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563fc12f65f0 0x563fc12f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc3a1da0740 4 client.26171 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2414544649 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fc1239cb0 con 0x563fc12f65f0 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.375+0000 7fc397e00640 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563fc12f65f0 0x563fc12f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.376+0000 7fc397e00640 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563fc12f65f0 0x563fc12f8a10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41ec75950ec38ea server_cookie=482a8e0822fc76ef in_seq=0 out_seq=0 2024-02-26T06:26:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.376+0000 7fc382000640 10 client.26171 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.377+0000 7fc382000640 1 -- 172.21.15.49:0/2414544649 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fc1239cb0 con 0x563fc12f65f0 2024-02-26T06:26:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.377+0000 7fc382000640 10 client.26171 handle_command_reply: tid=0 2024-02-26T06:26:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.377+0000 7fc3a1da0740 1 client.26171 shutdown 2024-02-26T06:26:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 msgr2=0x563fc12e4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e4050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x563fc12dda60 tx=0x7fc390016000 comp rx=0 tx=0).stop 2024-02-26T06:26:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2414544649 shutdown_connections 2024-02-26T06:26:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563fc12f65f0 0x563fc12f8a10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12e6540 0x563fc12e4050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 0x563fc12e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2414544649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12e35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2414544649 >> 172.21.15.49:0/2414544649 conn(0x563fc12a4480 msgr2=0x563fc12d9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc382000640 0 client.26171 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.378+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2414544649 shutdown_connections 2024-02-26T06:26:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.379+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2414544649 wait complete. 2024-02-26T06:26:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.379+0000 7fc3a1da0740 20 client.26171 trim_cache size 0 max 16384 2024-02-26T06:26:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.402+0000 7fc3a1da0740 1 Processor -- start 2024-02-26T06:26:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.402+0000 7fc3a1da0740 1 -- start start 2024-02-26T06:26:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.402+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12ed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12ddd30 0x563fc12ed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12e6540 0x563fc12d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc0f8be60 con 0x563fc12e6540 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc1215ca0 con 0x563fc12ddd30 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc12f29c0 con 0x563fc12dcfc0 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc397e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc397e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12ed300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58550/0 (socket says 172.21.15.49:58550) 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc397e00640 1 -- 172.21.15.49:0/2566505912 learned_addr learned my addr 172.21.15.49:0/2566505912 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.403+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12e6540 0x563fc12d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.404+0000 7fc397400640 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12ddd30 0x563fc12ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.404+0000 7fc397e00640 1 -- 172.21.15.49:0/2566505912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12ddd30 msgr2=0x563fc12ed840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.404+0000 7fc397e00640 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12ddd30 0x563fc12ed840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.404+0000 7fc397e00640 1 -- 172.21.15.49:0/2566505912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12e6540 msgr2=0x563fc12d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.404+0000 7fc397e00640 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12e6540 0x563fc12d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.404+0000 7fc397e00640 1 -- 172.21.15.49:0/2566505912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc11de500 con 0x563fc12dcfc0 2024-02-26T06:26:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.405+0000 7fc397e00640 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12ed300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc38802c3b0 tx=0x7fc38802c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38076831fca102e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.405+0000 7fc38cc00640 1 -- 172.21.15.49:0/2566505912 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38804fd70 con 0x563fc12dcfc0 2024-02-26T06:26:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.405+0000 7fc38cc00640 1 -- 172.21.15.49:0/2566505912 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc388004b10 con 0x563fc12dcfc0 2024-02-26T06:26:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.405+0000 7fc38cc00640 1 -- 172.21.15.49:0/2566505912 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38804f5a0 con 0x563fc12dcfc0 2024-02-26T06:26:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.406+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2566505912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 msgr2=0x563fc12ed300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.407+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12ed300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc38802c3b0 tx=0x7fc38802c340 comp rx=0 tx=0).stop 2024-02-26T06:26:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.407+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2566505912 shutdown_connections 2024-02-26T06:26:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.407+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12ddd30 0x563fc12ed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.407+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12e6540 0x563fc12d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.407+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2566505912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12dcfc0 0x563fc12ed300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.407+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2566505912 >> 172.21.15.49:0/2566505912 conn(0x563fc12a4480 msgr2=0x563fc12de900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.407+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2566505912 shutdown_connections 2024-02-26T06:26:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.407+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2566505912 wait complete. 2024-02-26T06:26:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.408+0000 7fc3a1da0740 1 -- start start 2024-02-26T06:26:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.408+0000 7fc3a1da0740 1 Processor -- start 2024-02-26T06:26:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.409+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12dcfc0 0x563fc12ed5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.409+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 0x563fc12e0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.409+0000 7fc3a1da0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12e6540 0x563fc12e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.409+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc12e1480 con 0x563fc12ddd30 2024-02-26T06:26:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.409+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc12e15f0 con 0x563fc12dcfc0 2024-02-26T06:26:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.409+0000 7fc3a1da0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc12df840 con 0x563fc12e6540 2024-02-26T06:26:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.409+0000 7fc3a1da0740 10 client.0 _ll_register_callbacks cb 0x7fc39000c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc397e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12dcfc0 0x563fc12ed5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc397e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12dcfc0 0x563fc12ed5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34298/0 (socket says 172.21.15.49:34298) 2024-02-26T06:26:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc397e00640 1 -- 172.21.15.49:0/2235545416 learned_addr learned my addr 172.21.15.49:0/2235545416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12e6540 0x563fc12e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc397400640 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 0x563fc12e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc39ce00640 1 -- 172.21.15.49:0/2235545416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12dcfc0 msgr2=0x563fc12ed5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12dcfc0 0x563fc12ed5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc39ce00640 1 -- 172.21.15.49:0/2235545416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 msgr2=0x563fc12e0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 0x563fc12e0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.410+0000 7fc39ce00640 1 -- 172.21.15.49:0/2235545416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fc11c4ad0 con 0x563fc12e6540 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.411+0000 7fc39ce00640 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12e6540 0x563fc12e0e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc39000a480 tx=0x7fc39002e860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffabb99747a004af server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.412+0000 7fc382000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.412+0000 7fc382000640 1 -- 172.21.15.49:0/2235545416 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39000a7b0 con 0x563fc12e6540 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.412+0000 7fc382000640 1 -- 172.21.15.49:0/2235545416 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc39002ea50 con 0x563fc12e6540 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.412+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2235545416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fc1268d00 con 0x563fc12e6540 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.413+0000 7fc382000640 1 -- 172.21.15.49:0/2235545416 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc39000ce60 con 0x563fc12e6540 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc3a1da0740 10 client.26215 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2235545416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563fc1204290 con 0x563fc12e6540 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc382000640 1 -- 172.21.15.49:0/2235545416 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563fc1268d00 con 0x563fc12e6540 2024-02-26T06:26:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc382000640 1 -- 172.21.15.49:0/2235545416 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc39002d040 con 0x563fc12e6540 2024-02-26T06:26:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc3a1da0740 10 client.26215 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc3a1da0740 10 client.26215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc3a1da0740 20 client.26215 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563fc134d150 0x563fc134f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc3a1da0740 4 client.26215 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:26:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.414+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2235545416 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563fc0f43600 con 0x563fc134d150 2024-02-26T06:26:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.415+0000 7fc397e00640 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563fc134d150 0x563fc134f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.415+0000 7fc397e00640 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563fc134d150 0x563fc134f570 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c13366f129fd8c server_cookie=cf81ea437671a75a in_seq=0 out_seq=0 2024-02-26T06:26:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.415+0000 7fc382000640 10 client.26215 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.426+0000 7fc382000640 1 -- 172.21.15.49:0/2235545416 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563fc0f43600 con 0x563fc134d150 2024-02-26T06:26:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.426+0000 7fc382000640 10 client.26215 handle_command_reply: tid=0 2024-02-26T06:26:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.426+0000 7fc3a1da0740 1 client.26215 shutdown 2024-02-26T06:26:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.426+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2235545416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12e6540 msgr2=0x563fc12e0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.426+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12e6540 0x563fc12e0e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc39000a480 tx=0x7fc39002e860 comp rx=0 tx=0).stop 2024-02-26T06:26:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2235545416 shutdown_connections 2024-02-26T06:26:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563fc134d150 0x563fc134f570 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563fc12e6540 0x563fc12e0e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563fc12ddd30 0x563fc12e0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc3a1da0740 1 --2- 172.21.15.49:0/2235545416 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc12dcfc0 0x563fc12ed5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2235545416 >> 172.21.15.49:0/2235545416 conn(0x563fc12a4480 msgr2=0x563fc12d77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc382000640 0 client.26215 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2235545416 shutdown_connections 2024-02-26T06:26:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.427+0000 7fc3a1da0740 1 -- 172.21.15.49:0/2235545416 wait complete. 2024-02-26T06:26:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.428+0000 7fc3a1da0740 20 client.26215 trim_cache size 0 max 16384 2024-02-26T06:26:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:26:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:26:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:26:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:26:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 -- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc37c0a2370 msgr2=0x7fc37c0a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc37c0a2370 0x7fc37c0a4830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3900104a0 tx=0x7fc390010960 comp rx=0 tx=0).stop 2024-02-26T06:26:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 -- 172.21.15.49:0/2970578256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc398102a50 msgr2=0x7fc39810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc398102a50 0x7fc39810aea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc38400d750 tx=0x7fc38400dc10 comp rx=0 tx=0).stop 2024-02-26T06:26:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 -- 172.21.15.49:0/2970578256 shutdown_connections 2024-02-26T06:26:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc37c0a2370 0x7fc37c0a4830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc398102a50 0x7fc39810aea0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc39810b550 0x7fc398106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 --2- 172.21.15.49:0/2970578256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc398102130 0x7fc39810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 -- 172.21.15.49:0/2970578256 >> 172.21.15.49:0/2970578256 conn(0x7fc3980fb990 msgr2=0x7fc39810bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 -- 172.21.15.49:0/2970578256 shutdown_connections 2024-02-26T06:26:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:32.430+0000 7fc39d800640 1 -- 172.21.15.49:0/2970578256 wait complete. 2024-02-26T06:26:33.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:32 smithi080 ceph-mon[36206]: pgmap v97: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 820 op/s 2024-02-26T06:26:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:32 smithi115 ceph-mon[36426]: pgmap v97: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 820 op/s 2024-02-26T06:26:33.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:32 smithi049 ceph-mon[31696]: pgmap v97: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 820 op/s 2024-02-26T06:26:33.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:26:33.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:26:33.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 -- 172.21.15.49:0/1425236681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86901021b0 msgr2=0x7f8690102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 --2- 172.21.15.49:0/1425236681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86901021b0 0x7f8690102590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8680009900 tx=0x7f868002bcd0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 -- 172.21.15.49:0/1425236681 shutdown_connections 2024-02-26T06:26:33.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 --2- 172.21.15.49:0/1425236681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f869010b5d0 0x7f869010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 --2- 172.21.15.49:0/1425236681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8690102ad0 0x7f869010afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 --2- 172.21.15.49:0/1425236681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86901021b0 0x7f8690102590 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 -- 172.21.15.49:0/1425236681 >> 172.21.15.49:0/1425236681 conn(0x7f86900fb9d0 msgr2=0x7f86900fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:33.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 -- 172.21.15.49:0/1425236681 shutdown_connections 2024-02-26T06:26:33.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.731+0000 7f868f200640 1 -- 172.21.15.49:0/1425236681 wait complete. 2024-02-26T06:26:33.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.733+0000 7f868f200640 1 Processor -- start 2024-02-26T06:26:33.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.733+0000 7f868f200640 1 -- start start 2024-02-26T06:26:33.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.734+0000 7f868f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86901021b0 0x7f869019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.734+0000 7f868f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8690102ad0 0x7f869019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.734+0000 7f868f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869010b5d0 0x7f869010a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.734+0000 7f868f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869010a830 con 0x7f8690102ad0 2024-02-26T06:26:33.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.734+0000 7f868f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869010a9a0 con 0x7f86901021b0 2024-02-26T06:26:33.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.734+0000 7f868f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869010ab60 con 0x7f869010b5d0 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.734+0000 7f868de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86901021b0 0x7f869019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.734+0000 7f868e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869010b5d0 0x7f869010a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.735+0000 7f868e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869010b5d0 0x7f869010a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58570/0 (socket says 172.21.15.49:58570) 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.735+0000 7f868e800640 1 -- 172.21.15.49:0/2185178134 learned_addr learned my addr 172.21.15.49:0/2185178134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.735+0000 7f868d400640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8690102ad0 0x7f869019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.735+0000 7f868e800640 1 -- 172.21.15.49:0/2185178134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86901021b0 msgr2=0x7f869019e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.735+0000 7f868e800640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86901021b0 0x7f869019e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.735+0000 7f868e800640 1 -- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8690102ad0 msgr2=0x7f869019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.735+0000 7f868e800640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8690102ad0 0x7f869019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.735+0000 7f868e800640 1 -- 172.21.15.49:0/2185178134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680009560 con 0x7f869010b5d0 2024-02-26T06:26:33.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.736+0000 7f868e800640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869010b5d0 0x7f869010a0a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8688009fd0 tx=0x7f868800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:33.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.736+0000 7f8686a00640 1 -- 172.21.15.49:0/2185178134 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800bea0 con 0x7f869010b5d0 2024-02-26T06:26:33.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.736+0000 7f8686a00640 1 -- 172.21.15.49:0/2185178134 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8688003e50 con 0x7f869010b5d0 2024-02-26T06:26:33.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.736+0000 7f8686a00640 1 -- 172.21.15.49:0/2185178134 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688010750 con 0x7f869010b5d0 2024-02-26T06:26:33.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.736+0000 7f868f200640 1 -- 172.21.15.49:0/2185178134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869010ae40 con 0x7f869010b5d0 2024-02-26T06:26:33.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.737+0000 7f868f200640 1 -- 172.21.15.49:0/2185178134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869019a070 con 0x7f869010b5d0 2024-02-26T06:26:33.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.740+0000 7f8686a00640 1 -- 172.21.15.49:0/2185178134 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8688004340 con 0x7f869010b5d0 2024-02-26T06:26:33.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.741+0000 7f8686a00640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86740a22b0 0x7f86740a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.741+0000 7f8697fdc740 1 Processor -- start 2024-02-26T06:26:33.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.741+0000 7f8697fdc740 1 -- start start 2024-02-26T06:26:33.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f8686a00640 1 -- 172.21.15.49:0/2185178134 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f86880ce0c0 con 0x7f869010b5d0 2024-02-26T06:26:33.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe82a140 0x557ebe828e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe8298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe834bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f8697fdc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe798120 con 0x557ebe82a140 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f8697fdc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe795650 con 0x557ebe8327d0 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f8697fdc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe790e30 con 0x557ebe829460 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f868de00640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86740a22b0 0x7f86740a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.742+0000 7f868e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe834bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe8298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe834bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34342/0 (socket says 172.21.15.49:34342) 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868e800640 1 -- 172.21.15.49:0/3986373419 learned_addr learned my addr 172.21.15.49:0/3986373419 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868de00640 1 --2- 172.21.15.49:0/3986373419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe82a140 0x557ebe828e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868e800640 1 -- 172.21.15.49:0/3986373419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 msgr2=0x557ebe8298e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868e800640 1 --2- 172.21.15.49:0/3986373419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe8298e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868e800640 1 -- 172.21.15.49:0/3986373419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe82a140 msgr2=0x557ebe828e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868e800640 1 --2- 172.21.15.49:0/3986373419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe82a140 0x557ebe828e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868e800640 1 -- 172.21.15.49:0/3986373419 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebe74a030 con 0x557ebe8327d0 2024-02-26T06:26:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.743+0000 7f868e800640 1 --2- 172.21.15.49:0/3986373419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe834bc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f868800dfd0 tx=0x7f868809a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eee905284d29f90 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.744+0000 7f868de00640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86740a22b0 0x7f86740a4770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8680006010 tx=0x7f868002c6d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.744+0000 7f867b400640 1 -- 172.21.15.49:0/3986373419 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868809ebb0 con 0x557ebe8327d0 2024-02-26T06:26:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.744+0000 7f867b400640 1 -- 172.21.15.49:0/3986373419 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86880a3200 con 0x557ebe8327d0 2024-02-26T06:26:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.745+0000 7f867b400640 1 -- 172.21.15.49:0/3986373419 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86880a2d70 con 0x557ebe8327d0 2024-02-26T06:26:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.746+0000 7f8697fdc740 1 -- 172.21.15.49:0/3986373419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 msgr2=0x557ebe834bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.746+0000 7f8697fdc740 1 --2- 172.21.15.49:0/3986373419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe834bc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f868800dfd0 tx=0x7f868809a480 comp rx=0 tx=0).stop 2024-02-26T06:26:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.746+0000 7f8697fdc740 1 -- 172.21.15.49:0/3986373419 shutdown_connections 2024-02-26T06:26:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.746+0000 7f8697fdc740 1 --2- 172.21.15.49:0/3986373419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe834bc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.746+0000 7f8697fdc740 1 --2- 172.21.15.49:0/3986373419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe8298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.746+0000 7f8697fdc740 1 --2- 172.21.15.49:0/3986373419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe82a140 0x557ebe828e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.746+0000 7f8697fdc740 1 -- 172.21.15.49:0/3986373419 >> 172.21.15.49:0/3986373419 conn(0x557ebe7f0570 msgr2=0x557ebe823eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.748+0000 7f8697fdc740 1 -- 172.21.15.49:0/3986373419 shutdown_connections 2024-02-26T06:26:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.748+0000 7f8697fdc740 1 -- 172.21.15.49:0/3986373419 wait complete. 2024-02-26T06:26:33.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.749+0000 7f8697fdc740 1 -- start start 2024-02-26T06:26:33.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.750+0000 7f8697fdc740 1 Processor -- start 2024-02-26T06:26:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.750+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe82f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe8327d0 0x557ebe830210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f8697fdc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe735e90 con 0x557ebe8327d0 2024-02-26T06:26:33.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f8697fdc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe735b30 con 0x557ebe82a140 2024-02-26T06:26:33.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe82f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f8697fdc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe4ff1c0 con 0x557ebe829460 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe82f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58588/0 (socket says 172.21.15.49:58588) 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868de00640 1 -- 172.21.15.49:0/160495812 learned_addr learned my addr 172.21.15.49:0/160495812 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f8697fdc740 10 client.0 _ll_register_callbacks cb 0x557ebe749930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868d400640 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868de00640 1 -- 172.21.15.49:0/160495812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 msgr2=0x557ebe82fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868de00640 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868de00640 1 -- 172.21.15.49:0/160495812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe8327d0 msgr2=0x557ebe830210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868de00640 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe8327d0 0x557ebe830210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.751+0000 7f868de00640 1 -- 172.21.15.49:0/160495812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869019a070 con 0x557ebe829460 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.752+0000 7f868de00640 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe82f790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f868002cc30 tx=0x7f8680005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d5d106adc71641 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.752+0000 7f866fe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.752+0000 7f866fe00640 1 -- 172.21.15.49:0/160495812 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680004a70 con 0x557ebe829460 2024-02-26T06:26:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.752+0000 7f866fe00640 1 -- 172.21.15.49:0/160495812 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f8680048040 con 0x557ebe829460 2024-02-26T06:26:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.752+0000 7f8697fdc740 1 -- 172.21.15.49:0/160495812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ebe4907e0 con 0x557ebe829460 2024-02-26T06:26:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.753+0000 7f866fe00640 1 -- 172.21.15.49:0/160495812 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8680005000 con 0x557ebe829460 2024-02-26T06:26:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.753+0000 7f8697fdc740 10 client.26227 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.753+0000 7f8697fdc740 1 -- 172.21.15.49:0/160495812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557ebe78f4b0 con 0x557ebe829460 2024-02-26T06:26:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.753+0000 7f866fe00640 1 -- 172.21.15.49:0/160495812 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557ebe4907e0 con 0x557ebe829460 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.753+0000 7f866fe00640 1 -- 172.21.15.49:0/160495812 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f868003c030 con 0x557ebe829460 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.754+0000 7f8697fdc740 10 client.26227 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.754+0000 7f8697fdc740 10 client.26227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.754+0000 7f8697fdc740 20 client.26227 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.754+0000 7f8697fdc740 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ebe842980 0x557ebe844da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.754+0000 7f8697fdc740 4 client.26227 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.754+0000 7f8697fdc740 1 -- 172.21.15.49:0/160495812 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ebe4cd720 con 0x557ebe842980 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.754+0000 7f868d400640 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ebe842980 0x557ebe844da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.755+0000 7f868d400640 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ebe842980 0x557ebe844da0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bab3d9b00087a9d server_cookie=3527011dd2d9667e in_seq=0 out_seq=0 2024-02-26T06:26:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.755+0000 7f866fe00640 10 client.26227 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.755+0000 7f866fe00640 1 -- 172.21.15.49:0/160495812 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ebe4cd720 con 0x557ebe842980 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.755+0000 7f866fe00640 10 client.26227 handle_command_reply: tid=0 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.755+0000 7f8697fdc740 1 client.26227 shutdown 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 -- 172.21.15.49:0/160495812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 msgr2=0x557ebe82f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe82f790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f868002cc30 tx=0x7f8680005e40 comp rx=0 tx=0).stop 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 -- 172.21.15.49:0/160495812 shutdown_connections 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ebe842980 0x557ebe844da0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f866fe00640 0 client.26227 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe8327d0 0x557ebe830210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 --2- 172.21.15.49:0/160495812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe829460 0x557ebe82f790 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 -- 172.21.15.49:0/160495812 >> 172.21.15.49:0/160495812 conn(0x557ebe7f0570 msgr2=0x557ebe825c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 -- 172.21.15.49:0/160495812 shutdown_connections 2024-02-26T06:26:33.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 1 -- 172.21.15.49:0/160495812 wait complete. 2024-02-26T06:26:33.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.756+0000 7f8697fdc740 20 client.26227 trim_cache size 0 max 16384 2024-02-26T06:26:33.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.789+0000 7f8697fdc740 1 Processor -- start 2024-02-26T06:26:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.789+0000 7f8697fdc740 1 -- start start 2024-02-26T06:26:33.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.790+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 0x557ebe831ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.790+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe82a140 0x557ebe830030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.790+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe839730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.790+0000 7f8697fdc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe83e750 con 0x557ebe829460 2024-02-26T06:26:33.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.790+0000 7f8697fdc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe83edc0 con 0x557ebe8327d0 2024-02-26T06:26:33.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.790+0000 7f8697fdc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe81e300 con 0x557ebe82a140 2024-02-26T06:26:33.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe839730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe839730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34368/0 (socket says 172.21.15.49:34368) 2024-02-26T06:26:33.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868e800640 1 -- 172.21.15.49:0/2621965208 learned_addr learned my addr 172.21.15.49:0/2621965208 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:33.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868d400640 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe82a140 0x557ebe830030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868de00640 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 0x557ebe831ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868e800640 1 -- 172.21.15.49:0/2621965208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe82a140 msgr2=0x557ebe830030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868e800640 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe82a140 0x557ebe830030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868e800640 1 -- 172.21.15.49:0/2621965208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 msgr2=0x557ebe831ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868e800640 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 0x557ebe831ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.791+0000 7f868e800640 1 -- 172.21.15.49:0/2621965208 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebe72b920 con 0x557ebe8327d0 2024-02-26T06:26:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.792+0000 7f868e800640 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe839730 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8688012040 tx=0x7f86880042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc1e1ef5ea9036f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:33.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.793+0000 7f867aa00640 1 -- 172.21.15.49:0/2621965208 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868809ebb0 con 0x557ebe8327d0 2024-02-26T06:26:33.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.793+0000 7f867aa00640 1 -- 172.21.15.49:0/2621965208 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8688004000 con 0x557ebe8327d0 2024-02-26T06:26:33.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.793+0000 7f867aa00640 1 -- 172.21.15.49:0/2621965208 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86880a2fd0 con 0x557ebe8327d0 2024-02-26T06:26:33.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.794+0000 7f8697fdc740 1 -- 172.21.15.49:0/2621965208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 msgr2=0x557ebe839730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.794+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe839730 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8688012040 tx=0x7f86880042b0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.796+0000 7f8697fdc740 1 -- 172.21.15.49:0/2621965208 shutdown_connections 2024-02-26T06:26:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.796+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe8327d0 0x557ebe839730 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.796+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe82a140 0x557ebe830030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.796+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2621965208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 0x557ebe831ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.796+0000 7f8697fdc740 1 -- 172.21.15.49:0/2621965208 >> 172.21.15.49:0/2621965208 conn(0x557ebe7f0570 msgr2=0x557ebe82e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:33.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.796+0000 7f8697fdc740 1 -- 172.21.15.49:0/2621965208 shutdown_connections 2024-02-26T06:26:33.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.796+0000 7f8697fdc740 1 -- 172.21.15.49:0/2621965208 wait complete. 2024-02-26T06:26:33.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.797+0000 7f8697fdc740 1 -- start start 2024-02-26T06:26:33.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.797+0000 7f8697fdc740 1 Processor -- start 2024-02-26T06:26:33.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.798+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 0x557ebe839a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.798+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.798+0000 7f8697fdc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe8327d0 0x557ebe82c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.798+0000 7f8697fdc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe82cc40 con 0x557ebe829460 2024-02-26T06:26:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.798+0000 7f8697fdc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe82cdb0 con 0x557ebe82a140 2024-02-26T06:26:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.798+0000 7f8697fdc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe82cf70 con 0x557ebe8327d0 2024-02-26T06:26:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.799+0000 7f8697fdc740 10 client.0 _ll_register_callbacks cb 0x7f8680009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.799+0000 7f868d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.800+0000 7f868d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34380/0 (socket says 172.21.15.49:34380) 2024-02-26T06:26:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.800+0000 7f868d400640 1 -- 172.21.15.49:0/2617683989 learned_addr learned my addr 172.21.15.49:0/2617683989 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.800+0000 7f868e800640 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe8327d0 0x557ebe82c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.801+0000 7f868e800640 1 -- 172.21.15.49:0/2617683989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 msgr2=0x557ebe82c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.801+0000 7f868e800640 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.801+0000 7f868e800640 1 -- 172.21.15.49:0/2617683989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 msgr2=0x557ebe839a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.801+0000 7f868e800640 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 0x557ebe839a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.801+0000 7f868e800640 1 -- 172.21.15.49:0/2617683989 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ebe74a030 con 0x557ebe8327d0 2024-02-26T06:26:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.801+0000 7f868e800640 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe8327d0 0x557ebe82c5e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8688098d10 tx=0x7f868809a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76dcc2b15372ddfa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.801+0000 7f866fe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.802+0000 7f866fe00640 1 -- 172.21.15.49:0/2617683989 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868809ebb0 con 0x557ebe8327d0 2024-02-26T06:26:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.802+0000 7f866fe00640 1 -- 172.21.15.49:0/2617683989 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f86880a2d70 con 0x557ebe8327d0 2024-02-26T06:26:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.802+0000 7f8697fdc740 1 -- 172.21.15.49:0/2617683989 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86880a3200 con 0x557ebe8327d0 2024-02-26T06:26:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.802+0000 7f866fe00640 1 -- 172.21.15.49:0/2617683989 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86880a2f70 con 0x557ebe8327d0 2024-02-26T06:26:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.802+0000 7f8697fdc740 10 client.26233 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.802+0000 7f8697fdc740 1 -- 172.21.15.49:0/2617683989 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557ebe751400 con 0x557ebe8327d0 2024-02-26T06:26:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.803+0000 7f866fe00640 1 -- 172.21.15.49:0/2617683989 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86880a3200 con 0x557ebe8327d0 2024-02-26T06:26:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.803+0000 7f866fe00640 1 -- 172.21.15.49:0/2617683989 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8688023070 con 0x557ebe8327d0 2024-02-26T06:26:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.803+0000 7f8697fdc740 10 client.26233 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.803+0000 7f8697fdc740 10 client.26233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.803+0000 7f8697fdc740 20 client.26233 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.803+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ebe89a090 0x557ebe89c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.803+0000 7f8697fdc740 4 client.26233 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:26:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.803+0000 7f8697fdc740 1 -- 172.21.15.49:0/2617683989 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ebe51c890 con 0x557ebe89a090 2024-02-26T06:26:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.805+0000 7f868de00640 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ebe89a090 0x557ebe89c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.811+0000 7f868de00640 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ebe89a090 0x557ebe89c4b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217a315fb496c9be server_cookie=18234da80af1c4f9 in_seq=0 out_seq=0 2024-02-26T06:26:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.811+0000 7f866fe00640 10 client.26233 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.811+0000 7f866fe00640 1 -- 172.21.15.49:0/2617683989 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557ebe51c890 con 0x557ebe89a090 2024-02-26T06:26:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.811+0000 7f866fe00640 10 client.26233 handle_command_reply: tid=0 2024-02-26T06:26:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.811+0000 7f8697fdc740 1 client.26233 shutdown 2024-02-26T06:26:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.812+0000 7f8697fdc740 1 -- 172.21.15.49:0/2617683989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe8327d0 msgr2=0x557ebe82c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.812+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe8327d0 0x557ebe82c5e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8688098d10 tx=0x7f868809a480 comp rx=0 tx=0).stop 2024-02-26T06:26:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.813+0000 7f8697fdc740 1 -- 172.21.15.49:0/2617683989 shutdown_connections 2024-02-26T06:26:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.813+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ebe89a090 0x557ebe89c4b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.813+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ebe82a140 0x557ebe82c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.813+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ebe8327d0 0x557ebe82c5e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.813+0000 7f8697fdc740 1 --2- 172.21.15.49:0/2617683989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ebe829460 0x557ebe839a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.813+0000 7f8697fdc740 1 -- 172.21.15.49:0/2617683989 >> 172.21.15.49:0/2617683989 conn(0x557ebe7f0570 msgr2=0x557ebe8226a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:33.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.813+0000 7f866fe00640 0 client.26233 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:33.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.814+0000 7f8697fdc740 1 -- 172.21.15.49:0/2617683989 shutdown_connections 2024-02-26T06:26:33.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.814+0000 7f8697fdc740 1 -- 172.21.15.49:0/2617683989 wait complete. 2024-02-26T06:26:33.815 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:26:33.815 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:26:33.815 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:26:33.815 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:26:33.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.814+0000 7f8697fdc740 20 client.26233 trim_cache size 0 max 16384 2024-02-26T06:26:33.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 -- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86740a22b0 msgr2=0x7f86740a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86740a22b0 0x7f86740a4770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8680006010 tx=0x7f868002c6d0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 -- 172.21.15.49:0/2185178134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869010b5d0 msgr2=0x7f869010a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869010b5d0 0x7f869010a0a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8688009fd0 tx=0x7f868800b260 comp rx=0 tx=0).stop 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 -- 172.21.15.49:0/2185178134 shutdown_connections 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86740a22b0 0x7f86740a4770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869010b5d0 0x7f869010a0a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8690102ad0 0x7f869019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 --2- 172.21.15.49:0/2185178134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86901021b0 0x7f869019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.815+0000 7f868f200640 1 -- 172.21.15.49:0/2185178134 >> 172.21.15.49:0/2185178134 conn(0x7f86900fb9d0 msgr2=0x7f86900fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.816+0000 7f868f200640 1 -- 172.21.15.49:0/2185178134 shutdown_connections 2024-02-26T06:26:33.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:33.816+0000 7f868f200640 1 -- 172.21.15.49:0/2185178134 wait complete. 2024-02-26T06:26:33.842 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:26:33.842 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:26:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.115+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1175183045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b80a28e0 msgr2=0x7fa8b80a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.115+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1175183045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b80a28e0 0x7fa8b80a4cd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0066f90 tx=0x7fa8c0096e40 comp rx=0 tx=0).stop 2024-02-26T06:26:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.116+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1175183045 shutdown_connections 2024-02-26T06:26:34.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.116+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1175183045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b80a28e0 0x7fa8b80a4cd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.116+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1175183045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b8007690 0x7fa8b80a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.116+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1175183045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8b8006d70 0x7fa8b8007150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.116+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1175183045 >> 172.21.15.49:0/1175183045 conn(0x7fa8b800cb40 msgr2=0x7fa8b800cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.116+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1175183045 shutdown_connections 2024-02-26T06:26:34.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.116+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1175183045 wait complete. 2024-02-26T06:26:34.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.117+0000 7fa8c5a00640 1 Processor -- start 2024-02-26T06:26:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.118+0000 7fa8c5a00640 1 -- start start 2024-02-26T06:26:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.118+0000 7fa8c5a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b8006d70 0x7fa8b8096000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.118+0000 7fa8c5a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b8007690 0x7fa8b8096540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.119+0000 7fa8c5a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8b80a28e0 0x7fa8b8096a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.119+0000 7fa8c5a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b813ea60 con 0x7fa8b80a28e0 2024-02-26T06:26:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.119+0000 7fa8c5a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b812e200 con 0x7fa8b8007690 2024-02-26T06:26:34.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.119+0000 7fa8c5a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b812e3c0 con 0x7fa8b8006d70 2024-02-26T06:26:34.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.119+0000 7fa8bf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b8007690 0x7fa8b8096540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.119+0000 7fa8bfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b8006d70 0x7fa8b8096000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.119+0000 7fa8bfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b8006d70 0x7fa8b8096000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58626/0 (socket says 172.21.15.49:58626) 2024-02-26T06:26:34.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.120+0000 7fa8bfe00640 1 -- 172.21.15.49:0/1191675903 learned_addr learned my addr 172.21.15.49:0/1191675903 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:34.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.120+0000 7fa8c5000640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8b80a28e0 0x7fa8b8096a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.120+0000 7fa8bf400640 1 -- 172.21.15.49:0/1191675903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b8006d70 msgr2=0x7fa8b8096000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.120+0000 7fa8bf400640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b8006d70 0x7fa8b8096000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.120+0000 7fa8bf400640 1 -- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8b80a28e0 msgr2=0x7fa8b8096a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.120+0000 7fa8bf400640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8b80a28e0 0x7fa8b8096a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.120+0000 7fa8bf400640 1 -- 172.21.15.49:0/1191675903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c004f750 con 0x7fa8b8007690 2024-02-26T06:26:34.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.120+0000 7fa8bf400640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b8007690 0x7fa8b8096540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00ea20 tx=0x7fa8ac00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.121+0000 7fa8bcc00640 1 -- 172.21.15.49:0/1191675903 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac00ccc0 con 0x7fa8b8007690 2024-02-26T06:26:34.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.121+0000 7fa8bcc00640 1 -- 172.21.15.49:0/1191675903 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8ac00ce20 con 0x7fa8b8007690 2024-02-26T06:26:34.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.121+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b812e6a0 con 0x7fa8b8007690 2024-02-26T06:26:34.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.121+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b812ec00 con 0x7fa8b8007690 2024-02-26T06:26:34.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.121+0000 7fa8bcc00640 1 -- 172.21.15.49:0/1191675903 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac018840 con 0x7fa8b8007690 2024-02-26T06:26:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.123+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a0005290 con 0x7fa8b8007690 2024-02-26T06:26:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.131+0000 7fa8bcc00640 1 -- 172.21.15.49:0/1191675903 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa8ac010430 con 0x7fa8b8007690 2024-02-26T06:26:34.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.135+0000 7fa8bcc00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa8a80a22b0 0x7fa8a80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.135+0000 7fa8bcc00640 1 -- 172.21.15.49:0/1191675903 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fa8ac014070 con 0x7fa8b8007690 2024-02-26T06:26:34.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.136+0000 7fa8bfe00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa8a80a22b0 0x7fa8a80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.138+0000 7fa8bcc00640 1 -- 172.21.15.49:0/1191675903 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8ac052d20 con 0x7fa8b8007690 2024-02-26T06:26:34.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.138+0000 7fa8bfe00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa8a80a22b0 0x7fa8a80a4770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009c90 tx=0x7fa8b4005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.580+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8a0005810 con 0x7fa8b8007690 2024-02-26T06:26:34.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.581+0000 7fa8bcc00640 1 -- 172.21.15.49:0/1191675903 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11337 (secure 0 0 0) 0x7fa8ac09a9b0 con 0x7fa8b8007690 2024-02-26T06:26:34.583 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:26:34.583 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:26:24.192774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:26:34.583 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 30 2024-02-26T06:26:34.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa8a80a22b0 msgr2=0x7fa8a80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa8a80a22b0 0x7fa8a80a4770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009c90 tx=0x7fa8b4005f90 comp rx=0 tx=0).stop 2024-02-26T06:26:34.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b8007690 msgr2=0x7fa8b8096540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b8007690 0x7fa8b8096540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00ea20 tx=0x7fa8ac00eee0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 shutdown_connections 2024-02-26T06:26:34.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa8a80a22b0 0x7fa8a80a4770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8b8007690 0x7fa8b8096540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8b80a28e0 0x7fa8b8096a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 --2- 172.21.15.49:0/1191675903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b8006d70 0x7fa8b8096000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.587+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 >> 172.21.15.49:0/1191675903 conn(0x7fa8b800cb40 msgr2=0x7fa8b80942c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.588+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 shutdown_connections 2024-02-26T06:26:34.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.588+0000 7fa8c5a00640 1 -- 172.21.15.49:0/1191675903 wait complete. 2024-02-26T06:26:34.612 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:26:34.612 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:26:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 -- 172.21.15.49:0/2739161574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f517c0756b0 msgr2=0x7f517c075a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 --2- 172.21.15.49:0/2739161574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f517c0756b0 0x7f517c075a90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f516c009930 tx=0x7f516c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 -- 172.21.15.49:0/2739161574 shutdown_connections 2024-02-26T06:26:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 --2- 172.21.15.49:0/2739161574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f517c111fc0 0x7f517c1143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 --2- 172.21.15.49:0/2739161574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f517c075fd0 0x7f517c111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 --2- 172.21.15.49:0/2739161574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f517c0756b0 0x7f517c075a90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 -- 172.21.15.49:0/2739161574 >> 172.21.15.49:0/2739161574 conn(0x7f517c0fedf0 msgr2=0x7f517c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 -- 172.21.15.49:0/2739161574 shutdown_connections 2024-02-26T06:26:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.922+0000 7f5183200640 1 -- 172.21.15.49:0/2739161574 wait complete. 2024-02-26T06:26:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.923+0000 7f5183200640 1 Processor -- start 2024-02-26T06:26:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f5183200640 1 -- start start 2024-02-26T06:26:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f5183200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f517c0756b0 0x7f517c19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f5183200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f517c075fd0 0x7f517c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f5183200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f517c111fc0 0x7f517c1a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f5183200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c1a2d00 con 0x7f517c0756b0 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f5183200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c1a2e70 con 0x7f517c111fc0 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f5183200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c1a3030 con 0x7f517c075fd0 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f517b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f517c075fd0 0x7f517c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f517b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f517c075fd0 0x7f517c19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58630/0 (socket says 172.21.15.49:58630) 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f517b400640 1 -- 172.21.15.49:0/237234950 learned_addr learned my addr 172.21.15.49:0/237234950 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f5181200640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f517c111fc0 0x7f517c1a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.924+0000 7f517be00640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f517c0756b0 0x7f517c19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.925+0000 7f517b400640 1 -- 172.21.15.49:0/237234950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f517c111fc0 msgr2=0x7f517c1a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.925+0000 7f517b400640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f517c111fc0 0x7f517c1a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.925+0000 7f517b400640 1 -- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f517c0756b0 msgr2=0x7f517c19de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.925+0000 7f517b400640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f517c0756b0 0x7f517c19de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.925+0000 7f517b400640 1 -- 172.21.15.49:0/237234950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c009560 con 0x7f517c075fd0 2024-02-26T06:26:34.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.925+0000 7f517be00640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f517c0756b0 0x7f517c19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.926+0000 7f517b400640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f517c075fd0 0x7f517c19e3d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f516800e9f0 tx=0x7f516800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.926+0000 7f5178c00640 1 -- 172.21.15.49:0/237234950 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168009700 con 0x7f517c075fd0 2024-02-26T06:26:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.926+0000 7f5183200640 1 -- 172.21.15.49:0/237234950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c1a3310 con 0x7f517c075fd0 2024-02-26T06:26:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.926+0000 7f5178c00640 1 -- 172.21.15.49:0/237234950 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f516800cd80 con 0x7f517c075fd0 2024-02-26T06:26:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.926+0000 7f5183200640 1 -- 172.21.15.49:0/237234950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c1aab90 con 0x7f517c075fd0 2024-02-26T06:26:34.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.926+0000 7f5178c00640 1 -- 172.21.15.49:0/237234950 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168009700 con 0x7f517c075fd0 2024-02-26T06:26:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.933+0000 7f5186019740 1 Processor -- start 2024-02-26T06:26:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.934+0000 7f5186019740 1 -- start start 2024-02-26T06:26:34.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5178c00640 1 -- 172.21.15.49:0/237234950 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f51680186e0 con 0x7f517c075fd0 2024-02-26T06:26:34.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5186019740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053019ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5186019740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 0x563053022bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5186019740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 0x563053025500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5186019740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563052f881e0 con 0x563053023110 2024-02-26T06:26:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5186019740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563052f85710 con 0x563053019c10 2024-02-26T06:26:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5186019740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563052f80ef0 con 0x56305301a530 2024-02-26T06:26:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5181200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053019ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5181200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053019ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34418/0 (socket says 172.21.15.49:34418) 2024-02-26T06:26:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.935+0000 7f5181200640 1 -- 172.21.15.49:0/3071937489 learned_addr learned my addr 172.21.15.49:0/3071937489 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.936+0000 7f5181200640 1 -- 172.21.15.49:0/3071937489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 msgr2=0x563053022bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.936+0000 7f5181200640 1 --2- 172.21.15.49:0/3071937489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 0x563053022bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.936+0000 7f5181200640 1 -- 172.21.15.49:0/3071937489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 msgr2=0x563053025500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.936+0000 7f5181200640 1 --2- 172.21.15.49:0/3071937489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 0x563053025500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.936+0000 7f5181200640 1 -- 172.21.15.49:0/3071937489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563052f42dc0 con 0x563053019c10 2024-02-26T06:26:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.936+0000 7f5181200640 1 --2- 172.21.15.49:0/3071937489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053019ff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5174009e30 tx=0x7f5174010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d52df01a2133ec8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.937+0000 7f5178c00640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51600a23c0 0x7f51600a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.937+0000 7f5178c00640 1 -- 172.21.15.49:0/237234950 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5168014070 con 0x7f517c075fd0 2024-02-26T06:26:34.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.937+0000 7f5171600640 1 -- 172.21.15.49:0/3071937489 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174004d20 con 0x563053019c10 2024-02-26T06:26:34.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.937+0000 7f5171600640 1 -- 172.21.15.49:0/3071937489 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5174004e80 con 0x563053019c10 2024-02-26T06:26:34.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.937+0000 7f5171600640 1 -- 172.21.15.49:0/3071937489 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174012910 con 0x563053019c10 2024-02-26T06:26:34.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.937+0000 7f517be00640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51600a23c0 0x7f51600a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.938+0000 7f5186019740 1 -- 172.21.15.49:0/3071937489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 msgr2=0x563053019ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.938+0000 7f5186019740 1 --2- 172.21.15.49:0/3071937489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053019ff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5174009e30 tx=0x7f5174010740 comp rx=0 tx=0).stop 2024-02-26T06:26:34.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.939+0000 7f5186019740 1 -- 172.21.15.49:0/3071937489 shutdown_connections 2024-02-26T06:26:34.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.939+0000 7f5186019740 1 --2- 172.21.15.49:0/3071937489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 0x563053025500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.939+0000 7f5186019740 1 --2- 172.21.15.49:0/3071937489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 0x563053022bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.939+0000 7f5186019740 1 --2- 172.21.15.49:0/3071937489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053019ff0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.939+0000 7f5186019740 1 -- 172.21.15.49:0/3071937489 >> 172.21.15.49:0/3071937489 conn(0x563053014340 msgr2=0x563053014720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.939+0000 7f517be00640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51600a23c0 0x7f51600a4880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f516c009900 tx=0x7f516c004880 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.939+0000 7f5186019740 1 -- 172.21.15.49:0/3071937489 shutdown_connections 2024-02-26T06:26:34.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.939+0000 7f5186019740 1 -- 172.21.15.49:0/3071937489 wait complete. 2024-02-26T06:26:34.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 1 -- start start 2024-02-26T06:26:34.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 1 Processor -- start 2024-02-26T06:26:34.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053020150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 0x563053020690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 0x563053020bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563052f26120 con 0x563053023110 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563052f25dc0 con 0x563053019c10 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563052cef1c0 con 0x56305301a530 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f5186019740 10 client.0 _ll_register_callbacks cb 0x563052f75280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.940+0000 7f517be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053020150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053020150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34428/0 (socket says 172.21.15.49:34428) 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517be00640 1 -- 172.21.15.49:0/3875706565 learned_addr learned my addr 172.21.15.49:0/3875706565 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517b400640 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 0x563053020690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f5181200640 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 0x563053020bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517b400640 1 -- 172.21.15.49:0/3875706565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 msgr2=0x563053020150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517b400640 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053020150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517b400640 1 -- 172.21.15.49:0/3875706565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 msgr2=0x563053020bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517b400640 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 0x563053020bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517b400640 1 -- 172.21.15.49:0/3875706565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5174009a60 con 0x56305301a530 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f517b400640 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 0x563053020690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f51680cb6f0 tx=0x7f516809a3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5794784c4af415 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f5166000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.941+0000 7f5166000640 1 -- 172.21.15.49:0/3875706565 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680b3450 con 0x56305301a530 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.942+0000 7f5166000640 1 -- 172.21.15.49:0/3875706565 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f51680cbe40 con 0x56305301a530 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.942+0000 7f5186019740 1 -- 172.21.15.49:0/3875706565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563052cbd720 con 0x56305301a530 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.942+0000 7f5166000640 1 -- 172.21.15.49:0/3875706565 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51680a3130 con 0x56305301a530 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.942+0000 7f5186019740 10 client.26248 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.942+0000 7f5186019740 1 -- 172.21.15.49:0/3875706565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563052f7f570 con 0x56305301a530 2024-02-26T06:26:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.942+0000 7f5166000640 1 -- 172.21.15.49:0/3875706565 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563052cbd720 con 0x56305301a530 2024-02-26T06:26:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.943+0000 7f5166000640 1 -- 172.21.15.49:0/3875706565 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5168099970 con 0x56305301a530 2024-02-26T06:26:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.943+0000 7f5186019740 10 client.26248 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.943+0000 7f5186019740 10 client.26248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:26:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.943+0000 7f5186019740 20 client.26248 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.943+0000 7f5186019740 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x563053033250 0x563053035670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.943+0000 7f5186019740 4 client.26248 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.943+0000 7f5186019740 1 -- 172.21.15.49:0/3875706565 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563052d0c890 con 0x563053033250 2024-02-26T06:26:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.943+0000 7f517be00640 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x563053033250 0x563053035670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.944+0000 7f517be00640 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x563053033250 0x563053035670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19ece35bb6d3f47 server_cookie=ef7e5fb46584cf79 in_seq=0 out_seq=0 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.945+0000 7f5166000640 10 client.26248 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.945+0000 7f5166000640 1 -- 172.21.15.49:0/3875706565 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563052d0c890 con 0x563053033250 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.945+0000 7f5166000640 10 client.26248 handle_command_reply: tid=0 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.945+0000 7f5186019740 1 client.26248 shutdown 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 -- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 msgr2=0x563053020690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 0x563053020690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f51680cb6f0 tx=0x7f516809a3e0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 -- 172.21.15.49:0/3875706565 shutdown_connections 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x563053033250 0x563053035670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053023110 0x563053020bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301a530 0x563053020690 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 --2- 172.21.15.49:0/3875706565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053019c10 0x563053020150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 -- 172.21.15.49:0/3875706565 >> 172.21.15.49:0/3875706565 conn(0x563053014340 msgr2=0x563053016090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5166000640 0 client.26248 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:26:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.946+0000 7f5186019740 1 -- 172.21.15.49:0/3875706565 shutdown_connections 2024-02-26T06:26:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.947+0000 7f5186019740 1 -- 172.21.15.49:0/3875706565 wait complete. 2024-02-26T06:26:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.947+0000 7f5186019740 20 client.26248 trim_cache size 0 max 16384 2024-02-26T06:26:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.971+0000 7f5186019740 1 Processor -- start 2024-02-26T06:26:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.971+0000 7f5186019740 1 -- start start 2024-02-26T06:26:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f5186019740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053022880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f5186019740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 0x56305301d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f5186019740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x56305301dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f5186019740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563053018d00 con 0x563053019c10 2024-02-26T06:26:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f5186019740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305300fc80 con 0x563053023110 2024-02-26T06:26:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f5186019740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305302fd10 con 0x56305301d910 2024-02-26T06:26:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f517be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053022880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f5181200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x56305301dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f517b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 0x56305301d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f517be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053022880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51740/0 (socket says 172.21.15.49:51740) 2024-02-26T06:26:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.972+0000 7f517be00640 1 -- 172.21.15.49:0/3774866515 learned_addr learned my addr 172.21.15.49:0/3774866515 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.973+0000 7f517be00640 1 -- 172.21.15.49:0/3774866515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 msgr2=0x56305301dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.973+0000 7f517be00640 1 --2- 172.21.15.49:0/3774866515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x56305301dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.973+0000 7f517be00640 1 -- 172.21.15.49:0/3774866515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 msgr2=0x56305301d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.973+0000 7f517be00640 1 --2- 172.21.15.49:0/3774866515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 0x56305301d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.973+0000 7f517be00640 1 -- 172.21.15.49:0/3774866515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563052f1ba80 con 0x563053019c10 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.973+0000 7f517b400640 1 --2- 172.21.15.49:0/3774866515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 0x56305301d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.973+0000 7f517be00640 1 --2- 172.21.15.49:0/3774866515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053022880 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f516c02c240 tx=0x7f516c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd49e97b09db4480 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.973+0000 7f5170c00640 1 -- 172.21.15.49:0/3774866515 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c02cb20 con 0x563053019c10 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5170c00640 1 -- 172.21.15.49:0/3774866515 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f516c02cc80 con 0x563053019c10 2024-02-26T06:26:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5170c00640 1 -- 172.21.15.49:0/3774866515 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c003850 con 0x563053019c10 2024-02-26T06:26:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5186019740 1 -- 172.21.15.49:0/3774866515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 msgr2=0x563053022880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5186019740 1 --2- 172.21.15.49:0/3774866515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053022880 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f516c02c240 tx=0x7f516c043000 comp rx=0 tx=0).stop 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5186019740 1 -- 172.21.15.49:0/3774866515 shutdown_connections 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5186019740 1 --2- 172.21.15.49:0/3774866515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 0x56305301d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5186019740 1 --2- 172.21.15.49:0/3774866515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x56305301dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5186019740 1 --2- 172.21.15.49:0/3774866515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053022880 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.974+0000 7f5186019740 1 -- 172.21.15.49:0/3774866515 >> 172.21.15.49:0/3774866515 conn(0x563053014340 msgr2=0x56305301b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.975+0000 7f5186019740 1 -- 172.21.15.49:0/3774866515 shutdown_connections 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.975+0000 7f5186019740 1 -- 172.21.15.49:0/3774866515 wait complete. 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.975+0000 7f5186019740 1 -- start start 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.975+0000 7f5186019740 1 Processor -- start 2024-02-26T06:26:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.975+0000 7f5186019740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053084050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.975+0000 7f5186019740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x563053084590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f5186019740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 0x5630530297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f5186019740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563053029e80 con 0x563053019c10 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f5186019740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563053029ff0 con 0x563053023110 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f5186019740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305302a1b0 con 0x56305301d910 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f5186019740 10 client.0 _ll_register_callbacks cb 0x7f51740095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f517b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x563053084590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f517b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x563053084590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58670/0 (socket says 172.21.15.49:58670) 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f517b400640 1 -- 172.21.15.49:0/2239828157 learned_addr learned my addr 172.21.15.49:0/2239828157 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.976+0000 7f517be00640 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053084050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.977+0000 7f517b400640 1 -- 172.21.15.49:0/2239828157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 msgr2=0x5630530297b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.977+0000 7f517b400640 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 0x5630530297b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.977+0000 7f517b400640 1 -- 172.21.15.49:0/2239828157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 msgr2=0x563053084050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.977+0000 7f517b400640 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053084050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.977+0000 7f517b400640 1 -- 172.21.15.49:0/2239828157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563052f7f570 con 0x56305301d910 2024-02-26T06:26:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.977+0000 7f517be00640 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053084050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.978+0000 7f517b400640 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x563053084590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x563053021720 tx=0x7f516809a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ceb4afd639c403 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.978+0000 7f5166000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.979+0000 7f5166000640 1 -- 172.21.15.49:0/2239828157 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168096c30 con 0x56305301d910 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.979+0000 7f5166000640 1 -- 172.21.15.49:0/2239828157 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f51680cbe40 con 0x56305301d910 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.979+0000 7f5186019740 1 -- 172.21.15.49:0/2239828157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563052c6f4e0 con 0x56305301d910 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.979+0000 7f5166000640 1 -- 172.21.15.49:0/2239828157 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f516809fba0 con 0x56305301d910 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.979+0000 7f5186019740 10 client.26254 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.979+0000 7f5186019740 1 -- 172.21.15.49:0/2239828157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f517c114e40 con 0x56305301d910 2024-02-26T06:26:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.980+0000 7f5166000640 1 -- 172.21.15.49:0/2239828157 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563052c6f4e0 con 0x56305301d910 2024-02-26T06:26:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.980+0000 7f5166000640 1 -- 172.21.15.49:0/2239828157 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f51680ba180 con 0x56305301d910 2024-02-26T06:26:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.981+0000 7f5186019740 10 client.26254 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.981+0000 7f5186019740 10 client.26254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:26:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.981+0000 7f5186019740 20 client.26254 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.981+0000 7f5186019740 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56305308bd30 0x56305308e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.981+0000 7f5186019740 4 client.26254 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:26:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.981+0000 7f5186019740 1 -- 172.21.15.49:0/2239828157 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563052c2c700 con 0x56305308bd30 2024-02-26T06:26:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.981+0000 7f517be00640 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56305308bd30 0x56305308e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.982+0000 7f517be00640 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56305308bd30 0x56305308e150 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36c7a8c1811287ec server_cookie=8d5c8e8739b2a491 in_seq=0 out_seq=0 2024-02-26T06:26:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.982+0000 7f5166000640 10 client.26254 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:26:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.982+0000 7f5166000640 1 -- 172.21.15.49:0/2239828157 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563052c2c700 con 0x56305308bd30 2024-02-26T06:26:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.982+0000 7f5166000640 10 client.26254 handle_command_reply: tid=0 2024-02-26T06:26:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.982+0000 7f5186019740 1 client.26254 shutdown 2024-02-26T06:26:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.982+0000 7f5186019740 1 -- 172.21.15.49:0/2239828157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 msgr2=0x563053084590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.982+0000 7f5186019740 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x563053084590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x563053021720 tx=0x7f516809a090 comp rx=0 tx=0).stop 2024-02-26T06:26:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5186019740 1 -- 172.21.15.49:0/2239828157 shutdown_connections 2024-02-26T06:26:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5186019740 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x56305308bd30 0x56305308e150 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5186019740 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563053023110 0x5630530297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5186019740 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305301d910 0x563053084590 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5186019740 1 --2- 172.21.15.49:0/2239828157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563053019c10 0x563053084050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5186019740 1 -- 172.21.15.49:0/2239828157 >> 172.21.15.49:0/2239828157 conn(0x563053014340 msgr2=0x5630530200f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5166000640 0 client.26254 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:26:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5186019740 1 -- 172.21.15.49:0/2239828157 shutdown_connections 2024-02-26T06:26:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.983+0000 7f5186019740 1 -- 172.21.15.49:0/2239828157 wait complete. 2024-02-26T06:26:34.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.984+0000 7f5186019740 20 client.26254 trim_cache size 0 max 16384 2024-02-26T06:26:34.985 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:26:34.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 -- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51600a23c0 msgr2=0x7f51600a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51600a23c0 0x7f51600a4880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f516c009900 tx=0x7f516c004880 comp rx=0 tx=0).stop 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 -- 172.21.15.49:0/237234950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f517c075fd0 msgr2=0x7f517c19e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f517c075fd0 0x7f517c19e3d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f516800e9f0 tx=0x7f516800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 -- 172.21.15.49:0/237234950 shutdown_connections 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51600a23c0 0x7f51600a4880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f517c111fc0 0x7f517c1a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f517c075fd0 0x7f517c19e3d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 --2- 172.21.15.49:0/237234950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f517c0756b0 0x7f517c19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 -- 172.21.15.49:0/237234950 >> 172.21.15.49:0/237234950 conn(0x7f517c0fedf0 msgr2=0x7f517c113680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.985+0000 7f5183200640 1 -- 172.21.15.49:0/237234950 shutdown_connections 2024-02-26T06:26:34.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:34.986+0000 7f5183200640 1 -- 172.21.15.49:0/237234950 wait complete. 2024-02-26T06:26:35.013 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:26:35.013 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:26:35.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.317+0000 7f76f1800640 1 -- 172.21.15.49:0/551970039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec112a70 msgr2=0x7f76ec114e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.317+0000 7f76f1800640 1 --2- 172.21.15.49:0/551970039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec112a70 0x7f76ec114e60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009ca0 tx=0x7f76e402c100 comp rx=0 tx=0).stop 2024-02-26T06:26:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.318+0000 7f76f1800640 1 -- 172.21.15.49:0/551970039 shutdown_connections 2024-02-26T06:26:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.318+0000 7f76f1800640 1 --2- 172.21.15.49:0/551970039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec112a70 0x7f76ec114e60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.318+0000 7f76f1800640 1 --2- 172.21.15.49:0/551970039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec10cd10 0x7f76ec10d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.318+0000 7f76f1800640 1 --2- 172.21.15.49:0/551970039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ec108a10 0x7f76ec10c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.318+0000 7f76f1800640 1 -- 172.21.15.49:0/551970039 >> 172.21.15.49:0/551970039 conn(0x7f76ec06c6c0 msgr2=0x7f76ec06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.318+0000 7f76f1800640 1 -- 172.21.15.49:0/551970039 shutdown_connections 2024-02-26T06:26:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.319+0000 7f76f1800640 1 -- 172.21.15.49:0/551970039 wait complete. 2024-02-26T06:26:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.320+0000 7f76f1800640 1 Processor -- start 2024-02-26T06:26:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.321+0000 7f76f1800640 1 -- start start 2024-02-26T06:26:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.321+0000 7f76f1800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec108a10 0x7f76ec1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.321+0000 7f76f1800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ec10cd10 0x7f76ec111c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.321+0000 7f76f1800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec112a70 0x7f76ec1a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.321+0000 7f76f1800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec1a6f90 con 0x7f76ec10cd10 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.321+0000 7f76f1800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec1a7100 con 0x7f76ec108a10 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.321+0000 7f76f1800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec1a72c0 con 0x7f76ec112a70 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.322+0000 7f76eaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec108a10 0x7f76ec1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.322+0000 7f76eaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec108a10 0x7f76ec1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34462/0 (socket says 172.21.15.49:34462) 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.322+0000 7f76eaa00640 1 -- 172.21.15.49:0/725271524 learned_addr learned my addr 172.21.15.49:0/725271524 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.322+0000 7f76eb400640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec112a70 0x7f76ec1a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.323+0000 7f76ea000640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ec10cd10 0x7f76ec111c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.324+0000 7f76eaa00640 1 -- 172.21.15.49:0/725271524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec112a70 msgr2=0x7f76ec1a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.324+0000 7f76eaa00640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec112a70 0x7f76ec1a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.324+0000 7f76eaa00640 1 -- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ec10cd10 msgr2=0x7f76ec111c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.324+0000 7f76eaa00640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ec10cd10 0x7f76ec111c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.325+0000 7f76eaa00640 1 -- 172.21.15.49:0/725271524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e4009900 con 0x7f76ec108a10 2024-02-26T06:26:35.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.327+0000 7f76eaa00640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec108a10 0x7f76ec1116c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00c920 tx=0x7f76dc00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.327+0000 7f76e3400640 1 -- 172.21.15.49:0/725271524 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc007c70 con 0x7f76ec108a10 2024-02-26T06:26:35.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.327+0000 7f76e3400640 1 -- 172.21.15.49:0/725271524 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76dc007dd0 con 0x7f76ec108a10 2024-02-26T06:26:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.327+0000 7f76e3400640 1 -- 172.21.15.49:0/725271524 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc018860 con 0x7f76ec108a10 2024-02-26T06:26:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.327+0000 7f76f1800640 1 -- 172.21.15.49:0/725271524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ec1a75a0 con 0x7f76ec108a10 2024-02-26T06:26:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.328+0000 7f76f1800640 1 -- 172.21.15.49:0/725271524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec1aee20 con 0x7f76ec108a10 2024-02-26T06:26:35.329 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:34 smithi080 ceph-mon[36206]: pgmap v98: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 MiB/s wr, 676 op/s 2024-02-26T06:26:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:34 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1191675903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:26:35.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.335+0000 7f76e3400640 1 -- 172.21.15.49:0/725271524 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f76dc00f450 con 0x7f76ec108a10 2024-02-26T06:26:35.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.336+0000 7f76f47f5740 1 Processor -- start 2024-02-26T06:26:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.337+0000 7f76f47f5740 1 -- start start 2024-02-26T06:26:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.338+0000 7f76e3400640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76d00a22b0 0x7f76d00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.338+0000 7f76e3400640 1 -- 172.21.15.49:0/725271524 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f76dc0ce0c0 con 0x7f76ec108a10 2024-02-26T06:26:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.338+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb87eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb87f530 0x55aaeb887bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76eb400640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76d00a22b0 0x7f76d00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb888110 0x55aaeb88a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76f47f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb7ed1e0 con 0x55aaeb87f530 2024-02-26T06:26:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76f47f5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb7ea710 con 0x55aaeb87ec10 2024-02-26T06:26:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76f47f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb7e5ef0 con 0x55aaeb888110 2024-02-26T06:26:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:34 smithi115 ceph-mon[36426]: pgmap v98: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 MiB/s wr, 676 op/s 2024-02-26T06:26:35.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:34 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1191675903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76eb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb888110 0x55aaeb88a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76eb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb888110 0x55aaeb88a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58698/0 (socket says 172.21.15.49:58698) 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76eb400640 1 -- 172.21.15.49:0/526261944 learned_addr learned my addr 172.21.15.49:0/526261944 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.339+0000 7f76eaa00640 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb87eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.340+0000 7f76ea000640 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb87f530 0x55aaeb887bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.340+0000 7f76eaa00640 1 -- 172.21.15.49:0/526261944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb888110 msgr2=0x55aaeb88a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.340+0000 7f76eaa00640 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb888110 0x55aaeb88a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.340+0000 7f76eaa00640 1 -- 172.21.15.49:0/526261944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb87f530 msgr2=0x55aaeb887bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.340+0000 7f76eaa00640 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb87f530 0x55aaeb887bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.340+0000 7f76eaa00640 1 -- 172.21.15.49:0/526261944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaeb7a7dc0 con 0x55aaeb87ec10 2024-02-26T06:26:35.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.341+0000 7f76eb400640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76d00a22b0 0x7f76d00a4770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098d0 tx=0x7f76e4032340 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.341+0000 7f76eaa00640 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb87eff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0cb8c0 tx=0x7f76dc09b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6e7bd6df52a94b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.342+0000 7f76d7e00640 1 -- 172.21.15.49:0/526261944 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0a3300 con 0x55aaeb87ec10 2024-02-26T06:26:35.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.342+0000 7f76d7e00640 1 -- 172.21.15.49:0/526261944 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76dc09be90 con 0x55aaeb87ec10 2024-02-26T06:26:35.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.342+0000 7f76d7e00640 1 -- 172.21.15.49:0/526261944 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00f700 con 0x55aaeb87ec10 2024-02-26T06:26:35.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 -- 172.21.15.49:0/526261944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 msgr2=0x55aaeb87eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb87eff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0cb8c0 tx=0x7f76dc09b260 comp rx=0 tx=0).stop 2024-02-26T06:26:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 -- 172.21.15.49:0/526261944 shutdown_connections 2024-02-26T06:26:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb888110 0x55aaeb88a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb87f530 0x55aaeb887bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 --2- 172.21.15.49:0/526261944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb87eff0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 -- 172.21.15.49:0/526261944 >> 172.21.15.49:0/526261944 conn(0x55aaeb879340 msgr2=0x55aaeb879720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 -- 172.21.15.49:0/526261944 shutdown_connections 2024-02-26T06:26:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.343+0000 7f76f47f5740 1 -- 172.21.15.49:0/526261944 wait complete. 2024-02-26T06:26:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.344+0000 7f76f47f5740 1 -- start start 2024-02-26T06:26:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.345+0000 7f76f47f5740 1 Processor -- start 2024-02-26T06:26:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb885170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8856b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76f47f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb78b120 con 0x55aaeb888110 2024-02-26T06:26:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76f47f5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb78adc0 con 0x55aaeb87ec10 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76f47f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb5541c0 con 0x55aaeb87f530 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76f47f5740 10 client.0 _ll_register_callbacks cb 0x55aaeb7da280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb885170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76ea000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8856b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb885170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34488/0 (socket says 172.21.15.49:34488) 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eaa00640 1 -- 172.21.15.49:0/2001960161 learned_addr learned my addr 172.21.15.49:0/2001960161 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eb400640 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eaa00640 1 -- 172.21.15.49:0/2001960161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 msgr2=0x55aaeb8856b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eaa00640 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8856b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eaa00640 1 -- 172.21.15.49:0/2001960161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 msgr2=0x55aaeb885bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eaa00640 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.346+0000 7f76eaa00640 1 -- 172.21.15.49:0/2001960161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec1aee20 con 0x55aaeb87ec10 2024-02-26T06:26:35.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.347+0000 7f76ea000640 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8856b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.347+0000 7f76eaa00640 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb885170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f76dc007590 tx=0x7f76dc09ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9f7013f07416a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.347+0000 7f76d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.347+0000 7f76d4c00640 1 -- 172.21.15.49:0/2001960161 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00f910 con 0x55aaeb87ec10 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.347+0000 7f76f47f5740 1 -- 172.21.15.49:0/2001960161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaeb4e57e0 con 0x55aaeb87ec10 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.348+0000 7f76d4c00640 1 -- 172.21.15.49:0/2001960161 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f76dc0a3300 con 0x55aaeb87ec10 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.348+0000 7f76d4c00640 1 -- 172.21.15.49:0/2001960161 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76dc09abe0 con 0x55aaeb87ec10 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.348+0000 7f76f47f5740 10 client.26228 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.348+0000 7f76f47f5740 1 -- 172.21.15.49:0/2001960161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aaeb7e4570 con 0x55aaeb87ec10 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.349+0000 7f76d4c00640 1 -- 172.21.15.49:0/2001960161 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aaeb4e57e0 con 0x55aaeb87ec10 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.349+0000 7f76d4c00640 1 -- 172.21.15.49:0/2001960161 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f76dc016020 con 0x55aaeb87ec10 2024-02-26T06:26:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.349+0000 7f76f47f5740 10 client.26228 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.349+0000 7f76f47f5740 10 client.26228 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:26:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.349+0000 7f76f47f5740 20 client.26228 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.349+0000 7f76f47f5740 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55aaeb897d50 0x55aaeb89a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.350+0000 7f76f47f5740 4 client.26228 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:35.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.350+0000 7f76f47f5740 1 -- 172.21.15.49:0/2001960161 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaeb522720 con 0x55aaeb897d50 2024-02-26T06:26:35.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.350+0000 7f76ea000640 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55aaeb897d50 0x55aaeb89a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.351+0000 7f76d4c00640 10 client.26228 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.352+0000 7f76ea000640 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55aaeb897d50 0x55aaeb89a140 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1cdea02448643 server_cookie=bc0e35f7a1bc4fc7 in_seq=0 out_seq=0 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.353+0000 7f76d4c00640 1 -- 172.21.15.49:0/2001960161 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aaeb522720 con 0x55aaeb897d50 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.353+0000 7f76d4c00640 10 client.26228 handle_command_reply: tid=0 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.353+0000 7f76f47f5740 1 client.26228 shutdown 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 -- 172.21.15.49:0/2001960161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 msgr2=0x55aaeb885170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb885170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f76dc007590 tx=0x7f76dc09ee30 comp rx=0 tx=0).stop 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 -- 172.21.15.49:0/2001960161 shutdown_connections 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55aaeb897d50 0x55aaeb89a140 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8856b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 --2- 172.21.15.49:0/2001960161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb87ec10 0x55aaeb885170 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 -- 172.21.15.49:0/2001960161 >> 172.21.15.49:0/2001960161 conn(0x55aaeb879340 msgr2=0x55aaeb87b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76d4c00640 0 client.26228 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:26:35.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 -- 172.21.15.49:0/2001960161 shutdown_connections 2024-02-26T06:26:35.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.354+0000 7f76f47f5740 1 -- 172.21.15.49:0/2001960161 wait complete. 2024-02-26T06:26:35.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.355+0000 7f76f47f5740 20 client.26228 trim_cache size 0 max 16384 2024-02-26T06:26:35.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.379+0000 7f76f47f5740 1 Processor -- start 2024-02-26T06:26:35.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.379+0000 7f76f47f5740 1 -- start start 2024-02-26T06:26:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.379+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.379+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.379+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb89ea60 0x55aaeb89eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.379+0000 7f76f47f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb894ba0 con 0x55aaeb888110 2024-02-26T06:26:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.379+0000 7f76f47f5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb87e3b0 con 0x55aaeb89ea60 2024-02-26T06:26:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.379+0000 7f76f47f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb885f80 con 0x55aaeb87f530 2024-02-26T06:26:35.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.380+0000 7f76eaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.380+0000 7f76eaa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58714/0 (socket says 172.21.15.49:58714) 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.380+0000 7f76eaa00640 1 -- 172.21.15.49:0/444979298 learned_addr learned my addr 172.21.15.49:0/444979298 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.380+0000 7f76eb400640 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb89ea60 0x55aaeb89eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.380+0000 7f76ea000640 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76eaa00640 1 -- 172.21.15.49:0/444979298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb89ea60 msgr2=0x55aaeb89eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76eaa00640 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb89ea60 0x55aaeb89eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76eaa00640 1 -- 172.21.15.49:0/444979298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 msgr2=0x55aaeb885a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76eaa00640 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76eaa00640 1 -- 172.21.15.49:0/444979298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaeb780a80 con 0x55aaeb87f530 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76ea000640 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:35.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76eaa00640 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8878a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76dc097530 tx=0x7f76dc09b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f59aa06b1ee85e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76d7400640 1 -- 172.21.15.49:0/444979298 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00f910 con 0x55aaeb87f530 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76d7400640 1 -- 172.21.15.49:0/444979298 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76dc0cbc00 con 0x55aaeb87f530 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.381+0000 7f76d7400640 1 -- 172.21.15.49:0/444979298 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0a2e70 con 0x55aaeb87f530 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.382+0000 7f76f47f5740 1 -- 172.21.15.49:0/444979298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 msgr2=0x55aaeb8878a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.382+0000 7f76f47f5740 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8878a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76dc097530 tx=0x7f76dc09b260 comp rx=0 tx=0).stop 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.382+0000 7f76f47f5740 1 -- 172.21.15.49:0/444979298 shutdown_connections 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.382+0000 7f76f47f5740 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb89ea60 0x55aaeb89eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.382+0000 7f76f47f5740 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb888110 0x55aaeb885a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.382+0000 7f76f47f5740 1 --2- 172.21.15.49:0/444979298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb87f530 0x55aaeb8878a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.382+0000 7f76f47f5740 1 -- 172.21.15.49:0/444979298 >> 172.21.15.49:0/444979298 conn(0x55aaeb879340 msgr2=0x55aaeb879720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.382+0000 7f76f47f5740 1 -- 172.21.15.49:0/444979298 shutdown_connections 2024-02-26T06:26:35.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.383+0000 7f76f47f5740 1 -- 172.21.15.49:0/444979298 wait complete. 2024-02-26T06:26:35.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.383+0000 7f76f47f5740 1 -- start start 2024-02-26T06:26:35.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.383+0000 7f76f47f5740 1 Processor -- start 2024-02-26T06:26:35.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb888110 0x55aaeb88f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb89ea60 0x55aaeb88d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76f47f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb88d940 0x55aaeb88ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76f47f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb88f7a0 con 0x55aaeb88d940 2024-02-26T06:26:35.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76f47f5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb8ee1f0 con 0x55aaeb888110 2024-02-26T06:26:35.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76f47f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaeb8ee3b0 con 0x55aaeb89ea60 2024-02-26T06:26:35.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76f47f5740 10 client.0 _ll_register_callbacks cb 0x7f76dc096e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:35.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76eaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb888110 0x55aaeb88f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76eaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb888110 0x55aaeb88f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34508/0 (socket says 172.21.15.49:34508) 2024-02-26T06:26:35.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76eaa00640 1 -- 172.21.15.49:0/551764128 learned_addr learned my addr 172.21.15.49:0/551764128 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.384+0000 7f76eb400640 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb88d940 0x55aaeb88ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.385+0000 7f76ea000640 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb89ea60 0x55aaeb88d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.385+0000 7f76eaa00640 1 -- 172.21.15.49:0/551764128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb89ea60 msgr2=0x55aaeb88d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.385+0000 7f76eaa00640 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb89ea60 0x55aaeb88d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.385+0000 7f76eaa00640 1 -- 172.21.15.49:0/551764128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb88d940 msgr2=0x55aaeb88ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.386+0000 7f76eaa00640 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb88d940 0x55aaeb88ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.386+0000 7f76eaa00640 1 -- 172.21.15.49:0/551764128 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaeb7e4570 con 0x55aaeb888110 2024-02-26T06:26:35.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.386+0000 7f76eb400640 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb88d940 0x55aaeb88ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:35.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.387+0000 7f76eaa00640 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb888110 0x55aaeb88f100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f76dc007590 tx=0x7f76dc099070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baf45d78167a7b4b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.387+0000 7f76d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:35.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.387+0000 7f76d4c00640 1 -- 172.21.15.49:0/551764128 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00f910 con 0x55aaeb888110 2024-02-26T06:26:35.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.387+0000 7f76d4c00640 1 -- 172.21.15.49:0/551764128 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f76dc0a3300 con 0x55aaeb888110 2024-02-26T06:26:35.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.387+0000 7f76f47f5740 1 -- 172.21.15.49:0/551764128 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76dc09be90 con 0x55aaeb888110 2024-02-26T06:26:35.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.388+0000 7f76d4c00640 1 -- 172.21.15.49:0/551764128 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76dc09a170 con 0x55aaeb888110 2024-02-26T06:26:35.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.388+0000 7f76f47f5740 10 client.26234 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:35.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.388+0000 7f76f47f5740 1 -- 172.21.15.49:0/551764128 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aaeb7a6620 con 0x55aaeb888110 2024-02-26T06:26:35.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76d4c00640 1 -- 172.21.15.49:0/551764128 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76dc09be90 con 0x55aaeb888110 2024-02-26T06:26:35.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76d4c00640 1 -- 172.21.15.49:0/551764128 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f76dc016070 con 0x55aaeb888110 2024-02-26T06:26:35.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76f47f5740 10 client.26234 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:35.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76f47f5740 10 client.26234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:26:35.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76f47f5740 20 client.26234 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:35.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76f47f5740 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55aaeb8ef680 0x55aaeb8f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76f47f5740 4 client.26234 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:26:35.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76f47f5740 1 -- 172.21.15.49:0/551764128 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aaeb571890 con 0x55aaeb8ef680 2024-02-26T06:26:35.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.389+0000 7f76ea000640 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55aaeb8ef680 0x55aaeb8f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.390+0000 7f76ea000640 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55aaeb8ef680 0x55aaeb8f1a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc8e9ccafb780248 server_cookie=913b25adbf634ed0 in_seq=0 out_seq=0 2024-02-26T06:26:35.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.390+0000 7f76d4c00640 10 client.26234 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:26:35.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.390+0000 7f76d4c00640 1 -- 172.21.15.49:0/551764128 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aaeb571890 con 0x55aaeb8ef680 2024-02-26T06:26:35.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.390+0000 7f76d4c00640 10 client.26234 handle_command_reply: tid=0 2024-02-26T06:26:35.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.390+0000 7f76f47f5740 1 client.26234 shutdown 2024-02-26T06:26:35.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 -- 172.21.15.49:0/551764128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb888110 msgr2=0x55aaeb88f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb888110 0x55aaeb88f100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f76dc007590 tx=0x7f76dc099070 comp rx=0 tx=0).stop 2024-02-26T06:26:35.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 -- 172.21.15.49:0/551764128 shutdown_connections 2024-02-26T06:26:35.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55aaeb8ef680 0x55aaeb8f1a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aaeb88d940 0x55aaeb88ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aaeb89ea60 0x55aaeb88d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 --2- 172.21.15.49:0/551764128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55aaeb888110 0x55aaeb88f100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 -- 172.21.15.49:0/551764128 >> 172.21.15.49:0/551764128 conn(0x55aaeb879340 msgr2=0x55aaeb880450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76d4c00640 0 client.26234 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:26:35.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 -- 172.21.15.49:0/551764128 shutdown_connections 2024-02-26T06:26:35.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.391+0000 7f76f47f5740 1 -- 172.21.15.49:0/551764128 wait complete. 2024-02-26T06:26:35.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.392+0000 7f76f47f5740 20 client.26234 trim_cache size 0 max 16384 2024-02-26T06:26:35.393 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 -- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76d00a22b0 msgr2=0x7f76d00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76d00a22b0 0x7f76d00a4770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098d0 tx=0x7f76e4032340 comp rx=0 tx=0).stop 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 -- 172.21.15.49:0/725271524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec108a10 msgr2=0x7f76ec1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec108a10 0x7f76ec1116c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00c920 tx=0x7f76dc00cde0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 -- 172.21.15.49:0/725271524 shutdown_connections 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f76d00a22b0 0x7f76d00a4770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec112a70 0x7f76ec1a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ec10cd10 0x7f76ec111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 --2- 172.21.15.49:0/725271524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76ec108a10 0x7f76ec1116c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.393+0000 7f76f1800640 1 -- 172.21.15.49:0/725271524 >> 172.21.15.49:0/725271524 conn(0x7f76ec06c6c0 msgr2=0x7f76ec1138e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.394+0000 7f76f1800640 1 -- 172.21.15.49:0/725271524 shutdown_connections 2024-02-26T06:26:35.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.394+0000 7f76f1800640 1 -- 172.21.15.49:0/725271524 wait complete. 2024-02-26T06:26:35.417 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:34 smithi049 ceph-mon[31696]: pgmap v98: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 MiB/s wr, 676 op/s 2024-02-26T06:26:35.417 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:34 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1191675903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:26:35.421 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:26:35.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.706+0000 7fe4ae800640 1 -- 172.21.15.49:0/1670422283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a8107710 msgr2=0x7fe4a8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.706+0000 7fe4ae800640 1 --2- 172.21.15.49:0/1670422283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a8107710 0x7fe4a8107b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe494009930 tx=0x7fe49402bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.707+0000 7fe4ae800640 1 -- 172.21.15.49:0/1670422283 shutdown_connections 2024-02-26T06:26:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.707+0000 7fe4ae800640 1 --2- 172.21.15.49:0/1670422283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4a81080d0 0x7fe4a81145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.707+0000 7fe4ae800640 1 --2- 172.21.15.49:0/1670422283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a8107710 0x7fe4a8107b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.707+0000 7fe4ae800640 1 --2- 172.21.15.49:0/1670422283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4a810d720 0x7fe4a810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.707+0000 7fe4ae800640 1 -- 172.21.15.49:0/1670422283 >> 172.21.15.49:0/1670422283 conn(0x7fe4a8075690 msgr2=0x7fe4a8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.707+0000 7fe4ae800640 1 -- 172.21.15.49:0/1670422283 shutdown_connections 2024-02-26T06:26:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.708+0000 7fe4ae800640 1 -- 172.21.15.49:0/1670422283 wait complete. 2024-02-26T06:26:35.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.709+0000 7fe4ae800640 1 Processor -- start 2024-02-26T06:26:35.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.710+0000 7fe4ae800640 1 -- start start 2024-02-26T06:26:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.710+0000 7fe4ae800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4a8107710 0x7fe4a819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.710+0000 7fe4ae800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a81080d0 0x7fe4a819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.710+0000 7fe4ae800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4a810d720 0x7fe4a81a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.710+0000 7fe4ae800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81a2da0 con 0x7fe4a810d720 2024-02-26T06:26:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.710+0000 7fe4ae800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81a2f10 con 0x7fe4a8107710 2024-02-26T06:26:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.710+0000 7fe4ae800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81a30d0 con 0x7fe4a81080d0 2024-02-26T06:26:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.711+0000 7fe4a6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a81080d0 0x7fe4a819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.711+0000 7fe4a6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a81080d0 0x7fe4a819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58748/0 (socket says 172.21.15.49:58748) 2024-02-26T06:26:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.711+0000 7fe4a6a00640 1 -- 172.21.15.49:0/2273628391 learned_addr learned my addr 172.21.15.49:0/2273628391 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.711+0000 7fe4a7400640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4a8107710 0x7fe4a819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.711+0000 7fe4a6a00640 1 -- 172.21.15.49:0/2273628391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4a8107710 msgr2=0x7fe4a819dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.711+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4a8107710 0x7fe4a819dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.711+0000 7fe4a6a00640 1 -- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4a810d720 msgr2=0x7fe4a81a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.712+0000 7fe4a7e00640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4a810d720 0x7fe4a81a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.713+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4a810d720 0x7fe4a81a27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.713+0000 7fe4a6a00640 1 -- 172.21.15.49:0/2273628391 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494009560 con 0x7fe4a81080d0 2024-02-26T06:26:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.713+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a81080d0 0x7fe4a819e420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe494000c00 tx=0x7fe494004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.714+0000 7fe49fe00640 1 -- 172.21.15.49:0/2273628391 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494004440 con 0x7fe4a81080d0 2024-02-26T06:26:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.715+0000 7fe4ae800640 1 -- 172.21.15.49:0/2273628391 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a81a3350 con 0x7fe4a81080d0 2024-02-26T06:26:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.715+0000 7fe49fe00640 1 -- 172.21.15.49:0/2273628391 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4940052d0 con 0x7fe4a81080d0 2024-02-26T06:26:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.715+0000 7fe4ae800640 1 -- 172.21.15.49:0/2273628391 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a81aac30 con 0x7fe4a81080d0 2024-02-26T06:26:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.715+0000 7fe49fe00640 1 -- 172.21.15.49:0/2273628391 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494004440 con 0x7fe4a81080d0 2024-02-26T06:26:35.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.722+0000 7fe4b1322740 1 Processor -- start 2024-02-26T06:26:35.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.722+0000 7fe4b1322740 1 -- start start 2024-02-26T06:26:35.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.722+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e3c10 0x56170e6e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.722+0000 7fe49fe00640 1 -- 172.21.15.49:0/2273628391 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe494003b30 con 0x7fe4a81080d0 2024-02-26T06:26:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e4530 0x56170e6ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 0x56170e6ef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4b1322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e6521e0 con 0x56170e6e4530 2024-02-26T06:26:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4b1322740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e64f710 con 0x56170e6e3c10 2024-02-26T06:26:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4b1322740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e64aef0 con 0x56170e6ed110 2024-02-26T06:26:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4a7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e3c10 0x56170e6e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4a7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e3c10 0x56170e6e3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34546/0 (socket says 172.21.15.49:34546) 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4a7e00640 1 -- 172.21.15.49:0/237608878 learned_addr learned my addr 172.21.15.49:0/237608878 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 0x56170e6ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4a6a00640 1 -- 172.21.15.49:0/237608878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e3c10 msgr2=0x56170e6e3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e3c10 0x56170e6e3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.723+0000 7fe4a7400640 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e4530 0x56170e6ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.724+0000 7fe4a6a00640 1 -- 172.21.15.49:0/237608878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e4530 msgr2=0x56170e6ecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.724+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e4530 0x56170e6ecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.724+0000 7fe4a6a00640 1 -- 172.21.15.49:0/237608878 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56170e60cdc0 con 0x56170e6ed110 2024-02-26T06:26:35.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.724+0000 7fe4a7e00640 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e3c10 0x56170e6e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:35.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.724+0000 7fe4a7400640 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e4530 0x56170e6ecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:35.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.725+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 0x56170e6ef500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe4940edba0 tx=0x7fe4940f8490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eab7d5d1eba7d3c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.726+0000 7fe49cc00640 1 -- 172.21.15.49:0/237608878 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940f8ea0 con 0x56170e6ed110 2024-02-26T06:26:35.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.726+0000 7fe49cc00640 1 -- 172.21.15.49:0/237608878 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4940f0c80 con 0x56170e6ed110 2024-02-26T06:26:35.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.726+0000 7fe49cc00640 1 -- 172.21.15.49:0/237608878 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940ff8e0 con 0x56170e6ed110 2024-02-26T06:26:35.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.726+0000 7fe49fe00640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe48c0a2310 0x7fe48c0a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.727+0000 7fe4b1322740 1 -- 172.21.15.49:0/237608878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 msgr2=0x56170e6ef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.727+0000 7fe4b1322740 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 0x56170e6ef500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe4940edba0 tx=0x7fe4940f8490 comp rx=0 tx=0).stop 2024-02-26T06:26:35.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.727+0000 7fe4a7e00640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe48c0a2310 0x7fe48c0a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.729+0000 7fe49fe00640 1 -- 172.21.15.49:0/2273628391 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe4940f0330 con 0x7fe4a81080d0 2024-02-26T06:26:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.729+0000 7fe4a7e00640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe48c0a2310 0x7fe48c0a47d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000f470 tx=0x7fe4a0007680 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.729+0000 7fe4b1322740 1 -- 172.21.15.49:0/237608878 shutdown_connections 2024-02-26T06:26:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.729+0000 7fe4b1322740 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 0x56170e6ef500 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.729+0000 7fe4b1322740 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e4530 0x56170e6ecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.729+0000 7fe4b1322740 1 --2- 172.21.15.49:0/237608878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e3c10 0x56170e6e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.729+0000 7fe4b1322740 1 -- 172.21.15.49:0/237608878 >> 172.21.15.49:0/237608878 conn(0x56170e6de340 msgr2=0x56170e6de720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.730+0000 7fe4b1322740 1 -- 172.21.15.49:0/237608878 shutdown_connections 2024-02-26T06:26:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.730+0000 7fe4b1322740 1 -- 172.21.15.49:0/237608878 wait complete. 2024-02-26T06:26:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.731+0000 7fe4b1322740 1 -- start start 2024-02-26T06:26:35.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.731+0000 7fe4b1322740 1 Processor -- start 2024-02-26T06:26:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.732+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.732+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.732+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.732+0000 7fe4b1322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e5f0120 con 0x56170e6e3c10 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.732+0000 7fe4b1322740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e5efdc0 con 0x56170e6ed110 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.732+0000 7fe4b1322740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e3b91c0 con 0x56170e6e4530 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.732+0000 7fe4b1322740 10 client.0 _ll_register_callbacks cb 0x56170e63f280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6eab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34562/0 (socket says 172.21.15.49:34562) 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7e00640 1 -- 172.21.15.49:0/2184296764 learned_addr learned my addr 172.21.15.49:0/2184296764 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7400640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7e00640 1 -- 172.21.15.49:0/2184296764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 msgr2=0x56170e6ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7e00640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7e00640 1 -- 172.21.15.49:0/2184296764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 msgr2=0x56170e6ea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7e00640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7e00640 1 -- 172.21.15.49:0/2184296764 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a81aac30 con 0x56170e6ed110 2024-02-26T06:26:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.733+0000 7fe4a7400640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe4a7e00640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6eab10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0007fd0 tx=0x7fe4a0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8291efb86f2105 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe491600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe491600640 1 -- 172.21.15.49:0/2184296764 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0012e10 con 0x56170e6ed110 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe4b1322740 1 -- 172.21.15.49:0/2184296764 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56170e34a7e0 con 0x56170e6ed110 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe491600640 1 -- 172.21.15.49:0/2184296764 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe4a000d040 con 0x56170e6ed110 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe491600640 1 -- 172.21.15.49:0/2184296764 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4a001f440 con 0x56170e6ed110 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe4b1322740 10 client.26246 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.734+0000 7fe4b1322740 1 -- 172.21.15.49:0/2184296764 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56170e649570 con 0x56170e6ed110 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.735+0000 7fe491600640 1 -- 172.21.15.49:0/2184296764 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56170e34a7e0 con 0x56170e6ed110 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.735+0000 7fe491600640 1 -- 172.21.15.49:0/2184296764 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe4a001e730 con 0x56170e6ed110 2024-02-26T06:26:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.735+0000 7fe4b1322740 10 client.26246 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.735+0000 7fe4b1322740 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.735+0000 7fe4b1322740 20 client.26246 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.735+0000 7fe4b1322740 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56170e6fd000 0x56170e6ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.735+0000 7fe4b1322740 4 client.26246 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.735+0000 7fe4b1322740 1 -- 172.21.15.49:0/2184296764 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56170e387720 con 0x56170e6fd000 2024-02-26T06:26:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.736+0000 7fe4a7400640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56170e6fd000 0x56170e6ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.737+0000 7fe491600640 10 client.26246 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.738+0000 7fe4a7400640 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56170e6fd000 0x56170e6ff420 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5b3b180b9db2e2 server_cookie=f3caa71d2c1bf196 in_seq=0 out_seq=0 2024-02-26T06:26:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.739+0000 7fe491600640 1 -- 172.21.15.49:0/2184296764 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56170e387720 con 0x56170e6fd000 2024-02-26T06:26:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.739+0000 7fe491600640 10 client.26246 handle_command_reply: tid=0 2024-02-26T06:26:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.739+0000 7fe4b1322740 1 client.26246 shutdown 2024-02-26T06:26:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe4b1322740 1 -- 172.21.15.49:0/2184296764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 msgr2=0x56170e6eab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe4b1322740 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6eab10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0007fd0 tx=0x7fe4a0004a40 comp rx=0 tx=0).stop 2024-02-26T06:26:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe4b1322740 1 -- 172.21.15.49:0/2184296764 shutdown_connections 2024-02-26T06:26:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe4b1322740 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56170e6fd000 0x56170e6ff420 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe4b1322740 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6eab10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe4b1322740 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe4b1322740 1 --2- 172.21.15.49:0/2184296764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe491600640 0 client.26246 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.740+0000 7fe4b1322740 1 -- 172.21.15.49:0/2184296764 >> 172.21.15.49:0/2184296764 conn(0x56170e6de340 msgr2=0x56170e6e0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.741+0000 7fe4b1322740 1 -- 172.21.15.49:0/2184296764 shutdown_connections 2024-02-26T06:26:35.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.741+0000 7fe4b1322740 1 -- 172.21.15.49:0/2184296764 wait complete. 2024-02-26T06:26:35.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.741+0000 7fe4b1322740 20 client.26246 trim_cache size 0 max 16384 2024-02-26T06:26:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.763+0000 7fe4b1322740 1 Processor -- start 2024-02-26T06:26:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.763+0000 7fe4b1322740 1 -- start start 2024-02-26T06:26:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6e8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4b1322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e6df340 con 0x56170e6e3c10 2024-02-26T06:26:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4b1322740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e6df570 con 0x56170e6ed110 2024-02-26T06:26:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4b1322740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e6f9a20 con 0x56170e6e4530 2024-02-26T06:26:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4a6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4a6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58778/0 (socket says 172.21.15.49:58778) 2024-02-26T06:26:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4a6a00640 1 -- 172.21.15.49:0/1861513437 learned_addr learned my addr 172.21.15.49:0/1861513437 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4a7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4a6a00640 1 -- 172.21.15.49:0/1861513437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 msgr2=0x56170e6e8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6e8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4a7400640 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.764+0000 7fe4a6a00640 1 -- 172.21.15.49:0/1861513437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 msgr2=0x56170e6ec7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:35.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.765+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ec7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.765+0000 7fe4a6a00640 1 -- 172.21.15.49:0/1861513437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56170e5e5a80 con 0x56170e6e4530 2024-02-26T06:26:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.765+0000 7fe4a7400640 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.766+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe4940ff4d0 tx=0x7fe4940ff460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f5a156f293faae server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.766+0000 7fe493e00640 1 -- 172.21.15.49:0/1861513437 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940f3950 con 0x56170e6e4530 2024-02-26T06:26:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.767+0000 7fe493e00640 1 -- 172.21.15.49:0/1861513437 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4940b90e0 con 0x56170e6e4530 2024-02-26T06:26:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.767+0000 7fe493e00640 1 -- 172.21.15.49:0/1861513437 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940f3b30 con 0x56170e6e4530 2024-02-26T06:26:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.768+0000 7fe4b1322740 1 -- 172.21.15.49:0/1861513437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 msgr2=0x56170e6ea930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.768+0000 7fe4b1322740 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe4940ff4d0 tx=0x7fe4940ff460 comp rx=0 tx=0).stop 2024-02-26T06:26:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.768+0000 7fe4b1322740 1 -- 172.21.15.49:0/1861513437 shutdown_connections 2024-02-26T06:26:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.768+0000 7fe4b1322740 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6ed110 0x56170e6e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.768+0000 7fe4b1322740 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6e4530 0x56170e6ea930 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.768+0000 7fe4b1322740 1 --2- 172.21.15.49:0/1861513437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.768+0000 7fe4b1322740 1 -- 172.21.15.49:0/1861513437 >> 172.21.15.49:0/1861513437 conn(0x56170e6de340 msgr2=0x56170e6e5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.769+0000 7fe4b1322740 1 -- 172.21.15.49:0/1861513437 shutdown_connections 2024-02-26T06:26:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.769+0000 7fe4b1322740 1 -- 172.21.15.49:0/1861513437 wait complete. 2024-02-26T06:26:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.769+0000 7fe4b1322740 1 -- start start 2024-02-26T06:26:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.770+0000 7fe4b1322740 1 Processor -- start 2024-02-26T06:26:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.771+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.771+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e4530 0x56170e6f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.771+0000 7fe4b1322740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 0x56170e6f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4a6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e4530 0x56170e6f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4a6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e4530 0x56170e6f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34580/0 (socket says 172.21.15.49:34580) 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4a6a00640 1 -- 172.21.15.49:0/145927369 learned_addr learned my addr 172.21.15.49:0/145927369 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e6f25a0 con 0x56170e6e3c10 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e6f2710 con 0x56170e6e4530 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170e6f28d0 con 0x56170e6ed110 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4b1322740 10 client.0 _ll_register_callbacks cb 0x7fe4940f3d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4a6a00640 1 -- 172.21.15.49:0/145927369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 msgr2=0x56170e6f1f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 0x56170e6f1f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.772+0000 7fe4a6a00640 1 -- 172.21.15.49:0/145927369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 msgr2=0x56170e6f3aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:26:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.773+0000 7fe4a7400640 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.773+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.774+0000 7fe4a6a00640 1 -- 172.21.15.49:0/145927369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56170e60cdc0 con 0x56170e6e4530 2024-02-26T06:26:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.775+0000 7fe4a6a00640 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e4530 0x56170e6f3fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe494006010 tx=0x7fe4940f8150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d127a6f3c0a8da server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.775+0000 7fe491600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.775+0000 7fe4a7400640 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.775+0000 7fe491600640 1 -- 172.21.15.49:0/145927369 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940f3950 con 0x56170e6e4530 2024-02-26T06:26:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.775+0000 7fe491600640 1 -- 172.21.15.49:0/145927369 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe494047e60 con 0x56170e6e4530 2024-02-26T06:26:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.775+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4940f0c80 con 0x56170e6e4530 2024-02-26T06:26:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.776+0000 7fe491600640 1 -- 172.21.15.49:0/145927369 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4940bcc10 con 0x56170e6e4530 2024-02-26T06:26:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.776+0000 7fe4b1322740 10 client.26252 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.776+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56170e60b620 con 0x56170e6e4530 2024-02-26T06:26:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.777+0000 7fe491600640 1 -- 172.21.15.49:0/145927369 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4940d5020 con 0x56170e6e4530 2024-02-26T06:26:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.777+0000 7fe491600640 1 -- 172.21.15.49:0/145927369 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe49403c070 con 0x56170e6e4530 2024-02-26T06:26:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.777+0000 7fe4b1322740 10 client.26252 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.777+0000 7fe4b1322740 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.777+0000 7fe4b1322740 20 client.26252 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.777+0000 7fe4b1322740 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56170e754160 0x56170e756580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.777+0000 7fe4b1322740 4 client.26252 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:26:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.777+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56170e3d6890 con 0x56170e754160 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.778+0000 7fe4a7400640 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56170e754160 0x56170e756580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.778+0000 7fe491600640 10 client.26252 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.778+0000 7fe4a7400640 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56170e754160 0x56170e756580 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5813fe3f30e35dcd server_cookie=f0f22c8312cbb027 in_seq=0 out_seq=0 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.778+0000 7fe491600640 1 -- 172.21.15.49:0/145927369 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56170e3d6890 con 0x56170e754160 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.778+0000 7fe491600640 10 client.26252 handle_command_reply: tid=0 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.778+0000 7fe4b1322740 1 client.26252 shutdown 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e4530 msgr2=0x56170e6f3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e4530 0x56170e6f3fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe494006010 tx=0x7fe4940f8150 comp rx=0 tx=0).stop 2024-02-26T06:26:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 shutdown_connections 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56170e754160 0x56170e756580 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56170e6e4530 0x56170e6f3fe0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56170e6ed110 0x56170e6f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 --2- 172.21.15.49:0/145927369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56170e6e3c10 0x56170e6f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 >> 172.21.15.49:0/145927369 conn(0x56170e6de340 msgr2=0x56170e6e8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe491600640 0 client.26252 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 shutdown_connections 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 1 -- 172.21.15.49:0/145927369 wait complete. 2024-02-26T06:26:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.779+0000 7fe4b1322740 20 client.26252 trim_cache size 0 max 16384 2024-02-26T06:26:35.782 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:26:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.780+0000 7fe4ae800640 1 -- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe48c0a2310 msgr2=0x7fe48c0a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.780+0000 7fe4ae800640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe48c0a2310 0x7fe48c0a47d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000f470 tx=0x7fe4a0007680 comp rx=0 tx=0).stop 2024-02-26T06:26:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.780+0000 7fe4ae800640 1 -- 172.21.15.49:0/2273628391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a81080d0 msgr2=0x7fe4a819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.780+0000 7fe4ae800640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a81080d0 0x7fe4a819e420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe494000c00 tx=0x7fe494004040 comp rx=0 tx=0).stop 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.781+0000 7fe4ae800640 1 -- 172.21.15.49:0/2273628391 shutdown_connections 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.781+0000 7fe4ae800640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe48c0a2310 0x7fe48c0a47d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.781+0000 7fe4ae800640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4a810d720 0x7fe4a81a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.781+0000 7fe4ae800640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a81080d0 0x7fe4a819e420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.781+0000 7fe4ae800640 1 --2- 172.21.15.49:0/2273628391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4a8107710 0x7fe4a819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.781+0000 7fe4ae800640 1 -- 172.21.15.49:0/2273628391 >> 172.21.15.49:0/2273628391 conn(0x7fe4a8075690 msgr2=0x7fe4a8075eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.781+0000 7fe4ae800640 1 -- 172.21.15.49:0/2273628391 shutdown_connections 2024-02-26T06:26:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:35.781+0000 7fe4ae800640 1 -- 172.21.15.49:0/2273628391 wait complete. 2024-02-26T06:26:36.304 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 165.0 s, 3177.4 tps, lat 31.360 ms stddev 33.753 2024-02-26T06:26:36.807 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:26:36.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:26:36.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:26:37.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.146+0000 7fe506800640 1 -- 172.21.15.49:0/2523469897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f809d500 msgr2=0x7fe4f80a0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.146+0000 7fe506800640 1 --2- 172.21.15.49:0/2523469897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f809d500 0x7fe4f80a0f30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009860 tx=0x7fe4f402bb70 comp rx=0 tx=0).stop 2024-02-26T06:26:37.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.147+0000 7fe506800640 1 -- 172.21.15.49:0/2523469897 shutdown_connections 2024-02-26T06:26:37.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.147+0000 7fe506800640 1 --2- 172.21.15.49:0/2523469897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4f80a1610 0x7fe4f80a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.147+0000 7fe506800640 1 --2- 172.21.15.49:0/2523469897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f809d500 0x7fe4f80a0f30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.147+0000 7fe506800640 1 --2- 172.21.15.49:0/2523469897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f809cb50 0x7fe4f809cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.147+0000 7fe506800640 1 -- 172.21.15.49:0/2523469897 >> 172.21.15.49:0/2523469897 conn(0x7fe4f80092a0 msgr2=0x7fe4f80096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.148+0000 7fe506800640 1 -- 172.21.15.49:0/2523469897 shutdown_connections 2024-02-26T06:26:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.148+0000 7fe506800640 1 -- 172.21.15.49:0/2523469897 wait complete. 2024-02-26T06:26:37.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.149+0000 7fe506800640 1 Processor -- start 2024-02-26T06:26:37.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.150+0000 7fe506800640 1 -- start start 2024-02-26T06:26:37.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.152+0000 7fe506800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4f809cb50 0x7fe4f8014ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.152+0000 7fe506800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f80a1610 0x7fe4f8015430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.152+0000 7fe506800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f800dfc0 0x7fe4f800e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.152+0000 7fe506800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f800ec00 con 0x7fe4f800dfc0 2024-02-26T06:26:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.152+0000 7fe506800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f800ed70 con 0x7fe4f809cb50 2024-02-26T06:26:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.152+0000 7fe506800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f8012a10 con 0x7fe4f80a1610 2024-02-26T06:26:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.153+0000 7fe505e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f800dfc0 0x7fe4f800e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.153+0000 7fe505e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f800dfc0 0x7fe4f800e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51876/0 (socket says 172.21.15.49:51876) 2024-02-26T06:26:37.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.153+0000 7fe505e00640 1 -- 172.21.15.49:0/2658740413 learned_addr learned my addr 172.21.15.49:0/2658740413 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.153+0000 7fe504a00640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f80a1610 0x7fe4f8015430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.154+0000 7fe504a00640 1 -- 172.21.15.49:0/2658740413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4f809cb50 msgr2=0x7fe4f8014ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.154+0000 7fe504a00640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4f809cb50 0x7fe4f8014ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.154+0000 7fe504a00640 1 -- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f800dfc0 msgr2=0x7fe4f800e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.154+0000 7fe504a00640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f800dfc0 0x7fe4f800e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.154+0000 7fe504a00640 1 -- 172.21.15.49:0/2658740413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4009490 con 0x7fe4f80a1610 2024-02-26T06:26:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.154+0000 7fe505e00640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f800dfc0 0x7fe4f800e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.154+0000 7fe504a00640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f80a1610 0x7fe4f8015430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402c0b0 tx=0x7fe4f4034440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.155+0000 7fe4fe000640 1 -- 172.21.15.49:0/2658740413 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4034a40 con 0x7fe4f80a1610 2024-02-26T06:26:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.155+0000 7fe4fe000640 1 -- 172.21.15.49:0/2658740413 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4f402c9f0 con 0x7fe4f80a1610 2024-02-26T06:26:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.155+0000 7fe4fe000640 1 -- 172.21.15.49:0/2658740413 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40035f0 con 0x7fe4f80a1610 2024-02-26T06:26:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.155+0000 7fe506800640 1 -- 172.21.15.49:0/2658740413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f8012c90 con 0x7fe4f80a1610 2024-02-26T06:26:37.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.155+0000 7fe506800640 1 -- 172.21.15.49:0/2658740413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f80130f0 con 0x7fe4f80a1610 2024-02-26T06:26:37.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.161+0000 7fe4fe000640 1 -- 172.21.15.49:0/2658740413 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe4f403b040 con 0x7fe4f80a1610 2024-02-26T06:26:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.163+0000 7fe4fe000640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe4e80a22b0 0x7fe4e80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.163+0000 7fe4fe000640 1 -- 172.21.15.49:0/2658740413 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe4f40efaa0 con 0x7fe4f80a1610 2024-02-26T06:26:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.164+0000 7fe505400640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe4e80a22b0 0x7fe4e80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.166+0000 7fe505400640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe4e80a22b0 0x7fe4e80a4770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe500066cc0 tx=0x7fe50004f890 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.167+0000 7fe50ac46740 1 Processor -- start 2024-02-26T06:26:37.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.168+0000 7fe50ac46740 1 -- start start 2024-02-26T06:26:37.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.168+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9c110 0x5565b7e9c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.168+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9ca30 0x5565b7ea50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.168+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.168+0000 7fe50ac46740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7e572d0 con 0x5565b7e9ca30 2024-02-26T06:26:37.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.168+0000 7fe50ac46740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7e09e20 con 0x5565b7e9c110 2024-02-26T06:26:37.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.168+0000 7fe50ac46740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7e07350 con 0x5565b7ea5610 2024-02-26T06:26:37.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.169+0000 7fe505e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.169+0000 7fe505e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58806/0 (socket says 172.21.15.49:58806) 2024-02-26T06:26:37.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.169+0000 7fe505e00640 1 -- 172.21.15.49:0/3536479691 learned_addr learned my addr 172.21.15.49:0/3536479691 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.169+0000 7fe505400640 1 --2- 172.21.15.49:0/3536479691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9c110 0x5565b7e9c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.170+0000 7fe505400640 1 -- 172.21.15.49:0/3536479691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 msgr2=0x5565b7ea7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.170+0000 7fe505400640 1 --2- 172.21.15.49:0/3536479691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.170+0000 7fe505400640 1 -- 172.21.15.49:0/3536479691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9ca30 msgr2=0x5565b7ea50d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:26:37.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.170+0000 7fe505400640 1 --2- 172.21.15.49:0/3536479691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9ca30 0x5565b7ea50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.170+0000 7fe505400640 1 -- 172.21.15.49:0/3536479691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565b7dcbc50 con 0x5565b7e9c110 2024-02-26T06:26:37.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.171+0000 7fe505400640 1 --2- 172.21.15.49:0/3536479691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9c110 0x5565b7e9c4f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe500066a40 tx=0x7fe50007e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6357529466b86b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.172+0000 7fe4ee600640 1 -- 172.21.15.49:0/3536479691 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500082610 con 0x5565b7e9c110 2024-02-26T06:26:37.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.172+0000 7fe4ee600640 1 -- 172.21.15.49:0/3536479691 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe500088040 con 0x5565b7e9c110 2024-02-26T06:26:37.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.172+0000 7fe4ee600640 1 -- 172.21.15.49:0/3536479691 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50008cae0 con 0x5565b7e9c110 2024-02-26T06:26:37.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 -- 172.21.15.49:0/3536479691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9c110 msgr2=0x5565b7e9c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 --2- 172.21.15.49:0/3536479691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9c110 0x5565b7e9c4f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe500066a40 tx=0x7fe50007e000 comp rx=0 tx=0).stop 2024-02-26T06:26:37.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 -- 172.21.15.49:0/3536479691 shutdown_connections 2024-02-26T06:26:37.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 --2- 172.21.15.49:0/3536479691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 --2- 172.21.15.49:0/3536479691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9ca30 0x5565b7ea50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 --2- 172.21.15.49:0/3536479691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9c110 0x5565b7e9c4f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 -- 172.21.15.49:0/3536479691 >> 172.21.15.49:0/3536479691 conn(0x5565b7e96840 msgr2=0x5565b7e96c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 -- 172.21.15.49:0/3536479691 shutdown_connections 2024-02-26T06:26:37.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.174+0000 7fe50ac46740 1 -- 172.21.15.49:0/3536479691 wait complete. 2024-02-26T06:26:37.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.175+0000 7fe50ac46740 1 -- start start 2024-02-26T06:26:37.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.178+0000 7fe50ac46740 1 Processor -- start 2024-02-26T06:26:37.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.179+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9c110 0x5565b7ea25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.179+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.179+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.179+0000 7fe50ac46740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7e02b30 con 0x5565b7e9c110 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.179+0000 7fe50ac46740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7da7fa0 con 0x5565b7e9ca30 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.179+0000 7fe50ac46740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7da7c40 con 0x5565b7ea5610 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.181+0000 7fe50ac46740 10 client.0 _ll_register_callbacks cb 0x5565b7b13c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.181+0000 7fe504a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.181+0000 7fe505e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.181+0000 7fe504a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34610/0 (socket says 172.21.15.49:34610) 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.181+0000 7fe504a00640 1 -- 172.21.15.49:0/2335239395 learned_addr learned my addr 172.21.15.49:0/2335239395 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.182+0000 7fe505400640 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9c110 0x5565b7ea25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.182+0000 7fe504a00640 1 -- 172.21.15.49:0/2335239395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 msgr2=0x5565b7ea3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.182+0000 7fe504a00640 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.182+0000 7fe504a00640 1 -- 172.21.15.49:0/2335239395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9c110 msgr2=0x5565b7ea25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.182+0000 7fe504a00640 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9c110 0x5565b7ea25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.182+0000 7fe504a00640 1 -- 172.21.15.49:0/2335239395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe500076a30 con 0x5565b7e9ca30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.183+0000 7fe504a00640 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2b30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40ed3c0 tx=0x7fe4f40bc170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=932d01ec9300f65f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.183+0000 7fe4e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.184+0000 7fe4e2a00640 1 -- 172.21.15.49:0/2335239395 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40bc5b0 con 0x5565b7e9ca30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.184+0000 7fe4e2a00640 1 -- 172.21.15.49:0/2335239395 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe4f40c0700 con 0x5565b7e9ca30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.184+0000 7fe50ac46740 1 -- 172.21.15.49:0/2335239395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565b7b02860 con 0x5565b7e9ca30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe4e2a00640 1 -- 172.21.15.49:0/2335239395 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4f40c4d50 con 0x5565b7e9ca30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe50ac46740 10 client.26258 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe50ac46740 1 -- 172.21.15.49:0/2335239395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5565b7e011b0 con 0x5565b7e9ca30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe4e2a00640 1 -- 172.21.15.49:0/2335239395 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5565b7b02860 con 0x5565b7e9ca30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe4e2a00640 1 -- 172.21.15.49:0/2335239395 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe4f40dad20 con 0x5565b7e9ca30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe50ac46740 10 client.26258 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:37.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe50ac46740 10 client.26258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:37.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe50ac46740 20 client.26258 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:37.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.185+0000 7fe50ac46740 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5565b7eb5720 0x5565b7eb7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.186+0000 7fe50ac46740 4 client.26258 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:37.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.186+0000 7fe50ac46740 1 -- 172.21.15.49:0/2335239395 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565b7b53410 con 0x5565b7eb5720 2024-02-26T06:26:37.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.186+0000 7fe505400640 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5565b7eb5720 0x5565b7eb7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.186+0000 7fe505400640 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5565b7eb5720 0x5565b7eb7b40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7efd8e7e7573b85 server_cookie=6ed656b2b8542108 in_seq=0 out_seq=0 2024-02-26T06:26:37.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.186+0000 7fe4e2a00640 10 client.26258 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:37.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.187+0000 7fe4e2a00640 1 -- 172.21.15.49:0/2335239395 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565b7b53410 con 0x5565b7eb5720 2024-02-26T06:26:37.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.187+0000 7fe4e2a00640 10 client.26258 handle_command_reply: tid=0 2024-02-26T06:26:37.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 client.26258 shutdown 2024-02-26T06:26:37.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 -- 172.21.15.49:0/2335239395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 msgr2=0x5565b7ea2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2b30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40ed3c0 tx=0x7fe4f40bc170 comp rx=0 tx=0).stop 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 -- 172.21.15.49:0/2335239395 shutdown_connections 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5565b7eb5720 0x5565b7eb7b40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2b30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7ea5610 0x5565b7ea3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 --2- 172.21.15.49:0/2335239395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9c110 0x5565b7ea25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe50ac46740 1 -- 172.21.15.49:0/2335239395 >> 172.21.15.49:0/2335239395 conn(0x5565b7e96840 msgr2=0x5565b7e98590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.188+0000 7fe4e2a00640 0 client.26258 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.189+0000 7fe50ac46740 1 -- 172.21.15.49:0/2335239395 shutdown_connections 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.189+0000 7fe50ac46740 1 -- 172.21.15.49:0/2335239395 wait complete. 2024-02-26T06:26:37.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.189+0000 7fe50ac46740 20 client.26258 trim_cache size 0 max 16384 2024-02-26T06:26:37.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.215+0000 7fe50ac46740 1 Processor -- start 2024-02-26T06:26:37.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe50ac46740 1 -- start start 2024-02-26T06:26:37.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7e98380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7ea5610 0x5565b7eac570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe50ac46740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7b4b0c0 con 0x5565b7e9ca30 2024-02-26T06:26:37.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe50ac46740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7e9bd70 con 0x5565b7ea5610 2024-02-26T06:26:37.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe50ac46740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7e97890 con 0x5565b7e9c110 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe505400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7e98380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe505400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7e98380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58830/0 (socket says 172.21.15.49:58830) 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.216+0000 7fe505400640 1 -- 172.21.15.49:0/224800623 learned_addr learned my addr 172.21.15.49:0/224800623 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.217+0000 7fe505e00640 1 --2- 172.21.15.49:0/224800623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7ea5610 0x5565b7eac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.217+0000 7fe505400640 1 -- 172.21.15.49:0/224800623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7ea5610 msgr2=0x5565b7eac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.217+0000 7fe505400640 1 --2- 172.21.15.49:0/224800623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7ea5610 0x5565b7eac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.217+0000 7fe505400640 1 -- 172.21.15.49:0/224800623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9ca30 msgr2=0x5565b7ea2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.217+0000 7fe505400640 1 --2- 172.21.15.49:0/224800623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.217+0000 7fe505400640 1 -- 172.21.15.49:0/224800623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565b7d9da60 con 0x5565b7e9c110 2024-02-26T06:26:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.218+0000 7fe505400640 1 --2- 172.21.15.49:0/224800623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7e98380 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe500066f90 tx=0x7fe500081000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5de1575d74a132 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.219+0000 7fe4edc00640 1 -- 172.21.15.49:0/224800623 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50008c810 con 0x5565b7e9c110 2024-02-26T06:26:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.219+0000 7fe4edc00640 1 -- 172.21.15.49:0/224800623 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5565b7b53410 con 0x5565b7e9c110 2024-02-26T06:26:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.219+0000 7fe4edc00640 1 -- 172.21.15.49:0/224800623 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500097cd0 con 0x5565b7e9c110 2024-02-26T06:26:37.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.220+0000 7fe50ac46740 1 -- 172.21.15.49:0/224800623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 msgr2=0x5565b7e98380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.220+0000 7fe50ac46740 1 --2- 172.21.15.49:0/224800623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7e98380 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe500066f90 tx=0x7fe500081000 comp rx=0 tx=0).stop 2024-02-26T06:26:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.220+0000 7fe50ac46740 1 -- 172.21.15.49:0/224800623 shutdown_connections 2024-02-26T06:26:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.220+0000 7fe50ac46740 1 --2- 172.21.15.49:0/224800623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7ea5610 0x5565b7eac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.220+0000 7fe50ac46740 1 --2- 172.21.15.49:0/224800623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7e9ca30 0x5565b7ea2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.220+0000 7fe50ac46740 1 --2- 172.21.15.49:0/224800623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7e98380 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.220+0000 7fe50ac46740 1 -- 172.21.15.49:0/224800623 >> 172.21.15.49:0/224800623 conn(0x5565b7e96840 msgr2=0x5565b7e97c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.221+0000 7fe50ac46740 1 -- 172.21.15.49:0/224800623 shutdown_connections 2024-02-26T06:26:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.221+0000 7fe50ac46740 1 -- 172.21.15.49:0/224800623 wait complete. 2024-02-26T06:26:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.222+0000 7fe50ac46740 1 -- start start 2024-02-26T06:26:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.222+0000 7fe50ac46740 1 Processor -- start 2024-02-26T06:26:37.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.223+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7f04f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.223+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7f054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.223+0000 7fe50ac46740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7ea5610 0x5565b7eab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.223+0000 7fe50ac46740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7eabe30 con 0x5565b7ea5610 2024-02-26T06:26:37.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.223+0000 7fe50ac46740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7eabfa0 con 0x5565b7e9ca30 2024-02-26T06:26:37.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.223+0000 7fe50ac46740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b7eac160 con 0x5565b7e9c110 2024-02-26T06:26:37.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7f04f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe50ac46740 10 client.0 _ll_register_callbacks cb 0x7fe500076980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7f04f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58838/0 (socket says 172.21.15.49:58838) 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505400640 1 -- 172.21.15.49:0/748806092 learned_addr learned my addr 172.21.15.49:0/748806092 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe504a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7f054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505e00640 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7ea5610 0x5565b7eab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505400640 1 -- 172.21.15.49:0/748806092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 msgr2=0x5565b7f054a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505400640 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7f054a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505400640 1 -- 172.21.15.49:0/748806092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7ea5610 msgr2=0x5565b7eab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505400640 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7ea5610 0x5565b7eab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.224+0000 7fe505400640 1 -- 172.21.15.49:0/748806092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565b7dcbc50 con 0x5565b7e9c110 2024-02-26T06:26:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.225+0000 7fe505400640 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7f04f60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe50004f940 tx=0x7fe50008c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3150934c7ce3154 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.225+0000 7fe4e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.226+0000 7fe4e2a00640 1 -- 172.21.15.49:0/748806092 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500097c50 con 0x5565b7e9c110 2024-02-26T06:26:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.226+0000 7fe4e2a00640 1 -- 172.21.15.49:0/748806092 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe500097760 con 0x5565b7e9c110 2024-02-26T06:26:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.226+0000 7fe50ac46740 1 -- 172.21.15.49:0/748806092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe500088040 con 0x5565b7e9c110 2024-02-26T06:26:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.227+0000 7fe4e2a00640 1 -- 172.21.15.49:0/748806092 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe500097db0 con 0x5565b7e9c110 2024-02-26T06:26:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.227+0000 7fe50ac46740 10 client.26302 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.227+0000 7fe50ac46740 1 -- 172.21.15.49:0/748806092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5565b7dc3670 con 0x5565b7e9c110 2024-02-26T06:26:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.227+0000 7fe4e2a00640 1 -- 172.21.15.49:0/748806092 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe500088040 con 0x5565b7e9c110 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.228+0000 7fe4e2a00640 1 -- 172.21.15.49:0/748806092 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe50008b070 con 0x5565b7e9c110 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.228+0000 7fe50ac46740 10 client.26302 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.228+0000 7fe50ac46740 10 client.26302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.228+0000 7fe50ac46740 20 client.26302 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.228+0000 7fe50ac46740 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5565b7f0da50 0x5565b7f0fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.228+0000 7fe50ac46740 4 client.26302 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.228+0000 7fe50ac46740 1 -- 172.21.15.49:0/748806092 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5565b7b3f7b0 con 0x5565b7f0da50 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.228+0000 7fe504a00640 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5565b7f0da50 0x5565b7f0fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.229+0000 7fe4e2a00640 10 client.26302 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.229+0000 7fe504a00640 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5565b7f0da50 0x5565b7f0fe70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa99d93c0ac0ed9 server_cookie=2513e89067c392f4 in_seq=0 out_seq=0 2024-02-26T06:26:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe4e2a00640 1 -- 172.21.15.49:0/748806092 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5565b7b3f7b0 con 0x5565b7f0da50 2024-02-26T06:26:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe4e2a00640 10 client.26302 handle_command_reply: tid=0 2024-02-26T06:26:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 client.26302 shutdown 2024-02-26T06:26:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 -- 172.21.15.49:0/748806092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 msgr2=0x5565b7f04f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7f04f60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe50004f940 tx=0x7fe50008c480 comp rx=0 tx=0).stop 2024-02-26T06:26:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 -- 172.21.15.49:0/748806092 shutdown_connections 2024-02-26T06:26:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5565b7f0da50 0x5565b7f0fe70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5565b7e9ca30 0x5565b7f054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565b7ea5610 0x5565b7eab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 --2- 172.21.15.49:0/748806092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565b7e9c110 0x5565b7f04f60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe50ac46740 1 -- 172.21.15.49:0/748806092 >> 172.21.15.49:0/748806092 conn(0x5565b7e96840 msgr2=0x5565b7e98500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.231+0000 7fe4e2a00640 0 client.26302 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.232+0000 7fe50ac46740 1 -- 172.21.15.49:0/748806092 shutdown_connections 2024-02-26T06:26:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.232+0000 7fe50ac46740 1 -- 172.21.15.49:0/748806092 wait complete. 2024-02-26T06:26:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.232+0000 7fe50ac46740 20 client.26302 trim_cache size 0 max 16384 2024-02-26T06:26:37.233 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:26:37.234 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:26:37.234 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "7300e20a-d834-4435-b2ad-1367b1db2942", 2024-02-26T06:26:37.234 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:26:37.234 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 -- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe4e80a22b0 msgr2=0x7fe4e80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe4e80a22b0 0x7fe4e80a4770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe500066cc0 tx=0x7fe50004f890 comp rx=0 tx=0).stop 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 -- 172.21.15.49:0/2658740413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f80a1610 msgr2=0x7fe4f8015430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f80a1610 0x7fe4f8015430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402c0b0 tx=0x7fe4f4034440 comp rx=0 tx=0).stop 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 -- 172.21.15.49:0/2658740413 shutdown_connections 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe4e80a22b0 0x7fe4e80a4770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f800dfc0 0x7fe4f800e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4f80a1610 0x7fe4f8015430 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 --2- 172.21.15.49:0/2658740413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4f809cb50 0x7fe4f8014ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 -- 172.21.15.49:0/2658740413 >> 172.21.15.49:0/2658740413 conn(0x7fe4f80092a0 msgr2=0x7fe4f809a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 -- 172.21.15.49:0/2658740413 shutdown_connections 2024-02-26T06:26:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.233+0000 7fe506800640 1 -- 172.21.15.49:0/2658740413 wait complete. 2024-02-26T06:26:37.249 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:36 smithi080 ceph-mon[36206]: pgmap v99: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 986 op/s 2024-02-26T06:26:37.249 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:36 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:26:37.260 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:36 smithi049 ceph-mon[31696]: pgmap v99: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 986 op/s 2024-02-26T06:26:37.260 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:36 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:26:37.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:26:37.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:26:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:36 smithi115 ceph-mon[36426]: pgmap v99: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 986 op/s 2024-02-26T06:26:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:36 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:26:37.580 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:26:37 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: 2024-02-26T06:26:37.248+0000 7f2b87eed700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...5,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_subtrans/ auth{1=1,2=1} v1262 ap=2 f(v4 m2024-02-26T06:26:16.475293+0000 5=5+0) n(v6 rc2024-02-26T06:26:16.475293+0000 b1024000 6=5+1) old_inodes=3 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25193=pAsLsXsFs/pAsLsXsFsx@14} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5582523e0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v252>,<0x10000000005/data v111>,<0x100000001fb/postgres v19>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v33>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v38>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v1262>,<0x10000000005/data v325>,<0x100000001fb/postgres v161>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v182>,<0x10000000001/sv_1 v200>,<0x10000000000/qa v189>,<0x1/volumes v175>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:25:39.752300+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:26:16.475293+0000 b1024000 6=5+1)","memory_value.dirstat":"f(v4 m2024-02-26T06:26:16.475293+0000 5=5+0)","memory_value.rstat":"n(v6 rc2024-02-26T06:26:16.475293+0000 b1024000 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-02-26T06:26:37.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.594+0000 7f9742c00640 1 -- 172.21.15.49:0/1491757845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c107230 msgr2=0x7f973c107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.594+0000 7f9742c00640 1 --2- 172.21.15.49:0/1491757845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c107230 0x7f973c107610 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f972c009890 tx=0x7f972c02bb70 comp rx=0 tx=0).stop 2024-02-26T06:26:37.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.595+0000 7f9742c00640 1 -- 172.21.15.49:0/1491757845 shutdown_connections 2024-02-26T06:26:37.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.595+0000 7f9742c00640 1 --2- 172.21.15.49:0/1491757845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c06a0b0 0x7f973c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.595+0000 7f9742c00640 1 --2- 172.21.15.49:0/1491757845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f973c0696f0 0x7f973c069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.595+0000 7f9742c00640 1 --2- 172.21.15.49:0/1491757845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c107230 0x7f973c107610 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.595+0000 7f9742c00640 1 -- 172.21.15.49:0/1491757845 >> 172.21.15.49:0/1491757845 conn(0x7f973c077060 msgr2=0x7f973c077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.595+0000 7f9742c00640 1 -- 172.21.15.49:0/1491757845 shutdown_connections 2024-02-26T06:26:37.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.595+0000 7f9742c00640 1 -- 172.21.15.49:0/1491757845 wait complete. 2024-02-26T06:26:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.597+0000 7f9742c00640 1 Processor -- start 2024-02-26T06:26:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.597+0000 7f9742c00640 1 -- start start 2024-02-26T06:26:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f9742c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f973c0696f0 0x7f973c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f9742c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c06a0b0 0x7f973c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f9742c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c107230 0x7f973c1a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f9742c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1a2bf0 con 0x7f973c107230 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f9742c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1a2d60 con 0x7f973c06a0b0 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f9742c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1a2f20 con 0x7f973c0696f0 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f973b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c06a0b0 0x7f973c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f973b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c06a0b0 0x7f973c19e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34650/0 (socket says 172.21.15.49:34650) 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.598+0000 7f973b400640 1 -- 172.21.15.49:0/3858029318 learned_addr learned my addr 172.21.15.49:0/3858029318 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.599+0000 7f973be00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f973c0696f0 0x7f973c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.599+0000 7f973b400640 1 -- 172.21.15.49:0/3858029318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f973c0696f0 msgr2=0x7f973c19dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.599+0000 7f973b400640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f973c0696f0 0x7f973c19dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.599+0000 7f973b400640 1 -- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c107230 msgr2=0x7f973c1a25f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:26:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.599+0000 7f973b400640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c107230 0x7f973c1a25f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.599+0000 7f973b400640 1 -- 172.21.15.49:0/3858029318 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c009490 con 0x7f973c06a0b0 2024-02-26T06:26:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.599+0000 7f973be00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f973c0696f0 0x7f973c19dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.601+0000 7f973b400640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c06a0b0 0x7f973c19e270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f972800e9c0 tx=0x7f972800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.601+0000 7f9738c00640 1 -- 172.21.15.49:0/3858029318 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800ccc0 con 0x7f973c06a0b0 2024-02-26T06:26:37.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.601+0000 7f9738c00640 1 -- 172.21.15.49:0/3858029318 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f972800ce20 con 0x7f973c06a0b0 2024-02-26T06:26:37.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.601+0000 7f9738c00640 1 -- 172.21.15.49:0/3858029318 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728010430 con 0x7f973c06a0b0 2024-02-26T06:26:37.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.601+0000 7f9742c00640 1 -- 172.21.15.49:0/3858029318 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c1a3200 con 0x7f973c06a0b0 2024-02-26T06:26:37.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.602+0000 7f9742c00640 1 -- 172.21.15.49:0/3858029318 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c194870 con 0x7f973c06a0b0 2024-02-26T06:26:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.608+0000 7f9738c00640 1 -- 172.21.15.49:0/3858029318 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9728005020 con 0x7f973c06a0b0 2024-02-26T06:26:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.611+0000 7f9738c00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f97200a2290 0x7f97200a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.612+0000 7f9745a5c740 1 Processor -- start 2024-02-26T06:26:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.612+0000 7f9738c00640 1 -- 172.21.15.49:0/3858029318 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9728014070 con 0x7f973c06a0b0 2024-02-26T06:26:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.612+0000 7f973be00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f97200a2290 0x7f97200a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.614+0000 7f9745a5c740 1 -- start start 2024-02-26T06:26:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.614+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48f70 0x557bbdb47cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.614+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb48710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.614+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb51600 0x557bbdb539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.614+0000 7f9745a5c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdab7120 con 0x557bbdb48f70 2024-02-26T06:26:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.614+0000 7f9745a5c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdab4650 con 0x557bbdb51600 2024-02-26T06:26:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.614+0000 7f9745a5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdaafe30 con 0x557bbdb48290 2024-02-26T06:26:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.615+0000 7f973b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb48710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.615+0000 7f973b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb48710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58854/0 (socket says 172.21.15.49:58854) 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.615+0000 7f973b400640 1 -- 172.21.15.49:0/2309140128 learned_addr learned my addr 172.21.15.49:0/2309140128 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.615+0000 7f9740c00640 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb51600 0x557bbdb539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.615+0000 7f973be00640 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48f70 0x557bbdb47cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.615+0000 7f973be00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f97200a2290 0x7f97200a4750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f972c032530 tx=0x7f972c000bc0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.615+0000 7f973b400640 1 -- 172.21.15.49:0/2309140128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb51600 msgr2=0x557bbdb539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.616+0000 7f973b400640 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb51600 0x557bbdb539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.616+0000 7f973b400640 1 -- 172.21.15.49:0/2309140128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48f70 msgr2=0x557bbdb47cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.616+0000 7f973b400640 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48f70 0x557bbdb47cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.616+0000 7f973b400640 1 -- 172.21.15.49:0/2309140128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bbda69030 con 0x557bbdb48290 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.616+0000 7f973b400640 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb48710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f97280972f0 tx=0x7f972809b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d402a78ba4843ac8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.616+0000 7f9731200640 1 -- 172.21.15.49:0/2309140128 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280a3630 con 0x557bbdb48290 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.616+0000 7f9731200640 1 -- 172.21.15.49:0/2309140128 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9728099190 con 0x557bbdb48290 2024-02-26T06:26:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.616+0000 7f9731200640 1 -- 172.21.15.49:0/2309140128 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728016330 con 0x557bbdb48290 2024-02-26T06:26:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 -- 172.21.15.49:0/2309140128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 msgr2=0x557bbdb48710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb48710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f97280972f0 tx=0x7f972809b650 comp rx=0 tx=0).stop 2024-02-26T06:26:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 -- 172.21.15.49:0/2309140128 shutdown_connections 2024-02-26T06:26:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb51600 0x557bbdb539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb48710 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2309140128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48f70 0x557bbdb47cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 -- 172.21.15.49:0/2309140128 >> 172.21.15.49:0/2309140128 conn(0x557bbdb0f570 msgr2=0x557bbdb42ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 -- 172.21.15.49:0/2309140128 shutdown_connections 2024-02-26T06:26:37.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.617+0000 7f9745a5c740 1 -- 172.21.15.49:0/2309140128 wait complete. 2024-02-26T06:26:37.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.619+0000 7f9745a5c740 1 -- start start 2024-02-26T06:26:37.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.620+0000 7f9745a5c740 1 Processor -- start 2024-02-26T06:26:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48290 0x557bbdb4e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48f70 0x557bbdb4eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f9745a5c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbda54e90 con 0x557bbdb48290 2024-02-26T06:26:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f9745a5c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbda54b30 con 0x557bbdb48f70 2024-02-26T06:26:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f9745a5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbd81e1c0 con 0x557bbdb51600 2024-02-26T06:26:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f9745a5c740 10 client.0 _ll_register_callbacks cb 0x557bbda68930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f973b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48f70 0x557bbdb4eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f973b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48f70 0x557bbdb4eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34660/0 (socket says 172.21.15.49:34660) 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.621+0000 7f973b400640 1 -- 172.21.15.49:0/963371348 learned_addr learned my addr 172.21.15.49:0/963371348 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.622+0000 7f9740c00640 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.622+0000 7f973b400640 1 -- 172.21.15.49:0/963371348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 msgr2=0x557bbdb4f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.622+0000 7f973b400640 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.622+0000 7f973b400640 1 -- 172.21.15.49:0/963371348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48290 msgr2=0x557bbdb4e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.622+0000 7f973b400640 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48290 0x557bbdb4e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.622+0000 7f973b400640 1 -- 172.21.15.49:0/963371348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c194870 con 0x557bbdb48f70 2024-02-26T06:26:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.622+0000 7f973b400640 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48f70 0x557bbdb4eb40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f97280cbca0 tx=0x7f9728009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b5fbbe7e6efd73 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.622+0000 7f9725600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:26:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.623+0000 7f9725600640 1 -- 172.21.15.49:0/963371348 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280053d0 con 0x557bbdb48f70 2024-02-26T06:26:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.623+0000 7f9725600640 1 -- 172.21.15.49:0/963371348 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9728097500 con 0x557bbdb48f70 2024-02-26T06:26:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.623+0000 7f9745a5c740 1 -- 172.21.15.49:0/963371348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bbd7af7e0 con 0x557bbdb48f70 2024-02-26T06:26:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.624+0000 7f9725600640 1 -- 172.21.15.49:0/963371348 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9728016330 con 0x557bbdb48f70 2024-02-26T06:26:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.624+0000 7f9745a5c740 10 client.26276 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.624+0000 7f9745a5c740 1 -- 172.21.15.49:0/963371348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557bbdaae4b0 con 0x557bbdb48f70 2024-02-26T06:26:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.625+0000 7f9725600640 1 -- 172.21.15.49:0/963371348 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97280a7030 con 0x557bbdb48f70 2024-02-26T06:26:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.625+0000 7f9725600640 1 -- 172.21.15.49:0/963371348 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f972801f060 con 0x557bbdb48f70 2024-02-26T06:26:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.625+0000 7f9745a5c740 10 client.26276 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.625+0000 7f9745a5c740 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.625+0000 7f9745a5c740 20 client.26276 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.625+0000 7f9745a5c740 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557bbdb72400 0x557bbdb74860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.625+0000 7f9745a5c740 4 client.26276 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.625+0000 7f9745a5c740 1 -- 172.21.15.49:0/963371348 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bbd7ec720 con 0x557bbdb72400 2024-02-26T06:26:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.626+0000 7f973be00640 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557bbdb72400 0x557bbdb74860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.628+0000 7f9725600640 10 client.26276 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:37.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.628+0000 7f973be00640 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557bbdb72400 0x557bbdb74860 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cfed9492add8645 server_cookie=c9ccb1d794300e47 in_seq=0 out_seq=0 2024-02-26T06:26:37.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.628+0000 7f9725600640 1 -- 172.21.15.49:0/963371348 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bbd7ec720 con 0x557bbdb72400 2024-02-26T06:26:37.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.628+0000 7f9725600640 10 client.26276 handle_command_reply: tid=0 2024-02-26T06:26:37.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.629+0000 7f9745a5c740 1 client.26276 shutdown 2024-02-26T06:26:37.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.629+0000 7f9745a5c740 1 -- 172.21.15.49:0/963371348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48f70 msgr2=0x557bbdb4eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.629+0000 7f9745a5c740 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48f70 0x557bbdb4eb40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f97280cbca0 tx=0x7f9728009f90 comp rx=0 tx=0).stop 2024-02-26T06:26:37.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.630+0000 7f9745a5c740 1 -- 172.21.15.49:0/963371348 shutdown_connections 2024-02-26T06:26:37.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.630+0000 7f9745a5c740 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557bbdb72400 0x557bbdb74860 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.630+0000 7f9745a5c740 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48f70 0x557bbdb4eb40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.630+0000 7f9745a5c740 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.630+0000 7f9745a5c740 1 --2- 172.21.15.49:0/963371348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb48290 0x557bbdb4e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.630+0000 7f9745a5c740 1 -- 172.21.15.49:0/963371348 >> 172.21.15.49:0/963371348 conn(0x557bbdb0f570 msgr2=0x557bbdb44a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.631+0000 7f9725600640 0 client.26276 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:37.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.632+0000 7f9745a5c740 1 -- 172.21.15.49:0/963371348 shutdown_connections 2024-02-26T06:26:37.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.632+0000 7f9745a5c740 1 -- 172.21.15.49:0/963371348 wait complete. 2024-02-26T06:26:37.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.632+0000 7f9745a5c740 20 client.26276 trim_cache size 0 max 16384 2024-02-26T06:26:37.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.664+0000 7f9745a5c740 1 Processor -- start 2024-02-26T06:26:37.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.665+0000 7f9745a5c740 1 -- start start 2024-02-26T06:26:37.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.665+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb50d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.665+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb51600 0x557bbdb69510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.665+0000 7f973be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb50d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.665+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.666+0000 7f973be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb50d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58874/0 (socket says 172.21.15.49:58874) 2024-02-26T06:26:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.666+0000 7f9745a5c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdb4a260 con 0x557bbdb51600 2024-02-26T06:26:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.666+0000 7f9745a5c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdb3e5b0 con 0x557bbdb44cf0 2024-02-26T06:26:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.666+0000 7f9745a5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdb3d2c0 con 0x557bbdb48290 2024-02-26T06:26:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.666+0000 7f973be00640 1 -- 172.21.15.49:0/3458443000 learned_addr learned my addr 172.21.15.49:0/3458443000 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.666+0000 7f9740c00640 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.666+0000 7f973b400640 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb51600 0x557bbdb69510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.667+0000 7f9740c00640 1 -- 172.21.15.49:0/3458443000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 msgr2=0x557bbdb50d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.667+0000 7f9740c00640 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb50d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.667+0000 7f9740c00640 1 -- 172.21.15.49:0/3458443000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb51600 msgr2=0x557bbdb69510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.667+0000 7f9740c00640 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb51600 0x557bbdb69510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.667+0000 7f9740c00640 1 -- 172.21.15.49:0/3458443000 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bbda4a920 con 0x557bbdb44cf0 2024-02-26T06:26:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.667+0000 7f973be00640 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb50d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.668+0000 7f9740c00640 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f973400ad80 tx=0x7f9734010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0e0c67366c1011 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.668+0000 7f9727e00640 1 -- 172.21.15.49:0/3458443000 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734015070 con 0x557bbdb44cf0 2024-02-26T06:26:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.669+0000 7f9727e00640 1 -- 172.21.15.49:0/3458443000 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9734003e50 con 0x557bbdb44cf0 2024-02-26T06:26:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.669+0000 7f9727e00640 1 -- 172.21.15.49:0/3458443000 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734015070 con 0x557bbdb44cf0 2024-02-26T06:26:37.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 -- 172.21.15.49:0/3458443000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb44cf0 msgr2=0x557bbdb69a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f973400ad80 tx=0x7f9734010800 comp rx=0 tx=0).stop 2024-02-26T06:26:37.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 -- 172.21.15.49:0/3458443000 shutdown_connections 2024-02-26T06:26:37.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb51600 0x557bbdb69510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 --2- 172.21.15.49:0/3458443000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb48290 0x557bbdb50d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 -- 172.21.15.49:0/3458443000 >> 172.21.15.49:0/3458443000 conn(0x557bbdb0f570 msgr2=0x557bbdb79200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 -- 172.21.15.49:0/3458443000 shutdown_connections 2024-02-26T06:26:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.670+0000 7f9745a5c740 1 -- 172.21.15.49:0/3458443000 wait complete. 2024-02-26T06:26:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.671+0000 7f9745a5c740 1 -- start start 2024-02-26T06:26:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.671+0000 7f9745a5c740 1 Processor -- start 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.672+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.672+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48290 0x557bbdb4b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.672+0000 7f9745a5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.672+0000 7f9745a5c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdb4be90 con 0x557bbdb44cf0 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.672+0000 7f9745a5c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdbc8980 con 0x557bbdb48290 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.672+0000 7f9745a5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bbdbc8ac0 con 0x557bbdb51600 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9745a5c740 10 client.0 _ll_register_callbacks cb 0x7f973400a2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9740c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9740c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58878/0 (socket says 172.21.15.49:58878) 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9740c00640 1 -- 172.21.15.49:0/2236263490 learned_addr learned my addr 172.21.15.49:0/2236263490 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f973be00640 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f973b400640 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48290 0x557bbdb4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9740c00640 1 -- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48290 msgr2=0x557bbdb4b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9740c00640 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48290 0x557bbdb4b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9740c00640 1 -- 172.21.15.49:0/2236263490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb44cf0 msgr2=0x557bbdb69a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9740c00640 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.673+0000 7f9740c00640 1 -- 172.21.15.49:0/2236263490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c10e8b0 con 0x557bbdb51600 2024-02-26T06:26:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.674+0000 7f973b400640 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48290 0x557bbdb4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.674+0000 7f9725600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.674+0000 7f9740c00640 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4b7c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9734005c10 tx=0x7f973400a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8600de4eae04c330 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.674+0000 7f9725600640 1 -- 172.21.15.49:0/2236263490 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734015040 con 0x557bbdb51600 2024-02-26T06:26:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.675+0000 7f9725600640 1 -- 172.21.15.49:0/2236263490 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f973400f980 con 0x557bbdb51600 2024-02-26T06:26:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.675+0000 7f9745a5c740 1 -- 172.21.15.49:0/2236263490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9728099190 con 0x557bbdb51600 2024-02-26T06:26:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.675+0000 7f9725600640 1 -- 172.21.15.49:0/2236263490 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9734004020 con 0x557bbdb51600 2024-02-26T06:26:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.675+0000 7f9745a5c740 10 client.26317 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.675+0000 7f9745a5c740 1 -- 172.21.15.49:0/2236263490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f973400aa30 con 0x557bbdb51600 2024-02-26T06:26:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.675+0000 7f9725600640 1 -- 172.21.15.49:0/2236263490 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9728099190 con 0x557bbdb51600 2024-02-26T06:26:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.675+0000 7f9725600640 1 -- 172.21.15.49:0/2236263490 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9734021070 con 0x557bbdb51600 2024-02-26T06:26:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.676+0000 7f9745a5c740 10 client.26317 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.676+0000 7f9745a5c740 10 client.26317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.676+0000 7f9745a5c740 20 client.26317 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.676+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557bbdbca240 0x557bbdbcc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.676+0000 7f9745a5c740 4 client.26317 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:26:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.676+0000 7f9745a5c740 1 -- 172.21.15.49:0/2236263490 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557bbd83b890 con 0x557bbdbca240 2024-02-26T06:26:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.676+0000 7f973be00640 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557bbdbca240 0x557bbdbcc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.678+0000 7f973be00640 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557bbdbca240 0x557bbdbcc660 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea54f54d363cb6f server_cookie=c31d8270004a5d01 in_seq=0 out_seq=0 2024-02-26T06:26:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.678+0000 7f9725600640 10 client.26317 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9725600640 1 -- 172.21.15.49:0/2236263490 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557bbd83b890 con 0x557bbdbca240 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9725600640 10 client.26317 handle_command_reply: tid=0 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 client.26317 shutdown 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 -- 172.21.15.49:0/2236263490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 msgr2=0x557bbdb4b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4b7c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9734005c10 tx=0x7f973400a8b0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 -- 172.21.15.49:0/2236263490 shutdown_connections 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557bbdbca240 0x557bbdbcc660 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557bbdb48290 0x557bbdb4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557bbdb51600 0x557bbdb4b7c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 --2- 172.21.15.49:0/2236263490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557bbdb44cf0 0x557bbdb69a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9745a5c740 1 -- 172.21.15.49:0/2236263490 >> 172.21.15.49:0/2236263490 conn(0x557bbdb0f570 msgr2=0x557bbdb4ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.679+0000 7f9725600640 0 client.26317 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.680+0000 7f9745a5c740 1 -- 172.21.15.49:0/2236263490 shutdown_connections 2024-02-26T06:26:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.680+0000 7f9745a5c740 1 -- 172.21.15.49:0/2236263490 wait complete. 2024-02-26T06:26:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.680+0000 7f9745a5c740 20 client.26317 trim_cache size 0 max 16384 2024-02-26T06:26:37.682 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:26:37.682 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:26:37.682 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:26:37.682 INFO:teuthology.orchestra.run.smithi049.stdout: "7300e20a-d834-4435-b2ad-1367b1db2942": { 2024-02-26T06:26:37.682 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:26:37.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "7300e20a-d834-4435-b2ad-1367b1db2942", 2024-02-26T06:26:37.682 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:26:37.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:26:37.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:26:37.683 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:26:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 -- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f97200a2290 msgr2=0x7f97200a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f97200a2290 0x7f97200a4750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f972c032530 tx=0x7f972c000bc0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 -- 172.21.15.49:0/3858029318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c06a0b0 msgr2=0x7f973c19e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c06a0b0 0x7f973c19e270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f972800e9c0 tx=0x7f972800ee80 comp rx=0 tx=0).stop 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 -- 172.21.15.49:0/3858029318 shutdown_connections 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f97200a2290 0x7f97200a4750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f973c06a0b0 0x7f973c19e270 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c107230 0x7f973c1a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 --2- 172.21.15.49:0/3858029318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f973c0696f0 0x7f973c19dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 -- 172.21.15.49:0/3858029318 >> 172.21.15.49:0/3858029318 conn(0x7f973c077060 msgr2=0x7f973c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 -- 172.21.15.49:0/3858029318 shutdown_connections 2024-02-26T06:26:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:26:37.681+0000 7f9742c00640 1 -- 172.21.15.49:0/3858029318 wait complete. 2024-02-26T06:26:37.709 INFO:tasks.cephfs.filesystem:scrub status for tag:7300e20a-d834-4435-b2ad-1367b1db2942 - {'path': '/', 'tag': '7300e20a-d834-4435-b2ad-1367b1db2942', 'options': 'recursive,force'} 2024-02-26T06:26:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:38 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:26:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:38 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:38 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:38 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:38 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:38 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:38 smithi080 ceph-mon[36206]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-02-26T06:26:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:38 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:38 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:26:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:38 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:38 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:38 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:38 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:38 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:38 smithi115 ceph-mon[36426]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-02-26T06:26:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:38 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:38 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:26:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:38 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:38 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:38 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:38 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:38 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:38 smithi049 ceph-mon[31696]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-02-26T06:26:38.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:38 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:26:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: pgmap v100: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 692 op/s 2024-02-26T06:26:39.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:26:39.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:26:39.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:39 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:26:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:26:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: pgmap v100: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 692 op/s 2024-02-26T06:26:39.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:26:39.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:26:39.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:39 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: pgmap v100: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 692 op/s 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:26:39.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:26:39.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:39 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:26:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 170.0 s, 2381.0 tps, lat 41.723 ms stddev 47.625 2024-02-26T06:26:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:41 smithi080 ceph-mon[36206]: pgmap v101: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 9.9 MiB/s wr, 922 op/s 2024-02-26T06:26:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:41 smithi115 ceph-mon[36426]: pgmap v101: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 9.9 MiB/s wr, 922 op/s 2024-02-26T06:26:41.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:41 smithi049 ceph-mon[31696]: pgmap v101: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 9.9 MiB/s wr, 922 op/s 2024-02-26T06:26:43.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:43 smithi080 ceph-mon[36206]: pgmap v102: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 8.1 MiB/s wr, 802 op/s 2024-02-26T06:26:43.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:43 smithi115 ceph-mon[36426]: pgmap v102: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 8.1 MiB/s wr, 802 op/s 2024-02-26T06:26:43.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:43 smithi049 ceph-mon[31696]: pgmap v102: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 8.1 MiB/s wr, 802 op/s 2024-02-26T06:26:45.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:45 smithi080 ceph-mon[36206]: pgmap v103: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-02-26T06:26:45.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:45 smithi115 ceph-mon[36426]: pgmap v103: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-02-26T06:26:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:45 smithi049 ceph-mon[31696]: pgmap v103: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-02-26T06:26:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.620+0000 7fb8fa800640 1 -- 172.21.15.49:0/2368712831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 msgr2=0x7fb8f4110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.620+0000 7fb8fa800640 1 --2- 172.21.15.49:0/2368712831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 0x7fb8f4110270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009ca0 tx=0x7fb8ec02c100 comp rx=0 tx=0).stop 2024-02-26T06:26:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.622+0000 7fb8fa800640 1 -- 172.21.15.49:0/2368712831 shutdown_connections 2024-02-26T06:26:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.623+0000 7fb8fa800640 1 --2- 172.21.15.49:0/2368712831 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8f4103f80 0x7fb8f4107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.623+0000 7fb8fa800640 1 --2- 172.21.15.49:0/2368712831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 0x7fb8f4110270 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.623+0000 7fb8fa800640 1 --2- 172.21.15.49:0/2368712831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f41035d0 0x7fb8f41039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.623+0000 7fb8fa800640 1 -- 172.21.15.49:0/2368712831 >> 172.21.15.49:0/2368712831 conn(0x7fb8f40fee30 msgr2=0x7fb8f4101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.624+0000 7fb8fa800640 1 -- 172.21.15.49:0/2368712831 shutdown_connections 2024-02-26T06:26:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.625+0000 7fb8fa800640 1 -- 172.21.15.49:0/2368712831 wait complete. 2024-02-26T06:26:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.627+0000 7fb8fa800640 1 Processor -- start 2024-02-26T06:26:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.628+0000 7fb8fa800640 1 -- start start 2024-02-26T06:26:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.628+0000 7fb8fa800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8f41035d0 0x7fb8f410ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.628+0000 7fb8fa800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 0x7fb8f410f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.628+0000 7fb8fa800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f411a740 0x7fb8f410f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.628+0000 7fb8fa800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f41b2aa0 con 0x7fb8f411a740 2024-02-26T06:26:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.628+0000 7fb8fa800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f41b2c10 con 0x7fb8f41035d0 2024-02-26T06:26:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.628+0000 7fb8fa800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f406aba0 con 0x7fb8f41086a0 2024-02-26T06:26:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.629+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 0x7fb8f410f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.629+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 0x7fb8f410f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:36008/0 (socket says 172.21.15.49:36008) 2024-02-26T06:26:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.629+0000 7fb8f2a00640 1 -- 172.21.15.49:0/690700228 learned_addr learned my addr 172.21.15.49:0/690700228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.629+0000 7fb8f3400640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8f41035d0 0x7fb8f410ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.630+0000 7fb8f3400640 1 -- 172.21.15.49:0/690700228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 msgr2=0x7fb8f410f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.630+0000 7fb8f3400640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 0x7fb8f410f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.630+0000 7fb8f3400640 1 -- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f411a740 msgr2=0x7fb8f410f670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:26:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.630+0000 7fb8f3400640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f411a740 0x7fb8f410f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.630+0000 7fb8f3400640 1 -- 172.21.15.49:0/690700228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec009900 con 0x7fb8f41035d0 2024-02-26T06:26:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.632+0000 7fb8f3400640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8f41035d0 0x7fb8f410ebb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4005fd0 tx=0x7fb8e400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.635+0000 7fb8ebe00640 1 -- 172.21.15.49:0/690700228 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4009c30 con 0x7fb8f41035d0 2024-02-26T06:26:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.636+0000 7fb8ebe00640 1 -- 172.21.15.49:0/690700228 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8e4004510 con 0x7fb8f41035d0 2024-02-26T06:26:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.636+0000 7fb8ebe00640 1 -- 172.21.15.49:0/690700228 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4003e00 con 0x7fb8f41035d0 2024-02-26T06:26:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.636+0000 7fb8fa800640 1 -- 172.21.15.49:0/690700228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f406ae80 con 0x7fb8f41035d0 2024-02-26T06:26:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.636+0000 7fb8fa800640 1 -- 172.21.15.49:0/690700228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f406b340 con 0x7fb8f41035d0 2024-02-26T06:26:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.645+0000 7fb8fd620740 1 Processor -- start 2024-02-26T06:26:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.646+0000 7fb8fd620740 1 -- start start 2024-02-26T06:26:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.646+0000 7fb8ebe00640 1 -- 172.21.15.49:0/690700228 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb8e4010460 con 0x7fb8f41035d0 2024-02-26T06:26:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.646+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f93af0 0x560e58f92840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.646+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f92d80 0x560e58f93200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.646+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f9c300 0x560e58f9e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.646+0000 7fb8fd620740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58f4e5f0 con 0x560e58f93af0 2024-02-26T06:26:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.646+0000 7fb8fd620740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58f00890 con 0x560e58f9c300 2024-02-26T06:26:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.646+0000 7fb8fd620740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58efddc0 con 0x560e58f92d80 2024-02-26T06:26:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.647+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f9c300 0x560e58f9e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.647+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f9c300 0x560e58f9e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35480/0 (socket says 172.21.15.49:35480) 2024-02-26T06:26:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.647+0000 7fb8f2a00640 1 -- 172.21.15.49:0/1245104302 learned_addr learned my addr 172.21.15.49:0/1245104302 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.648+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1245104302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f92d80 0x560e58f93200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.648+0000 7fb8f3400640 1 -- 172.21.15.49:0/1245104302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f9c300 msgr2=0x560e58f9e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.648+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1245104302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f9c300 0x560e58f9e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.648+0000 7fb8f3400640 1 -- 172.21.15.49:0/1245104302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f93af0 msgr2=0x560e58f92840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.648+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1245104302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f93af0 0x560e58f92840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.648+0000 7fb8f3400640 1 -- 172.21.15.49:0/1245104302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e58e7aad0 con 0x560e58f92d80 2024-02-26T06:26:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.649+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1245104302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f92d80 0x560e58f93200 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40cb440 tx=0x7fb8e40d67e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4020713123d2ea server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.650+0000 7fb8ebe00640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8d80a23a0 0x7fb8d80a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.651+0000 7fb8e8c00640 1 -- 172.21.15.49:0/1245104302 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4096980 con 0x560e58f92d80 2024-02-26T06:26:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.651+0000 7fb8f3e00640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8d80a23a0 0x7fb8d80a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.651+0000 7fb8e8c00640 1 -- 172.21.15.49:0/1245104302 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8e409a4b0 con 0x560e58f92d80 2024-02-26T06:26:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.651+0000 7fb8e8c00640 1 -- 172.21.15.49:0/1245104302 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4094fd0 con 0x560e58f92d80 2024-02-26T06:26:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.651+0000 7fb8ebe00640 1 -- 172.21.15.49:0/690700228 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb8e40cdce0 con 0x7fb8f41035d0 2024-02-26T06:26:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.652+0000 7fb8fd620740 1 -- 172.21.15.49:0/1245104302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f92d80 msgr2=0x560e58f93200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.652+0000 7fb8f3e00640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8d80a23a0 0x7fb8d80a4860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec037040 tx=0x7fb8ec02c620 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.652+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1245104302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f92d80 0x560e58f93200 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40cb440 tx=0x7fb8e40d67e0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.653+0000 7fb8fd620740 1 -- 172.21.15.49:0/1245104302 shutdown_connections 2024-02-26T06:26:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.653+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1245104302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f9c300 0x560e58f9e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.653+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1245104302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f92d80 0x560e58f93200 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.653+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1245104302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f93af0 0x560e58f92840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.653+0000 7fb8fd620740 1 -- 172.21.15.49:0/1245104302 >> 172.21.15.49:0/1245104302 conn(0x560e58f5a480 msgr2=0x560e58f8da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.654+0000 7fb8fd620740 1 -- 172.21.15.49:0/1245104302 shutdown_connections 2024-02-26T06:26:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.654+0000 7fb8fd620740 1 -- 172.21.15.49:0/1245104302 wait complete. 2024-02-26T06:26:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.655+0000 7fb8fd620740 1 -- start start 2024-02-26T06:26:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.655+0000 7fb8fd620740 1 Processor -- start 2024-02-26T06:26:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.658+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f993c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.658+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 0x560e58f99900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.658+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.660+0000 7fb8fd620740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58ef95a0 con 0x560e58f92d80 2024-02-26T06:26:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.660+0000 7fb8fd620740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58e9eb20 con 0x560e58f93af0 2024-02-26T06:26:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.660+0000 7fb8fd620740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58e9e7c0 con 0x560e58f9c300 2024-02-26T06:26:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.660+0000 7fb8fd620740 10 client.0 _ll_register_callbacks cb 0x560e58eb2520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.660+0000 7fb8f3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.660+0000 7fb8f3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f99e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:36028/0 (socket says 172.21.15.49:36028) 2024-02-26T06:26:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.660+0000 7fb8f3e00640 1 -- 172.21.15.49:0/1504650404 learned_addr learned my addr 172.21.15.49:0/1504650404 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.661+0000 7fb8f3e00640 1 -- 172.21.15.49:0/1504650404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 msgr2=0x560e58f99900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:26:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.661+0000 7fb8f3e00640 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 0x560e58f99900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.661+0000 7fb8f3e00640 1 -- 172.21.15.49:0/1504650404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 msgr2=0x560e58f993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.661+0000 7fb8f3e00640 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.661+0000 7fb8f3e00640 1 -- 172.21.15.49:0/1504650404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f406b340 con 0x560e58f9c300 2024-02-26T06:26:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.662+0000 7fb8f3e00640 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f99e40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec032640 tx=0x7fb8ec009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0c6596bd91b83dc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:45.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.662+0000 7fb8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.663+0000 7fb8dd600640 1 -- 172.21.15.49:0/1504650404 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec003970 con 0x560e58f9c300 2024-02-26T06:26:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.663+0000 7fb8fd620740 1 -- 172.21.15.49:0/1504650404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e58ecbca0 con 0x560e58f9c300 2024-02-26T06:26:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.664+0000 7fb8dd600640 1 -- 172.21.15.49:0/1504650404 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb8ec003ad0 con 0x560e58f9c300 2024-02-26T06:26:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.664+0000 7fb8dd600640 1 -- 172.21.15.49:0/1504650404 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8ec003c90 con 0x560e58f9c300 2024-02-26T06:26:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.665+0000 7fb8fd620740 10 client.26329 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.665+0000 7fb8fd620740 1 -- 172.21.15.49:0/1504650404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560e58ef7c20 con 0x560e58f9c300 2024-02-26T06:26:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.665+0000 7fb8dd600640 1 -- 172.21.15.49:0/1504650404 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560e58ecbca0 con 0x560e58f9c300 2024-02-26T06:26:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.666+0000 7fb8dd600640 1 -- 172.21.15.49:0/1504650404 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb8ec04a710 con 0x560e58f9c300 2024-02-26T06:26:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.666+0000 7fb8fd620740 10 client.26329 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.666+0000 7fb8fd620740 10 client.26329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.666+0000 7fb8fd620740 20 client.26329 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.666+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560e58fac4d0 0x560e58fae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.666+0000 7fb8fd620740 4 client.26329 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.666+0000 7fb8fd620740 1 -- 172.21.15.49:0/1504650404 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e58eefcb0 con 0x560e58fac4d0 2024-02-26T06:26:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.666+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560e58fac4d0 0x560e58fae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.668+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560e58fac4d0 0x560e58fae910 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53d4b6ac1535b9f server_cookie=6cbb4eadc7693e6d in_seq=0 out_seq=0 2024-02-26T06:26:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.668+0000 7fb8dd600640 10 client.26329 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.668+0000 7fb8dd600640 1 -- 172.21.15.49:0/1504650404 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e58eefcb0 con 0x560e58fac4d0 2024-02-26T06:26:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.668+0000 7fb8dd600640 10 client.26329 handle_command_reply: tid=0 2024-02-26T06:26:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.669+0000 7fb8fd620740 1 client.26329 shutdown 2024-02-26T06:26:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.670+0000 7fb8fd620740 1 -- 172.21.15.49:0/1504650404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 msgr2=0x560e58f99e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.670+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f99e40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec032640 tx=0x7fb8ec009840 comp rx=0 tx=0).stop 2024-02-26T06:26:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 1 -- 172.21.15.49:0/1504650404 shutdown_connections 2024-02-26T06:26:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560e58fac4d0 0x560e58fae910 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 0x560e58f99900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f99e40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1504650404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 1 -- 172.21.15.49:0/1504650404 >> 172.21.15.49:0/1504650404 conn(0x560e58f5a480 msgr2=0x560e58f8f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8dd600640 0 client.26329 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 1 -- 172.21.15.49:0/1504650404 shutdown_connections 2024-02-26T06:26:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 1 -- 172.21.15.49:0/1504650404 wait complete. 2024-02-26T06:26:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.671+0000 7fb8fd620740 20 client.26329 trim_cache size 0 max 16384 2024-02-26T06:26:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.702+0000 7fb8fd620740 1 Processor -- start 2024-02-26T06:26:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.702+0000 7fb8fd620740 1 -- start start 2024-02-26T06:26:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.702+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f9baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.702+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 0x560e58f99c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.702+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f96620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.702+0000 7fb8fd620740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58c41e60 con 0x560e58f92d80 2024-02-26T06:26:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.702+0000 7fb8fd620740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58f940e0 con 0x560e58f93af0 2024-02-26T06:26:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.702+0000 7fb8fd620740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58f8e870 con 0x560e58f9c300 2024-02-26T06:26:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.703+0000 7fb8f3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f9baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.703+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 0x560e58f99c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.703+0000 7fb8f3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f9baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49362/0 (socket says 172.21.15.49:49362) 2024-02-26T06:26:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.703+0000 7fb8f3400640 1 -- 172.21.15.49:0/1705308762 learned_addr learned my addr 172.21.15.49:0/1705308762 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.703+0000 7fb8f3e00640 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f96620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.704+0000 7fb8f2a00640 1 -- 172.21.15.49:0/1705308762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 msgr2=0x560e58f96620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.704+0000 7fb8f2a00640 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f96620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.704+0000 7fb8f2a00640 1 -- 172.21.15.49:0/1705308762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 msgr2=0x560e58f9baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.704+0000 7fb8f2a00640 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f9baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.704+0000 7fb8f2a00640 1 -- 172.21.15.49:0/1705308762 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e58e94500 con 0x560e58f93af0 2024-02-26T06:26:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.704+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f9baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.704+0000 7fb8f2a00640 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 0x560e58f99c60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x560e58f98c20 tx=0x7fb8e0012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ba39b45f5e1cb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.705+0000 7fb8dfe00640 1 -- 172.21.15.49:0/1705308762 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e001b640 con 0x560e58f93af0 2024-02-26T06:26:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.705+0000 7fb8dfe00640 1 -- 172.21.15.49:0/1705308762 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8e000be50 con 0x560e58f93af0 2024-02-26T06:26:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.705+0000 7fb8dfe00640 1 -- 172.21.15.49:0/1705308762 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0019750 con 0x560e58f93af0 2024-02-26T06:26:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 -- 172.21.15.49:0/1705308762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 msgr2=0x560e58f99c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 0x560e58f99c60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x560e58f98c20 tx=0x7fb8e0012820 comp rx=0 tx=0).stop 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 -- 172.21.15.49:0/1705308762 shutdown_connections 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f93af0 0x560e58f99c60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f9c300 0x560e58f96620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1705308762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f92d80 0x560e58f9baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 -- 172.21.15.49:0/1705308762 >> 172.21.15.49:0/1705308762 conn(0x560e58f5a480 msgr2=0x560e58f97c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 -- 172.21.15.49:0/1705308762 shutdown_connections 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.706+0000 7fb8fd620740 1 -- 172.21.15.49:0/1705308762 wait complete. 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.707+0000 7fb8fd620740 1 -- start start 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.707+0000 7fb8fd620740 1 Processor -- start 2024-02-26T06:26:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f92d80 0x560e58fa3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f93af0 0x560e58fa1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8fd620740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f9c300 0x560e58fa1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8fd620740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58fa2090 con 0x560e58f9c300 2024-02-26T06:26:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8fd620740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58fa2200 con 0x560e58f92d80 2024-02-26T06:26:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8fd620740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e58fa23c0 con 0x560e58f93af0 2024-02-26T06:26:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8fd620740 10 client.0 _ll_register_callbacks cb 0x7fb8ec037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f93af0 0x560e58fa1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f93af0 0x560e58fa1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:36044/0 (socket says 172.21.15.49:36044) 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8f2a00640 1 -- 172.21.15.49:0/1567993877 learned_addr learned my addr 172.21.15.49:0/1567993877 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f92d80 0x560e58fa3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8f2a00640 1 -- 172.21.15.49:0/1567993877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f92d80 msgr2=0x560e58fa3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8f2a00640 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f92d80 0x560e58fa3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.708+0000 7fb8f2a00640 1 -- 172.21.15.49:0/1567993877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f9c300 msgr2=0x560e58fa1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.709+0000 7fb8f2a00640 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f9c300 0x560e58fa1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.709+0000 7fb8f2a00640 1 -- 172.21.15.49:0/1567993877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e58e7aad0 con 0x560e58f93af0 2024-02-26T06:26:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.709+0000 7fb8f2a00640 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f93af0 0x560e58fa1610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0004600 tx=0x7fb8e0005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91826db7ce0ecb6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.709+0000 7fb8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.710+0000 7fb8dd600640 1 -- 172.21.15.49:0/1567993877 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00046c0 con 0x560e58f93af0 2024-02-26T06:26:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.710+0000 7fb8dd600640 1 -- 172.21.15.49:0/1567993877 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb8e0004820 con 0x560e58f93af0 2024-02-26T06:26:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.710+0000 7fb8fd620740 1 -- 172.21.15.49:0/1567993877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e58f1ed00 con 0x560e58f93af0 2024-02-26T06:26:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.710+0000 7fb8dd600640 1 -- 172.21.15.49:0/1567993877 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8e00049e0 con 0x560e58f93af0 2024-02-26T06:26:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.710+0000 7fb8fd620740 10 client.26335 fetch_fsmap learned FSMap version 30 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8fd620740 1 -- 172.21.15.49:0/1567993877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb8e000b9b0 con 0x560e58f93af0 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8dd600640 1 -- 172.21.15.49:0/1567993877 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560e58f1ed00 con 0x560e58f93af0 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8dd600640 1 -- 172.21.15.49:0/1567993877 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb8e0026440 con 0x560e58f93af0 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8fd620740 10 client.26335 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8fd620740 10 client.26335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8fd620740 20 client.26335 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560e59003f40 0x560e59006360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8fd620740 4 client.26335 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.711+0000 7fb8fd620740 1 -- 172.21.15.49:0/1567993877 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e58bf9600 con 0x560e59003f40 2024-02-26T06:26:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.712+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560e59003f40 0x560e59006360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.712+0000 7fb8f3400640 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560e59003f40 0x560e59006360 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=757abf0ec541a816 server_cookie=959c74ce4f01d433 in_seq=0 out_seq=0 2024-02-26T06:26:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.712+0000 7fb8dd600640 10 client.26335 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.725+0000 7fb8dd600640 1 -- 172.21.15.49:0/1567993877 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e58bf9600 con 0x560e59003f40 2024-02-26T06:26:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.725+0000 7fb8dd600640 10 client.26335 handle_command_reply: tid=0 2024-02-26T06:26:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 client.26335 shutdown 2024-02-26T06:26:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 -- 172.21.15.49:0/1567993877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f93af0 msgr2=0x560e58fa1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f93af0 0x560e58fa1610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0004600 tx=0x7fb8e0005270 comp rx=0 tx=0).stop 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 -- 172.21.15.49:0/1567993877 shutdown_connections 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560e59003f40 0x560e59006360 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8dd600640 0 client.26335 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e58f9c300 0x560e58fa1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e58f93af0 0x560e58fa1610 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 --2- 172.21.15.49:0/1567993877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e58f92d80 0x560e58fa3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.726+0000 7fb8fd620740 1 -- 172.21.15.49:0/1567993877 >> 172.21.15.49:0/1567993877 conn(0x560e58f5a480 msgr2=0x560e58f8cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.727+0000 7fb8fd620740 1 -- 172.21.15.49:0/1567993877 shutdown_connections 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.727+0000 7fb8fd620740 1 -- 172.21.15.49:0/1567993877 wait complete. 2024-02-26T06:26:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.727+0000 7fb8fd620740 20 client.26335 trim_cache size 0 max 16384 2024-02-26T06:26:45.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:26:45.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:26:45.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:26:45.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:26:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 -- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8d80a23a0 msgr2=0x7fb8d80a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8d80a23a0 0x7fb8d80a4860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec037040 tx=0x7fb8ec02c620 comp rx=0 tx=0).stop 2024-02-26T06:26:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 -- 172.21.15.49:0/690700228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8f41035d0 msgr2=0x7fb8f410ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8f41035d0 0x7fb8f410ebb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4005fd0 tx=0x7fb8e400dde0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 -- 172.21.15.49:0/690700228 shutdown_connections 2024-02-26T06:26:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8d80a23a0 0x7fb8d80a4860 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f411a740 0x7fb8f410f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f41086a0 0x7fb8f410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 --2- 172.21.15.49:0/690700228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8f41035d0 0x7fb8f410ebb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.729+0000 7fb8fa800640 1 -- 172.21.15.49:0/690700228 >> 172.21.15.49:0/690700228 conn(0x7fb8f40fee30 msgr2=0x7fb8f40ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.730+0000 7fb8fa800640 1 -- 172.21.15.49:0/690700228 shutdown_connections 2024-02-26T06:26:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:45.730+0000 7fb8fa800640 1 -- 172.21.15.49:0/690700228 wait complete. 2024-02-26T06:26:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 175.0 s, 2933.7 tps, lat 33.912 ms stddev 37.614 2024-02-26T06:26:46.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:46 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:26:46.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:46 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:26:46.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:46 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:26:47.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:47 smithi080 ceph-mon[36206]: pgmap v104: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 10 MiB/s wr, 990 op/s 2024-02-26T06:26:47.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:47 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:26:47.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:47 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:26:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:47 smithi115 ceph-mon[36426]: pgmap v104: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 10 MiB/s wr, 990 op/s 2024-02-26T06:26:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:47 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:26:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:47 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:26:47.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:47 smithi049 ceph-mon[31696]: pgmap v104: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 10 MiB/s wr, 990 op/s 2024-02-26T06:26:47.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:47 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:26:47.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:47 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:26:49.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:49 smithi080 ceph-mon[36206]: pgmap v105: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.7 MiB/s wr, 679 op/s 2024-02-26T06:26:49.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:49 smithi115 ceph-mon[36426]: pgmap v105: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.7 MiB/s wr, 679 op/s 2024-02-26T06:26:49.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:49 smithi049 ceph-mon[31696]: pgmap v105: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.7 MiB/s wr, 679 op/s 2024-02-26T06:26:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 180.0 s, 2544.5 tps, lat 39.724 ms stddev 45.775 2024-02-26T06:26:51.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:51 smithi080 ceph-mon[36206]: pgmap v106: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-02-26T06:26:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:51 smithi115 ceph-mon[36426]: pgmap v106: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-02-26T06:26:51.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:51 smithi049 ceph-mon[31696]: pgmap v106: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-02-26T06:26:52.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:52 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:52.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:52 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:52.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:52 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:53.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:53 smithi080 ceph-mon[36206]: pgmap v107: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 751 op/s 2024-02-26T06:26:53.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:53 smithi115 ceph-mon[36426]: pgmap v107: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 751 op/s 2024-02-26T06:26:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:53 smithi049 ceph-mon[31696]: pgmap v107: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 751 op/s 2024-02-26T06:26:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:55 smithi080 ceph-mon[36206]: pgmap v108: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.6 MiB/s wr, 604 op/s 2024-02-26T06:26:55.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:55 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:55 smithi115 ceph-mon[36426]: pgmap v108: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.6 MiB/s wr, 604 op/s 2024-02-26T06:26:55.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:55 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:55 smithi049 ceph-mon[31696]: pgmap v108: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.6 MiB/s wr, 604 op/s 2024-02-26T06:26:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:55 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:26:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 185.0 s, 2694.8 tps, lat 37.130 ms stddev 42.849 2024-02-26T06:26:57.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:57 smithi080 ceph-mon[36206]: pgmap v109: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 903 op/s 2024-02-26T06:26:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:57 smithi115 ceph-mon[36426]: pgmap v109: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 903 op/s 2024-02-26T06:26:57.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:57 smithi049 ceph-mon[31696]: pgmap v109: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 903 op/s 2024-02-26T06:26:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.921+0000 7f59eba00640 1 -- 172.21.15.49:0/351770888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 msgr2=0x7f59e4101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.921+0000 7f59eba00640 1 --2- 172.21.15.49:0/351770888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 0x7f59e4101780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009900 tx=0x7f59d802bc40 comp rx=0 tx=0).stop 2024-02-26T06:26:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.921+0000 7f59eba00640 1 -- 172.21.15.49:0/351770888 shutdown_connections 2024-02-26T06:26:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.921+0000 7f59eba00640 1 --2- 172.21.15.49:0/351770888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4106470 0x7f59e410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.921+0000 7f59eba00640 1 --2- 172.21.15.49:0/351770888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59e4101d50 0x7f59e4105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.921+0000 7f59eba00640 1 --2- 172.21.15.49:0/351770888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 0x7f59e4101780 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.921+0000 7f59eba00640 1 -- 172.21.15.49:0/351770888 >> 172.21.15.49:0/351770888 conn(0x7f59e40fcc40 msgr2=0x7f59e40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.922+0000 7f59eba00640 1 -- 172.21.15.49:0/351770888 shutdown_connections 2024-02-26T06:26:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.922+0000 7f59eba00640 1 -- 172.21.15.49:0/351770888 wait complete. 2024-02-26T06:26:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.923+0000 7f59eba00640 1 Processor -- start 2024-02-26T06:26:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.923+0000 7f59eba00640 1 -- start start 2024-02-26T06:26:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.923+0000 7f59eba00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 0x7f59e410e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.923+0000 7f59eba00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4106470 0x7f59e410eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59eba00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59e410f0d0 0x7f59e41a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59eba00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4112830 con 0x7f59e410f0d0 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59eba00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41aa420 con 0x7f59e41013a0 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59eba00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41aa5e0 con 0x7f59e4106470 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 0x7f59e410e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 0x7f59e410e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51938/0 (socket says 172.21.15.49:51938) 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 -- 172.21.15.49:0/298760695 learned_addr learned my addr 172.21.15.49:0/298760695 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e3e00640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4106470 0x7f59e410eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9a00640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59e410f0d0 0x7f59e41a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 -- 172.21.15.49:0/298760695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4106470 msgr2=0x7f59e410eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4106470 0x7f59e410eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 -- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59e410f0d0 msgr2=0x7f59e41a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59e410f0d0 0x7f59e41a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 -- 172.21.15.49:0/298760695 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d8009560 con 0x7f59e41013a0 2024-02-26T06:26:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.924+0000 7f59e9000640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 0x7f59e410e650 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f59d802c250 tx=0x7f59d802c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.925+0000 7f59e1600640 1 -- 172.21.15.49:0/298760695 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8004410 con 0x7f59e41013a0 2024-02-26T06:26:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.925+0000 7f59e1600640 1 -- 172.21.15.49:0/298760695 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59d80052d0 con 0x7f59e41013a0 2024-02-26T06:26:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.925+0000 7f59e1600640 1 -- 172.21.15.49:0/298760695 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8003ca0 con 0x7f59e41013a0 2024-02-26T06:26:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.925+0000 7f59eba00640 1 -- 172.21.15.49:0/298760695 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e41aa860 con 0x7f59e41013a0 2024-02-26T06:26:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.925+0000 7f59eba00640 1 -- 172.21.15.49:0/298760695 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e41aad20 con 0x7f59e41013a0 2024-02-26T06:26:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.928+0000 7f59e1600640 1 -- 172.21.15.49:0/298760695 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f59d8005440 con 0x7f59e41013a0 2024-02-26T06:26:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.929+0000 7f59ee991740 1 Processor -- start 2024-02-26T06:26:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.930+0000 7f59ee991740 1 -- start start 2024-02-26T06:26:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.930+0000 7f59ee991740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713583af0 0x563713582840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.931+0000 7f59ee991740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713582d80 0x563713583200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.931+0000 7f59ee991740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x56371358e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.931+0000 7f59ee991740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371353e5f0 con 0x563713583af0 2024-02-26T06:26:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.931+0000 7f59ee991740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637134f0890 con 0x563713582d80 2024-02-26T06:26:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.931+0000 7f59ee991740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637134eddc0 con 0x56371358c300 2024-02-26T06:26:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.932+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x56371358e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.932+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x56371358e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42118/0 (socket says 172.21.15.49:42118) 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.932+0000 7f59e3e00640 1 -- 172.21.15.49:0/1529594965 learned_addr learned my addr 172.21.15.49:0/1529594965 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.932+0000 7f59e9a00640 1 --2- 172.21.15.49:0/1529594965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713583af0 0x563713582840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.933+0000 7f59e9a00640 1 -- 172.21.15.49:0/1529594965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 msgr2=0x56371358e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.933+0000 7f59e9a00640 1 --2- 172.21.15.49:0/1529594965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x56371358e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.933+0000 7f59e9a00640 1 -- 172.21.15.49:0/1529594965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713582d80 msgr2=0x563713583200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.933+0000 7f59e9a00640 1 --2- 172.21.15.49:0/1529594965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713582d80 0x563713583200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.933+0000 7f59e9a00640 1 -- 172.21.15.49:0/1529594965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56371346aad0 con 0x563713583af0 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.934+0000 7f59e9a00640 1 --2- 172.21.15.49:0/1529594965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713583af0 0x563713582840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009e50 tx=0x7f59dc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a225ef9748943a29 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.934+0000 7f59d6000640 1 -- 172.21.15.49:0/1529594965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc003b70 con 0x563713583af0 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.934+0000 7f59d6000640 1 -- 172.21.15.49:0/1529594965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59dc015070 con 0x563713583af0 2024-02-26T06:26:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.934+0000 7f59d6000640 1 -- 172.21.15.49:0/1529594965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc011800 con 0x563713583af0 2024-02-26T06:26:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.935+0000 7f59e1600640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f59cc0a23d0 0x7f59cc0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.935+0000 7f59ee991740 1 -- 172.21.15.49:0/1529594965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713583af0 msgr2=0x563713582840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.935+0000 7f59ee991740 1 --2- 172.21.15.49:0/1529594965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713583af0 0x563713582840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009e50 tx=0x7f59dc0107a0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.936+0000 7f59e1600640 1 -- 172.21.15.49:0/298760695 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f59d80f0230 con 0x7f59e41013a0 2024-02-26T06:26:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.936+0000 7f59e9000640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f59cc0a23d0 0x7f59cc0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.938+0000 7f59ee991740 1 -- 172.21.15.49:0/1529594965 shutdown_connections 2024-02-26T06:26:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.938+0000 7f59ee991740 1 --2- 172.21.15.49:0/1529594965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713582d80 0x563713583200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.938+0000 7f59ee991740 1 --2- 172.21.15.49:0/1529594965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x56371358e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.938+0000 7f59ee991740 1 --2- 172.21.15.49:0/1529594965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713583af0 0x563713582840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.938+0000 7f59ee991740 1 -- 172.21.15.49:0/1529594965 >> 172.21.15.49:0/1529594965 conn(0x56371354a480 msgr2=0x56371357da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.938+0000 7f59ee991740 1 -- 172.21.15.49:0/1529594965 shutdown_connections 2024-02-26T06:26:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.938+0000 7f59ee991740 1 -- 172.21.15.49:0/1529594965 wait complete. 2024-02-26T06:26:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.939+0000 7f59e9000640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f59cc0a23d0 0x7f59cc0a4890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x563713584250 tx=0x7f59d802bc40 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.939+0000 7f59ee991740 1 -- start start 2024-02-26T06:26:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.939+0000 7f59ee991740 1 Processor -- start 2024-02-26T06:26:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.941+0000 7f59ee991740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713582d80 0x563713589410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.943+0000 7f59e9a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713582d80 0x563713589410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.943+0000 7f59e9a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713582d80 0x563713589410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42134/0 (socket says 172.21.15.49:42134) 2024-02-26T06:26:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.942+0000 7f59ee991740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x563713589950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.943+0000 7f59e9a00640 1 -- 172.21.15.49:0/3866914441 learned_addr learned my addr 172.21.15.49:0/3866914441 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.943+0000 7f59ee991740 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56371358c300 0x563713589e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.943+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637134e95a0 con 0x56371358c300 2024-02-26T06:26:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.943+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371348eb20 con 0x563713583af0 2024-02-26T06:26:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.943+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371348e7c0 con 0x563713582d80 2024-02-26T06:26:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.943+0000 7f59ee991740 10 client.0 _ll_register_callbacks cb 0x5637134a2520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.948+0000 7f59e3e00640 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x563713589950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.948+0000 7f59e9a00640 1 -- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 msgr2=0x563713589950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.948+0000 7f59e9a00640 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x563713589950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.948+0000 7f59e9a00640 1 -- 172.21.15.49:0/3866914441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56371358c300 msgr2=0x563713589e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.948+0000 7f59e9a00640 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56371358c300 0x563713589e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.948+0000 7f59e9a00640 1 -- 172.21.15.49:0/3866914441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc009b30 con 0x563713582d80 2024-02-26T06:26:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.948+0000 7f59e3e00640 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x563713589950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.949+0000 7f59e9a00640 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713582d80 0x563713589410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f59dc005b20 tx=0x7f59dc003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d02a6c21e3432e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.949+0000 7f59caa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.953+0000 7f59caa00640 1 -- 172.21.15.49:0/3866914441 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc01f030 con 0x563713582d80 2024-02-26T06:26:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.953+0000 7f59caa00640 1 -- 172.21.15.49:0/3866914441 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f59dc020070 con 0x563713582d80 2024-02-26T06:26:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.953+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637134bbca0 con 0x563713582d80 2024-02-26T06:26:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.953+0000 7f59caa00640 1 -- 172.21.15.49:0/3866914441 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59dc0046c0 con 0x563713582d80 2024-02-26T06:26:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.953+0000 7f59ee991740 10 client.26347 fetch_fsmap learned FSMap version 32 2024-02-26T06:26:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.953+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5637134e7c20 con 0x563713582d80 2024-02-26T06:26:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.953+0000 7f59caa00640 1 -- 172.21.15.49:0/3866914441 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5637134bbca0 con 0x563713582d80 2024-02-26T06:26:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.955+0000 7f59caa00640 1 -- 172.21.15.49:0/3866914441 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f59dc02e780 con 0x563713582d80 2024-02-26T06:26:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.955+0000 7f59ee991740 10 client.26347 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:26:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.955+0000 7f59ee991740 10 client.26347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.955+0000 7f59ee991740 20 client.26347 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.955+0000 7f59ee991740 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56371359c260 0x56371359e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.955+0000 7f59ee991740 4 client.26347 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:26:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.955+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637134dfcb0 con 0x56371359c260 2024-02-26T06:26:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.956+0000 7f59e3e00640 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56371359c260 0x56371359e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.957+0000 7f59e3e00640 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56371359c260 0x56371359e650 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9bb80fa036313b server_cookie=a34625521762745f in_seq=0 out_seq=0 2024-02-26T06:26:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.957+0000 7f59caa00640 10 client.26347 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.958+0000 7f59caa00640 1 -- 172.21.15.49:0/3866914441 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5637134dfcb0 con 0x56371359c260 2024-02-26T06:26:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.958+0000 7f59caa00640 10 client.26347 handle_command_reply: tid=0 2024-02-26T06:26:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.958+0000 7f59ee991740 1 client.26347 shutdown 2024-02-26T06:26:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.959+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713582d80 msgr2=0x563713589410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.959+0000 7f59ee991740 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713582d80 0x563713589410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f59dc005b20 tx=0x7f59dc003f00 comp rx=0 tx=0).stop 2024-02-26T06:26:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.960+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 shutdown_connections 2024-02-26T06:26:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.960+0000 7f59ee991740 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56371359c260 0x56371359e650 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.960+0000 7f59ee991740 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x563713589950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.960+0000 7f59ee991740 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56371358c300 0x563713589e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.960+0000 7f59ee991740 1 --2- 172.21.15.49:0/3866914441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713582d80 0x563713589410 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.960+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 >> 172.21.15.49:0/3866914441 conn(0x56371354a480 msgr2=0x56371357f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.960+0000 7f59caa00640 0 client.26347 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.963+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 shutdown_connections 2024-02-26T06:26:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.963+0000 7f59ee991740 1 -- 172.21.15.49:0/3866914441 wait complete. 2024-02-26T06:26:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.964+0000 7f59ee991740 20 client.26347 trim_cache size 0 max 16384 2024-02-26T06:26:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.995+0000 7f59ee991740 1 Processor -- start 2024-02-26T06:26:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.995+0000 7f59ee991740 1 -- start start 2024-02-26T06:26:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.995+0000 7f59ee991740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x56371358bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.995+0000 7f59ee991740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x5637135866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.995+0000 7f59ee991740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x563713586bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.995+0000 7f59ee991740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563713231e60 con 0x563713582d80 2024-02-26T06:26:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.995+0000 7f59ee991740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637134dfce0 con 0x563713583af0 2024-02-26T06:26:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.995+0000 7f59ee991740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563713589200 con 0x56371358c300 2024-02-26T06:26:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x5637135866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x5637135866b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51970/0 (socket says 172.21.15.49:51970) 2024-02-26T06:26:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e3e00640 1 -- 172.21.15.49:0/3776032112 learned_addr learned my addr 172.21.15.49:0/3776032112 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e9a00640 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x563713586bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e9a00640 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x56371358bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e3e00640 1 -- 172.21.15.49:0/3776032112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 msgr2=0x563713586bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e3e00640 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x563713586bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e3e00640 1 -- 172.21.15.49:0/3776032112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 msgr2=0x56371358bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e3e00640 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x56371358bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e3e00640 1 -- 172.21.15.49:0/3776032112 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563713585070 con 0x563713583af0 2024-02-26T06:26:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.997+0000 7f59e9a00640 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x56371358bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.998+0000 7f59e3e00640 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x5637135866b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f59d000ed80 tx=0x7f59d000e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbce625a2600715 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.998+0000 7f59d5600640 1 -- 172.21.15.49:0/3776032112 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0017040 con 0x563713583af0 2024-02-26T06:26:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.998+0000 7f59d5600640 1 -- 172.21.15.49:0/3776032112 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59d001d070 con 0x563713583af0 2024-02-26T06:26:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.998+0000 7f59d5600640 1 -- 172.21.15.49:0/3776032112 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0019650 con 0x563713583af0 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 -- 172.21.15.49:0/3776032112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 msgr2=0x5637135866b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x5637135866b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f59d000ed80 tx=0x7f59d000e3b0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 -- 172.21.15.49:0/3776032112 shutdown_connections 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563713583af0 0x5637135866b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56371358c300 0x563713586bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 --2- 172.21.15.49:0/3776032112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x56371358bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 -- 172.21.15.49:0/3776032112 >> 172.21.15.49:0/3776032112 conn(0x56371354a480 msgr2=0x563713587d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 -- 172.21.15.49:0/3776032112 shutdown_connections 2024-02-26T06:26:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:58.999+0000 7f59ee991740 1 -- 172.21.15.49:0/3776032112 wait complete. 2024-02-26T06:26:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.000+0000 7f59ee991740 1 -- start start 2024-02-26T06:26:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.000+0000 7f59ee991740 1 Processor -- start 2024-02-26T06:26:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.001+0000 7f59ee991740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x563713591e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.001+0000 7f59ee991740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713583af0 0x5637135923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.001+0000 7f59ee991740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56371358c300 0x563713592930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.001+0000 7f59ee991740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637135f2900 con 0x563713582d80 2024-02-26T06:26:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.001+0000 7f59ee991740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637135f2a70 con 0x56371358c300 2024-02-26T06:26:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.001+0000 7f59ee991740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637135f2c30 con 0x563713583af0 2024-02-26T06:26:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.001+0000 7f59ee991740 10 client.0 _ll_register_callbacks cb 0x7f59dc009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:26:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.001+0000 7f59e9a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56371358c300 0x563713592930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.002+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713583af0 0x5637135923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.002+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713583af0 0x5637135923b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42152/0 (socket says 172.21.15.49:42152) 2024-02-26T06:26:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.002+0000 7f59e3e00640 1 -- 172.21.15.49:0/681427782 learned_addr learned my addr 172.21.15.49:0/681427782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:26:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.002+0000 7f59e9a00640 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x563713591e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.002+0000 7f59e3e00640 1 -- 172.21.15.49:0/681427782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56371358c300 msgr2=0x563713592930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.002+0000 7f59e3e00640 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56371358c300 0x563713592930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.002+0000 7f59e3e00640 1 -- 172.21.15.49:0/681427782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 msgr2=0x563713591e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:26:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.002+0000 7f59e3e00640 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x563713591e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.003+0000 7f59e3e00640 1 -- 172.21.15.49:0/681427782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d000e9d0 con 0x563713583af0 2024-02-26T06:26:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.003+0000 7f59e3e00640 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713583af0 0x5637135923b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f59d00057c0 tx=0x7f59d00131f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7763856e7048db5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.003+0000 7f59caa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:26:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.004+0000 7f59caa00640 1 -- 172.21.15.49:0/681427782 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00195d0 con 0x563713583af0 2024-02-26T06:26:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.004+0000 7f59caa00640 1 -- 172.21.15.49:0/681427782 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f59d0019730 con 0x563713583af0 2024-02-26T06:26:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.004+0000 7f59ee991740 1 -- 172.21.15.49:0/681427782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56371350ed00 con 0x563713583af0 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.004+0000 7f59caa00640 1 -- 172.21.15.49:0/681427782 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59d0019930 con 0x563713583af0 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.004+0000 7f59ee991740 10 client.26353 fetch_fsmap learned FSMap version 32 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.004+0000 7f59ee991740 1 -- 172.21.15.49:0/681427782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563713484500 con 0x563713583af0 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.005+0000 7f59caa00640 1 -- 172.21.15.49:0/681427782 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56371350ed00 con 0x563713583af0 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.005+0000 7f59caa00640 1 -- 172.21.15.49:0/681427782 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f59d002b540 con 0x563713583af0 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.005+0000 7f59ee991740 10 client.26353 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.005+0000 7f59ee991740 10 client.26353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.005+0000 7f59ee991740 20 client.26353 populate_metadata read hostname 'smithi049' 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.005+0000 7f59ee991740 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5637135f42e0 0x5637135f6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.005+0000 7f59ee991740 4 client.26353 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:26:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.005+0000 7f59ee991740 1 -- 172.21.15.49:0/681427782 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5637131e9600 con 0x5637135f42e0 2024-02-26T06:26:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.006+0000 7f59e9a00640 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5637135f42e0 0x5637135f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.006+0000 7f59e9a00640 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5637135f42e0 0x5637135f6700 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=540dd32e5f1133c2 server_cookie=336454aab25470cc in_seq=0 out_seq=0 2024-02-26T06:26:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.006+0000 7f59caa00640 10 client.26353 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.020+0000 7f59caa00640 1 -- 172.21.15.49:0/681427782 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5637131e9600 con 0x5637135f42e0 2024-02-26T06:26:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.020+0000 7f59caa00640 10 client.26353 handle_command_reply: tid=0 2024-02-26T06:26:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.020+0000 7f59ee991740 1 client.26353 shutdown 2024-02-26T06:26:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.021+0000 7f59ee991740 1 -- 172.21.15.49:0/681427782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713583af0 msgr2=0x5637135923b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.021+0000 7f59ee991740 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713583af0 0x5637135923b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f59d00057c0 tx=0x7f59d00131f0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.021+0000 7f59ee991740 1 -- 172.21.15.49:0/681427782 shutdown_connections 2024-02-26T06:26:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.021+0000 7f59ee991740 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5637135f42e0 0x5637135f6700 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.021+0000 7f59ee991740 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56371358c300 0x563713592930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.021+0000 7f59ee991740 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563713583af0 0x5637135923b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.021+0000 7f59ee991740 1 --2- 172.21.15.49:0/681427782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563713582d80 0x563713591e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.021+0000 7f59ee991740 1 -- 172.21.15.49:0/681427782 >> 172.21.15.49:0/681427782 conn(0x56371354a480 msgr2=0x56371357c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.022+0000 7f59caa00640 0 client.26353 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.022+0000 7f59ee991740 1 -- 172.21.15.49:0/681427782 shutdown_connections 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.022+0000 7f59ee991740 1 -- 172.21.15.49:0/681427782 wait complete. 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.022+0000 7f59ee991740 20 client.26353 trim_cache size 0 max 16384 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:26:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.023+0000 7f59eba00640 1 -- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f59cc0a23d0 msgr2=0x7f59cc0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.023+0000 7f59eba00640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f59cc0a23d0 0x7f59cc0a4890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x563713584250 tx=0x7f59d802bc40 comp rx=0 tx=0).stop 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.023+0000 7f59eba00640 1 -- 172.21.15.49:0/298760695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 msgr2=0x7f59e410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.023+0000 7f59eba00640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 0x7f59e410e650 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f59d802c250 tx=0x7f59d802c6b0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.024+0000 7f59eba00640 1 -- 172.21.15.49:0/298760695 shutdown_connections 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.024+0000 7f59eba00640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f59cc0a23d0 0x7f59cc0a4890 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.024+0000 7f59eba00640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59e410f0d0 0x7f59e41a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.024+0000 7f59eba00640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4106470 0x7f59e410eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.024+0000 7f59eba00640 1 --2- 172.21.15.49:0/298760695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59e41013a0 0x7f59e410e650 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.024+0000 7f59eba00640 1 -- 172.21.15.49:0/298760695 >> 172.21.15.49:0/298760695 conn(0x7f59e40fcc40 msgr2=0x7f59e40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.025+0000 7f59eba00640 1 -- 172.21.15.49:0/298760695 shutdown_connections 2024-02-26T06:26:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:26:59.025+0000 7f59eba00640 1 -- 172.21.15.49:0/298760695 wait complete. 2024-02-26T06:26:59.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:26:59 smithi080 ceph-mon[36206]: pgmap v110: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2024-02-26T06:26:59.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:26:59 smithi115 ceph-mon[36426]: pgmap v110: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2024-02-26T06:26:59.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:26:59 smithi049 ceph-mon[31696]: pgmap v110: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2024-02-26T06:27:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 190.0 s, 2603.5 tps, lat 32.747 ms stddev 39.326 2024-02-26T06:27:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:01 smithi080 ceph-mon[36206]: pgmap v111: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 815 op/s 2024-02-26T06:27:01.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:01 smithi115 ceph-mon[36426]: pgmap v111: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 815 op/s 2024-02-26T06:27:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:01 smithi049 ceph-mon[31696]: pgmap v111: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 815 op/s 2024-02-26T06:27:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:03 smithi115 ceph-mon[36426]: pgmap v112: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-02-26T06:27:03.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:03 smithi049 ceph-mon[31696]: pgmap v112: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-02-26T06:27:03.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:03 smithi080 ceph-mon[36206]: pgmap v112: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-02-26T06:27:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:05 smithi115 ceph-mon[36426]: pgmap v113: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 MiB/s wr, 617 op/s 2024-02-26T06:27:05.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:05 smithi049 ceph-mon[31696]: pgmap v113: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 MiB/s wr, 617 op/s 2024-02-26T06:27:05.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:05 smithi080 ceph-mon[36206]: pgmap v113: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 MiB/s wr, 617 op/s 2024-02-26T06:27:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 195.0 s, 1790.6 tps, lat 64.024 ms stddev 264.971 2024-02-26T06:27:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:07 smithi049 ceph-mon[31696]: pgmap v114: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.61k op/s 2024-02-26T06:27:07.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:07 smithi080 ceph-mon[36206]: pgmap v114: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.61k op/s 2024-02-26T06:27:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:07 smithi115 ceph-mon[36426]: pgmap v114: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.61k op/s 2024-02-26T06:27:07.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:27:07.710 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:27:08.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.012+0000 7fdcd5800640 1 -- 172.21.15.49:0/3187130568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd010d730 msgr2=0x7fdcd010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.012+0000 7fdcd5800640 1 --2- 172.21.15.49:0/3187130568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd010d730 0x7fdcd010db10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009c10 tx=0x7fdcc802bef0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.013+0000 7fdcd5800640 1 -- 172.21.15.49:0/3187130568 shutdown_connections 2024-02-26T06:27:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.013+0000 7fdcd5800640 1 --2- 172.21.15.49:0/3187130568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcd0107720 0x7fdcd0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.013+0000 7fdcd5800640 1 --2- 172.21.15.49:0/3187130568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcd01080e0 0x7fdcd01145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.013+0000 7fdcd5800640 1 --2- 172.21.15.49:0/3187130568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd010d730 0x7fdcd010db10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.013+0000 7fdcd5800640 1 -- 172.21.15.49:0/3187130568 >> 172.21.15.49:0/3187130568 conn(0x7fdcd00756c0 msgr2=0x7fdcd0077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.014+0000 7fdcd5800640 1 -- 172.21.15.49:0/3187130568 shutdown_connections 2024-02-26T06:27:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.014+0000 7fdcd5800640 1 -- 172.21.15.49:0/3187130568 wait complete. 2024-02-26T06:27:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.015+0000 7fdcd5800640 1 Processor -- start 2024-02-26T06:27:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.015+0000 7fdcd5800640 1 -- start start 2024-02-26T06:27:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0107720 0x7fdcd019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcd01080e0 0x7fdcd019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcd010d730 0x7fdcd01a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdcd5800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01a2c80 con 0x7fdcd0107720 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdcd5800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01a2df0 con 0x7fdcd01080e0 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdcd5800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01a2fb0 con 0x7fdcd010d730 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdcce000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcd01080e0 0x7fdcd019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdccea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0107720 0x7fdcd019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdccea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0107720 0x7fdcd019de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54262/0 (socket says 172.21.15.49:54262) 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdccea00640 1 -- 172.21.15.49:0/2957513251 learned_addr learned my addr 172.21.15.49:0/2957513251 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdcce000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcd01080e0 0x7fdcd019e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57684/0 (socket says 172.21.15.49:57684) 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.016+0000 7fdccf400640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcd010d730 0x7fdcd01a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.017+0000 7fdccea00640 1 -- 172.21.15.49:0/2957513251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcd010d730 msgr2=0x7fdcd01a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.017+0000 7fdccea00640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcd010d730 0x7fdcd01a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.017+0000 7fdccea00640 1 -- 172.21.15.49:0/2957513251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcd01080e0 msgr2=0x7fdcd019e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.017+0000 7fdccea00640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcd01080e0 0x7fdcd019e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.017+0000 7fdccea00640 1 -- 172.21.15.49:0/2957513251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc009490 con 0x7fdcd0107720 2024-02-26T06:27:08.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.017+0000 7fdccf400640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcd010d730 0x7fdcd01a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:08.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.017+0000 7fdccea00640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0107720 0x7fdcd019de10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8000f80 tx=0x7fdcc8033610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:08.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.019+0000 7fdcc7400640 1 -- 172.21.15.49:0/2957513251 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8003d80 con 0x7fdcd0107720 2024-02-26T06:27:08.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.019+0000 7fdcc7400640 1 -- 172.21.15.49:0/2957513251 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcc802cdd0 con 0x7fdcd0107720 2024-02-26T06:27:08.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.019+0000 7fdcc7400640 1 -- 172.21.15.49:0/2957513251 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc803d7e0 con 0x7fdcd0107720 2024-02-26T06:27:08.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.020+0000 7fdcd5800640 1 -- 172.21.15.49:0/2957513251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc0009560 con 0x7fdcd0107720 2024-02-26T06:27:08.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.021+0000 7fdcd5800640 1 -- 172.21.15.49:0/2957513251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc8009840 con 0x7fdcd0107720 2024-02-26T06:27:08.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.026+0000 7fdcc7400640 1 -- 172.21.15.49:0/2957513251 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdcc8004040 con 0x7fdcd0107720 2024-02-26T06:27:08.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.026+0000 7fdcd87b3740 1 Processor -- start 2024-02-26T06:27:08.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdcd87b3740 1 -- start start 2024-02-26T06:27:08.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2539cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda253a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda25459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda24a9120 con 0x55cda2543600 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda24a6650 con 0x55cda253a290 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda24a1e30 con 0x55cda253af70 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdccea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda253a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdccea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda253a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57700/0 (socket says 172.21.15.49:57700) 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.027+0000 7fdccea00640 1 -- 172.21.15.49:0/4062660692 learned_addr learned my addr 172.21.15.49:0/4062660692 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:08.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdccf400640 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2539cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdcce000640 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda25459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdccea00640 1 -- 172.21.15.49:0/4062660692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 msgr2=0x55cda2539cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:08.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdccea00640 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2539cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdccea00640 1 -- 172.21.15.49:0/4062660692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 msgr2=0x55cda25459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdccea00640 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda25459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdccea00640 1 -- 172.21.15.49:0/4062660692 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cda245b030 con 0x55cda253a290 2024-02-26T06:27:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdccea00640 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda253a710 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ed390 tx=0x7fdcc80f86b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109422a1188ab832 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.028+0000 7fdcbbe00640 1 -- 172.21.15.49:0/4062660692 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80fb460 con 0x55cda253a290 2024-02-26T06:27:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.029+0000 7fdcbbe00640 1 -- 172.21.15.49:0/4062660692 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcc80fbaa0 con 0x55cda253a290 2024-02-26T06:27:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.029+0000 7fdcbbe00640 1 -- 172.21.15.49:0/4062660692 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80fa8b0 con 0x55cda253a290 2024-02-26T06:27:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.029+0000 7fdcc7400640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdcb40a2380 0x7fdcb40a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.029+0000 7fdcc7400640 1 -- 172.21.15.49:0/2957513251 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fdcc80efbb0 con 0x7fdcd0107720 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.029+0000 7fdcd87b3740 1 -- 172.21.15.49:0/4062660692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 msgr2=0x55cda253a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.029+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda253a710 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ed390 tx=0x7fdcc80f86b0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.030+0000 7fdccf400640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdcb40a2380 0x7fdcb40a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.030+0000 7fdcd87b3740 1 -- 172.21.15.49:0/4062660692 shutdown_connections 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.030+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda253a710 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.030+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda25459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.030+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/4062660692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2539cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.030+0000 7fdcd87b3740 1 -- 172.21.15.49:0/4062660692 >> 172.21.15.49:0/4062660692 conn(0x55cda2501570 msgr2=0x55cda2534ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.031+0000 7fdcd87b3740 1 -- 172.21.15.49:0/4062660692 shutdown_connections 2024-02-26T06:27:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.031+0000 7fdcd87b3740 1 -- 172.21.15.49:0/4062660692 wait complete. 2024-02-26T06:27:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.031+0000 7fdccf400640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdcb40a2380 0x7fdcb40a4840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009530 tx=0x7fdcc0009260 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.031+0000 7fdcd87b3740 1 -- start start 2024-02-26T06:27:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.032+0000 7fdcd87b3740 1 Processor -- start 2024-02-26T06:27:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.032+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda25405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.032+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2540af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda2541030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda2446e90 con 0x55cda2543600 2024-02-26T06:27:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda2446b30 con 0x55cda253a290 2024-02-26T06:27:08.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda22101c0 con 0x55cda253af70 2024-02-26T06:27:08.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdcd87b3740 10 client.0 _ll_register_callbacks cb 0x55cda245a930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:08.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdccea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda25405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdccea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda25405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57708/0 (socket says 172.21.15.49:57708) 2024-02-26T06:27:08.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdccea00640 1 -- 172.21.15.49:0/3894841312 learned_addr learned my addr 172.21.15.49:0/3894841312 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:08.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdcce000640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2540af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdccf400640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda2541030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.033+0000 7fdccea00640 1 -- 172.21.15.49:0/3894841312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 msgr2=0x55cda2540af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdccea00640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2540af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdccea00640 1 -- 172.21.15.49:0/3894841312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 msgr2=0x55cda2541030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdccea00640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda2541030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdccea00640 1 -- 172.21.15.49:0/3894841312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc8009840 con 0x55cda253a290 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdcce000640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2540af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdccf400640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda2541030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdccea00640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda25405b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80b9090 tx=0x7fdcc802bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeeabb4dadb58d83 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdcb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3894841312 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80fa3c0 con 0x55cda253a290 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3894841312 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fdcc80bcdc0 con 0x55cda253a290 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.034+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3894841312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cda21a17e0 con 0x55cda253a290 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3894841312 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcc80f8eb0 con 0x55cda253a290 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcd87b3740 10 client.26324 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:08.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3894841312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cda24a04b0 con 0x55cda253a290 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3894841312 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cda21a17e0 con 0x55cda253a290 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3894841312 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fdcc80b8d30 con 0x55cda253a290 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcd87b3740 10 client.26324 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcd87b3740 10 client.26324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcd87b3740 20 client.26324 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cda2552fb0 0x55cda25553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcd87b3740 4 client.26324 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.035+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3894841312 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cda21de720 con 0x55cda2552fb0 2024-02-26T06:27:08.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.036+0000 7fdcce000640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cda2552fb0 0x55cda25553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.039+0000 7fdcce000640 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cda2552fb0 0x55cda25553f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f101efd5a85615a server_cookie=b65c7db20b1133b3 in_seq=0 out_seq=0 2024-02-26T06:27:08.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.039+0000 7fdcb8c00640 10 client.26324 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:08.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3894841312 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cda21de720 con 0x55cda2552fb0 2024-02-26T06:27:08.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcb8c00640 10 client.26324 handle_command_reply: tid=0 2024-02-26T06:27:08.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 client.26324 shutdown 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3894841312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 msgr2=0x55cda25405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda25405b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80b9090 tx=0x7fdcc802bef0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3894841312 shutdown_connections 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cda2552fb0 0x55cda25553f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda2543600 0x55cda2541030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda2540af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3894841312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda25405b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcb8c00640 0 client.26324 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.040+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3894841312 >> 172.21.15.49:0/3894841312 conn(0x55cda2501570 msgr2=0x55cda2536a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.041+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3894841312 shutdown_connections 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.041+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3894841312 wait complete. 2024-02-26T06:27:08.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.041+0000 7fdcd87b3740 20 client.26324 trim_cache size 0 max 16384 2024-02-26T06:27:08.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.063+0000 7fdcd87b3740 1 Processor -- start 2024-02-26T06:27:08.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.064+0000 7fdcd87b3740 1 -- start start 2024-02-26T06:27:08.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda2542ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253af70 0x55cda253f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda2543600 0x55cda2540e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda21a17e0 con 0x55cda253af70 2024-02-26T06:27:08.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda2535ae0 con 0x55cda253a290 2024-02-26T06:27:08.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda2539b00 con 0x55cda2543600 2024-02-26T06:27:08.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdccf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda2543600 0x55cda2540e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdccf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda2543600 0x55cda2540e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39674/0 (socket says 172.21.15.49:39674) 2024-02-26T06:27:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.065+0000 7fdccf400640 1 -- 172.21.15.49:0/2627785531 learned_addr learned my addr 172.21.15.49:0/2627785531 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:08.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.066+0000 7fdccea00640 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda2542ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.066+0000 7fdcce000640 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253af70 0x55cda253f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.067+0000 7fdccf400640 1 -- 172.21.15.49:0/2627785531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 msgr2=0x55cda2542ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.067+0000 7fdccf400640 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda2542ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.067+0000 7fdccf400640 1 -- 172.21.15.49:0/2627785531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253af70 msgr2=0x55cda253f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.067+0000 7fdccf400640 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253af70 0x55cda253f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.067+0000 7fdccf400640 1 -- 172.21.15.49:0/2627785531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cda243c920 con 0x55cda2543600 2024-02-26T06:27:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.068+0000 7fdccf400640 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda2543600 0x55cda2540e50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00045a0 tx=0x7fdcc0016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a69654a314ddac9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.068+0000 7fdcbb400640 1 -- 172.21.15.49:0/2627785531 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0016070 con 0x55cda2543600 2024-02-26T06:27:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.068+0000 7fdcbb400640 1 -- 172.21.15.49:0/2627785531 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0023040 con 0x55cda2543600 2024-02-26T06:27:08.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.068+0000 7fdcbb400640 1 -- 172.21.15.49:0/2627785531 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc001c740 con 0x55cda2543600 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 -- 172.21.15.49:0/2627785531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda2543600 msgr2=0x55cda2540e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda2543600 0x55cda2540e50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00045a0 tx=0x7fdcc0016000 comp rx=0 tx=0).stop 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 -- 172.21.15.49:0/2627785531 shutdown_connections 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda2543600 0x55cda2540e50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253af70 0x55cda253f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/2627785531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda253a290 0x55cda2542ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 -- 172.21.15.49:0/2627785531 >> 172.21.15.49:0/2627785531 conn(0x55cda2501570 msgr2=0x55cda253ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 -- 172.21.15.49:0/2627785531 shutdown_connections 2024-02-26T06:27:08.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.069+0000 7fdcd87b3740 1 -- 172.21.15.49:0/2627785531 wait complete. 2024-02-26T06:27:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.070+0000 7fdcd87b3740 1 -- start start 2024-02-26T06:27:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.070+0000 7fdcd87b3740 1 Processor -- start 2024-02-26T06:27:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.071+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253a290 0x55cda2548b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.071+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda25490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.071+0000 7fdcd87b3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda2543600 0x55cda25495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.071+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda254aac0 con 0x55cda253a290 2024-02-26T06:27:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.071+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda25a94e0 con 0x55cda2543600 2024-02-26T06:27:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.071+0000 7fdcd87b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda25a9650 con 0x55cda253af70 2024-02-26T06:27:08.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.071+0000 7fdcd87b3740 10 client.0 _ll_register_callbacks cb 0x7fdcc80f3a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.072+0000 7fdccf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda2543600 0x55cda25495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.072+0000 7fdccf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda2543600 0x55cda25495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57722/0 (socket says 172.21.15.49:57722) 2024-02-26T06:27:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.072+0000 7fdccf400640 1 -- 172.21.15.49:0/3792153115 learned_addr learned my addr 172.21.15.49:0/3792153115 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.072+0000 7fdcce000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda25490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.072+0000 7fdccea00640 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253a290 0x55cda2548b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.072+0000 7fdccf400640 1 -- 172.21.15.49:0/3792153115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 msgr2=0x55cda25490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.072+0000 7fdccf400640 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda25490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.072+0000 7fdccf400640 1 -- 172.21.15.49:0/3792153115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253a290 msgr2=0x55cda2548b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:08.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.073+0000 7fdccf400640 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253a290 0x55cda2548b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.073+0000 7fdccf400640 1 -- 172.21.15.49:0/3792153115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc0004290 con 0x55cda2543600 2024-02-26T06:27:08.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.073+0000 7fdccea00640 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253a290 0x55cda2548b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:08.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.075+0000 7fdccf400640 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda2543600 0x55cda25495f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0005890 tx=0x7fdcc0009fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77b209bbb473c75e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:08.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.075+0000 7fdcb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:08.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.076+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3792153115 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0005610 con 0x55cda2543600 2024-02-26T06:27:08.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.076+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3792153115 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fdcc000f540 con 0x55cda2543600 2024-02-26T06:27:08.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.077+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3792153115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcc80fbaa0 con 0x55cda2543600 2024-02-26T06:27:08.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.077+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3792153115 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcc000f700 con 0x55cda2543600 2024-02-26T06:27:08.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.077+0000 7fdcd87b3740 10 client.26330 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:08.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.077+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3792153115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cda2462400 con 0x55cda2543600 2024-02-26T06:27:08.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3792153115 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcc80fbaa0 con 0x55cda2543600 2024-02-26T06:27:08.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3792153115 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fdcc0030460 con 0x55cda2543600 2024-02-26T06:27:08.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdcd87b3740 10 client.26330 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:08.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdcd87b3740 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:08.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdcd87b3740 20 client.26330 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:08.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cda25aad60 0x55cda25ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdcd87b3740 4 client.26330 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:27:08.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3792153115 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cda222d890 con 0x55cda25aad60 2024-02-26T06:27:08.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.078+0000 7fdccea00640 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cda25aad60 0x55cda25ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.079+0000 7fdccea00640 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cda25aad60 0x55cda25ad180 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b521ebcdb77b5a89 server_cookie=dfc1e405e5be71be in_seq=0 out_seq=0 2024-02-26T06:27:08.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.079+0000 7fdcb8c00640 10 client.26330 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:08.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.079+0000 7fdcb8c00640 1 -- 172.21.15.49:0/3792153115 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cda222d890 con 0x55cda25aad60 2024-02-26T06:27:08.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.080+0000 7fdcb8c00640 10 client.26330 handle_command_reply: tid=0 2024-02-26T06:27:08.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.080+0000 7fdcd87b3740 1 client.26330 shutdown 2024-02-26T06:27:08.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.080+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3792153115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda2543600 msgr2=0x55cda25495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.080+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda2543600 0x55cda25495f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0005890 tx=0x7fdcc0009fc0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3792153115 shutdown_connections 2024-02-26T06:27:08.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cda25aad60 0x55cda25ad180 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cda2543600 0x55cda25495f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cda253af70 0x55cda25490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcd87b3740 1 --2- 172.21.15.49:0/3792153115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cda253a290 0x55cda2548b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3792153115 >> 172.21.15.49:0/3792153115 conn(0x55cda2501570 msgr2=0x55cda2533570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:08.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcb8c00640 0 client.26330 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:08.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3792153115 shutdown_connections 2024-02-26T06:27:08.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.081+0000 7fdcd87b3740 1 -- 172.21.15.49:0/3792153115 wait complete. 2024-02-26T06:27:08.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.082+0000 7fdcd87b3740 20 client.26330 trim_cache size 0 max 16384 2024-02-26T06:27:08.083 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:27:08.083 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:27:08.083 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:27:08.083 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 -- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdcb40a2380 msgr2=0x7fdcb40a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdcb40a2380 0x7fdcb40a4840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009530 tx=0x7fdcc0009260 comp rx=0 tx=0).stop 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 -- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0107720 msgr2=0x7fdcd019de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0107720 0x7fdcd019de10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8000f80 tx=0x7fdcc8033610 comp rx=0 tx=0).stop 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 -- 172.21.15.49:0/2957513251 shutdown_connections 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdcb40a2380 0x7fdcb40a4840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcd01080e0 0x7fdcd019e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcd010d730 0x7fdcd01a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 --2- 172.21.15.49:0/2957513251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0107720 0x7fdcd019de10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 -- 172.21.15.49:0/2957513251 >> 172.21.15.49:0/2957513251 conn(0x7fdcd00756c0 msgr2=0x7fdcd0075e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 -- 172.21.15.49:0/2957513251 shutdown_connections 2024-02-26T06:27:08.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.083+0000 7fdcd5800640 1 -- 172.21.15.49:0/2957513251 wait complete. 2024-02-26T06:27:08.109 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:27:08.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:27:08.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.403+0000 7ff11a600640 1 -- 172.21.15.49:0/2070821640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff114106450 msgr2=0x7ff11410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.403+0000 7ff11a600640 1 --2- 172.21.15.49:0/2070821640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff114106450 0x7ff11410e020 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff10000fb10 tx=0x7ff1000341b0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.403+0000 7ff11a600640 1 -- 172.21.15.49:0/2070821640 shutdown_connections 2024-02-26T06:27:08.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.404+0000 7ff11a600640 1 --2- 172.21.15.49:0/2070821640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff114106450 0x7ff11410e020 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.404+0000 7ff11a600640 1 --2- 172.21.15.49:0/2070821640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff114101d30 0x7ff114105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.404+0000 7ff11a600640 1 --2- 172.21.15.49:0/2070821640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff114101380 0x7ff114101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.404+0000 7ff11a600640 1 -- 172.21.15.49:0/2070821640 >> 172.21.15.49:0/2070821640 conn(0x7ff1140fcc20 msgr2=0x7ff1140ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:08.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.404+0000 7ff11a600640 1 -- 172.21.15.49:0/2070821640 shutdown_connections 2024-02-26T06:27:08.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.404+0000 7ff11a600640 1 -- 172.21.15.49:0/2070821640 wait complete. 2024-02-26T06:27:08.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.406+0000 7ff11a600640 1 Processor -- start 2024-02-26T06:27:08.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.406+0000 7ff11a600640 1 -- start start 2024-02-26T06:27:08.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff11a600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff114101380 0x7ff114195510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff11a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff114106450 0x7ff114195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff11a600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff114199d80 0x7ff11419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff11a600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419a770 con 0x7ff114106450 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff11a600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419a8b0 con 0x7ff114199d80 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff11a600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419aa70 con 0x7ff114101380 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff113e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff114199d80 0x7ff11419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff113400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff114101380 0x7ff114195510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff113e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff114199d80 0x7ff11419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57736/0 (socket says 172.21.15.49:57736) 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.407+0000 7ff113e00640 1 -- 172.21.15.49:0/4127834229 learned_addr learned my addr 172.21.15.49:0/4127834229 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.408+0000 7ff112a00640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff114106450 0x7ff114195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.408+0000 7ff113e00640 1 -- 172.21.15.49:0/4127834229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff114101380 msgr2=0x7ff114195510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.408+0000 7ff113e00640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff114101380 0x7ff114195510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.408+0000 7ff113e00640 1 -- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff114106450 msgr2=0x7ff114195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.408+0000 7ff113e00640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff114106450 0x7ff114195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.408+0000 7ff113e00640 1 -- 172.21.15.49:0/4127834229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c009840 con 0x7ff114199d80 2024-02-26T06:27:08.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.408+0000 7ff112a00640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff114106450 0x7ff114195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:08.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.408+0000 7ff113e00640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff114199d80 0x7ff11419a230 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff1000057c0 tx=0x7ff100005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:08.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.409+0000 7ff10be00640 1 -- 172.21.15.49:0/4127834229 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100004a50 con 0x7ff114199d80 2024-02-26T06:27:08.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.409+0000 7ff10be00640 1 -- 172.21.15.49:0/4127834229 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff100004bb0 con 0x7ff114199d80 2024-02-26T06:27:08.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.409+0000 7ff10be00640 1 -- 172.21.15.49:0/4127834229 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100046820 con 0x7ff114199d80 2024-02-26T06:27:08.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.409+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10000f770 con 0x7ff114199d80 2024-02-26T06:27:08.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.409+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11406ade0 con 0x7ff114199d80 2024-02-26T06:27:08.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.411+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1140683c0 con 0x7ff114199d80 2024-02-26T06:27:08.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.413+0000 7ff10be00640 1 -- 172.21.15.49:0/4127834229 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff100004d20 con 0x7ff114199d80 2024-02-26T06:27:08.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.414+0000 7ff10be00640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff0f80a2330 0x7ff0f80a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:08.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.415+0000 7ff10be00640 1 -- 172.21.15.49:0/4127834229 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7ff1000f8a10 con 0x7ff114199d80 2024-02-26T06:27:08.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.415+0000 7ff113400640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff0f80a2330 0x7ff0f80a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:08.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.417+0000 7ff113400640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff0f80a2330 0x7ff0f80a47f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff10c005fd0 tx=0x7ff10c009500 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:08.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.418+0000 7ff10be00640 1 -- 172.21.15.49:0/4127834229 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1000c1760 con 0x7ff114199d80 2024-02-26T06:27:08.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.812+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff114196810 con 0x7ff114199d80 2024-02-26T06:27:08.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.813+0000 7ff10be00640 1 -- 172.21.15.49:0/4127834229 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11332 (secure 0 0 0) 0x7ff1000c4e20 con 0x7ff114199d80 2024-02-26T06:27:08.814 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:27:08.814 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:26:53.232438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:27:08.814 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 32 2024-02-26T06:27:08.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff0f80a2330 msgr2=0x7ff0f80a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff0f80a2330 0x7ff0f80a47f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff10c005fd0 tx=0x7ff10c009500 comp rx=0 tx=0).stop 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff114199d80 msgr2=0x7ff11419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff114199d80 0x7ff11419a230 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff1000057c0 tx=0x7ff100005720 comp rx=0 tx=0).stop 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 shutdown_connections 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff0f80a2330 0x7ff0f80a47f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff114199d80 0x7ff11419a230 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff114106450 0x7ff114195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 --2- 172.21.15.49:0/4127834229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff114101380 0x7ff114195510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 >> 172.21.15.49:0/4127834229 conn(0x7ff1140fcc20 msgr2=0x7ff1140fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 shutdown_connections 2024-02-26T06:27:08.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:08.818+0000 7ff11a600640 1 -- 172.21.15.49:0/4127834229 wait complete. 2024-02-26T06:27:08.843 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:27:08.843 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:27:09.098 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:09 smithi049 ceph-mon[31696]: pgmap v115: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 1.31k op/s 2024-02-26T06:27:09.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.125+0000 7f82b5600640 1 -- 172.21.15.49:0/1058965982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82b00685f0 msgr2=0x7f82b00689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.125+0000 7f82b5600640 1 --2- 172.21.15.49:0/1058965982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82b00685f0 0x7f82b00689d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009930 tx=0x7f82a002bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:09.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.125+0000 7f82b5600640 1 -- 172.21.15.49:0/1058965982 shutdown_connections 2024-02-26T06:27:09.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.125+0000 7f82b5600640 1 --2- 172.21.15.49:0/1058965982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82b01064c0 0x7f82b010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.125+0000 7f82b5600640 1 --2- 172.21.15.49:0/1058965982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b0068fa0 0x7f82b0105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.125+0000 7f82b5600640 1 --2- 172.21.15.49:0/1058965982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82b00685f0 0x7f82b00689d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.125+0000 7f82b5600640 1 -- 172.21.15.49:0/1058965982 >> 172.21.15.49:0/1058965982 conn(0x7f82b0077180 msgr2=0x7f82b0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.125+0000 7f82b5600640 1 -- 172.21.15.49:0/1058965982 shutdown_connections 2024-02-26T06:27:09.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.126+0000 7f82b5600640 1 -- 172.21.15.49:0/1058965982 wait complete. 2024-02-26T06:27:09.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.127+0000 7f82b5600640 1 Processor -- start 2024-02-26T06:27:09.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.127+0000 7f82b5600640 1 -- start start 2024-02-26T06:27:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.127+0000 7f82b5600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82b00685f0 0x7f82b019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.127+0000 7f82b5600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b01064c0 0x7f82b019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.127+0000 7f82b5600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82b0199800 0x7f82b0199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.127+0000 7f82b5600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b019a2b0 con 0x7f82b00685f0 2024-02-26T06:27:09.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.127+0000 7f82b5600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b019a420 con 0x7f82b0199800 2024-02-26T06:27:09.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.127+0000 7f82b5600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b019a590 con 0x7f82b01064c0 2024-02-26T06:27:09.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.128+0000 7f82af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82b0199800 0x7f82b0199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.128+0000 7f82af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82b0199800 0x7f82b0199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57758/0 (socket says 172.21.15.49:57758) 2024-02-26T06:27:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.128+0000 7f82af400640 1 -- 172.21.15.49:0/1174676766 learned_addr learned my addr 172.21.15.49:0/1174676766 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.128+0000 7f82ae000640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b01064c0 0x7f82b019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.128+0000 7f82aea00640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82b00685f0 0x7f82b019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82af400640 1 -- 172.21.15.49:0/1174676766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b01064c0 msgr2=0x7f82b019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82af400640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b01064c0 0x7f82b019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82af400640 1 -- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82b00685f0 msgr2=0x7f82b019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82af400640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82b00685f0 0x7f82b019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82af400640 1 -- 172.21.15.49:0/1174676766 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a0009560 con 0x7f82b0199800 2024-02-26T06:27:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82af400640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82b0199800 0x7f82b0199cb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f82a800eb10 tx=0x7f82a800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82a7400640 1 -- 172.21.15.49:0/1174676766 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8004b20 con 0x7f82b0199800 2024-02-26T06:27:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82a7400640 1 -- 172.21.15.49:0/1174676766 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82a8004c80 con 0x7f82b0199800 2024-02-26T06:27:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82a7400640 1 -- 172.21.15.49:0/1174676766 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8010650 con 0x7f82b0199800 2024-02-26T06:27:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.129+0000 7f82b5600640 1 -- 172.21.15.49:0/1174676766 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b01a6540 con 0x7f82b0199800 2024-02-26T06:27:09.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.130+0000 7f82b5600640 1 -- 172.21.15.49:0/1174676766 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b01a6970 con 0x7f82b0199800 2024-02-26T06:27:09.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.134+0000 7f82b842a740 1 Processor -- start 2024-02-26T06:27:09.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.135+0000 7f82a7400640 1 -- 172.21.15.49:0/1174676766 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f82a8005020 con 0x7f82b0199800 2024-02-26T06:27:09.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.136+0000 7f82b842a740 1 -- start start 2024-02-26T06:27:09.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.136+0000 7f82b842a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b40800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82b842a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82b842a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b49920 0x55db18b4bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82b842a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18aa61e0 con 0x55db18b40d40 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82b842a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18aa3710 con 0x55db18b40420 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82b842a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18a9eef0 con 0x55db18b49920 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82ae000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b40800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82ae000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b40800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57760/0 (socket says 172.21.15.49:57760) 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82ae000640 1 -- 172.21.15.49:0/4151277616 learned_addr learned my addr 172.21.15.49:0/4151277616 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.137+0000 7f82aea00640 1 --2- 172.21.15.49:0/4151277616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82ae000640 1 -- 172.21.15.49:0/4151277616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b49920 msgr2=0x55db18b4bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82ae000640 1 --2- 172.21.15.49:0/4151277616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b49920 0x55db18b4bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82ae000640 1 -- 172.21.15.49:0/4151277616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 msgr2=0x55db18b493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82ae000640 1 --2- 172.21.15.49:0/4151277616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b493e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82ae000640 1 -- 172.21.15.49:0/4151277616 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db18a60dc0 con 0x55db18b40420 2024-02-26T06:27:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82a7400640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f82940a22b0 0x7f82940a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82a7400640 1 -- 172.21.15.49:0/1174676766 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f82a8014070 con 0x7f82b0199800 2024-02-26T06:27:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82ae000640 1 --2- 172.21.15.49:0/4151277616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b40800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f829c009ca0 tx=0x7f829c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=112a418715bf5948 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.138+0000 7f82aea00640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f82940a22b0 0x7f82940a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.139+0000 7f829b400640 1 -- 172.21.15.49:0/4151277616 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c004030 con 0x55db18b40420 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.139+0000 7f829b400640 1 -- 172.21.15.49:0/4151277616 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f829c005020 con 0x55db18b40420 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.139+0000 7f829b400640 1 -- 172.21.15.49:0/4151277616 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c004bc0 con 0x55db18b40420 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.139+0000 7f82b842a740 1 -- 172.21.15.49:0/4151277616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 msgr2=0x55db18b40800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 --2- 172.21.15.49:0/4151277616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b40800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f829c009ca0 tx=0x7f829c011810 comp rx=0 tx=0).stop 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 -- 172.21.15.49:0/4151277616 shutdown_connections 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82aea00640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f82940a22b0 0x7f82940a4770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000c00 tx=0x7f82a002e040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 --2- 172.21.15.49:0/4151277616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b49920 0x55db18b4bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 --2- 172.21.15.49:0/4151277616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 --2- 172.21.15.49:0/4151277616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b40800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 -- 172.21.15.49:0/4151277616 >> 172.21.15.49:0/4151277616 conn(0x55db18b3ab50 msgr2=0x55db18b3af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 -- 172.21.15.49:0/4151277616 shutdown_connections 2024-02-26T06:27:09.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 -- 172.21.15.49:0/4151277616 wait complete. 2024-02-26T06:27:09.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.140+0000 7f82b842a740 1 -- start start 2024-02-26T06:27:09.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.141+0000 7f82b842a740 1 Processor -- start 2024-02-26T06:27:09.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82b842a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b46920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82b842a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40d40 0x55db18b46e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82b842a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18b473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82aea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b46920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40d40 0x55db18b46e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40d40 0x55db18b46e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39744/0 (socket says 172.21.15.49:39744) 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18a44120 con 0x55db18b49920 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18a43dc0 con 0x55db18b40420 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1880d1c0 con 0x55db18b40d40 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82ae000640 1 -- 172.21.15.49:0/482388375 learned_addr learned my addr 172.21.15.49:0/482388375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.142+0000 7f82b842a740 10 client.0 _ll_register_callbacks cb 0x55db18a93280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:09.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.143+0000 7f82aea00640 1 -- 172.21.15.49:0/482388375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40d40 msgr2=0x55db18b46e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.143+0000 7f82aea00640 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40d40 0x55db18b46e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.143+0000 7f82aea00640 1 -- 172.21.15.49:0/482388375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 msgr2=0x55db18b473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.143+0000 7f82af400640 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18b473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.143+0000 7f82aea00640 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18b473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.143+0000 7f82aea00640 1 -- 172.21.15.49:0/482388375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c009870 con 0x55db18b40420 2024-02-26T06:27:09.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.143+0000 7f82ae000640 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40d40 0x55db18b46e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:09.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.144+0000 7f82aea00640 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b46920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f82a002c240 tx=0x7f82a002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26c62ca41e217e7a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.144+0000 7f828fe00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.144+0000 7f828fe00640 1 -- 172.21.15.49:0/482388375 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a002cea0 con 0x55db18b40420 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.144+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db1879e7e0 con 0x55db18b40420 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.144+0000 7f828fe00640 1 -- 172.21.15.49:0/482388375 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f82a002cbd0 con 0x55db18b40420 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.144+0000 7f828fe00640 1 -- 172.21.15.49:0/482388375 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82a0004df0 con 0x55db18b40420 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.144+0000 7f82b842a740 10 client.26348 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.144+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55db18a9d570 con 0x55db18b40420 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.145+0000 7f828fe00640 1 -- 172.21.15.49:0/482388375 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55db1879e7e0 con 0x55db18b40420 2024-02-26T06:27:09.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.145+0000 7f828fe00640 1 -- 172.21.15.49:0/482388375 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f82a0003a90 con 0x55db18b40420 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.145+0000 7f82b842a740 10 client.26348 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.145+0000 7f82b842a740 10 client.26348 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.145+0000 7f82b842a740 20 client.26348 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.145+0000 7f82b842a740 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55db18b599b0 0x55db18b5bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.145+0000 7f82b842a740 4 client.26348 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.145+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db187db720 con 0x55db18b599b0 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.146+0000 7f82ae000640 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55db18b599b0 0x55db18b5bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.147+0000 7f828fe00640 10 client.26348 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:27:09.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.147+0000 7f82ae000640 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55db18b599b0 0x55db18b5bdd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be61c0e20df87973 server_cookie=c30ba580d711f133 in_seq=0 out_seq=0 2024-02-26T06:27:09.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.150+0000 7f828fe00640 1 -- 172.21.15.49:0/482388375 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db187db720 con 0x55db18b599b0 2024-02-26T06:27:09.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.150+0000 7f828fe00640 10 client.26348 handle_command_reply: tid=0 2024-02-26T06:27:09.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.150+0000 7f82b842a740 1 client.26348 shutdown 2024-02-26T06:27:09.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.150+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 msgr2=0x55db18b46920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.150+0000 7f82b842a740 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b46920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f82a002c240 tx=0x7f82a002bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:09.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 shutdown_connections 2024-02-26T06:27:09.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55db18b599b0 0x55db18b5bdd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18b473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40d40 0x55db18b46e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 1 --2- 172.21.15.49:0/482388375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40420 0x55db18b46920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 >> 172.21.15.49:0/482388375 conn(0x55db18b3ab50 msgr2=0x55db18b3c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f828fe00640 0 client.26348 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:27:09.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 shutdown_connections 2024-02-26T06:27:09.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 1 -- 172.21.15.49:0/482388375 wait complete. 2024-02-26T06:27:09.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.151+0000 7f82b842a740 20 client.26348 trim_cache size 0 max 16384 2024-02-26T06:27:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82b842a740 1 Processor -- start 2024-02-26T06:27:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82b842a740 1 -- start start 2024-02-26T06:27:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82b842a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82b842a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82b842a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b49920 0x55db18b507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82b842a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18b3f840 con 0x55db18b40d40 2024-02-26T06:27:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82b842a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18b3b950 con 0x55db18b49920 2024-02-26T06:27:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82b842a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18b564b0 con 0x55db18b40420 2024-02-26T06:27:09.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b49920 0x55db18b507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b49920 0x55db18b507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57770/0 (socket says 172.21.15.49:57770) 2024-02-26T06:27:09.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.173+0000 7f82af400640 1 -- 172.21.15.49:0/525401917 learned_addr learned my addr 172.21.15.49:0/525401917 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.174+0000 7f82aea00640 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.174+0000 7f82ae000640 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.175+0000 7f82af400640 1 -- 172.21.15.49:0/525401917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 msgr2=0x55db18b471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.175+0000 7f82af400640 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.175+0000 7f82af400640 1 -- 172.21.15.49:0/525401917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 msgr2=0x55db18b312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.175+0000 7f82af400640 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b312c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.175+0000 7f82af400640 1 -- 172.21.15.49:0/525401917 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db18a39a80 con 0x55db18b49920 2024-02-26T06:27:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.175+0000 7f82ae000640 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:09.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.176+0000 7f82af400640 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b49920 0x55db18b507d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f82a8097a90 tx=0x7f82a8009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eada6154a2f222f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.176+0000 7f829aa00640 1 -- 172.21.15.49:0/525401917 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a809b0c0 con 0x55db18b49920 2024-02-26T06:27:09.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.176+0000 7f829aa00640 1 -- 172.21.15.49:0/525401917 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82a809f210 con 0x55db18b49920 2024-02-26T06:27:09.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.176+0000 7f829aa00640 1 -- 172.21.15.49:0/525401917 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a809b320 con 0x55db18b49920 2024-02-26T06:27:09.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.177+0000 7f82b842a740 1 -- 172.21.15.49:0/525401917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b49920 msgr2=0x55db18b507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.177+0000 7f82b842a740 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b49920 0x55db18b507d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f82a8097a90 tx=0x7f82a8009a70 comp rx=0 tx=0).stop 2024-02-26T06:27:09.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.178+0000 7f82b842a740 1 -- 172.21.15.49:0/525401917 shutdown_connections 2024-02-26T06:27:09.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.178+0000 7f82b842a740 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b49920 0x55db18b507d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.178+0000 7f82b842a740 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b40d40 0x55db18b312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.178+0000 7f82b842a740 1 --2- 172.21.15.49:0/525401917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.178+0000 7f82b842a740 1 -- 172.21.15.49:0/525401917 >> 172.21.15.49:0/525401917 conn(0x55db18b3ab50 msgr2=0x55db18b30bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.178+0000 7f82b842a740 1 -- 172.21.15.49:0/525401917 shutdown_connections 2024-02-26T06:27:09.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.178+0000 7f82b842a740 1 -- 172.21.15.49:0/525401917 wait complete. 2024-02-26T06:27:09.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.178+0000 7f82b842a740 1 -- start start 2024-02-26T06:27:09.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.179+0000 7f82b842a740 1 Processor -- start 2024-02-26T06:27:09.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.179+0000 7f82b842a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b4fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.179+0000 7f82b842a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40d40 0x55db18b50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.179+0000 7f82b842a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18ba9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.179+0000 7f82b842a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18baa520 con 0x55db18b49920 2024-02-26T06:27:09.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.179+0000 7f82b842a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18baa690 con 0x55db18b40d40 2024-02-26T06:27:09.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.179+0000 7f82b842a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db18baa850 con 0x55db18b40420 2024-02-26T06:27:09.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.180+0000 7f82b842a740 10 client.0 _ll_register_callbacks cb 0x7f829c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:09.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.180+0000 7f82aea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b4fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.180+0000 7f82aea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b4fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39762/0 (socket says 172.21.15.49:39762) 2024-02-26T06:27:09.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.180+0000 7f82aea00640 1 -- 172.21.15.49:0/473777602 learned_addr learned my addr 172.21.15.49:0/473777602 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.180+0000 7f82ae000640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40d40 0x55db18b50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.181+0000 7f82af400640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18ba9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.181+0000 7f82ae000640 1 -- 172.21.15.49:0/473777602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 msgr2=0x55db18b4fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.181+0000 7f82ae000640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b4fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.181+0000 7f82ae000640 1 -- 172.21.15.49:0/473777602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 msgr2=0x55db18ba9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.181+0000 7f82ae000640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18ba9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.181+0000 7f82ae000640 1 -- 172.21.15.49:0/473777602 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b01a6970 con 0x55db18b40d40 2024-02-26T06:27:09.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.181+0000 7f82aea00640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b4fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:09.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.181+0000 7f82af400640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18ba9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:09.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.182+0000 7f82ae000640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40d40 0x55db18b50090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55db18b47ef0 tx=0x7f829c009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fb9913ea0f9a898 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.182+0000 7f828fe00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:09.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.183+0000 7f828fe00640 1 -- 172.21.15.49:0/473777602 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c004040 con 0x55db18b40d40 2024-02-26T06:27:09.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.183+0000 7f828fe00640 1 -- 172.21.15.49:0/473777602 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f829c0041a0 con 0x55db18b40d40 2024-02-26T06:27:09.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.183+0000 7f82b842a740 1 -- 172.21.15.49:0/473777602 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f829c005020 con 0x55db18b40d40 2024-02-26T06:27:09.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.183+0000 7f828fe00640 1 -- 172.21.15.49:0/473777602 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f829c004360 con 0x55db18b40d40 2024-02-26T06:27:09.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.184+0000 7f82b842a740 10 client.26354 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:09.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.184+0000 7f82b842a740 1 -- 172.21.15.49:0/473777602 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55db18a5f620 con 0x55db18b40d40 2024-02-26T06:27:09.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.184+0000 7f828fe00640 1 -- 172.21.15.49:0/473777602 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f829c005020 con 0x55db18b40d40 2024-02-26T06:27:09.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.184+0000 7f828fe00640 1 -- 172.21.15.49:0/473777602 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f829c021040 con 0x55db18b40d40 2024-02-26T06:27:09.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.186+0000 7f82b842a740 10 client.26354 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:09.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.186+0000 7f82b842a740 10 client.26354 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:27:09.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.186+0000 7f82b842a740 20 client.26354 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:09.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.186+0000 7f82b842a740 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55db18bb2040 0x55db18bb4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.186+0000 7f82b842a740 4 client.26354 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:27:09.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.186+0000 7f82b842a740 1 -- 172.21.15.49:0/473777602 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55db1882a890 con 0x55db18bb2040 2024-02-26T06:27:09.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.188+0000 7f82aea00640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55db18bb2040 0x55db18bb4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.189+0000 7f82aea00640 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55db18bb2040 0x55db18bb4460 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f105cab0a268b1e2 server_cookie=5de41e54df77aa7a in_seq=0 out_seq=0 2024-02-26T06:27:09.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.189+0000 7f828fe00640 10 client.26354 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:27:09.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.189+0000 7f828fe00640 1 -- 172.21.15.49:0/473777602 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55db1882a890 con 0x55db18bb2040 2024-02-26T06:27:09.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.189+0000 7f828fe00640 10 client.26354 handle_command_reply: tid=0 2024-02-26T06:27:09.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.189+0000 7f82b842a740 1 client.26354 shutdown 2024-02-26T06:27:09.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f82b842a740 1 -- 172.21.15.49:0/473777602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40d40 msgr2=0x55db18b50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f82b842a740 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40d40 0x55db18b50090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55db18b47ef0 tx=0x7f829c009ee0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f82b842a740 1 -- 172.21.15.49:0/473777602 shutdown_connections 2024-02-26T06:27:09.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f82b842a740 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x55db18bb2040 0x55db18bb4460 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f82b842a740 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db18b40d40 0x55db18b50090 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55db18b47ef0 tx=0x7f829c009ee0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f82b842a740 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db18b49920 0x55db18ba9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f82b842a740 1 --2- 172.21.15.49:0/473777602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db18b40420 0x55db18b4fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f82b842a740 1 -- 172.21.15.49:0/473777602 >> 172.21.15.49:0/473777602 conn(0x55db18b3ab50 msgr2=0x55db18b30bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.190+0000 7f828fe00640 0 client.26354 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:27:09.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.191+0000 7f82b842a740 1 -- 172.21.15.49:0/473777602 shutdown_connections 2024-02-26T06:27:09.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.191+0000 7f82b842a740 1 -- 172.21.15.49:0/473777602 wait complete. 2024-02-26T06:27:09.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.191+0000 7f82b842a740 20 client.26354 trim_cache size 0 max 16384 2024-02-26T06:27:09.192 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.192+0000 7f82b5600640 1 -- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f82940a22b0 msgr2=0x7f82940a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.192+0000 7f82b5600640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f82940a22b0 0x7f82940a4770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000c00 tx=0x7f82a002e040 comp rx=0 tx=0).stop 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.192+0000 7f82b5600640 1 -- 172.21.15.49:0/1174676766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82b0199800 msgr2=0x7f82b0199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.192+0000 7f82b5600640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82b0199800 0x7f82b0199cb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f82a800eb10 tx=0x7f82a800c2d0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.193+0000 7f82b5600640 1 -- 172.21.15.49:0/1174676766 shutdown_connections 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.193+0000 7f82b5600640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f82940a22b0 0x7f82940a4770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.193+0000 7f82b5600640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82b0199800 0x7f82b0199cb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.193+0000 7f82b5600640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b01064c0 0x7f82b019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.193+0000 7f82b5600640 1 --2- 172.21.15.49:0/1174676766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82b00685f0 0x7f82b019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.193+0000 7f82b5600640 1 -- 172.21.15.49:0/1174676766 >> 172.21.15.49:0/1174676766 conn(0x7f82b0077180 msgr2=0x7f82b010c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.193+0000 7f82b5600640 1 -- 172.21.15.49:0/1174676766 shutdown_connections 2024-02-26T06:27:09.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.193+0000 7f82b5600640 1 -- 172.21.15.49:0/1174676766 wait complete. 2024-02-26T06:27:09.219 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:27:09.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:27:09.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:09 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4127834229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:09.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.520+0000 7fd3db400640 1 -- 172.21.15.49:0/3915308797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 msgr2=0x7fd3d40ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.520+0000 7fd3db400640 1 --2- 172.21.15.49:0/3915308797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 0x7fd3d40ffcd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009900 tx=0x7fd3c802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.521+0000 7fd3db400640 1 -- 172.21.15.49:0/3915308797 shutdown_connections 2024-02-26T06:27:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.521+0000 7fd3db400640 1 --2- 172.21.15.49:0/3915308797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3d4106500 0x7fd3d410e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.521+0000 7fd3db400640 1 --2- 172.21.15.49:0/3915308797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3d41002a0 0x7fd3d4105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.521+0000 7fd3db400640 1 --2- 172.21.15.49:0/3915308797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 0x7fd3d40ffcd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.521+0000 7fd3db400640 1 -- 172.21.15.49:0/3915308797 >> 172.21.15.49:0/3915308797 conn(0x7fd3d40765a0 msgr2=0x7fd3d40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.521+0000 7fd3db400640 1 -- 172.21.15.49:0/3915308797 shutdown_connections 2024-02-26T06:27:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.521+0000 7fd3db400640 1 -- 172.21.15.49:0/3915308797 wait complete. 2024-02-26T06:27:09.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.522+0000 7fd3db400640 1 Processor -- start 2024-02-26T06:27:09.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.522+0000 7fd3db400640 1 -- start start 2024-02-26T06:27:09.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.523+0000 7fd3db400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 0x7fd3d419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.524+0000 7fd3db400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3d4106500 0x7fd3d419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.524+0000 7fd3db400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3d4197600 0x7fd3d4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.524+0000 7fd3db400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4197ff0 con 0x7fd3d4106500 2024-02-26T06:27:09.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.524+0000 7fd3db400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4198160 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.524+0000 7fd3db400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4198320 con 0x7fd3d4197600 2024-02-26T06:27:09.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.525+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 0x7fd3d419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.525+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 0x7fd3d419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57800/0 (socket says 172.21.15.49:57800) 2024-02-26T06:27:09.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.525+0000 7fd3d8a00640 1 -- 172.21.15.49:0/830548008 learned_addr learned my addr 172.21.15.49:0/830548008 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.525+0000 7fd3d9400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3d4197600 0x7fd3d4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.525+0000 7fd3d8a00640 1 -- 172.21.15.49:0/830548008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3d4197600 msgr2=0x7fd3d4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.525+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3d4106500 0x7fd3d419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.526+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3d4197600 0x7fd3d4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.526+0000 7fd3d8a00640 1 -- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3d4106500 msgr2=0x7fd3d419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.526+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3d4106500 0x7fd3d419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.526+0000 7fd3d8a00640 1 -- 172.21.15.49:0/830548008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c8009560 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.526+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3d4106500 0x7fd3d419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:09.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.527+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 0x7fd3d419d450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000c00 tx=0x7fd3c802c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.527+0000 7fd3d1600640 1 -- 172.21.15.49:0/830548008 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8004410 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.527+0000 7fd3d1600640 1 -- 172.21.15.49:0/830548008 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3c80052d0 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.527+0000 7fd3d1600640 1 -- 172.21.15.49:0/830548008 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8003ca0 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.527+0000 7fd3db400640 1 -- 172.21.15.49:0/830548008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d4100790 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.528+0000 7fd3db400640 1 -- 172.21.15.49:0/830548008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d4100c10 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.534+0000 7fd3de367740 1 Processor -- start 2024-02-26T06:27:09.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.534+0000 7fd3de367740 1 -- start start 2024-02-26T06:27:09.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.534+0000 7fd3de367740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf40800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.534+0000 7fd3de367740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.535+0000 7fd3de367740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf4bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.535+0000 7fd3de367740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cea61e0 con 0x56309cf40420 2024-02-26T06:27:09.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.535+0000 7fd3de367740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cea3710 con 0x56309cf49920 2024-02-26T06:27:09.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.535+0000 7fd3de367740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309ce9eef0 con 0x56309cf40d40 2024-02-26T06:27:09.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.536+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf4bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.536+0000 7fd3d9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf40800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.536+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf4bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57814/0 (socket says 172.21.15.49:57814) 2024-02-26T06:27:09.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.536+0000 7fd3d9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf40800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54406/0 (socket says 172.21.15.49:54406) 2024-02-26T06:27:09.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.536+0000 7fd3d3e00640 1 -- 172.21.15.49:0/3783983186 learned_addr learned my addr 172.21.15.49:0/3783983186 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.537+0000 7fd3d9400640 1 -- 172.21.15.49:0/3783983186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 msgr2=0x56309cf493e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:09.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.537+0000 7fd3d9400640 1 --2- 172.21.15.49:0/3783983186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf493e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.537+0000 7fd3d9400640 1 -- 172.21.15.49:0/3783983186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 msgr2=0x56309cf4bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.537+0000 7fd3d9400640 1 --2- 172.21.15.49:0/3783983186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf4bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.537+0000 7fd3d9400640 1 -- 172.21.15.49:0/3783983186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56309ce60dc0 con 0x56309cf40420 2024-02-26T06:27:09.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.537+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/3783983186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf4bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:09.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.538+0000 7fd3d9400640 1 --2- 172.21.15.49:0/3783983186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf40800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009e10 tx=0x7fd3cc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b9f41dfdaf5143 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.538+0000 7fd3d1600640 1 -- 172.21.15.49:0/830548008 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd3c8005440 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.539+0000 7fd3c5c00640 1 -- 172.21.15.49:0/3783983186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc003b70 con 0x56309cf40420 2024-02-26T06:27:09.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.539+0000 7fd3c5c00640 1 -- 172.21.15.49:0/3783983186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3cc003cd0 con 0x56309cf40420 2024-02-26T06:27:09.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.539+0000 7fd3c5c00640 1 -- 172.21.15.49:0/3783983186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc011800 con 0x56309cf40420 2024-02-26T06:27:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.539+0000 7fd3de367740 1 -- 172.21.15.49:0/3783983186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 msgr2=0x56309cf40800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.540+0000 7fd3de367740 1 --2- 172.21.15.49:0/3783983186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf40800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009e10 tx=0x7fd3cc00f730 comp rx=0 tx=0).stop 2024-02-26T06:27:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.540+0000 7fd3de367740 1 -- 172.21.15.49:0/3783983186 shutdown_connections 2024-02-26T06:27:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.540+0000 7fd3de367740 1 --2- 172.21.15.49:0/3783983186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf4bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.540+0000 7fd3de367740 1 --2- 172.21.15.49:0/3783983186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.540+0000 7fd3de367740 1 --2- 172.21.15.49:0/3783983186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf40800 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.540+0000 7fd3de367740 1 -- 172.21.15.49:0/3783983186 >> 172.21.15.49:0/3783983186 conn(0x56309cf3ab50 msgr2=0x56309cf3af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.540+0000 7fd3de367740 1 -- 172.21.15.49:0/3783983186 shutdown_connections 2024-02-26T06:27:09.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.540+0000 7fd3de367740 1 -- 172.21.15.49:0/3783983186 wait complete. 2024-02-26T06:27:09.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.541+0000 7fd3de367740 1 -- start start 2024-02-26T06:27:09.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.541+0000 7fd3d1600640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd3bc0a2240 0x7fd3bc0a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.541+0000 7fd3de367740 1 Processor -- start 2024-02-26T06:27:09.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.541+0000 7fd3d1600640 1 -- 172.21.15.49:0/830548008 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd3c80f07a0 con 0x7fd3d40ff8f0 2024-02-26T06:27:09.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.542+0000 7fd3d9400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd3bc0a2240 0x7fd3bc0a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3de367740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf40420 0x56309cf46980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3de367740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf46ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3de367740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf49920 0x56309cf47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3de367740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309ce44120 con 0x56309cf40d40 2024-02-26T06:27:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3de367740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309ce43dc0 con 0x56309cf40420 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3de367740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cc0d1c0 con 0x56309cf49920 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3de367740 10 client.0 _ll_register_callbacks cb 0x56309ce93280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf40420 0x56309cf46980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf40420 0x56309cf46980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57818/0 (socket says 172.21.15.49:57818) 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3d8a00640 1 -- 172.21.15.49:0/2992036931 learned_addr learned my addr 172.21.15.49:0/2992036931 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3d9400640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf49920 0x56309cf47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.543+0000 7fd3d8a00640 1 -- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf49920 msgr2=0x56309cf47400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf46ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf49920 0x56309cf47400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d8a00640 1 -- 172.21.15.49:0/2992036931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 msgr2=0x56309cf46ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf46ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d8a00640 1 -- 172.21.15.49:0/2992036931 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc009af0 con 0x56309cf40420 2024-02-26T06:27:09.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d9400640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf49920 0x56309cf47400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:09.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d9400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd3bc0a2240 0x7fd3bc0a4700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00fc30 tx=0x7fd3cc005bd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf46ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:09.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf40420 0x56309cf46980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005ec0 tx=0x7fd3c80035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d596d71bf00462 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3ba000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:09.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3ba000640 1 -- 172.21.15.49:0/2992036931 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c803c560 con 0x56309cf40420 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3ba000640 1 -- 172.21.15.49:0/2992036931 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd3c80bd250 con 0x56309cf40420 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.544+0000 7fd3de367740 1 -- 172.21.15.49:0/2992036931 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56309cb9e7e0 con 0x56309cf40420 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3ba000640 1 -- 172.21.15.49:0/2992036931 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3c80c13a0 con 0x56309cf40420 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3de367740 10 client.26366 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3de367740 1 -- 172.21.15.49:0/2992036931 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56309ce9d570 con 0x56309cf40420 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3ba000640 1 -- 172.21.15.49:0/2992036931 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56309cb9e7e0 con 0x56309cf40420 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3ba000640 1 -- 172.21.15.49:0/2992036931 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd3c803c070 con 0x56309cf40420 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3de367740 10 client.26366 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3de367740 10 client.26366 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:27:09.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3de367740 20 client.26366 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:09.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3de367740 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56309cf6a5d0 0x56309cf6ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3de367740 4 client.26366 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:09.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.545+0000 7fd3de367740 1 -- 172.21.15.49:0/2992036931 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56309cbdb720 con 0x56309cf6a5d0 2024-02-26T06:27:09.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.546+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56309cf6a5d0 0x56309cf6ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.546+0000 7fd3ba000640 10 client.26366 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:27:09.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.547+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56309cf6a5d0 0x56309cf6ca00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=506a1505076cfb87 server_cookie=8157868d2d291a6 in_seq=0 out_seq=0 2024-02-26T06:27:09.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.548+0000 7fd3ba000640 1 -- 172.21.15.49:0/2992036931 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56309cbdb720 con 0x56309cf6a5d0 2024-02-26T06:27:09.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.548+0000 7fd3ba000640 10 client.26366 handle_command_reply: tid=0 2024-02-26T06:27:09.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.548+0000 7fd3de367740 1 client.26366 shutdown 2024-02-26T06:27:09.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 -- 172.21.15.49:0/2992036931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf40420 msgr2=0x56309cf46980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf40420 0x56309cf46980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005ec0 tx=0x7fd3c80035f0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 -- 172.21.15.49:0/2992036931 shutdown_connections 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56309cf6a5d0 0x56309cf6ca00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf49920 0x56309cf47400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf46ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 --2- 172.21.15.49:0/2992036931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf40420 0x56309cf46980 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 -- 172.21.15.49:0/2992036931 >> 172.21.15.49:0/2992036931 conn(0x56309cf3ab50 msgr2=0x56309cf3c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3ba000640 0 client.26366 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 -- 172.21.15.49:0/2992036931 shutdown_connections 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.550+0000 7fd3de367740 1 -- 172.21.15.49:0/2992036931 wait complete. 2024-02-26T06:27:09.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.551+0000 7fd3de367740 20 client.26366 trim_cache size 0 max 16384 2024-02-26T06:27:09.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.566+0000 7fd3de367740 1 Processor -- start 2024-02-26T06:27:09.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.566+0000 7fd3de367740 1 -- start start 2024-02-26T06:27:09.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3de367740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3de367740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf47240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3de367740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf45270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3de367740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cf67470 con 0x56309cf40420 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3de367740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cf71c00 con 0x56309cf49920 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3de367740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cf44ef0 con 0x56309cf40d40 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf47240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3d8a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3d9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf45270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf47240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39820/0 (socket says 172.21.15.49:39820) 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.567+0000 7fd3d3e00640 1 -- 172.21.15.49:0/3723868659 learned_addr learned my addr 172.21.15.49:0/3723868659 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3d3e00640 1 -- 172.21.15.49:0/3723868659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 msgr2=0x56309cf45270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/3723868659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf45270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3d3e00640 1 -- 172.21.15.49:0/3723868659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 msgr2=0x56309cf490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/3723868659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3d3e00640 1 -- 172.21.15.49:0/3723868659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56309ce39a80 con 0x56309cf40d40 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/3723868659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf47240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000aac0 tx=0x7fd3c0016800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d846228868958a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3c5200640 1 -- 172.21.15.49:0/3723868659 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000adc0 con 0x56309cf40d40 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3c5200640 1 -- 172.21.15.49:0/3723868659 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3c001e070 con 0x56309cf40d40 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.568+0000 7fd3c5200640 1 -- 172.21.15.49:0/3723868659 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00215e0 con 0x56309cf40d40 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.569+0000 7fd3de367740 1 -- 172.21.15.49:0/3723868659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 msgr2=0x56309cf47240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.569+0000 7fd3de367740 1 --2- 172.21.15.49:0/3723868659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf47240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000aac0 tx=0x7fd3c0016800 comp rx=0 tx=0).stop 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.569+0000 7fd3de367740 1 -- 172.21.15.49:0/3723868659 shutdown_connections 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.569+0000 7fd3de367740 1 --2- 172.21.15.49:0/3723868659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf45270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.569+0000 7fd3de367740 1 --2- 172.21.15.49:0/3723868659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40d40 0x56309cf47240 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.569+0000 7fd3de367740 1 --2- 172.21.15.49:0/3723868659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40420 0x56309cf490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.569+0000 7fd3de367740 1 -- 172.21.15.49:0/3723868659 >> 172.21.15.49:0/3723868659 conn(0x56309cf3ab50 msgr2=0x56309cf41c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.570+0000 7fd3de367740 1 -- 172.21.15.49:0/3723868659 shutdown_connections 2024-02-26T06:27:09.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.570+0000 7fd3de367740 1 -- 172.21.15.49:0/3723868659 wait complete. 2024-02-26T06:27:09.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.571+0000 7fd3de367740 1 -- start start 2024-02-26T06:27:09.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.572+0000 7fd3de367740 1 Processor -- start 2024-02-26T06:27:09.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.574+0000 7fd3de367740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40420 0x56309cf61d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.574+0000 7fd3de367740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.574+0000 7fd3de367740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.574+0000 7fd3de367740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cf60ea0 con 0x56309cf40d40 2024-02-26T06:27:09.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.574+0000 7fd3de367740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cfc0b00 con 0x56309cf49920 2024-02-26T06:27:09.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.574+0000 7fd3de367740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309cfc0c40 con 0x56309cf40420 2024-02-26T06:27:09.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.574+0000 7fd3de367740 10 client.0 _ll_register_callbacks cb 0x7fd3cc0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.574+0000 7fd3d9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.575+0000 7fd3d9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57838/0 (socket says 172.21.15.49:57838) 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.575+0000 7fd3d9400640 1 -- 172.21.15.49:0/1206030011 learned_addr learned my addr 172.21.15.49:0/1206030011 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.575+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40420 0x56309cf61d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.575+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.575+0000 7fd3d8a00640 1 -- 172.21.15.49:0/1206030011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 msgr2=0x56309cf607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.575+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.575+0000 7fd3d8a00640 1 -- 172.21.15.49:0/1206030011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 msgr2=0x56309cf600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.575+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.576+0000 7fd3d8a00640 1 -- 172.21.15.49:0/1206030011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c000a7f0 con 0x56309cf40420 2024-02-26T06:27:09.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.576+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:09.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.576+0000 7fd3d8a00640 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40420 0x56309cf61d10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80b96b0 tx=0x7fd3c80b9620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=384c304fc2bfa42c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.576+0000 7fd3ba000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:09.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.577+0000 7fd3ba000640 1 -- 172.21.15.49:0/1206030011 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80bc960 con 0x56309cf40420 2024-02-26T06:27:09.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.577+0000 7fd3ba000640 1 -- 172.21.15.49:0/1206030011 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd3c809d020 con 0x56309cf40420 2024-02-26T06:27:09.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.577+0000 7fd3de367740 1 -- 172.21.15.49:0/1206030011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3cc003cd0 con 0x56309cf40420 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.577+0000 7fd3ba000640 1 -- 172.21.15.49:0/1206030011 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3c803c560 con 0x56309cf40420 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.577+0000 7fd3de367740 10 client.26401 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.577+0000 7fd3de367740 1 -- 172.21.15.49:0/1206030011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56309ce5f620 con 0x56309cf40420 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.577+0000 7fd3ba000640 1 -- 172.21.15.49:0/1206030011 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3cc003cd0 con 0x56309cf40420 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.578+0000 7fd3ba000640 1 -- 172.21.15.49:0/1206030011 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd3c80cb050 con 0x56309cf40420 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.578+0000 7fd3de367740 10 client.26401 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.578+0000 7fd3de367740 10 client.26401 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.578+0000 7fd3de367740 20 client.26401 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.578+0000 7fd3de367740 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56309cfc1f00 0x56309cfc42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.579+0000 7fd3de367740 4 client.26401 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:27:09.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.579+0000 7fd3de367740 1 -- 172.21.15.49:0/1206030011 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56309cc2a890 con 0x56309cfc1f00 2024-02-26T06:27:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:09 smithi080 ceph-mon[36206]: pgmap v115: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 1.31k op/s 2024-02-26T06:27:09.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:09 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4127834229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:09.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.579+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56309cfc1f00 0x56309cfc42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.580+0000 7fd3d3e00640 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56309cfc1f00 0x56309cfc42f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee23c98ba67e6b3b server_cookie=c04450e609286a31 in_seq=0 out_seq=0 2024-02-26T06:27:09.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.580+0000 7fd3ba000640 10 client.26401 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:27:09.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.580+0000 7fd3ba000640 1 -- 172.21.15.49:0/1206030011 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56309cc2a890 con 0x56309cfc1f00 2024-02-26T06:27:09.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.580+0000 7fd3ba000640 10 client.26401 handle_command_reply: tid=0 2024-02-26T06:27:09.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.580+0000 7fd3de367740 1 client.26401 shutdown 2024-02-26T06:27:09.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 -- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40420 msgr2=0x56309cf61d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40420 0x56309cf61d10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80b96b0 tx=0x7fd3c80b9620 comp rx=0 tx=0).stop 2024-02-26T06:27:09.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 -- 172.21.15.49:0/1206030011 shutdown_connections 2024-02-26T06:27:09.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x56309cfc1f00 0x56309cfc42f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56309cf49920 0x56309cf607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56309cf40d40 0x56309cf600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3ba000640 0 client.26401 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:27:09.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 --2- 172.21.15.49:0/1206030011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56309cf40420 0x56309cf61d10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 -- 172.21.15.49:0/1206030011 >> 172.21.15.49:0/1206030011 conn(0x56309cf3ab50 msgr2=0x56309cf45230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 -- 172.21.15.49:0/1206030011 shutdown_connections 2024-02-26T06:27:09.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 1 -- 172.21.15.49:0/1206030011 wait complete. 2024-02-26T06:27:09.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.581+0000 7fd3de367740 20 client.26401 trim_cache size 0 max 16384 2024-02-26T06:27:09.586 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:27:09.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 -- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd3bc0a2240 msgr2=0x7fd3bc0a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd3bc0a2240 0x7fd3bc0a4700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00fc30 tx=0x7fd3cc005bd0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 -- 172.21.15.49:0/830548008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 msgr2=0x7fd3d419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 0x7fd3d419d450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000c00 tx=0x7fd3c802c6a0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 -- 172.21.15.49:0/830548008 shutdown_connections 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd3bc0a2240 0x7fd3bc0a4700 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3d4197600 0x7fd3d4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3d4106500 0x7fd3d419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 --2- 172.21.15.49:0/830548008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3d40ff8f0 0x7fd3d419d450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 -- 172.21.15.49:0/830548008 >> 172.21.15.49:0/830548008 conn(0x7fd3d40765a0 msgr2=0x7fd3d40fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.582+0000 7fd3db400640 1 -- 172.21.15.49:0/830548008 shutdown_connections 2024-02-26T06:27:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.583+0000 7fd3db400640 1 -- 172.21.15.49:0/830548008 wait complete. 2024-02-26T06:27:09.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:09 smithi115 ceph-mon[36426]: pgmap v115: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 1.31k op/s 2024-02-26T06:27:09.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:09 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4127834229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:09.609 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:27:09.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:27:09.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.905+0000 7fb797c00640 1 -- 172.21.15.49:0/938713788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb790105370 msgr2=0x7fb79010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.905+0000 7fb797c00640 1 --2- 172.21.15.49:0/938713788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb790105370 0x7fb79010cf40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb788009900 tx=0x7fb78802bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:09.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.905+0000 7fb797c00640 1 -- 172.21.15.49:0/938713788 shutdown_connections 2024-02-26T06:27:09.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.905+0000 7fb797c00640 1 --2- 172.21.15.49:0/938713788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb790105370 0x7fb79010cf40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.905+0000 7fb797c00640 1 --2- 172.21.15.49:0/938713788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb790100ca0 0x7fb790104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.905+0000 7fb797c00640 1 --2- 172.21.15.49:0/938713788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7901002f0 0x7fb7901006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.905+0000 7fb797c00640 1 -- 172.21.15.49:0/938713788 >> 172.21.15.49:0/938713788 conn(0x7fb7900771e0 msgr2=0x7fb7900775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.906+0000 7fb797c00640 1 -- 172.21.15.49:0/938713788 shutdown_connections 2024-02-26T06:27:09.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.906+0000 7fb797c00640 1 -- 172.21.15.49:0/938713788 wait complete. 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.907+0000 7fb797c00640 1 Processor -- start 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.908+0000 7fb797c00640 1 -- start start 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.908+0000 7fb797c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7901002f0 0x7fb79019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.908+0000 7fb797c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb790105370 0x7fb79019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.908+0000 7fb797c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79010bef0 0x7fb79010c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.908+0000 7fb797c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79010cb30 con 0x7fb7901002f0 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.908+0000 7fb797c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79010cca0 con 0x7fb790105370 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.908+0000 7fb797c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79019df60 con 0x7fb79010bef0 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79010bef0 0x7fb79010c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb787e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb790105370 0x7fb79019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79010bef0 0x7fb79010c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39854/0 (socket says 172.21.15.49:39854) 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb787e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb790105370 0x7fb79019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57858/0 (socket says 172.21.15.49:57858) 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 -- 172.21.15.49:0/1400557924 learned_addr learned my addr 172.21.15.49:0/1400557924 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795200640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7901002f0 0x7fb79019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 -- 172.21.15.49:0/1400557924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb790105370 msgr2=0x7fb79019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb790105370 0x7fb79019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 -- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7901002f0 msgr2=0x7fb79019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7901002f0 0x7fb79019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 -- 172.21.15.49:0/1400557924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788009560 con 0x7fb79010bef0 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795200640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7901002f0 0x7fb79019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.909+0000 7fb795c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79010bef0 0x7fb79010c3a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb788009230 tx=0x7fb78802c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.910+0000 7fb78e000640 1 -- 172.21.15.49:0/1400557924 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788004350 con 0x7fb79010bef0 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.910+0000 7fb78e000640 1 -- 172.21.15.49:0/1400557924 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7880044b0 con 0x7fb79010bef0 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.910+0000 7fb797c00640 1 -- 172.21.15.49:0/1400557924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79019e1e0 con 0x7fb79010bef0 2024-02-26T06:27:09.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.910+0000 7fb797c00640 1 -- 172.21.15.49:0/1400557924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79019e6f0 con 0x7fb79010bef0 2024-02-26T06:27:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.910+0000 7fb78e000640 1 -- 172.21.15.49:0/1400557924 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788003c60 con 0x7fb79010bef0 2024-02-26T06:27:09.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.914+0000 7fb78e000640 1 -- 172.21.15.49:0/1400557924 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb788004df0 con 0x7fb79010bef0 2024-02-26T06:27:09.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.916+0000 7fb78e000640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7700a2400 0x7fb7700a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.916+0000 7fb79aa5c740 1 Processor -- start 2024-02-26T06:27:09.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.916+0000 7fb78e000640 1 -- 172.21.15.49:0/1400557924 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb7880f06b0 con 0x7fb79010bef0 2024-02-26T06:27:09.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.916+0000 7fb79aa5c740 1 -- start start 2024-02-26T06:27:09.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.916+0000 7fb795200640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7700a2400 0x7fb7700a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.919+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.919+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebffabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.919+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 0x5636ebffd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.919+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebf601e0 con 0x5636ebff2530 2024-02-26T06:27:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.919+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebf5d710 con 0x5636ebffb110 2024-02-26T06:27:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.919+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebf58ef0 con 0x5636ebff1c10 2024-02-26T06:27:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.920+0000 7fb795200640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7700a2400 0x7fb7700a48c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7800058c0 tx=0x7fb780005ec0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.920+0000 7fb787e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebffabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.920+0000 7fb787e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebffabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54472/0 (socket says 172.21.15.49:54472) 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.920+0000 7fb787e00640 1 -- 172.21.15.49:0/346633182 learned_addr learned my addr 172.21.15.49:0/346633182 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.920+0000 7fb795200640 1 --2- 172.21.15.49:0/346633182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.921+0000 7fb787e00640 1 -- 172.21.15.49:0/346633182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 msgr2=0x5636ebff1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.921+0000 7fb787e00640 1 --2- 172.21.15.49:0/346633182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.921+0000 7fb787e00640 1 -- 172.21.15.49:0/346633182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 msgr2=0x5636ebffd500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.921+0000 7fb787e00640 1 --2- 172.21.15.49:0/346633182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 0x5636ebffd500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.921+0000 7fb787e00640 1 -- 172.21.15.49:0/346633182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ebf1adc0 con 0x5636ebff2530 2024-02-26T06:27:09.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.921+0000 7fb787e00640 1 --2- 172.21.15.49:0/346633182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebffabd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb778009f60 tx=0x7fb778009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34e42b0e0c29f47 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.922+0000 7fb786000640 1 -- 172.21.15.49:0/346633182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778004d20 con 0x5636ebff2530 2024-02-26T06:27:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.922+0000 7fb786000640 1 -- 172.21.15.49:0/346633182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb778004e80 con 0x5636ebff2530 2024-02-26T06:27:09.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb786000640 1 -- 172.21.15.49:0/346633182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778005510 con 0x5636ebff2530 2024-02-26T06:27:09.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 -- 172.21.15.49:0/346633182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 msgr2=0x5636ebffabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/346633182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebffabd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb778009f60 tx=0x7fb778009f90 comp rx=0 tx=0).stop 2024-02-26T06:27:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 -- 172.21.15.49:0/346633182 shutdown_connections 2024-02-26T06:27:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/346633182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 0x5636ebffd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/346633182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebffabd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/346633182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 -- 172.21.15.49:0/346633182 >> 172.21.15.49:0/346633182 conn(0x5636ebfec340 msgr2=0x5636ebfec720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 -- 172.21.15.49:0/346633182 shutdown_connections 2024-02-26T06:27:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.923+0000 7fb79aa5c740 1 -- 172.21.15.49:0/346633182 wait complete. 2024-02-26T06:27:09.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.925+0000 7fb79aa5c740 1 -- start start 2024-02-26T06:27:09.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.926+0000 7fb79aa5c740 1 Processor -- start 2024-02-26T06:27:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebff85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 0x5636ebff8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebefe120 con 0x5636ebff2530 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebefddc0 con 0x5636ebffb110 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebcc71c0 con 0x5636ebff1c10 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb79aa5c740 10 client.0 _ll_register_callbacks cb 0x5636ebf4d280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb795200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb795200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39874/0 (socket says 172.21.15.49:39874) 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb795200640 1 -- 172.21.15.49:0/1363582671 learned_addr learned my addr 172.21.15.49:0/1363582671 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.927+0000 7fb795c00640 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 0x5636ebff8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb795200640 1 -- 172.21.15.49:0/1363582671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 msgr2=0x5636ebff8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb795200640 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 0x5636ebff8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb795200640 1 -- 172.21.15.49:0/1363582671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 msgr2=0x5636ebff85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb795200640 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebff85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb795200640 1 -- 172.21.15.49:0/1363582671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb778009b90 con 0x5636ebff1c10 2024-02-26T06:27:09.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb795200640 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff8080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb780010990 tx=0x7fb7800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ad676bbe0c2e04d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb77ea00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:09.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb77ea00640 1 -- 172.21.15.49:0/1363582671 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78000fd50 con 0x5636ebff1c10 2024-02-26T06:27:09.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1363582671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636ebc95720 con 0x5636ebff1c10 2024-02-26T06:27:09.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.928+0000 7fb77ea00640 1 -- 172.21.15.49:0/1363582671 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb78001e430 con 0x5636ebff1c10 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb77ea00640 1 -- 172.21.15.49:0/1363582671 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb78000feb0 con 0x5636ebff1c10 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb79aa5c740 10 client.26413 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1363582671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5636ebf57570 con 0x5636ebff1c10 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb77ea00640 1 -- 172.21.15.49:0/1363582671 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5636ebc95720 con 0x5636ebff1c10 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb77ea00640 1 -- 172.21.15.49:0/1363582671 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb78002d070 con 0x5636ebff1c10 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb79aa5c740 10 client.26413 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb79aa5c740 10 client.26413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb79aa5c740 20 client.26413 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5636ec00b1c0 0x5636ec00d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb79aa5c740 4 client.26413 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:09.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1363582671 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636ebce4890 con 0x5636ec00b1c0 2024-02-26T06:27:09.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.929+0000 7fb787e00640 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5636ec00b1c0 0x5636ec00d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.931+0000 7fb787e00640 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5636ec00b1c0 0x5636ec00d5e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9072573043e6d68 server_cookie=3d8131b6e3c9c649 in_seq=0 out_seq=0 2024-02-26T06:27:09.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.931+0000 7fb77ea00640 10 client.26413 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:09.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.931+0000 7fb77ea00640 1 -- 172.21.15.49:0/1363582671 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636ebce4890 con 0x5636ec00b1c0 2024-02-26T06:27:09.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.931+0000 7fb77ea00640 10 client.26413 handle_command_reply: tid=0 2024-02-26T06:27:09.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.932+0000 7fb79aa5c740 1 client.26413 shutdown 2024-02-26T06:27:09.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.932+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1363582671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 msgr2=0x5636ebff8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.932+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff8080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb780010990 tx=0x7fb7800046c0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1363582671 shutdown_connections 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5636ec00b1c0 0x5636ec00d5e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebffb110 0x5636ebff8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebff2530 0x5636ebff85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1363582671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff1c10 0x5636ebff8080 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1363582671 >> 172.21.15.49:0/1363582671 conn(0x5636ebfec340 msgr2=0x5636ebfee090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb77ea00640 0 client.26413 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1363582671 shutdown_connections 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.933+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1363582671 wait complete. 2024-02-26T06:27:09.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.934+0000 7fb79aa5c740 20 client.26413 trim_cache size 0 max 16384 2024-02-26T06:27:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.961+0000 7fb79aa5c740 1 Processor -- start 2024-02-26T06:27:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.961+0000 7fb79aa5c740 1 -- start start 2024-02-26T06:27:09.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ebffa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ebff8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec011f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebfe7ab0 con 0x5636ebffb110 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebff1560 con 0x5636ebff1c10 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ebfe7c10 con 0x5636ebff2530 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb787e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ebff8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb787e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ebff8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:59972/0 (socket says 172.21.15.49:59972) 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb787e00640 1 -- 172.21.15.49:0/367824739 learned_addr learned my addr 172.21.15.49:0/367824739 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.962+0000 7fb795200640 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ebffa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb787e00640 1 -- 172.21.15.49:0/367824739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 msgr2=0x5636ebffa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb795c00640 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec011f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb787e00640 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ebffa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb787e00640 1 -- 172.21.15.49:0/367824739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 msgr2=0x5636ec011f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb787e00640 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec011f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb787e00640 1 -- 172.21.15.49:0/367824739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ebef3a80 con 0x5636ebff2530 2024-02-26T06:27:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb795200640 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ebffa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb795c00640 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec011f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.963+0000 7fb787e00640 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ebff8920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb778005020 tx=0x7fb778005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d364b030a14728d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb785600640 1 -- 172.21.15.49:0/367824739 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77801e040 con 0x5636ebff2530 2024-02-26T06:27:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb785600640 1 -- 172.21.15.49:0/367824739 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb77801f040 con 0x5636ebff2530 2024-02-26T06:27:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb785600640 1 -- 172.21.15.49:0/367824739 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7780280f0 con 0x5636ebff2530 2024-02-26T06:27:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb79aa5c740 1 -- 172.21.15.49:0/367824739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 msgr2=0x5636ebff8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ebff8920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb778005020 tx=0x7fb778005050 comp rx=0 tx=0).stop 2024-02-26T06:27:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb79aa5c740 1 -- 172.21.15.49:0/367824739 shutdown_connections 2024-02-26T06:27:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec011f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ebff8920 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.964+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/367824739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ebffa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.965+0000 7fb79aa5c740 1 -- 172.21.15.49:0/367824739 >> 172.21.15.49:0/367824739 conn(0x5636ebfec340 msgr2=0x5636ebfec720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.965+0000 7fb79aa5c740 1 -- 172.21.15.49:0/367824739 shutdown_connections 2024-02-26T06:27:09.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.965+0000 7fb79aa5c740 1 -- 172.21.15.49:0/367824739 wait complete. 2024-02-26T06:27:09.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.966+0000 7fb79aa5c740 1 -- start start 2024-02-26T06:27:09.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.967+0000 7fb79aa5c740 1 Processor -- start 2024-02-26T06:27:09.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.968+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ec000090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.968+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ec0005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.968+0000 7fb79aa5c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec0606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.968+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ec000e80 con 0x5636ebffb110 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.968+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ec060c20 con 0x5636ebff1c10 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.968+0000 7fb79aa5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ec060d60 con 0x5636ebff2530 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.968+0000 7fb79aa5c740 10 client.0 _ll_register_callbacks cb 0x7fb778009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ec000090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ec000090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52982/0 (socket says 172.21.15.49:52982) 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795200640 1 -- 172.21.15.49:0/1289627972 learned_addr learned my addr 172.21.15.49:0/1289627972 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795c00640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec0606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795200640 1 -- 172.21.15.49:0/1289627972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 msgr2=0x5636ec0005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb787e00640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ec0005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795200640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ec0005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795200640 1 -- 172.21.15.49:0/1289627972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 msgr2=0x5636ec0606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795200640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec0606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795200640 1 -- 172.21.15.49:0/1289627972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636ebf1adc0 con 0x5636ebff1c10 2024-02-26T06:27:09.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb795c00640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec0606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:09.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.969+0000 7fb787e00640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ec0005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:09.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.970+0000 7fb795200640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ec000090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb780000c00 tx=0x7fb78002f6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18f31ea99f33733 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:09.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.970+0000 7fb77ea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:09.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.971+0000 7fb77ea00640 1 -- 172.21.15.49:0/1289627972 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78000f790 con 0x5636ebff1c10 2024-02-26T06:27:09.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.971+0000 7fb77ea00640 1 -- 172.21.15.49:0/1289627972 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb780034070 con 0x5636ebff1c10 2024-02-26T06:27:09.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.971+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1289627972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636ebc587e0 con 0x5636ebff1c10 2024-02-26T06:27:09.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.972+0000 7fb77ea00640 1 -- 172.21.15.49:0/1289627972 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb780005030 con 0x5636ebff1c10 2024-02-26T06:27:09.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.972+0000 7fb79aa5c740 10 client.26381 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:09.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.972+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1289627972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5636ebf19620 con 0x5636ebff1c10 2024-02-26T06:27:09.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.972+0000 7fb77ea00640 1 -- 172.21.15.49:0/1289627972 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5636ebc587e0 con 0x5636ebff1c10 2024-02-26T06:27:09.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.973+0000 7fb77ea00640 1 -- 172.21.15.49:0/1289627972 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb78002d040 con 0x5636ebff1c10 2024-02-26T06:27:09.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.973+0000 7fb79aa5c740 10 client.26381 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:09.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.973+0000 7fb79aa5c740 10 client.26381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:09.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.973+0000 7fb79aa5c740 20 client.26381 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:09.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.973+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5636ec062330 0x5636ec064790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:09.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.973+0000 7fb79aa5c740 4 client.26381 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:27:09.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.973+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1289627972 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5636ebc474e0 con 0x5636ec062330 2024-02-26T06:27:09.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.973+0000 7fb787e00640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5636ec062330 0x5636ec064790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:09.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.974+0000 7fb787e00640 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5636ec062330 0x5636ec064790 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb6aae906446a85 server_cookie=28ebec040df8f26e in_seq=0 out_seq=0 2024-02-26T06:27:09.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.974+0000 7fb77ea00640 10 client.26381 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:09.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.975+0000 7fb77ea00640 1 -- 172.21.15.49:0/1289627972 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5636ebc474e0 con 0x5636ec062330 2024-02-26T06:27:09.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.975+0000 7fb77ea00640 10 client.26381 handle_command_reply: tid=0 2024-02-26T06:27:09.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.975+0000 7fb79aa5c740 1 client.26381 shutdown 2024-02-26T06:27:09.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1289627972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 msgr2=0x5636ec000090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ec000090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb780000c00 tx=0x7fb78002f6d0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1289627972 shutdown_connections 2024-02-26T06:27:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5636ec062330 0x5636ec064790 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636ebffb110 0x5636ec0606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5636ebff2530 0x5636ec0005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 --2- 172.21.15.49:0/1289627972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636ebff1c10 0x5636ec000090 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb77ea00640 0 client.26381 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1289627972 >> 172.21.15.49:0/1289627972 conn(0x5636ebfec340 msgr2=0x5636ebff33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1289627972 shutdown_connections 2024-02-26T06:27:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.976+0000 7fb79aa5c740 1 -- 172.21.15.49:0/1289627972 wait complete. 2024-02-26T06:27:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.977+0000 7fb79aa5c740 20 client.26381 trim_cache size 0 max 16384 2024-02-26T06:27:09.978 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.977+0000 7fb797c00640 1 -- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7700a2400 msgr2=0x7fb7700a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.977+0000 7fb797c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7700a2400 0x7fb7700a48c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7800058c0 tx=0x7fb780005ec0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.977+0000 7fb797c00640 1 -- 172.21.15.49:0/1400557924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79010bef0 msgr2=0x7fb79010c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.977+0000 7fb797c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79010bef0 0x7fb79010c3a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb788009230 tx=0x7fb78802c6d0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.978+0000 7fb797c00640 1 -- 172.21.15.49:0/1400557924 shutdown_connections 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.978+0000 7fb797c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7700a2400 0x7fb7700a48c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.978+0000 7fb797c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb790105370 0x7fb79019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.978+0000 7fb797c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79010bef0 0x7fb79010c3a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.978+0000 7fb797c00640 1 --2- 172.21.15.49:0/1400557924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7901002f0 0x7fb79019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.978+0000 7fb797c00640 1 -- 172.21.15.49:0/1400557924 >> 172.21.15.49:0/1400557924 conn(0x7fb7900771e0 msgr2=0x7fb7900fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.978+0000 7fb797c00640 1 -- 172.21.15.49:0/1400557924 shutdown_connections 2024-02-26T06:27:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:09.978+0000 7fb797c00640 1 -- 172.21.15.49:0/1400557924 wait complete. 2024-02-26T06:27:11.007 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:27:11.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:11.007 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:27:11.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 -- 172.21.15.49:0/517517169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0900688c0 msgr2=0x7fd090068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 --2- 172.21.15.49:0/517517169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0900688c0 0x7fd090068d20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd080009860 tx=0x7fd08002bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 -- 172.21.15.49:0/517517169 shutdown_connections 2024-02-26T06:27:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 --2- 172.21.15.49:0/517517169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd090103c40 0x7fd0901040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 --2- 172.21.15.49:0/517517169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0900688c0 0x7fd090068d20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 --2- 172.21.15.49:0/517517169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0901060d0 0x7fd090068380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 -- 172.21.15.49:0/517517169 >> 172.21.15.49:0/517517169 conn(0x7fd090077180 msgr2=0x7fd090077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 -- 172.21.15.49:0/517517169 shutdown_connections 2024-02-26T06:27:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.299+0000 7fd094a00640 1 -- 172.21.15.49:0/517517169 wait complete. 2024-02-26T06:27:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.301+0000 7fd094a00640 1 Processor -- start 2024-02-26T06:27:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.301+0000 7fd094a00640 1 -- start start 2024-02-26T06:27:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.301+0000 7fd094a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0900688c0 0x7fd0901a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.301+0000 7fd094a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd090103c40 0x7fd0901a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.301+0000 7fd094a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0901060d0 0x7fd09019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.301+0000 7fd094a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09019e190 con 0x7fd0900688c0 2024-02-26T06:27:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.301+0000 7fd094a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09019e300 con 0x7fd090103c40 2024-02-26T06:27:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.301+0000 7fd094a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09019e4c0 con 0x7fd0901060d0 2024-02-26T06:27:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0900688c0 0x7fd0901a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0900688c0 0x7fd0901a3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49404/0 (socket says 172.21.15.49:49404) 2024-02-26T06:27:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 -- 172.21.15.49:0/1385625077 learned_addr learned my addr 172.21.15.49:0/1385625077 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08e800640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0901060d0 0x7fd09019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08d400640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd090103c40 0x7fd0901a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 -- 172.21.15.49:0/1385625077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0901060d0 msgr2=0x7fd09019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0901060d0 0x7fd09019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 -- 172.21.15.49:0/1385625077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd090103c40 msgr2=0x7fd0901a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd090103c40 0x7fd0901a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 -- 172.21.15.49:0/1385625077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c009490 con 0x7fd0900688c0 2024-02-26T06:27:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08d400640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd090103c40 0x7fd0901a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 200.0 s, 2107.8 tps, lat 47.154 ms stddev 51.940 2024-02-26T06:27:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.302+0000 7fd08de00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0900688c0 0x7fd0901a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00ea60 tx=0x7fd07c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.303+0000 7fd086a00640 1 -- 172.21.15.49:0/1385625077 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00cde0 con 0x7fd0900688c0 2024-02-26T06:27:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.303+0000 7fd094a00640 1 -- 172.21.15.49:0/1385625077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd080009490 con 0x7fd0900688c0 2024-02-26T06:27:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.303+0000 7fd086a00640 1 -- 172.21.15.49:0/1385625077 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd07c0035f0 con 0x7fd0900688c0 2024-02-26T06:27:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.303+0000 7fd086a00640 1 -- 172.21.15.49:0/1385625077 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c010690 con 0x7fd0900688c0 2024-02-26T06:27:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.304+0000 7fd094a00640 1 -- 172.21.15.49:0/1385625077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0901aa800 con 0x7fd0900688c0 2024-02-26T06:27:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.311+0000 7fd086a00640 1 -- 172.21.15.49:0/1385625077 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd07c0107f0 con 0x7fd0900688c0 2024-02-26T06:27:11.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.313+0000 7fd09785d740 1 Processor -- start 2024-02-26T06:27:11.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.313+0000 7fd09785d740 1 -- start start 2024-02-26T06:27:11.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.315+0000 7fd09785d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.315+0000 7fd086a00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd0740a22b0 0x7fd0740a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.315+0000 7fd09785d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.315+0000 7fd09785d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.315+0000 7fd09785d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b4a82d0 con 0x55e67b4f6410 2024-02-26T06:27:11.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.315+0000 7fd09785d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b45ae20 con 0x55e67b4ed830 2024-02-26T06:27:11.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.315+0000 7fd09785d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b458350 con 0x55e67b4ecf10 2024-02-26T06:27:11.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.315+0000 7fd086a00640 1 -- 172.21.15.49:0/1385625077 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd07c014070 con 0x7fd0900688c0 2024-02-26T06:27:11.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.316+0000 7fd08d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.316+0000 7fd08d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53014/0 (socket says 172.21.15.49:53014) 2024-02-26T06:27:11.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.316+0000 7fd08d400640 1 -- 172.21.15.49:0/2082435165 learned_addr learned my addr 172.21.15.49:0/2082435165 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.316+0000 7fd08e800640 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.317+0000 7fd08e800640 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.317+0000 7fd08de00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd0740a22b0 0x7fd0740a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.318+0000 7fd08d400640 1 -- 172.21.15.49:0/2082435165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 msgr2=0x55e67b4ed2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.318+0000 7fd08d400640 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4ed2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.318+0000 7fd08d400640 1 -- 172.21.15.49:0/2082435165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 msgr2=0x55e67b4f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.318+0000 7fd08d400640 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.318+0000 7fd08d400640 1 -- 172.21.15.49:0/2082435165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e67b41cc50 con 0x55e67b4ed830 2024-02-26T06:27:11.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.318+0000 7fd08de00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd0740a22b0 0x7fd0740a4770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd090100c90 tx=0x7fd07c00ef90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.320+0000 7fd08d400640 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f5ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd080000c00 tx=0x7fd080035c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e9127569d184bc0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.321+0000 7fd07b000640 1 -- 172.21.15.49:0/2082435165 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080004040 con 0x55e67b4ed830 2024-02-26T06:27:11.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.321+0000 7fd07b000640 1 -- 172.21.15.49:0/2082435165 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd080034440 con 0x55e67b4ed830 2024-02-26T06:27:11.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.321+0000 7fd07b000640 1 -- 172.21.15.49:0/2082435165 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800055a0 con 0x55e67b4ed830 2024-02-26T06:27:11.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.321+0000 7fd09785d740 1 -- 172.21.15.49:0/2082435165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 msgr2=0x55e67b4f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.321+0000 7fd09785d740 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f5ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd080000c00 tx=0x7fd080035c20 comp rx=0 tx=0).stop 2024-02-26T06:27:11.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.322+0000 7fd09785d740 1 -- 172.21.15.49:0/2082435165 shutdown_connections 2024-02-26T06:27:11.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.322+0000 7fd09785d740 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f5ed0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.322+0000 7fd09785d740 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.322+0000 7fd09785d740 1 --2- 172.21.15.49:0/2082435165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4ed2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.322+0000 7fd09785d740 1 -- 172.21.15.49:0/2082435165 >> 172.21.15.49:0/2082435165 conn(0x55e67b4e7640 msgr2=0x55e67b4e7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.322+0000 7fd09785d740 1 -- 172.21.15.49:0/2082435165 shutdown_connections 2024-02-26T06:27:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.322+0000 7fd09785d740 1 -- 172.21.15.49:0/2082435165 wait complete. 2024-02-26T06:27:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.323+0000 7fd09785d740 1 -- start start 2024-02-26T06:27:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.323+0000 7fd09785d740 1 Processor -- start 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.323+0000 7fd09785d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.323+0000 7fd09785d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.323+0000 7fd09785d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.323+0000 7fd09785d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b453b30 con 0x55e67b4f6410 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.323+0000 7fd09785d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b3f8fa0 con 0x55e67b4ed830 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.323+0000 7fd09785d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b3f8c40 con 0x55e67b4ecf10 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd09785d740 10 client.0 _ll_register_callbacks cb 0x55e67b164c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53030/0 (socket says 172.21.15.49:53030) 2024-02-26T06:27:11.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08d400640 1 -- 172.21.15.49:0/3292685612 learned_addr learned my addr 172.21.15.49:0/3292685612 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08d400640 1 -- 172.21.15.49:0/3292685612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 msgr2=0x55e67b4f33d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08d400640 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f33d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08d400640 1 -- 172.21.15.49:0/3292685612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 msgr2=0x55e67b4f3e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08e800640 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08d400640 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08d400640 1 -- 172.21.15.49:0/3292685612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd090114f10 con 0x55e67b4ed830 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.324+0000 7fd08e800640 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.325+0000 7fd08d400640 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f3910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd080037040 tx=0x7fd080005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b3588cf42b9206 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.325+0000 7fd06f400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.325+0000 7fd06f400640 1 -- 172.21.15.49:0/3292685612 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080034a80 con 0x55e67b4ed830 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.325+0000 7fd09785d740 1 -- 172.21.15.49:0/3292685612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e67b153860 con 0x55e67b4ed830 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.325+0000 7fd06f400640 1 -- 172.21.15.49:0/3292685612 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd080004030 con 0x55e67b4ed830 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.325+0000 7fd06f400640 1 -- 172.21.15.49:0/3292685612 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0800041f0 con 0x55e67b4ed830 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.326+0000 7fd09785d740 10 client.26393 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.326+0000 7fd09785d740 1 -- 172.21.15.49:0/3292685612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e67b4521b0 con 0x55e67b4ed830 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.326+0000 7fd06f400640 1 -- 172.21.15.49:0/3292685612 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd08005c020 con 0x55e67b4ed830 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.326+0000 7fd06f400640 1 -- 172.21.15.49:0/3292685612 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd080004880 con 0x55e67b4ed830 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.326+0000 7fd09785d740 10 client.26393 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.326+0000 7fd09785d740 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.327+0000 7fd09785d740 20 client.26393 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:11.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.327+0000 7fd09785d740 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e67b506260 0x55e67b508650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.327+0000 7fd09785d740 4 client.26393 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:11.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.327+0000 7fd09785d740 1 -- 172.21.15.49:0/3292685612 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e67b1a4410 con 0x55e67b506260 2024-02-26T06:27:11.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.327+0000 7fd08e800640 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e67b506260 0x55e67b508650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.328+0000 7fd08e800640 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e67b506260 0x55e67b508650 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26d22a3cc496827 server_cookie=a90d15c98ff12357 in_seq=0 out_seq=0 2024-02-26T06:27:11.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.328+0000 7fd06f400640 10 client.26393 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:11.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.330+0000 7fd06f400640 1 -- 172.21.15.49:0/3292685612 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e67b1a4410 con 0x55e67b506260 2024-02-26T06:27:11.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.330+0000 7fd06f400640 10 client.26393 handle_command_reply: tid=0 2024-02-26T06:27:11.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.330+0000 7fd09785d740 1 client.26393 shutdown 2024-02-26T06:27:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd09785d740 1 -- 172.21.15.49:0/3292685612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 msgr2=0x55e67b4f3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd09785d740 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f3910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd080037040 tx=0x7fd080005700 comp rx=0 tx=0).stop 2024-02-26T06:27:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd09785d740 1 -- 172.21.15.49:0/3292685612 shutdown_connections 2024-02-26T06:27:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd09785d740 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e67b506260 0x55e67b508650 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd09785d740 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4f3910 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd09785d740 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd09785d740 1 --2- 172.21.15.49:0/3292685612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd06f400640 0 client.26393 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.331+0000 7fd09785d740 1 -- 172.21.15.49:0/3292685612 >> 172.21.15.49:0/3292685612 conn(0x55e67b4e7640 msgr2=0x55e67b4e9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.333+0000 7fd09785d740 1 -- 172.21.15.49:0/3292685612 shutdown_connections 2024-02-26T06:27:11.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.333+0000 7fd09785d740 1 -- 172.21.15.49:0/3292685612 wait complete. 2024-02-26T06:27:11.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.334+0000 7fd09785d740 20 client.26393 trim_cache size 0 max 16384 2024-02-26T06:27:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.368+0000 7fd09785d740 1 Processor -- start 2024-02-26T06:27:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.369+0000 7fd09785d740 1 -- start start 2024-02-26T06:27:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.370+0000 7fd09785d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.370+0000 7fd09785d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.370+0000 7fd09785d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.370+0000 7fd09785d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b19c0c0 con 0x55e67b4f6410 2024-02-26T06:27:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.370+0000 7fd09785d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b4ecb70 con 0x55e67b4ed830 2024-02-26T06:27:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.370+0000 7fd09785d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b4e8640 con 0x55e67b4ecf10 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60020/0 (socket says 172.21.15.49:60020) 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 -- 172.21.15.49:0/1960187089 learned_addr learned my addr 172.21.15.49:0/1960187089 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08d400640 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 -- 172.21.15.49:0/1960187089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 msgr2=0x55e67b4fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 -- 172.21.15.49:0/1960187089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 msgr2=0x55e67b4fd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4fd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.371+0000 7fd08e800640 1 -- 172.21.15.49:0/1960187089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e67b3eea60 con 0x55e67b4ecf10 2024-02-26T06:27:11.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.373+0000 7fd08e800640 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f3c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd088009fd0 tx=0x7fd0880095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f63d0e4a6bff30c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.373+0000 7fd07a600640 1 -- 172.21.15.49:0/1960187089 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088013040 con 0x55e67b4ecf10 2024-02-26T06:27:11.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.373+0000 7fd07a600640 1 -- 172.21.15.49:0/1960187089 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd088026070 con 0x55e67b4ecf10 2024-02-26T06:27:11.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.373+0000 7fd07a600640 1 -- 172.21.15.49:0/1960187089 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088004290 con 0x55e67b4ecf10 2024-02-26T06:27:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 -- 172.21.15.49:0/1960187089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 msgr2=0x55e67b4f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f3c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd088009fd0 tx=0x7fd0880095c0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 -- 172.21.15.49:0/1960187089 shutdown_connections 2024-02-26T06:27:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4ed830 0x55e67b4fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4f6410 0x55e67b4fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 --2- 172.21.15.49:0/1960187089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b4f3c70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 -- 172.21.15.49:0/1960187089 >> 172.21.15.49:0/1960187089 conn(0x55e67b4e7640 msgr2=0x55e67b4e66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 -- 172.21.15.49:0/1960187089 shutdown_connections 2024-02-26T06:27:11.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 -- 172.21.15.49:0/1960187089 wait complete. 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.374+0000 7fd09785d740 1 -- start start 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.375+0000 7fd09785d740 1 Processor -- start 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.375+0000 7fd09785d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b5526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.375+0000 7fd09785d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4ed830 0x55e67b552bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.375+0000 7fd09785d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4f6410 0x55e67b55d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.375+0000 7fd09785d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b553380 con 0x55e67b4ed830 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.375+0000 7fd09785d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b5534f0 con 0x55e67b4f6410 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.375+0000 7fd09785d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67b55dc70 con 0x55e67b4ecf10 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.375+0000 7fd09785d740 10 client.0 _ll_register_callbacks cb 0x7fd080037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4f6410 0x55e67b55d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4f6410 0x55e67b55d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53050/0 (socket says 172.21.15.49:53050) 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 -- 172.21.15.49:0/3510557519 learned_addr learned my addr 172.21.15.49:0/3510557519 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b5526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08d400640 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4ed830 0x55e67b552bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 -- 172.21.15.49:0/3510557519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 msgr2=0x55e67b5526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b5526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 -- 172.21.15.49:0/3510557519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4ed830 msgr2=0x55e67b552bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4ed830 0x55e67b552bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 -- 172.21.15.49:0/3510557519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08800ea70 con 0x55e67b4f6410 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd08e800640 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4f6410 0x55e67b55d730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd088009430 tx=0x7fd088003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a501fa33b8d869 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.376+0000 7fd06f400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.377+0000 7fd06f400640 1 -- 172.21.15.49:0/3510557519 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08802f070 con 0x55e67b4f6410 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.377+0000 7fd06f400640 1 -- 172.21.15.49:0/3510557519 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd088004a40 con 0x55e67b4f6410 2024-02-26T06:27:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.377+0000 7fd09785d740 1 -- 172.21.15.49:0/3510557519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd080034440 con 0x55e67b4f6410 2024-02-26T06:27:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.378+0000 7fd06f400640 1 -- 172.21.15.49:0/3510557519 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd088013070 con 0x55e67b4f6410 2024-02-26T06:27:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.378+0000 7fd09785d740 10 client.26399 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.378+0000 7fd09785d740 1 -- 172.21.15.49:0/3510557519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e67b414670 con 0x55e67b4f6410 2024-02-26T06:27:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.379+0000 7fd06f400640 1 -- 172.21.15.49:0/3510557519 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd080034440 con 0x55e67b4f6410 2024-02-26T06:27:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.379+0000 7fd06f400640 1 -- 172.21.15.49:0/3510557519 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd0880052c0 con 0x55e67b4f6410 2024-02-26T06:27:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.379+0000 7fd09785d740 10 client.26399 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.379+0000 7fd09785d740 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.379+0000 7fd09785d740 20 client.26399 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:11.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.379+0000 7fd09785d740 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e67b55f290 0x55e67b561680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.379+0000 7fd09785d740 4 client.26399 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:27:11.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.379+0000 7fd09785d740 1 -- 172.21.15.49:0/3510557519 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e67b1907b0 con 0x55e67b55f290 2024-02-26T06:27:11.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.380+0000 7fd08d400640 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e67b55f290 0x55e67b561680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.382+0000 7fd06f400640 10 client.26399 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:11.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.382+0000 7fd08d400640 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e67b55f290 0x55e67b561680 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da58e03498944c3 server_cookie=3c6072884e7db630 in_seq=0 out_seq=0 2024-02-26T06:27:11.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.383+0000 7fd06f400640 1 -- 172.21.15.49:0/3510557519 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e67b1907b0 con 0x55e67b55f290 2024-02-26T06:27:11.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.383+0000 7fd06f400640 10 client.26399 handle_command_reply: tid=0 2024-02-26T06:27:11.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.383+0000 7fd09785d740 1 client.26399 shutdown 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 -- 172.21.15.49:0/3510557519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4f6410 msgr2=0x55e67b55d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4f6410 0x55e67b55d730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd088009430 tx=0x7fd088003e10 comp rx=0 tx=0).stop 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 -- 172.21.15.49:0/3510557519 shutdown_connections 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e67b55f290 0x55e67b561680 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e67b4f6410 0x55e67b55d730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e67b4ed830 0x55e67b552bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 --2- 172.21.15.49:0/3510557519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e67b4ecf10 0x55e67b5526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 -- 172.21.15.49:0/3510557519 >> 172.21.15.49:0/3510557519 conn(0x55e67b4e7640 msgr2=0x55e67b4e8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd06f400640 0 client.26399 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:11.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 -- 172.21.15.49:0/3510557519 shutdown_connections 2024-02-26T06:27:11.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 1 -- 172.21.15.49:0/3510557519 wait complete. 2024-02-26T06:27:11.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.384+0000 7fd09785d740 20 client.26399 trim_cache size 0 max 16384 2024-02-26T06:27:11.387 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:27:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:27:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "b97233ab-4649-49e3-a865-665d9b6518eb", 2024-02-26T06:27:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:27:11.387 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:27:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.385+0000 7fd094a00640 1 -- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd0740a22b0 msgr2=0x7fd0740a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.385+0000 7fd094a00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd0740a22b0 0x7fd0740a4770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd090100c90 tx=0x7fd07c00ef90 comp rx=0 tx=0).stop 2024-02-26T06:27:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.385+0000 7fd094a00640 1 -- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0900688c0 msgr2=0x7fd0901a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.385+0000 7fd094a00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0900688c0 0x7fd0901a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00ea60 tx=0x7fd07c00ef20 comp rx=0 tx=0).stop 2024-02-26T06:27:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.385+0000 7fd094a00640 1 -- 172.21.15.49:0/1385625077 shutdown_connections 2024-02-26T06:27:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.386+0000 7fd094a00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd0740a22b0 0x7fd0740a4770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.386+0000 7fd094a00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd090103c40 0x7fd0901a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.386+0000 7fd094a00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0901060d0 0x7fd09019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.386+0000 7fd094a00640 1 --2- 172.21.15.49:0/1385625077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0900688c0 0x7fd0901a3940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.386+0000 7fd094a00640 1 -- 172.21.15.49:0/1385625077 >> 172.21.15.49:0/1385625077 conn(0x7fd090077180 msgr2=0x7fd090101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.386+0000 7fd094a00640 1 -- 172.21.15.49:0/1385625077 shutdown_connections 2024-02-26T06:27:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.386+0000 7fd094a00640 1 -- 172.21.15.49:0/1385625077 wait complete. 2024-02-26T06:27:11.409 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:11 smithi049 ceph-mon[31696]: pgmap v116: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.45k op/s 2024-02-26T06:27:11.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:27:11.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:27:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:11 smithi080 ceph-mon[36206]: pgmap v116: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.45k op/s 2024-02-26T06:27:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:11 smithi115 ceph-mon[36426]: pgmap v116: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.45k op/s 2024-02-26T06:27:11.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 -- 172.21.15.49:0/2926931177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e4c105080 msgr2=0x7f7e4c105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 --2- 172.21.15.49:0/2926931177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e4c105080 0x7f7e4c105460 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009890 tx=0x7f7e3c02ff80 comp rx=0 tx=0).stop 2024-02-26T06:27:11.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 -- 172.21.15.49:0/2926931177 shutdown_connections 2024-02-26T06:27:11.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 --2- 172.21.15.49:0/2926931177 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e4c1015b0 0x7f7e4c10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 --2- 172.21.15.49:0/2926931177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e4c100bf0 0x7f7e4c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 --2- 172.21.15.49:0/2926931177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e4c105080 0x7f7e4c105460 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 -- 172.21.15.49:0/2926931177 >> 172.21.15.49:0/2926931177 conn(0x7f7e4c0761d0 msgr2=0x7f7e4c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 -- 172.21.15.49:0/2926931177 shutdown_connections 2024-02-26T06:27:11.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.694+0000 7f7e51c00640 1 -- 172.21.15.49:0/2926931177 wait complete. 2024-02-26T06:27:11.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.696+0000 7f7e51c00640 1 Processor -- start 2024-02-26T06:27:11.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.696+0000 7f7e51c00640 1 -- start start 2024-02-26T06:27:11.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.697+0000 7f7e51c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e4c100bf0 0x7f7e4c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e51c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e4c1015b0 0x7f7e4c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e51c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e4c105080 0x7f7e4c1a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e51c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c1a2d10 con 0x7f7e4c100bf0 2024-02-26T06:27:11.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e51c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c1a2e80 con 0x7f7e4c105080 2024-02-26T06:27:11.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e51c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c1a3040 con 0x7f7e4c1015b0 2024-02-26T06:27:11.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e4a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e4c1015b0 0x7f7e4c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e4a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e4c1015b0 0x7f7e4c19e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60032/0 (socket says 172.21.15.49:60032) 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e4a000640 1 -- 172.21.15.49:0/1611762876 learned_addr learned my addr 172.21.15.49:0/1611762876 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e4b400640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e4c105080 0x7f7e4c1a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.698+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e4c100bf0 0x7f7e4c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.699+0000 7f7e4a000640 1 -- 172.21.15.49:0/1611762876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e4c105080 msgr2=0x7f7e4c1a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.699+0000 7f7e4a000640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e4c105080 0x7f7e4c1a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.699+0000 7f7e4a000640 1 -- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e4c100bf0 msgr2=0x7f7e4c19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.699+0000 7f7e4a000640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e4c100bf0 0x7f7e4c19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.699+0000 7f7e4a000640 1 -- 172.21.15.49:0/1611762876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c009490 con 0x7f7e4c1015b0 2024-02-26T06:27:11.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.699+0000 7f7e4a000640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e4c1015b0 0x7f7e4c19e390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800e9c0 tx=0x7f7e3800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.700+0000 7f7e43400640 1 -- 172.21.15.49:0/1611762876 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800ccc0 con 0x7f7e4c1015b0 2024-02-26T06:27:11.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.700+0000 7f7e51c00640 1 -- 172.21.15.49:0/1611762876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4c1a3320 con 0x7f7e4c1015b0 2024-02-26T06:27:11.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.700+0000 7f7e43400640 1 -- 172.21.15.49:0/1611762876 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e3800ce20 con 0x7f7e4c1015b0 2024-02-26T06:27:11.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.700+0000 7f7e43400640 1 -- 172.21.15.49:0/1611762876 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38010430 con 0x7f7e4c1015b0 2024-02-26T06:27:11.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.700+0000 7f7e51c00640 1 -- 172.21.15.49:0/1611762876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c1aaba0 con 0x7f7e4c1015b0 2024-02-26T06:27:11.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.707+0000 7f7e54b97740 1 Processor -- start 2024-02-26T06:27:11.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.707+0000 7f7e43400640 1 -- 172.21.15.49:0/1611762876 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7e380046c0 con 0x7f7e4c1015b0 2024-02-26T06:27:11.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.707+0000 7f7e54b97740 1 -- start start 2024-02-26T06:27:11.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.709+0000 7f7e43400640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7e300a23c0 0x7f7e300a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.710+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295f70 0x557ce7294cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295290 0x557ce7295710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 0x557ce72a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e54b97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce7204120 con 0x557ce729e600 2024-02-26T06:27:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e54b97740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce7201650 con 0x557ce7295290 2024-02-26T06:27:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e54b97740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce71fce30 con 0x557ce7295f70 2024-02-26T06:27:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e4b400640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7e300a23c0 0x7f7e300a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e43400640 1 -- 172.21.15.49:0/1611762876 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7e38014070 con 0x7f7e4c1015b0 2024-02-26T06:27:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e4a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295290 0x557ce7295710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e4a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295290 0x557ce7295710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53072/0 (socket says 172.21.15.49:53072) 2024-02-26T06:27:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e4a000640 1 -- 172.21.15.49:0/3861389835 learned_addr learned my addr 172.21.15.49:0/3861389835 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295f70 0x557ce7294cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e4a000640 1 -- 172.21.15.49:0/3861389835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295f70 msgr2=0x557ce7294cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e4a000640 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295f70 0x557ce7294cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.711+0000 7f7e4a000640 1 -- 172.21.15.49:0/3861389835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 msgr2=0x557ce72a09f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:11.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.712+0000 7f7e4b400640 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 0x557ce72a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.712+0000 7f7e4a000640 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 0x557ce72a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.712+0000 7f7e4a000640 1 -- 172.21.15.49:0/3861389835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ce71b6030 con 0x557ce7295290 2024-02-26T06:27:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.712+0000 7f7e4a000640 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295290 0x557ce7295710 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e38000c00 tx=0x7f7e3809ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d7128c40ac06c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.713+0000 7f7e4b400640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7e300a23c0 0x7f7e300a4880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e44005fd0 tx=0x7f7e44009b00 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.713+0000 7f7e37e00640 1 -- 172.21.15.49:0/3861389835 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38004970 con 0x557ce7295290 2024-02-26T06:27:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.713+0000 7f7e37e00640 1 -- 172.21.15.49:0/3861389835 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e380971f0 con 0x557ce7295290 2024-02-26T06:27:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.713+0000 7f7e37e00640 1 -- 172.21.15.49:0/3861389835 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380a2e90 con 0x557ce7295290 2024-02-26T06:27:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.714+0000 7f7e54b97740 1 -- 172.21.15.49:0/3861389835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295290 msgr2=0x557ce7295710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.714+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295290 0x557ce7295710 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e38000c00 tx=0x7f7e3809ab80 comp rx=0 tx=0).stop 2024-02-26T06:27:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.714+0000 7f7e54b97740 1 -- 172.21.15.49:0/3861389835 shutdown_connections 2024-02-26T06:27:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.714+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295290 0x557ce7295710 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.714+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 0x557ce72a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.714+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3861389835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295f70 0x557ce7294cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.714+0000 7f7e54b97740 1 -- 172.21.15.49:0/3861389835 >> 172.21.15.49:0/3861389835 conn(0x557ce725c570 msgr2=0x557ce728fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.714+0000 7f7e54b97740 1 -- 172.21.15.49:0/3861389835 shutdown_connections 2024-02-26T06:27:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.715+0000 7f7e54b97740 1 -- 172.21.15.49:0/3861389835 wait complete. 2024-02-26T06:27:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.715+0000 7f7e54b97740 1 -- start start 2024-02-26T06:27:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.716+0000 7f7e54b97740 1 Processor -- start 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.716+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295290 0x557ce729b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.716+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce729e600 0x557ce729c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e54b97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce71a1e90 con 0x557ce7295290 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e54b97740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce71a1b30 con 0x557ce7295f70 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e54b97740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce6f6b1c0 con 0x557ce729e600 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e54b97740 10 client.0 _ll_register_callbacks cb 0x557ce71b5930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e4aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295290 0x557ce729b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e4a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e4aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295290 0x557ce729b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49490/0 (socket says 172.21.15.49:49490) 2024-02-26T06:27:11.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.717+0000 7f7e4aa00640 1 -- 172.21.15.49:0/276993673 learned_addr learned my addr 172.21.15.49:0/276993673 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.718+0000 7f7e4b400640 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce729e600 0x557ce729c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.718+0000 7f7e4a000640 1 -- 172.21.15.49:0/276993673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce729e600 msgr2=0x557ce729c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.718+0000 7f7e4a000640 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce729e600 0x557ce729c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.718+0000 7f7e4a000640 1 -- 172.21.15.49:0/276993673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295290 msgr2=0x557ce729b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.718+0000 7f7e4a000640 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295290 0x557ce729b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.719+0000 7f7e4a000640 1 -- 172.21.15.49:0/276993673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c1aaba0 con 0x557ce7295f70 2024-02-26T06:27:11.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.720+0000 7f7e4a000640 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729baf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e38000c00 tx=0x7f7e3809a280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de35f445579ba103 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.720+0000 7f7e34c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:11.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.720+0000 7f7e34c00640 1 -- 172.21.15.49:0/276993673 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38096e20 con 0x557ce7295f70 2024-02-26T06:27:11.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.720+0000 7f7e34c00640 1 -- 172.21.15.49:0/276993673 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7e38004970 con 0x557ce7295f70 2024-02-26T06:27:11.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.720+0000 7f7e54b97740 1 -- 172.21.15.49:0/276993673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ce6efc7e0 con 0x557ce7295f70 2024-02-26T06:27:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.721+0000 7f7e34c00640 1 -- 172.21.15.49:0/276993673 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e380968d0 con 0x557ce7295f70 2024-02-26T06:27:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.721+0000 7f7e54b97740 10 client.26411 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.721+0000 7f7e54b97740 1 -- 172.21.15.49:0/276993673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557ce71fb4b0 con 0x557ce7295f70 2024-02-26T06:27:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.721+0000 7f7e34c00640 1 -- 172.21.15.49:0/276993673 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557ce6efc7e0 con 0x557ce7295f70 2024-02-26T06:27:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.721+0000 7f7e34c00640 1 -- 172.21.15.49:0/276993673 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7e3801f060 con 0x557ce7295f70 2024-02-26T06:27:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.721+0000 7f7e54b97740 10 client.26411 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.721+0000 7f7e54b97740 10 client.26411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.722+0000 7f7e54b97740 20 client.26411 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.722+0000 7f7e54b97740 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ce72ae150 0x557ce72b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.722+0000 7f7e54b97740 4 client.26411 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.722+0000 7f7e54b97740 1 -- 172.21.15.49:0/276993673 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ce6f39720 con 0x557ce72ae150 2024-02-26T06:27:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.722+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ce72ae150 0x557ce72b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.723+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ce72ae150 0x557ce72b0540 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3d7d71c0f807fc server_cookie=589ef1ddf3b7e5ac in_seq=0 out_seq=0 2024-02-26T06:27:11.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.723+0000 7f7e34c00640 10 client.26411 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:11.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e34c00640 1 -- 172.21.15.49:0/276993673 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ce6f39720 con 0x557ce72ae150 2024-02-26T06:27:11.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e34c00640 10 client.26411 handle_command_reply: tid=0 2024-02-26T06:27:11.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 client.26411 shutdown 2024-02-26T06:27:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 -- 172.21.15.49:0/276993673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 msgr2=0x557ce729baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729baf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e38000c00 tx=0x7f7e3809a280 comp rx=0 tx=0).stop 2024-02-26T06:27:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 -- 172.21.15.49:0/276993673 shutdown_connections 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ce72ae150 0x557ce72b0540 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729baf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e34c00640 0 client.26411 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce729e600 0x557ce729c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 --2- 172.21.15.49:0/276993673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295290 0x557ce729b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.724+0000 7f7e54b97740 1 -- 172.21.15.49:0/276993673 >> 172.21.15.49:0/276993673 conn(0x557ce725c570 msgr2=0x557ce7291a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.725+0000 7f7e54b97740 1 -- 172.21.15.49:0/276993673 shutdown_connections 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.725+0000 7f7e54b97740 1 -- 172.21.15.49:0/276993673 wait complete. 2024-02-26T06:27:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.725+0000 7f7e54b97740 20 client.26411 trim_cache size 0 max 16384 2024-02-26T06:27:11.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.754+0000 7f7e54b97740 1 Processor -- start 2024-02-26T06:27:11.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.754+0000 7f7e54b97740 1 -- start start 2024-02-26T06:27:11.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.755+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 0x557ce728ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e54b97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce72a1350 con 0x557ce729e600 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e54b97740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce72b5710 con 0x557ce7295f70 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e54b97740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce72942c0 con 0x557ce7295290 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e4a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e4a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53094/0 (socket says 172.21.15.49:53094) 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e4a000640 1 -- 172.21.15.49:0/3672801542 learned_addr learned my addr 172.21.15.49:0/3672801542 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.756+0000 7f7e4aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.757+0000 7f7e4b400640 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 0x557ce728ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.757+0000 7f7e4aa00640 1 -- 172.21.15.49:0/3672801542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 msgr2=0x557ce729be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.757+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.757+0000 7f7e4aa00640 1 -- 172.21.15.49:0/3672801542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 msgr2=0x557ce728ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.757+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 0x557ce728ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.757+0000 7f7e4aa00640 1 -- 172.21.15.49:0/3672801542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ce7197920 con 0x557ce7295290 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.757+0000 7f7e4a000640 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.757+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72968d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x557ce72b0b20 tx=0x7f7e3c010410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed3c545d72a00102 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.758+0000 7f7e37400640 1 -- 172.21.15.49:0/3672801542 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c036600 con 0x557ce7295290 2024-02-26T06:27:11.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.758+0000 7f7e37400640 1 -- 172.21.15.49:0/3672801542 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c036760 con 0x557ce7295290 2024-02-26T06:27:11.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.758+0000 7f7e37400640 1 -- 172.21.15.49:0/3672801542 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c005330 con 0x557ce7295290 2024-02-26T06:27:11.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.760+0000 7f7e54b97740 1 -- 172.21.15.49:0/3672801542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 msgr2=0x557ce72968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.760+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72968d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x557ce72b0b20 tx=0x7f7e3c010410 comp rx=0 tx=0).stop 2024-02-26T06:27:11.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.760+0000 7f7e54b97740 1 -- 172.21.15.49:0/3672801542 shutdown_connections 2024-02-26T06:27:11.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.760+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce7295f70 0x557ce729be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.760+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce729e600 0x557ce728ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.760+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3672801542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72968d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.760+0000 7f7e54b97740 1 -- 172.21.15.49:0/3672801542 >> 172.21.15.49:0/3672801542 conn(0x557ce725c570 msgr2=0x557ce728e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.761+0000 7f7e54b97740 1 -- 172.21.15.49:0/3672801542 shutdown_connections 2024-02-26T06:27:11.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.761+0000 7f7e54b97740 1 -- 172.21.15.49:0/3672801542 wait complete. 2024-02-26T06:27:11.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.762+0000 7f7e54b97740 1 -- start start 2024-02-26T06:27:11.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.762+0000 7f7e54b97740 1 Processor -- start 2024-02-26T06:27:11.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.763+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295f70 0x557ce72a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e54b97740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce729e600 0x557ce72a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e54b97740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce72a3af0 con 0x557ce7295f70 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e54b97740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce72a3c60 con 0x557ce729e600 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e54b97740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ce72a3dd0 con 0x557ce7295290 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e4a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295f70 0x557ce72a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e4b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce729e600 0x557ce72a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e4b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce729e600 0x557ce72a35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53100/0 (socket says 172.21.15.49:53100) 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.764+0000 7f7e4b400640 1 -- 172.21.15.49:0/3031473195 learned_addr learned my addr 172.21.15.49:0/3031473195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.765+0000 7f7e54b97740 10 client.0 _ll_register_callbacks cb 0x7f7e380b1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.765+0000 7f7e4b400640 1 -- 172.21.15.49:0/3031473195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 msgr2=0x557ce72a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.765+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.765+0000 7f7e4b400640 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.765+0000 7f7e4b400640 1 -- 172.21.15.49:0/3031473195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295f70 msgr2=0x557ce72a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.765+0000 7f7e4b400640 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295f70 0x557ce72a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.765+0000 7f7e4b400640 1 -- 172.21.15.49:0/3031473195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ce71fb4b0 con 0x557ce729e600 2024-02-26T06:27:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.766+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.766+0000 7f7e54b97740 1 -- 172.21.15.49:0/3031473195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e380971f0 con 0x557ce729e600 2024-02-26T06:27:11.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.766+0000 7f7e4a000640 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295f70 0x557ce72a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:11.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.767+0000 7f7e4b400640 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce729e600 0x557ce72a35b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7e44003ec0 tx=0x7f7e44009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ca39d852114ac5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.768+0000 7f7e34c00640 10 client.26417 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:11.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.768+0000 7f7e34c00640 1 -- 172.21.15.49:0/3031473195 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400f7e0 con 0x557ce729e600 2024-02-26T06:27:11.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.768+0000 7f7e34c00640 1 -- 172.21.15.49:0/3031473195 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7e4401c040 con 0x557ce729e600 2024-02-26T06:27:11.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.768+0000 7f7e34c00640 1 -- 172.21.15.49:0/3031473195 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e4400f940 con 0x557ce729e600 2024-02-26T06:27:11.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.768+0000 7f7e54b97740 10 client.26417 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:11.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.768+0000 7f7e54b97740 1 -- 172.21.15.49:0/3031473195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x7f7e4c0ffae0 con 0x557ce729e600 2024-02-26T06:27:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.769+0000 7f7e34c00640 1 -- 172.21.15.49:0/3031473195 <== mon.2 v2:172.21.15.80:3300/0 4 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7e4400faa0 con 0x557ce729e600 2024-02-26T06:27:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.769+0000 7f7e54b97740 10 client.26417 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.769+0000 7f7e54b97740 10 client.26417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.769+0000 7f7e54b97740 20 client.26417 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.769+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ce73064c0 0x557ce73088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.769+0000 7f7e54b97740 4 client.26417 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:27:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.769+0000 7f7e54b97740 1 -- 172.21.15.49:0/3031473195 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ce6f88890 con 0x557ce73064c0 2024-02-26T06:27:11.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.770+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ce73064c0 0x557ce73088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.770+0000 7f7e34c00640 10 client.26417 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.770+0000 7f7e4aa00640 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ce73064c0 0x557ce73088e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86dcb7cdeddde16d server_cookie=d353e72b858f7e55 in_seq=0 out_seq=0 2024-02-26T06:27:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.770+0000 7f7e34c00640 1 -- 172.21.15.49:0/3031473195 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557ce6f88890 con 0x557ce73064c0 2024-02-26T06:27:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.770+0000 7f7e34c00640 10 client.26417 handle_command_reply: tid=0 2024-02-26T06:27:11.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.771+0000 7f7e54b97740 1 client.26417 shutdown 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.771+0000 7f7e54b97740 1 -- 172.21.15.49:0/3031473195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce729e600 msgr2=0x557ce72a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.771+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce729e600 0x557ce72a35b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7e44003ec0 tx=0x7f7e44009500 comp rx=0 tx=0).stop 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.772+0000 7f7e54b97740 1 -- 172.21.15.49:0/3031473195 shutdown_connections 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.772+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557ce73064c0 0x557ce73088e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.772+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557ce729e600 0x557ce72a35b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.772+0000 7f7e34c00640 0 client.26417 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.772+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557ce7295f70 0x557ce72a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.772+0000 7f7e54b97740 1 --2- 172.21.15.49:0/3031473195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ce7295290 0x557ce72a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.772+0000 7f7e54b97740 1 -- 172.21.15.49:0/3031473195 >> 172.21.15.49:0/3031473195 conn(0x557ce725c570 msgr2=0x557ce728ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.773+0000 7f7e54b97740 1 -- 172.21.15.49:0/3031473195 shutdown_connections 2024-02-26T06:27:11.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.773+0000 7f7e54b97740 1 -- 172.21.15.49:0/3031473195 wait complete. 2024-02-26T06:27:11.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.773+0000 7f7e54b97740 20 client.26417 trim_cache size 0 max 16384 2024-02-26T06:27:11.774 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:27:11.774 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:27:11.775 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:27:11.775 INFO:teuthology.orchestra.run.smithi049.stdout: "b97233ab-4649-49e3-a865-665d9b6518eb": { 2024-02-26T06:27:11.775 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:27:11.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "b97233ab-4649-49e3-a865-665d9b6518eb", 2024-02-26T06:27:11.775 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:27:11.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:27:11.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:27:11.775 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 -- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7e300a23c0 msgr2=0x7f7e300a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7e300a23c0 0x7f7e300a4880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e44005fd0 tx=0x7f7e44009b00 comp rx=0 tx=0).stop 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 -- 172.21.15.49:0/1611762876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e4c1015b0 msgr2=0x7f7e4c19e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e4c1015b0 0x7f7e4c19e390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800e9c0 tx=0x7f7e3800ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 -- 172.21.15.49:0/1611762876 shutdown_connections 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7e300a23c0 0x7f7e300a4880 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e4c105080 0x7f7e4c1a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e4c1015b0 0x7f7e4c19e390 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 --2- 172.21.15.49:0/1611762876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e4c100bf0 0x7f7e4c19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 -- 172.21.15.49:0/1611762876 >> 172.21.15.49:0/1611762876 conn(0x7f7e4c0761d0 msgr2=0x7f7e4c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 -- 172.21.15.49:0/1611762876 shutdown_connections 2024-02-26T06:27:11.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:11.774+0000 7f7e51c00640 1 -- 172.21.15.49:0/1611762876 wait complete. 2024-02-26T06:27:11.797 INFO:tasks.cephfs.filesystem:scrub status for tag:b97233ab-4649-49e3-a865-665d9b6518eb - {'path': '/', 'tag': 'b97233ab-4649-49e3-a865-665d9b6518eb', 'options': 'recursive,force'} 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.216+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2515271281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68107260 msgr2=0x7f9c68107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.216+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2515271281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68107260 0x7f9c68107640 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009900 tx=0x7f9c5c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.217+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2515271281 shutdown_connections 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.217+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2515271281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c680ffc10 0x7f9c6810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.217+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2515271281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c680ff250 0x7f9c680ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.217+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2515271281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c68107260 0x7f9c68107640 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.217+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2515271281 >> 172.21.15.49:0/2515271281 conn(0x7f9c68076210 msgr2=0x7f9c680fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.217+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2515271281 shutdown_connections 2024-02-26T06:27:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.217+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2515271281 wait complete. 2024-02-26T06:27:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.219+0000 7f9c6fc00640 1 Processor -- start 2024-02-26T06:27:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.219+0000 7f9c6fc00640 1 -- start start 2024-02-26T06:27:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.219+0000 7f9c6fc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c680ff250 0x7f9c68100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.219+0000 7f9c6fc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c680ffc10 0x7f9c68101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c6fc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68107260 0x7f9c68101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c6fc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c681a5a30 con 0x7f9c68107260 2024-02-26T06:27:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c6fc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c681a5ba0 con 0x7f9c680ff250 2024-02-26T06:27:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c6fc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c681a5d60 con 0x7f9c680ffc10 2024-02-26T06:27:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c680ff250 0x7f9c68100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c680ff250 0x7f9c68100cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53126/0 (socket says 172.21.15.49:53126) 2024-02-26T06:27:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c6d200640 1 -- 172.21.15.49:0/2991156170 learned_addr learned my addr 172.21.15.49:0/2991156170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c67e00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c680ffc10 0x7f9c68101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.220+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68107260 0x7f9c68101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.221+0000 7f9c6d200640 1 -- 172.21.15.49:0/2991156170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c680ffc10 msgr2=0x7f9c68101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.221+0000 7f9c6d200640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c680ffc10 0x7f9c68101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.221+0000 7f9c6d200640 1 -- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68107260 msgr2=0x7f9c68101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.221+0000 7f9c6d200640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68107260 0x7f9c68101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.221+0000 7f9c6d200640 1 -- 172.21.15.49:0/2991156170 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c009560 con 0x7f9c680ff250 2024-02-26T06:27:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.221+0000 7f9c67e00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c680ffc10 0x7f9c68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.222+0000 7f9c6d200640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c680ff250 0x7f9c68100cc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c005940 tx=0x7f9c5c02c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.222+0000 7f9c65600640 1 -- 172.21.15.49:0/2991156170 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c004350 con 0x7f9c680ff250 2024-02-26T06:27:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.222+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2991156170 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c681a5fe0 con 0x7f9c680ff250 2024-02-26T06:27:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.222+0000 7f9c65600640 1 -- 172.21.15.49:0/2991156170 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c0044b0 con 0x7f9c680ff250 2024-02-26T06:27:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.222+0000 7f9c65600640 1 -- 172.21.15.49:0/2991156170 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c003d30 con 0x7f9c680ff250 2024-02-26T06:27:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.222+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2991156170 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c681a6460 con 0x7f9c680ff250 2024-02-26T06:27:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.226+0000 7f9c72b94740 1 Processor -- start 2024-02-26T06:27:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.226+0000 7f9c72b94740 1 -- start start 2024-02-26T06:27:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.226+0000 7f9c65600640 1 -- 172.21.15.49:0/2991156170 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9c5c03e450 con 0x7f9c680ff250 2024-02-26T06:27:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.226+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11dd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.226+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11de200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.227+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.227+0000 7f9c72b94740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e11995f0 con 0x55f2e11efb10 2024-02-26T06:27:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.227+0000 7f9c72b94740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e114b890 con 0x55f2e11ddd80 2024-02-26T06:27:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.227+0000 7f9c72b94740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e1148dc0 con 0x55f2e11deaf0 2024-02-26T06:27:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.227+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11de200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.227+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11de200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53136/0 (socket says 172.21.15.49:53136) 2024-02-26T06:27:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.227+0000 7f9c6d200640 1 -- 172.21.15.49:0/2626720568 learned_addr learned my addr 172.21.15.49:0/2626720568 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.228+0000 7f9c6d200640 1 -- 172.21.15.49:0/2626720568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 msgr2=0x55f2e11dd840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:27:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.228+0000 7f9c6d200640 1 --2- 172.21.15.49:0/2626720568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11dd840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.228+0000 7f9c6d200640 1 -- 172.21.15.49:0/2626720568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 msgr2=0x55f2e11f1f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.228+0000 7f9c65600640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c500a2300 0x7f9c500a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.228+0000 7f9c65600640 1 -- 172.21.15.49:0/2991156170 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9c5c0f0060 con 0x7f9c680ff250 2024-02-26T06:27:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.230+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c500a2300 0x7f9c500a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.230+0000 7f9c67e00640 1 --2- 172.21.15.49:0/2626720568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.230+0000 7f9c6d200640 1 --2- 172.21.15.49:0/2626720568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.230+0000 7f9c6d200640 1 -- 172.21.15.49:0/2626720568 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2e10c5ad0 con 0x55f2e11ddd80 2024-02-26T06:27:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.231+0000 7f9c6d200640 1 --2- 172.21.15.49:0/2626720568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11de200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0ed7b0 tx=0x7f9c5c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=729a30ef879e7187 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.231+0000 7f9c5a000640 1 -- 172.21.15.49:0/2626720568 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0b8e90 con 0x55f2e11ddd80 2024-02-26T06:27:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.231+0000 7f9c5a000640 1 -- 172.21.15.49:0/2626720568 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c0f3be0 con 0x55f2e11ddd80 2024-02-26T06:27:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.231+0000 7f9c5a000640 1 -- 172.21.15.49:0/2626720568 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0c5340 con 0x55f2e11ddd80 2024-02-26T06:27:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.232+0000 7f9c72b94740 1 -- 172.21.15.49:0/2626720568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 msgr2=0x55f2e11de200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.232+0000 7f9c72b94740 1 --2- 172.21.15.49:0/2626720568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11de200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0ed7b0 tx=0x7f9c5c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.233+0000 7f9c72b94740 1 -- 172.21.15.49:0/2626720568 shutdown_connections 2024-02-26T06:27:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.233+0000 7f9c72b94740 1 --2- 172.21.15.49:0/2626720568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11de200 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.233+0000 7f9c72b94740 1 --2- 172.21.15.49:0/2626720568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.233+0000 7f9c72b94740 1 --2- 172.21.15.49:0/2626720568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11dd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.233+0000 7f9c72b94740 1 -- 172.21.15.49:0/2626720568 >> 172.21.15.49:0/2626720568 conn(0x55f2e11a5480 msgr2=0x55f2e11d8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.233+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c500a2300 0x7f9c500a47c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005fd0 tx=0x7f9c60009500 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.233+0000 7f9c72b94740 1 -- 172.21.15.49:0/2626720568 shutdown_connections 2024-02-26T06:27:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.233+0000 7f9c72b94740 1 -- 172.21.15.49:0/2626720568 wait complete. 2024-02-26T06:27:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.235+0000 7f9c72b94740 1 -- start start 2024-02-26T06:27:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.235+0000 7f9c72b94740 1 Processor -- start 2024-02-26T06:27:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.236+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11ecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.236+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.237+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.237+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11ecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.238+0000 7f9c6dc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.238+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11ecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53140/0 (socket says 172.21.15.49:53140) 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.238+0000 7f9c67e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.238+0000 7f9c6dc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11ed590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49556/0 (socket says 172.21.15.49:49556) 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.238+0000 7f9c67e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11ed050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60090/0 (socket says 172.21.15.49:60090) 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.239+0000 7f9c6d200640 1 -- 172.21.15.49:0/3888498970 learned_addr learned my addr 172.21.15.49:0/3888498970 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e11445a0 con 0x55f2e11efb10 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e10e9b20 con 0x55f2e11ddd80 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e10e97c0 con 0x55f2e11deaf0 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c72b94740 10 client.0 _ll_register_callbacks cb 0x55f2e10fd520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c6d200640 1 -- 172.21.15.49:0/3888498970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 msgr2=0x55f2e11ed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c6d200640 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11ed050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c6d200640 1 -- 172.21.15.49:0/3888498970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 msgr2=0x55f2e11ed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c6d200640 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11ed590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.240+0000 7f9c6d200640 1 -- 172.21.15.49:0/3888498970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c681a6460 con 0x55f2e11ddd80 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c6d200640 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11ecb10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c006010 tx=0x7f9c5c0bc730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce82ede8b1abd4d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c4ea00640 1 -- 172.21.15.49:0/3888498970 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0b8e90 con 0x55f2e11ddd80 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c4ea00640 1 -- 172.21.15.49:0/3888498970 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9c5c0bde40 con 0x55f2e11ddd80 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2e1116ca0 con 0x55f2e11ddd80 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c4ea00640 1 -- 172.21.15.49:0/3888498970 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c5c0bc550 con 0x55f2e11ddd80 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c72b94740 10 client.26429 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f2e1142c20 con 0x55f2e11ddd80 2024-02-26T06:27:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.241+0000 7f9c4ea00640 1 -- 172.21.15.49:0/3888498970 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f2e1116ca0 con 0x55f2e11ddd80 2024-02-26T06:27:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.242+0000 7f9c4ea00640 1 -- 172.21.15.49:0/3888498970 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9c5c03c070 con 0x55f2e11ddd80 2024-02-26T06:27:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.242+0000 7f9c72b94740 10 client.26429 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.242+0000 7f9c72b94740 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.242+0000 7f9c72b94740 20 client.26429 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.242+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f2e11ff440 0x55f2e1201890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.243+0000 7f9c72b94740 4 client.26429 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.243+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2e113acb0 con 0x55f2e11ff440 2024-02-26T06:27:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.243+0000 7f9c67e00640 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f2e11ff440 0x55f2e1201890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.245+0000 7f9c67e00640 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f2e11ff440 0x55f2e1201890 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1336cb6b17eee34a server_cookie=94cd3ad1086e5797 in_seq=0 out_seq=0 2024-02-26T06:27:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.245+0000 7f9c4ea00640 10 client.26429 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.245+0000 7f9c4ea00640 1 -- 172.21.15.49:0/3888498970 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2e113acb0 con 0x55f2e11ff440 2024-02-26T06:27:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.245+0000 7f9c4ea00640 10 client.26429 handle_command_reply: tid=0 2024-02-26T06:27:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 client.26429 shutdown 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 msgr2=0x55f2e11ecb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11ecb10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c006010 tx=0x7f9c5c0bc730 comp rx=0 tx=0).stop 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 shutdown_connections 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f2e11ff440 0x55f2e1201890 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3888498970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11ddd80 0x55f2e11ecb10 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c006010 tx=0x7f9c5c0bc730 comp rx=0 tx=0).stop 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 >> 172.21.15.49:0/3888498970 conn(0x55f2e11a5480 msgr2=0x55f2e11da7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.246+0000 7f9c4ea00640 0 client.26429 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.247+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 shutdown_connections 2024-02-26T06:27:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.248+0000 7f9c72b94740 1 -- 172.21.15.49:0/3888498970 wait complete. 2024-02-26T06:27:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.248+0000 7f9c72b94740 20 client.26429 trim_cache size 0 max 16384 2024-02-26T06:27:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.269+0000 7f9c72b94740 1 Processor -- start 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.269+0000 7f9c72b94740 1 -- start start 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11deaf0 0x55f2e11ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11eb920 0x55f2e11e9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c72b94740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e11d8d70 con 0x55f2e11efb10 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c72b94740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e0e8ce60 con 0x55f2e11deaf0 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c72b94740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e113acb0 con 0x55f2e11eb920 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11deaf0 0x55f2e11ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11deaf0 0x55f2e11ef240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53152/0 (socket says 172.21.15.49:53152) 2024-02-26T06:27:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c6d200640 1 -- 172.21.15.49:0/3753425631 learned_addr learned my addr 172.21.15.49:0/3753425631 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.270+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11eb920 0x55f2e11e9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.271+0000 7f9c6d200640 1 -- 172.21.15.49:0/3753425631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11eb920 msgr2=0x55f2e11e9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.271+0000 7f9c6d200640 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11eb920 0x55f2e11e9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.271+0000 7f9c67e00640 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.271+0000 7f9c6d200640 1 -- 172.21.15.49:0/3753425631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 msgr2=0x55f2e11eb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.271+0000 7f9c6d200640 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11eb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.271+0000 7f9c6d200640 1 -- 172.21.15.49:0/3753425631 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2e10df500 con 0x55f2e11deaf0 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.271+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11eb920 0x55f2e11e9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.271+0000 7f9c6d200640 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11deaf0 0x55f2e11ef240 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0b95c0 tx=0x7f9c5c0037d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569b4faa82ac71d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.272+0000 7f9c59600640 1 -- 172.21.15.49:0/3753425631 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0f3ea0 con 0x55f2e11deaf0 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.272+0000 7f9c59600640 1 -- 172.21.15.49:0/3753425631 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c0035f0 con 0x55f2e11deaf0 2024-02-26T06:27:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.272+0000 7f9c59600640 1 -- 172.21.15.49:0/3753425631 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0f3ea0 con 0x55f2e11deaf0 2024-02-26T06:27:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 -- 172.21.15.49:0/3753425631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11deaf0 msgr2=0x55f2e11ef240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11deaf0 0x55f2e11ef240 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0b95c0 tx=0x7f9c5c0037d0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 -- 172.21.15.49:0/3753425631 shutdown_connections 2024-02-26T06:27:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11eb920 0x55f2e11e9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11efb10 0x55f2e11eb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 --2- 172.21.15.49:0/3753425631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11deaf0 0x55f2e11ef240 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 -- 172.21.15.49:0/3753425631 >> 172.21.15.49:0/3753425631 conn(0x55f2e11a5480 msgr2=0x55f2e11e7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 -- 172.21.15.49:0/3753425631 shutdown_connections 2024-02-26T06:27:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.273+0000 7f9c72b94740 1 -- 172.21.15.49:0/3753425631 wait complete. 2024-02-26T06:27:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.274+0000 7f9c72b94740 1 -- start start 2024-02-26T06:27:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.274+0000 7f9c72b94740 1 Processor -- start 2024-02-26T06:27:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.275+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.275+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11eb920 0x55f2e11f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.275+0000 7f9c72b94740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11efb10 0x55f2e11f5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c72b94740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e11f5b00 con 0x55f2e11eb920 2024-02-26T06:27:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c72b94740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e11f5c70 con 0x55f2e11efb10 2024-02-26T06:27:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c72b94740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e12559c0 con 0x55f2e11deaf0 2024-02-26T06:27:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c72b94740 10 client.0 _ll_register_callbacks cb 0x7f9c5c0f3df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c67e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11eb920 0x55f2e11f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c6dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11efb10 0x55f2e11f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c6dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11efb10 0x55f2e11f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53158/0 (socket says 172.21.15.49:53158) 2024-02-26T06:27:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c6dc00640 1 -- 172.21.15.49:0/481958914 learned_addr learned my addr 172.21.15.49:0/481958914 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.276+0000 7f9c6d200640 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c67e00640 1 -- 172.21.15.49:0/481958914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 msgr2=0x55f2e11f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c67e00640 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c67e00640 1 -- 172.21.15.49:0/481958914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11efb10 msgr2=0x55f2e11f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c67e00640 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11efb10 0x55f2e11f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c67e00640 1 -- 172.21.15.49:0/481958914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2e1142c20 con 0x55f2e11eb920 2024-02-26T06:27:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c6d200640 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c6dc00640 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11efb10 0x55f2e11f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c67e00640 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11eb920 0x55f2e11f4ee0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55f2e12021e0 tx=0x7f9c54004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb94e5834aa5d70 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.277+0000 7f9c4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:27:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.278+0000 7f9c4ea00640 1 -- 172.21.15.49:0/481958914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54018040 con 0x55f2e11eb920 2024-02-26T06:27:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.278+0000 7f9c4ea00640 1 -- 172.21.15.49:0/481958914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9c5401d5a0 con 0x55f2e11eb920 2024-02-26T06:27:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.278+0000 7f9c72b94740 1 -- 172.21.15.49:0/481958914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2e1169d00 con 0x55f2e11eb920 2024-02-26T06:27:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.278+0000 7f9c4ea00640 1 -- 172.21.15.49:0/481958914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c54013ea0 con 0x55f2e11eb920 2024-02-26T06:27:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.279+0000 7f9c72b94740 10 client.16074 fetch_fsmap learned FSMap version 32 2024-02-26T06:27:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.279+0000 7f9c72b94740 1 -- 172.21.15.49:0/481958914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f2e1105290 con 0x55f2e11eb920 2024-02-26T06:27:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.279+0000 7f9c4ea00640 1 -- 172.21.15.49:0/481958914 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f2e1169d00 con 0x55f2e11eb920 2024-02-26T06:27:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.279+0000 7f9c4ea00640 1 -- 172.21.15.49:0/481958914 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9c54026450 con 0x55f2e11eb920 2024-02-26T06:27:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.280+0000 7f9c72b94740 10 client.16074 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T06:27:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.280+0000 7f9c72b94740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.280+0000 7f9c72b94740 20 client.16074 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.280+0000 7f9c72b94740 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f2e1257040 0x55f2e1259430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.280+0000 7f9c72b94740 4 client.16074 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:27:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.280+0000 7f9c72b94740 1 -- 172.21.15.49:0/481958914 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2e0e44600 con 0x55f2e1257040 2024-02-26T06:27:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.281+0000 7f9c6d200640 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f2e1257040 0x55f2e1259430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.282+0000 7f9c4ea00640 10 client.16074 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.282+0000 7f9c6d200640 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f2e1257040 0x55f2e1259430 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb718ee5cafa2a0 server_cookie=b3d496954484cd5e in_seq=0 out_seq=0 2024-02-26T06:27:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.296+0000 7f9c4ea00640 1 -- 172.21.15.49:0/481958914 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2e0e44600 con 0x55f2e1257040 2024-02-26T06:27:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.296+0000 7f9c4ea00640 10 client.16074 handle_command_reply: tid=0 2024-02-26T06:27:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 client.16074 shutdown 2024-02-26T06:27:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 -- 172.21.15.49:0/481958914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11eb920 msgr2=0x55f2e11f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11eb920 0x55f2e11f4ee0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55f2e12021e0 tx=0x7f9c54004410 comp rx=0 tx=0).stop 2024-02-26T06:27:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 -- 172.21.15.49:0/481958914 shutdown_connections 2024-02-26T06:27:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f2e1257040 0x55f2e1259430 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c4ea00640 0 client.16074 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f2e11efb10 0x55f2e11f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f2e11eb920 0x55f2e11f4ee0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 --2- 172.21.15.49:0/481958914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f2e11deaf0 0x55f2e11f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 -- 172.21.15.49:0/481958914 >> 172.21.15.49:0/481958914 conn(0x55f2e11a5480 msgr2=0x55f2e11d7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.297+0000 7f9c72b94740 1 -- 172.21.15.49:0/481958914 shutdown_connections 2024-02-26T06:27:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.298+0000 7f9c72b94740 1 -- 172.21.15.49:0/481958914 wait complete. 2024-02-26T06:27:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.298+0000 7f9c72b94740 20 client.16074 trim_cache size 0 max 16384 2024-02-26T06:27:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:27:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:27:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:27:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:27:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.299+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c500a2300 msgr2=0x7f9c500a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.299+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c500a2300 0x7f9c500a47c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005fd0 tx=0x7f9c60009500 comp rx=0 tx=0).stop 2024-02-26T06:27:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.299+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2991156170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c680ff250 msgr2=0x7f9c68100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.299+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c680ff250 0x7f9c68100cc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c005940 tx=0x7f9c5c02c670 comp rx=0 tx=0).stop 2024-02-26T06:27:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.300+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2991156170 shutdown_connections 2024-02-26T06:27:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.300+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c500a2300 0x7f9c500a47c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.300+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c68107260 0x7f9c68101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.300+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c680ffc10 0x7f9c68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.300+0000 7f9c6fc00640 1 --2- 172.21.15.49:0/2991156170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c680ff250 0x7f9c68100cc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.300+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2991156170 >> 172.21.15.49:0/2991156170 conn(0x7f9c68076210 msgr2=0x7f9c680fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.300+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2991156170 shutdown_connections 2024-02-26T06:27:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:12.300+0000 7f9c6fc00640 1 -- 172.21.15.49:0/2991156170 wait complete. 2024-02-26T06:27:12.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:12 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:27:12.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:12 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:12.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:12 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:12.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:12 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:12 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:27:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:12 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:12 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:12 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:12 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:27:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:12 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:12 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:12 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: pgmap v117: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.39k op/s 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:13 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:13.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:13.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: pgmap v117: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.39k op/s 2024-02-26T06:27:13.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:13 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:13.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:13.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:13.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:13.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:13.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: pgmap v117: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.39k op/s 2024-02-26T06:27:13.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:13.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:13 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:15.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:15 smithi049 ceph-mon[31696]: pgmap v118: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.28k op/s 2024-02-26T06:27:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:15 smithi080 ceph-mon[36206]: pgmap v118: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.28k op/s 2024-02-26T06:27:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:15 smithi115 ceph-mon[36426]: pgmap v118: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.28k op/s 2024-02-26T06:27:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 205.0 s, 2209.2 tps, lat 45.034 ms stddev 53.315 2024-02-26T06:27:17.390 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:17 smithi115 ceph-mon[36426]: pgmap v119: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-02-26T06:27:17.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:17 smithi049 ceph-mon[31696]: pgmap v119: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-02-26T06:27:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:17 smithi080 ceph-mon[36206]: pgmap v119: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-02-26T06:27:19.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:19 smithi049 ceph-mon[31696]: pgmap v120: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.3 MiB/s wr, 646 op/s 2024-02-26T06:27:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:19 smithi080 ceph-mon[36206]: pgmap v120: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.3 MiB/s wr, 646 op/s 2024-02-26T06:27:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:19 smithi115 ceph-mon[36426]: pgmap v120: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.3 MiB/s wr, 646 op/s 2024-02-26T06:27:20.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:20 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:20.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:20 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:20.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:20 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 210.0 s, 2260.2 tps, lat 44.597 ms stddev 48.580 2024-02-26T06:27:21.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:21 smithi049 ceph-mon[31696]: pgmap v121: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.6 MiB/s wr, 831 op/s 2024-02-26T06:27:21.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:21 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:27:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:21 smithi080 ceph-mon[36206]: pgmap v121: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.6 MiB/s wr, 831 op/s 2024-02-26T06:27:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:21 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:27:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:21 smithi115 ceph-mon[36426]: pgmap v121: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.6 MiB/s wr, 831 op/s 2024-02-26T06:27:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:21 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:27:22.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:22 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:27:22.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:22 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:27:22.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:22 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:27:22.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:22 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:27:22.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:22 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:27:22.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:22 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:27:23.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:23 smithi049 ceph-mon[31696]: pgmap v122: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.8 MiB/s wr, 843 op/s 2024-02-26T06:27:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:23 smithi080 ceph-mon[36206]: pgmap v122: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.8 MiB/s wr, 843 op/s 2024-02-26T06:27:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:23 smithi115 ceph-mon[36426]: pgmap v122: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.8 MiB/s wr, 843 op/s 2024-02-26T06:27:25.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:25 smithi049 ceph-mon[31696]: pgmap v123: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.1 MiB/s wr, 694 op/s 2024-02-26T06:27:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.490+0000 7fe1e5000640 1 -- 172.21.15.49:0/3492349923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e01013a0 msgr2=0x7fe1e0101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.490+0000 7fe1e5000640 1 --2- 172.21.15.49:0/3492349923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e01013a0 0x7fe1e0101780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009900 tx=0x7fe1d002bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.490+0000 7fe1e5000640 1 -- 172.21.15.49:0/3492349923 shutdown_connections 2024-02-26T06:27:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.490+0000 7fe1e5000640 1 --2- 172.21.15.49:0/3492349923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e0106470 0x7fe1e010e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.490+0000 7fe1e5000640 1 --2- 172.21.15.49:0/3492349923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e0101d50 0x7fe1e0105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.490+0000 7fe1e5000640 1 --2- 172.21.15.49:0/3492349923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e01013a0 0x7fe1e0101780 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.490+0000 7fe1e5000640 1 -- 172.21.15.49:0/3492349923 >> 172.21.15.49:0/3492349923 conn(0x7fe1e00fcc40 msgr2=0x7fe1e00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.491+0000 7fe1e5000640 1 -- 172.21.15.49:0/3492349923 shutdown_connections 2024-02-26T06:27:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.491+0000 7fe1e5000640 1 -- 172.21.15.49:0/3492349923 wait complete. 2024-02-26T06:27:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.493+0000 7fe1e5000640 1 Processor -- start 2024-02-26T06:27:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.493+0000 7fe1e5000640 1 -- start start 2024-02-26T06:27:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.494+0000 7fe1e5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e01013a0 0x7fe1e010dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.494+0000 7fe1e5000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e0106470 0x7fe1e010e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.494+0000 7fe1e5000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010aca0 0x7fe1e010b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.494+0000 7fe1e5000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e010b8e0 con 0x7fe1e0106470 2024-02-26T06:27:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.494+0000 7fe1e5000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e01aa5b0 con 0x7fe1e01013a0 2024-02-26T06:27:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.494+0000 7fe1e5000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e01aa6f0 con 0x7fe1e010aca0 2024-02-26T06:27:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.494+0000 7fe1de800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010aca0 0x7fe1e010b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.494+0000 7fe1de800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010aca0 0x7fe1e010b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46216/0 (socket says 172.21.15.49:46216) 2024-02-26T06:27:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.495+0000 7fe1de800640 1 -- 172.21.15.49:0/970951771 learned_addr learned my addr 172.21.15.49:0/970951771 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.495+0000 7fe1dd400640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e0106470 0x7fe1e010e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.495+0000 7fe1dde00640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e01013a0 0x7fe1e010dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.495+0000 7fe1de800640 1 -- 172.21.15.49:0/970951771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e01013a0 msgr2=0x7fe1e010dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.495+0000 7fe1de800640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e01013a0 0x7fe1e010dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.495+0000 7fe1de800640 1 -- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e0106470 msgr2=0x7fe1e010e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.495+0000 7fe1de800640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e0106470 0x7fe1e010e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.495+0000 7fe1de800640 1 -- 172.21.15.49:0/970951771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0009560 con 0x7fe1e010aca0 2024-02-26T06:27:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.496+0000 7fe1de800640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010aca0 0x7fe1e010b150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800db20 tx=0x7fe1d800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.497+0000 7fe1d6a00640 1 -- 172.21.15.49:0/970951771 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800bea0 con 0x7fe1e010aca0 2024-02-26T06:27:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.497+0000 7fe1e5000640 1 -- 172.21.15.49:0/970951771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e01aa890 con 0x7fe1e010aca0 2024-02-26T06:27:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.497+0000 7fe1d6a00640 1 -- 172.21.15.49:0/970951771 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1d8003e50 con 0x7fe1e010aca0 2024-02-26T06:27:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.497+0000 7fe1d6a00640 1 -- 172.21.15.49:0/970951771 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80106c0 con 0x7fe1e010aca0 2024-02-26T06:27:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.497+0000 7fe1e5000640 1 -- 172.21.15.49:0/970951771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e01aada0 con 0x7fe1e010aca0 2024-02-26T06:27:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.503+0000 7fe1e7f9d740 1 Processor -- start 2024-02-26T06:27:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.503+0000 7fe1d6a00640 1 -- 172.21.15.49:0/970951771 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe1d80042d0 con 0x7fe1e010aca0 2024-02-26T06:27:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.504+0000 7fe1e7f9d740 1 -- start start 2024-02-26T06:27:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.505+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c3b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.505+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 0x561472c3c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.505+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.506+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472bf75f0 con 0x561472c3bd80 2024-02-26T06:27:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.506+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472ba9890 con 0x561472c3caf0 2024-02-26T06:27:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.506+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472ba6dc0 con 0x561472c45300 2024-02-26T06:27:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.506+0000 7fe1dd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c3b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c3b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:40498/0 (socket says 172.21.15.49:40498) 2024-02-26T06:27:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dd400640 1 -- 172.21.15.49:0/1133788323 learned_addr learned my addr 172.21.15.49:0/1133788323 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dd400640 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dde00640 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 0x561472c3c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dd400640 1 -- 172.21.15.49:0/1133788323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 msgr2=0x561472c3b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dd400640 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c3b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dd400640 1 -- 172.21.15.49:0/1133788323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 msgr2=0x561472c3c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dd400640 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 0x561472c3c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.507+0000 7fe1dd400640 1 -- 172.21.15.49:0/1133788323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561472b23ad0 con 0x561472c45300 2024-02-26T06:27:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.508+0000 7fe1dd400640 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c476f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc004630 tx=0x7fe1cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=550b93000192dbd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.508+0000 7fe1d6a00640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe1c40a2350 0x7fe1c40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.508+0000 7fe1cb400640 1 -- 172.21.15.49:0/1133788323 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc011780 con 0x561472c45300 2024-02-26T06:27:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.508+0000 7fe1cb400640 1 -- 172.21.15.49:0/1133788323 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1cc01a020 con 0x561472c45300 2024-02-26T06:27:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.508+0000 7fe1cb400640 1 -- 172.21.15.49:0/1133788323 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0129c0 con 0x561472c45300 2024-02-26T06:27:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.508+0000 7fe1d6a00640 1 -- 172.21.15.49:0/970951771 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe1d80cde30 con 0x7fe1e010aca0 2024-02-26T06:27:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.509+0000 7fe1dde00640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe1c40a2350 0x7fe1c40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.510+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1133788323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 msgr2=0x561472c476f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.510+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c476f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc004630 tx=0x7fe1cc004730 comp rx=0 tx=0).stop 2024-02-26T06:27:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.511+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1133788323 shutdown_connections 2024-02-26T06:27:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.511+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c476f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.511+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 0x561472c3c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.511+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1133788323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c3b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.511+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1133788323 >> 172.21.15.49:0/1133788323 conn(0x561472c03480 msgr2=0x561472c36a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.511+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1133788323 shutdown_connections 2024-02-26T06:27:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.511+0000 7fe1dde00640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe1c40a2350 0x7fe1c40a4810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0006010 tx=0x7fe1d0005a20 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.511+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1133788323 wait complete. 2024-02-26T06:27:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.512+0000 7fe1e7f9d740 1 -- start start 2024-02-26T06:27:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.512+0000 7fe1e7f9d740 1 Processor -- start 2024-02-26T06:27:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 0x561472c423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c42920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c42e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472ba25a0 con 0x561472c3bd80 2024-02-26T06:27:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472b47b20 con 0x561472c3caf0 2024-02-26T06:27:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472b477c0 con 0x561472c45300 2024-02-26T06:27:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1e7f9d740 10 client.0 _ll_register_callbacks cb 0x561472b5b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1dd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c42920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1dd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c42920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:40508/0 (socket says 172.21.15.49:40508) 2024-02-26T06:27:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1dd400640 1 -- 172.21.15.49:0/3413106843 learned_addr learned my addr 172.21.15.49:0/3413106843 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.513+0000 7fe1de800640 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c42e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.514+0000 7fe1dd400640 1 -- 172.21.15.49:0/3413106843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 msgr2=0x561472c42e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.514+0000 7fe1dd400640 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c42e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.514+0000 7fe1dd400640 1 -- 172.21.15.49:0/3413106843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 msgr2=0x561472c423e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.515+0000 7fe1dd400640 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 0x561472c423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.515+0000 7fe1dd400640 1 -- 172.21.15.49:0/3413106843 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc014600 con 0x561472c3caf0 2024-02-26T06:27:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.517+0000 7fe1dd400640 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c42920 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0126a0 tx=0x7fe1cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b02ce214e4acdaf9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.517+0000 7fe1bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.517+0000 7fe1bfe00640 1 -- 172.21.15.49:0/3413106843 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc011b60 con 0x561472c3caf0 2024-02-26T06:27:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.517+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/3413106843 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561472b74ca0 con 0x561472c3caf0 2024-02-26T06:27:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.517+0000 7fe1bfe00640 1 -- 172.21.15.49:0/3413106843 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe1cc000e40 con 0x561472c3caf0 2024-02-26T06:27:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.518+0000 7fe1bfe00640 1 -- 172.21.15.49:0/3413106843 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1cc014490 con 0x561472c3caf0 2024-02-26T06:27:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.518+0000 7fe1e7f9d740 10 client.26447 fetch_fsmap learned FSMap version 34 2024-02-26T06:27:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.518+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/3413106843 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561472ba0c20 con 0x561472c3caf0 2024-02-26T06:27:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.518+0000 7fe1bfe00640 1 -- 172.21.15.49:0/3413106843 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561472b74ca0 con 0x561472c3caf0 2024-02-26T06:27:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.519+0000 7fe1bfe00640 1 -- 172.21.15.49:0/3413106843 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 34) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe1cc033630 con 0x561472c3caf0 2024-02-26T06:27:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.519+0000 7fe1e7f9d740 10 client.26447 fetch_fsmap finished waiting for FSMap version 34 2024-02-26T06:27:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.519+0000 7fe1e7f9d740 10 client.26447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.519+0000 7fe1e7f9d740 20 client.26447 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.519+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561472c55270 0x561472c57680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.519+0000 7fe1e7f9d740 4 client.26447 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.519+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/3413106843 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561472b98cb0 con 0x561472c55270 2024-02-26T06:27:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.520+0000 7fe1dde00640 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561472c55270 0x561472c57680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.520+0000 7fe1dde00640 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561472c55270 0x561472c57680 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156000ae4e1a58e server_cookie=3fec7d707e90ee6 in_seq=0 out_seq=0 2024-02-26T06:27:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.520+0000 7fe1bfe00640 10 client.26447 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.521+0000 7fe1bfe00640 1 -- 172.21.15.49:0/3413106843 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561472b98cb0 con 0x561472c55270 2024-02-26T06:27:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.521+0000 7fe1bfe00640 10 client.26447 handle_command_reply: tid=0 2024-02-26T06:27:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.522+0000 7fe1e7f9d740 1 client.26447 shutdown 2024-02-26T06:27:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.522+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/3413106843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 msgr2=0x561472c42920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.523+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c42920 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0126a0 tx=0x7fe1cc0046c0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.523+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/3413106843 shutdown_connections 2024-02-26T06:27:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.523+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561472c55270 0x561472c57680 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.523+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c3caf0 0x561472c42920 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.524+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c45300 0x561472c42e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.524+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/3413106843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3bd80 0x561472c423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.524+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/3413106843 >> 172.21.15.49:0/3413106843 conn(0x561472c03480 msgr2=0x561472c387c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.524+0000 7fe1bfe00640 0 client.26447 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.524+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/3413106843 shutdown_connections 2024-02-26T06:27:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.525+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/3413106843 wait complete. 2024-02-26T06:27:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.525+0000 7fe1e7f9d740 20 client.26447 trim_cache size 0 max 16384 2024-02-26T06:27:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.556+0000 7fe1e7f9d740 1 Processor -- start 2024-02-26T06:27:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.556+0000 7fe1e7f9d740 1 -- start start 2024-02-26T06:27:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.557+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c44b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.557+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 0x561472c40cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.557+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c41210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.557+0000 7fe1dde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c44b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.558+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614728eae60 con 0x561472c3caf0 2024-02-26T06:27:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.558+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472c28670 con 0x561472c45300 2024-02-26T06:27:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.558+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472c55070 con 0x561472c3bd80 2024-02-26T06:27:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.558+0000 7fe1de800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c41210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.558+0000 7fe1de800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c41210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:40516/0 (socket says 172.21.15.49:40516) 2024-02-26T06:27:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.558+0000 7fe1de800640 1 -- 172.21.15.49:0/1125864964 learned_addr learned my addr 172.21.15.49:0/1125864964 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.558+0000 7fe1dd400640 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 0x561472c40cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.559+0000 7fe1de800640 1 -- 172.21.15.49:0/1125864964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 msgr2=0x561472c44b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.559+0000 7fe1de800640 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c44b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.559+0000 7fe1de800640 1 -- 172.21.15.49:0/1125864964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 msgr2=0x561472c40cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.559+0000 7fe1de800640 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 0x561472c40cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.559+0000 7fe1de800640 1 -- 172.21.15.49:0/1125864964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561472b3d500 con 0x561472c45300 2024-02-26T06:27:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.559+0000 7fe1dd400640 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 0x561472c40cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.560+0000 7fe1de800640 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c41210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800b760 tx=0x7fe1d8085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55cf0476bd218e04 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.560+0000 7fe1caa00640 1 -- 172.21.15.49:0/1125864964 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d809e920 con 0x561472c45300 2024-02-26T06:27:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.560+0000 7fe1caa00640 1 -- 172.21.15.49:0/1125864964 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1d80a2f70 con 0x561472c45300 2024-02-26T06:27:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.560+0000 7fe1caa00640 1 -- 172.21.15.49:0/1125864964 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8096e40 con 0x561472c45300 2024-02-26T06:27:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1125864964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 msgr2=0x561472c41210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c41210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800b760 tx=0x7fe1d8085040 comp rx=0 tx=0).stop 2024-02-26T06:27:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1125864964 shutdown_connections 2024-02-26T06:27:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c41210 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 0x561472c40cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1125864964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c44b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1125864964 >> 172.21.15.49:0/1125864964 conn(0x561472c03480 msgr2=0x561472c365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1125864964 shutdown_connections 2024-02-26T06:27:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.562+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1125864964 wait complete. 2024-02-26T06:27:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.563+0000 7fe1e7f9d740 1 -- start start 2024-02-26T06:27:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.563+0000 7fe1e7f9d740 1 Processor -- start 2024-02-26T06:27:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c4aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 0x561472c4aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1e7f9d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472c4ca50 con 0x561472c3caf0 2024-02-26T06:27:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472cab780 con 0x561472c45300 2024-02-26T06:27:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1e7f9d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472cab8c0 con 0x561472c3bd80 2024-02-26T06:27:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1e7f9d740 10 client.0 _ll_register_callbacks cb 0x7fe1cc00ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1dde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c4aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1dde00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c4aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46262/0 (socket says 172.21.15.49:46262) 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.565+0000 7fe1dde00640 1 -- 172.21.15.49:0/1153713370 learned_addr learned my addr 172.21.15.49:0/1153713370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.566+0000 7fe1de800640 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.566+0000 7fe1dde00640 1 -- 172.21.15.49:0/1153713370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 msgr2=0x561472c4b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.566+0000 7fe1dde00640 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c4b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.566+0000 7fe1dde00640 1 -- 172.21.15.49:0/1153713370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 msgr2=0x561472c4aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.566+0000 7fe1dde00640 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 0x561472c4aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.566+0000 7fe1dde00640 1 -- 172.21.15.49:0/1153713370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e01aada0 con 0x561472c3bd80 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.567+0000 7fe1dde00640 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c4aab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009230 tx=0x7fe1d00049e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=561399e5696c4bbb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.567+0000 7fe1bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.567+0000 7fe1bfe00640 1 -- 172.21.15.49:0/1153713370 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d002dea0 con 0x561472c3bd80 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.568+0000 7fe1bfe00640 1 -- 172.21.15.49:0/1153713370 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe1d00035f0 con 0x561472c3bd80 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.568+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1153713370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561472bc7d00 con 0x561472c3bd80 2024-02-26T06:27:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.568+0000 7fe1bfe00640 1 -- 172.21.15.49:0/1153713370 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1d0004510 con 0x561472c3bd80 2024-02-26T06:27:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.568+0000 7fe1e7f9d740 10 client.26473 fetch_fsmap learned FSMap version 34 2024-02-26T06:27:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.568+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1153713370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561472b63290 con 0x561472c3bd80 2024-02-26T06:27:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.570+0000 7fe1bfe00640 1 -- 172.21.15.49:0/1153713370 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561472bc7d00 con 0x561472c3bd80 2024-02-26T06:27:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.570+0000 7fe1bfe00640 1 -- 172.21.15.49:0/1153713370 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe1d0046040 con 0x561472c3bd80 2024-02-26T06:27:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.571+0000 7fe1e7f9d740 10 client.26473 fetch_fsmap finished waiting for FSMap version 34 2024-02-26T06:27:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.571+0000 7fe1e7f9d740 10 client.26473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.571+0000 7fe1e7f9d740 20 client.26473 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.571+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561472cad0a0 0x561472caf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.571+0000 7fe1e7f9d740 4 client.26473 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:27:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.571+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1153713370 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5614728a2600 con 0x561472cad0a0 2024-02-26T06:27:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.571+0000 7fe1dd400640 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561472cad0a0 0x561472caf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.572+0000 7fe1dd400640 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561472cad0a0 0x561472caf490 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f1ead0a519ea39 server_cookie=3f4d81327584f454 in_seq=0 out_seq=0 2024-02-26T06:27:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.573+0000 7fe1bfe00640 10 client.26473 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:25.579 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:25 smithi080 ceph-mon[36206]: pgmap v123: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.1 MiB/s wr, 694 op/s 2024-02-26T06:27:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.584+0000 7fe1bfe00640 1 -- 172.21.15.49:0/1153713370 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5614728a2600 con 0x561472cad0a0 2024-02-26T06:27:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1bfe00640 10 client.26473 handle_command_reply: tid=0 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 client.26473 shutdown 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1153713370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 msgr2=0x561472c4aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c4aab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009230 tx=0x7fe1d00049e0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1153713370 shutdown_connections 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561472cad0a0 0x561472caf490 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561472c45300 0x561472c4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561472c3caf0 0x561472c4aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 --2- 172.21.15.49:0/1153713370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561472c3bd80 0x561472c4aab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1bfe00640 0 client.26473 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.585+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1153713370 >> 172.21.15.49:0/1153713370 conn(0x561472c03480 msgr2=0x561472c365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.586+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1153713370 shutdown_connections 2024-02-26T06:27:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.586+0000 7fe1e7f9d740 1 -- 172.21.15.49:0/1153713370 wait complete. 2024-02-26T06:27:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.586+0000 7fe1e7f9d740 20 client.26473 trim_cache size 0 max 16384 2024-02-26T06:27:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:27:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:27:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:27:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 -- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe1c40a2350 msgr2=0x7fe1c40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe1c40a2350 0x7fe1c40a4810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0006010 tx=0x7fe1d0005a20 comp rx=0 tx=0).stop 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 -- 172.21.15.49:0/970951771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010aca0 msgr2=0x7fe1e010b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010aca0 0x7fe1e010b150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800db20 tx=0x7fe1d800b260 comp rx=0 tx=0).stop 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 -- 172.21.15.49:0/970951771 shutdown_connections 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe1c40a2350 0x7fe1c40a4810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010aca0 0x7fe1e010b150 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e0106470 0x7fe1e010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 --2- 172.21.15.49:0/970951771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e01013a0 0x7fe1e010dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.588+0000 7fe1e5000640 1 -- 172.21.15.49:0/970951771 >> 172.21.15.49:0/970951771 conn(0x7fe1e00fcc40 msgr2=0x7fe1e00fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.589+0000 7fe1e5000640 1 -- 172.21.15.49:0/970951771 shutdown_connections 2024-02-26T06:27:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:25.589+0000 7fe1e5000640 1 -- 172.21.15.49:0/970951771 wait complete. 2024-02-26T06:27:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:25 smithi115 ceph-mon[36426]: pgmap v123: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.1 MiB/s wr, 694 op/s 2024-02-26T06:27:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 215.0 s, 2357.2 tps, lat 42.362 ms stddev 47.324 2024-02-26T06:27:26.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:26 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:26.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:26 smithi080 ceph-mon[36206]: pgmap v124: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 9.7 MiB/s wr, 972 op/s 2024-02-26T06:27:26.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:26 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:26.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:26 smithi115 ceph-mon[36426]: pgmap v124: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 9.7 MiB/s wr, 972 op/s 2024-02-26T06:27:26.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:26 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:26.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:26 smithi049 ceph-mon[31696]: pgmap v124: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 9.7 MiB/s wr, 972 op/s 2024-02-26T06:27:28.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:28 smithi080 ceph-mon[36206]: pgmap v125: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 611 op/s 2024-02-26T06:27:28.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:28 smithi115 ceph-mon[36426]: pgmap v125: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 611 op/s 2024-02-26T06:27:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:28 smithi049 ceph-mon[31696]: pgmap v125: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 611 op/s 2024-02-26T06:27:30.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:30 smithi080 ceph-mon[36206]: pgmap v126: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.4 MiB/s wr, 837 op/s 2024-02-26T06:27:30.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:30 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:30 smithi115 ceph-mon[36426]: pgmap v126: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.4 MiB/s wr, 837 op/s 2024-02-26T06:27:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:30 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:30.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:30 smithi049 ceph-mon[31696]: pgmap v126: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.4 MiB/s wr, 837 op/s 2024-02-26T06:27:30.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:30 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:27:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 220.0 s, 2746.4 tps, lat 36.230 ms stddev 42.896 2024-02-26T06:27:32.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:32 smithi080 ceph-mon[36206]: pgmap v127: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 758 op/s 2024-02-26T06:27:32.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:32 smithi115 ceph-mon[36426]: pgmap v127: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 758 op/s 2024-02-26T06:27:32.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:32 smithi049 ceph-mon[31696]: pgmap v127: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 758 op/s 2024-02-26T06:27:34.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:34 smithi080 ceph-mon[36206]: pgmap v128: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-02-26T06:27:34.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:34 smithi115 ceph-mon[36426]: pgmap v128: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-02-26T06:27:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:34 smithi049 ceph-mon[31696]: pgmap v128: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-02-26T06:27:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 225.0 s, 2753.4 tps, lat 36.272 ms stddev 44.440 2024-02-26T06:27:36.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:36 smithi080 ceph-mon[36206]: pgmap v129: 97 pgs: 97 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-02-26T06:27:36.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:36 smithi115 ceph-mon[36426]: pgmap v129: 97 pgs: 97 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-02-26T06:27:36.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:36 smithi049 ceph-mon[31696]: pgmap v129: 97 pgs: 97 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-02-26T06:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.800+0000 7f1f3e800640 1 -- 172.21.15.49:0/953205790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f38103d90 msgr2=0x7f1f38107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.800+0000 7f1f3e800640 1 --2- 172.21.15.49:0/953205790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f38103d90 0x7f1f38107dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009930 tx=0x7f1f2402bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.801+0000 7f1f3e800640 1 -- 172.21.15.49:0/953205790 shutdown_connections 2024-02-26T06:27:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.801+0000 7f1f3e800640 1 --2- 172.21.15.49:0/953205790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f38103d90 0x7f1f38107dd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.801+0000 7f1f3e800640 1 --2- 172.21.15.49:0/953205790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f381084b0 0x7f1f38110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.801+0000 7f1f3e800640 1 --2- 172.21.15.49:0/953205790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f381033e0 0x7f1f381037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.801+0000 7f1f3e800640 1 -- 172.21.15.49:0/953205790 >> 172.21.15.49:0/953205790 conn(0x7f1f380fec60 msgr2=0x7f1f38101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.801+0000 7f1f3e800640 1 -- 172.21.15.49:0/953205790 shutdown_connections 2024-02-26T06:27:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.801+0000 7f1f3e800640 1 -- 172.21.15.49:0/953205790 wait complete. 2024-02-26T06:27:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.803+0000 7f1f3e800640 1 Processor -- start 2024-02-26T06:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.804+0000 7f1f3e800640 1 -- start start 2024-02-26T06:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.804+0000 7f1f3e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f381033e0 0x7f1f3810ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.804+0000 7f1f3e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f381084b0 0x7f1f3810e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.804+0000 7f1f3e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f3810e690 0x7f1f3810eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.804+0000 7f1f3e800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381aa1c0 con 0x7f1f381084b0 2024-02-26T06:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.804+0000 7f1f3e800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38112a60 con 0x7f1f3810e690 2024-02-26T06:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.804+0000 7f1f3e800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38112c20 con 0x7f1f381033e0 2024-02-26T06:27:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f381033e0 0x7f1f3810ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f381033e0 0x7f1f3810ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53020/0 (socket says 172.21.15.49:53020) 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3d400640 1 -- 172.21.15.49:0/3301733022 learned_addr learned my addr 172.21.15.49:0/3301733022 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3de00640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f3810e690 0x7f1f3810eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f381084b0 0x7f1f3810e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3de00640 1 -- 172.21.15.49:0/3301733022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f381033e0 msgr2=0x7f1f3810ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3de00640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f381033e0 0x7f1f3810ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3de00640 1 -- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f381084b0 msgr2=0x7f1f3810e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3de00640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f381084b0 0x7f1f3810e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.805+0000 7f1f3de00640 1 -- 172.21.15.49:0/3301733022 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f24009560 con 0x7f1f3810e690 2024-02-26T06:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.806+0000 7f1f3de00640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f3810e690 0x7f1f3810eb10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000da10 tx=0x7f1f3000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.806+0000 7f1f36000640 1 -- 172.21.15.49:0/3301733022 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3000bd70 con 0x7f1f3810e690 2024-02-26T06:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.806+0000 7f1f36000640 1 -- 172.21.15.49:0/3301733022 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f30003e50 con 0x7f1f3810e690 2024-02-26T06:27:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.806+0000 7f1f36000640 1 -- 172.21.15.49:0/3301733022 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f300106c0 con 0x7f1f3810e690 2024-02-26T06:27:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.807+0000 7f1f3e800640 1 -- 172.21.15.49:0/3301733022 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f38112f00 con 0x7f1f3810e690 2024-02-26T06:27:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.807+0000 7f1f3e800640 1 -- 172.21.15.49:0/3301733022 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f38113390 con 0x7f1f3810e690 2024-02-26T06:27:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.812+0000 7f1f42dd3740 1 Processor -- start 2024-02-26T06:27:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.821+0000 7f1f36000640 1 -- 172.21.15.49:0/3301733022 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1f30004000 con 0x7f1f3810e690 2024-02-26T06:27:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.821+0000 7f1f42dd3740 1 -- start start 2024-02-26T06:27:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.822+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756612a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.822+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756613440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.822+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55875661c540 0x55875661e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.822+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587565ce5f0 con 0x55875661c540 2024-02-26T06:27:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.822+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558756580890 con 0x558756612fc0 2024-02-26T06:27:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.822+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875657ddc0 con 0x558756613d30 2024-02-26T06:27:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.823+0000 7f1f3ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756612a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.823+0000 7f1f3ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756612a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53026/0 (socket says 172.21.15.49:53026) 2024-02-26T06:27:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.823+0000 7f1f3ca00640 1 -- 172.21.15.49:0/148020886 learned_addr learned my addr 172.21.15.49:0/148020886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.823+0000 7f1f3d400640 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756613440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.823+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55875661c540 0x55875661e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.824+0000 7f1f3ca00640 1 -- 172.21.15.49:0/148020886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 msgr2=0x558756613440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.824+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756613440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.824+0000 7f1f3ca00640 1 -- 172.21.15.49:0/148020886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55875661c540 msgr2=0x55875661e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.824+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55875661c540 0x55875661e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.824+0000 7f1f3ca00640 1 -- 172.21.15.49:0/148020886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587564faad0 con 0x558756613d30 2024-02-26T06:27:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.825+0000 7f1f36000640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1f200a22f0 0x7f1f200a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.825+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756612a80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f24005ec0 tx=0x7f1f24005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545371ac134b3cd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.825+0000 7f1f36000640 1 -- 172.21.15.49:0/3301733022 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1f300ce3f0 con 0x7f1f3810e690 2024-02-26T06:27:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.826+0000 7f1f2a600640 1 -- 172.21.15.49:0/148020886 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2402cc10 con 0x558756613d30 2024-02-26T06:27:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.826+0000 7f1f2a600640 1 -- 172.21.15.49:0/148020886 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f24004510 con 0x558756613d30 2024-02-26T06:27:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.826+0000 7f1f2a600640 1 -- 172.21.15.49:0/148020886 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2402cc10 con 0x558756613d30 2024-02-26T06:27:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.826+0000 7f1f3d400640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1f200a22f0 0x7f1f200a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.827+0000 7f1f42dd3740 1 -- 172.21.15.49:0/148020886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 msgr2=0x558756612a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.827+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756612a80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f24005ec0 tx=0x7f1f24005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.827+0000 7f1f42dd3740 1 -- 172.21.15.49:0/148020886 shutdown_connections 2024-02-26T06:27:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.827+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756613440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.827+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55875661c540 0x55875661e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.827+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/148020886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756612a80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.827+0000 7f1f42dd3740 1 -- 172.21.15.49:0/148020886 >> 172.21.15.49:0/148020886 conn(0x5587565da480 msgr2=0x55875660dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.827+0000 7f1f42dd3740 1 -- 172.21.15.49:0/148020886 shutdown_connections 2024-02-26T06:27:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.828+0000 7f1f3d400640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1f200a22f0 0x7f1f200a47b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x558756614490 tx=0x7f1f2c00a380 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.829+0000 7f1f42dd3740 1 -- 172.21.15.49:0/148020886 wait complete. 2024-02-26T06:27:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.830+0000 7f1f42dd3740 1 -- start start 2024-02-26T06:27:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.830+0000 7f1f42dd3740 1 Processor -- start 2024-02-26T06:27:38.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:38 smithi080 ceph-mon[36206]: pgmap v130: 97 pgs: 97 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 639 op/s 2024-02-26T06:27:38.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:38 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:27:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.831+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 0x5587566195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.831+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756613d30 0x558756619af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.831+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55875661c540 0x55875661a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.831+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587565795a0 con 0x558756612fc0 2024-02-26T06:27:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.831+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875651eb20 con 0x558756613d30 2024-02-26T06:27:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.831+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875651e7c0 con 0x55875661c540 2024-02-26T06:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.831+0000 7f1f42dd3740 10 client.0 _ll_register_callbacks cb 0x558756532520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.832+0000 7f1f3ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756613d30 0x558756619af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.832+0000 7f1f3ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756613d30 0x558756619af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:37128/0 (socket says 172.21.15.49:37128) 2024-02-26T06:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.832+0000 7f1f3ca00640 1 -- 172.21.15.49:0/1554761746 learned_addr learned my addr 172.21.15.49:0/1554761746 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.832+0000 7f1f3de00640 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55875661c540 0x55875661a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.832+0000 7f1f3d400640 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 0x5587566195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f3de00640 1 -- 172.21.15.49:0/1554761746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756613d30 msgr2=0x558756619af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f3de00640 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756613d30 0x558756619af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f3de00640 1 -- 172.21.15.49:0/1554761746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 msgr2=0x5587566195b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f3de00640 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 0x5587566195b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f3de00640 1 -- 172.21.15.49:0/1554761746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f38110890 con 0x55875661c540 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f3de00640 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55875661c540 0x55875661a030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f300cbb20 tx=0x7f1f3009a810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8fe85ff18507e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1554761746 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f300b2070 con 0x55875661c540 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1554761746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55875654bca0 con 0x55875661c540 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.833+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1554761746 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1f300aee40 con 0x55875661c540 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1554761746 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f3009ef40 con 0x55875661c540 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f42dd3740 10 client.26482 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1554761746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558756577c20 con 0x55875661c540 2024-02-26T06:27:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1554761746 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55875654bca0 con 0x55875661c540 2024-02-26T06:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1554761746 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1f3001a070 con 0x55875661c540 2024-02-26T06:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f42dd3740 10 client.26482 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f42dd3740 10 client.26482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f42dd3740 20 client.26482 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55875662c360 0x55875662e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f42dd3740 4 client.26482 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.834+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1554761746 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55875656fcb0 con 0x55875662c360 2024-02-26T06:27:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.835+0000 7f1f3d400640 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55875662c360 0x55875662e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.837+0000 7f1f3d400640 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55875662c360 0x55875662e750 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737678f580265625 server_cookie=d72695d1efd531e2 in_seq=0 out_seq=0 2024-02-26T06:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.837+0000 7f1f1ea00640 10 client.26482 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.838+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1554761746 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55875656fcb0 con 0x55875662c360 2024-02-26T06:27:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.838+0000 7f1f1ea00640 10 client.26482 handle_command_reply: tid=0 2024-02-26T06:27:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.838+0000 7f1f42dd3740 1 client.26482 shutdown 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1554761746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55875661c540 msgr2=0x55875661a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55875661c540 0x55875661a030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f300cbb20 tx=0x7f1f3009a810 comp rx=0 tx=0).stop 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1554761746 shutdown_connections 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55875662c360 0x55875662e750 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756613d30 0x558756619af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f1ea00640 0 client.26482 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55875661c540 0x55875661a030 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1554761746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 0x5587566195b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1554761746 >> 172.21.15.49:0/1554761746 conn(0x5587565da480 msgr2=0x55875660fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1554761746 shutdown_connections 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1554761746 wait complete. 2024-02-26T06:27:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.839+0000 7f1f42dd3740 20 client.26482 trim_cache size 0 max 16384 2024-02-26T06:27:38.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:38 smithi115 ceph-mon[36426]: pgmap v130: 97 pgs: 97 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 639 op/s 2024-02-26T06:27:38.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:38 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:27:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.871+0000 7f1f42dd3740 1 Processor -- start 2024-02-26T06:27:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.871+0000 7f1f42dd3740 1 -- start start 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5587565da480 0x55875661bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 0x558756616830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756616d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587562c1e60 con 0x558756612fc0 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558756612580 con 0x5587565da480 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f42dd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875656fcb0 con 0x558756613d30 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f3de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756616d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f3de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756616d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53034/0 (socket says 172.21.15.49:53034) 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f3de00640 1 -- 172.21.15.49:0/3101756453 learned_addr learned my addr 172.21.15.49:0/3101756453 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 0x558756616830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.872+0000 7f1f3d400640 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5587565da480 0x55875661bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.873+0000 7f1f3de00640 1 -- 172.21.15.49:0/3101756453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5587565da480 msgr2=0x55875661bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.873+0000 7f1f3de00640 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5587565da480 0x55875661bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.873+0000 7f1f3de00640 1 -- 172.21.15.49:0/3101756453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 msgr2=0x558756616830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.873+0000 7f1f3de00640 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 0x558756616830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.873+0000 7f1f3de00640 1 -- 172.21.15.49:0/3101756453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558756514500 con 0x558756613d30 2024-02-26T06:27:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.874+0000 7f1f3d400640 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5587565da480 0x55875661bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.874+0000 7f1f3de00640 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756616d70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f30097250 tx=0x7f1f300044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94035337fc8e198e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.875+0000 7f1f29c00640 1 -- 172.21.15.49:0/3101756453 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f300b2070 con 0x558756613d30 2024-02-26T06:27:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.875+0000 7f1f29c00640 1 -- 172.21.15.49:0/3101756453 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f3009abc0 con 0x558756613d30 2024-02-26T06:27:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.875+0000 7f1f29c00640 1 -- 172.21.15.49:0/3101756453 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3009ef40 con 0x558756613d30 2024-02-26T06:27:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.876+0000 7f1f42dd3740 1 -- 172.21.15.49:0/3101756453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 msgr2=0x558756616d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.876+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756616d70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f30097250 tx=0x7f1f300044f0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.877+0000 7f1f42dd3740 1 -- 172.21.15.49:0/3101756453 shutdown_connections 2024-02-26T06:27:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.877+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756616d70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.877+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558756612fc0 0x558756616830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.877+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/3101756453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5587565da480 0x55875661bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.877+0000 7f1f42dd3740 1 -- 172.21.15.49:0/3101756453 >> 172.21.15.49:0/3101756453 conn(0x558756609140 msgr2=0x558756617e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.877+0000 7f1f42dd3740 1 -- 172.21.15.49:0/3101756453 shutdown_connections 2024-02-26T06:27:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.878+0000 7f1f42dd3740 1 -- 172.21.15.49:0/3101756453 wait complete. 2024-02-26T06:27:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.879+0000 7f1f42dd3740 1 -- start start 2024-02-26T06:27:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.880+0000 7f1f42dd3740 1 Processor -- start 2024-02-26T06:27:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.882+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5587565da480 0x558756622520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.882+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756622a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.882+0000 7f1f3d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5587565da480 0x558756622520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.882+0000 7f1f3d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5587565da480 0x558756622520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38138/0 (socket says 172.21.15.49:38138) 2024-02-26T06:27:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.883+0000 7f1f3ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756622a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.883+0000 7f1f3ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756622a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:37148/0 (socket says 172.21.15.49:37148) 2024-02-26T06:27:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.883+0000 7f1f42dd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756682840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.884+0000 7f1f3de00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756682840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.884+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558756682d80 con 0x5587565da480 2024-02-26T06:27:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.884+0000 7f1f3d400640 1 -- 172.21.15.49:0/1659516254 learned_addr learned my addr 172.21.15.49:0/1659516254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.884+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558756682ec0 con 0x558756612fc0 2024-02-26T06:27:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.884+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558756683000 con 0x558756613d30 2024-02-26T06:27:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.884+0000 7f1f42dd3740 10 client.0 _ll_register_callbacks cb 0x7f1f300092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.885+0000 7f1f3ca00640 1 -- 172.21.15.49:0/1659516254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 msgr2=0x558756682840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.885+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756682840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.885+0000 7f1f3ca00640 1 -- 172.21.15.49:0/1659516254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5587565da480 msgr2=0x558756622520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.885+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5587565da480 0x558756622520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.885+0000 7f1f3ca00640 1 -- 172.21.15.49:0/1659516254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c01b530 con 0x558756612fc0 2024-02-26T06:27:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.886+0000 7f1f3ca00640 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756622a60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1f2402c240 tx=0x7f1f240044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2220ae4616a0d88a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.886+0000 7f1f1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.887+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1659516254 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24053070 con 0x558756612fc0 2024-02-26T06:27:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.888+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1659516254 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1f24004870 con 0x558756612fc0 2024-02-26T06:27:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.888+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55875659ed00 con 0x558756612fc0 2024-02-26T06:27:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.889+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1659516254 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f24003800 con 0x558756612fc0 2024-02-26T06:27:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.891+0000 7f1f42dd3740 10 client.26471 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.891+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558756577c20 con 0x558756612fc0 2024-02-26T06:27:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.891+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1659516254 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55875659ed00 con 0x558756612fc0 2024-02-26T06:27:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.892+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1659516254 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1f24056020 con 0x558756612fc0 2024-02-26T06:27:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.892+0000 7f1f42dd3740 10 client.26471 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.892+0000 7f1f42dd3740 10 client.26471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.892+0000 7f1f42dd3740 20 client.26471 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.892+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55875661c540 0x558756686450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.892+0000 7f1f42dd3740 4 client.26471 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:27:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.892+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558756279600 con 0x55875661c540 2024-02-26T06:27:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.893+0000 7f1f3d400640 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55875661c540 0x558756686450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.893+0000 7f1f3d400640 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55875661c540 0x558756686450 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e06f24c9c00863f server_cookie=6ba423511b865f12 in_seq=0 out_seq=0 2024-02-26T06:27:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.893+0000 7f1f1ea00640 10 client.26471 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.905+0000 7f1f1ea00640 1 -- 172.21.15.49:0/1659516254 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558756279600 con 0x55875661c540 2024-02-26T06:27:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.905+0000 7f1f1ea00640 10 client.26471 handle_command_reply: tid=0 2024-02-26T06:27:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.905+0000 7f1f42dd3740 1 client.26471 shutdown 2024-02-26T06:27:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 msgr2=0x558756622a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756622a60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1f2402c240 tx=0x7f1f240044c0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:27:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:27:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:27:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:27:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 shutdown_connections 2024-02-26T06:27:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55875661c540 0x558756686450 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558756612fc0 0x558756622a60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558756613d30 0x558756682840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f42dd3740 1 --2- 172.21.15.49:0/1659516254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5587565da480 0x558756622520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f1ea00640 0 client.26471 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.908+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 >> 172.21.15.49:0/1659516254 conn(0x558756609140 msgr2=0x558756619550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.909+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 shutdown_connections 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.909+0000 7f1f42dd3740 1 -- 172.21.15.49:0/1659516254 wait complete. 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.910+0000 7f1f42dd3740 20 client.26471 trim_cache size 0 max 16384 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.911+0000 7f1f3e800640 1 -- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1f200a22f0 msgr2=0x7f1f200a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.911+0000 7f1f3e800640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1f200a22f0 0x7f1f200a47b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x558756614490 tx=0x7f1f2c00a380 comp rx=0 tx=0).stop 2024-02-26T06:27:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.911+0000 7f1f3e800640 1 -- 172.21.15.49:0/3301733022 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f3810e690 msgr2=0x7f1f3810eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.911+0000 7f1f3e800640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f3810e690 0x7f1f3810eb10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000da10 tx=0x7f1f3000ded0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.912+0000 7f1f3e800640 1 -- 172.21.15.49:0/3301733022 shutdown_connections 2024-02-26T06:27:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.912+0000 7f1f3e800640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1f200a22f0 0x7f1f200a47b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.912+0000 7f1f3e800640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f3810e690 0x7f1f3810eb10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.912+0000 7f1f3e800640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f381084b0 0x7f1f3810e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.912+0000 7f1f3e800640 1 --2- 172.21.15.49:0/3301733022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f381033e0 0x7f1f3810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.912+0000 7f1f3e800640 1 -- 172.21.15.49:0/3301733022 >> 172.21.15.49:0/3301733022 conn(0x7f1f380fec60 msgr2=0x7f1f38100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.913+0000 7f1f3e800640 1 -- 172.21.15.49:0/3301733022 shutdown_connections 2024-02-26T06:27:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:38.913+0000 7f1f3e800640 1 -- 172.21.15.49:0/3301733022 wait complete. 2024-02-26T06:27:38.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:38 smithi049 ceph-mon[31696]: pgmap v130: 97 pgs: 97 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 639 op/s 2024-02-26T06:27:38.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:38 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:27:40.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:40 smithi080 ceph-mon[36206]: pgmap v131: 97 pgs: 97 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.1 MiB/s wr, 793 op/s 2024-02-26T06:27:40.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:40 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:27:40.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:40 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:27:40.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:40 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:27:40.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:40 smithi115 ceph-mon[36426]: pgmap v131: 97 pgs: 97 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.1 MiB/s wr, 793 op/s 2024-02-26T06:27:40.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:40 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:27:40.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:40 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:27:40.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:40 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:27:40.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:40 smithi049 ceph-mon[31696]: pgmap v131: 97 pgs: 97 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.1 MiB/s wr, 793 op/s 2024-02-26T06:27:40.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:40 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:27:40.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:40 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:27:40.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:40 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:27:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 230.0 s, 2602.9 tps, lat 38.629 ms stddev 40.273 2024-02-26T06:27:41.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:27:41.799 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:27:42.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.099+0000 7f95f6200640 1 -- 172.21.15.49:0/315700024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f00fffa0 msgr2=0x7f95f0100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.099+0000 7f95f6200640 1 --2- 172.21.15.49:0/315700024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f00fffa0 0x7f95f0100380 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009900 tx=0x7f95e802bd20 comp rx=0 tx=0).stop 2024-02-26T06:27:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.102+0000 7f95f6200640 1 -- 172.21.15.49:0/315700024 shutdown_connections 2024-02-26T06:27:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.102+0000 7f95f6200640 1 --2- 172.21.15.49:0/315700024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95f01053a0 0x7f95f0105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.102+0000 7f95f6200640 1 --2- 172.21.15.49:0/315700024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f01008c0 0x7f95f0104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.102+0000 7f95f6200640 1 --2- 172.21.15.49:0/315700024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f00fffa0 0x7f95f0100380 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.102+0000 7f95f6200640 1 -- 172.21.15.49:0/315700024 >> 172.21.15.49:0/315700024 conn(0x7f95f00fb9d0 msgr2=0x7f95f00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.102+0000 7f95f6200640 1 -- 172.21.15.49:0/315700024 shutdown_connections 2024-02-26T06:27:42.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.102+0000 7f95f6200640 1 -- 172.21.15.49:0/315700024 wait complete. 2024-02-26T06:27:42.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.105+0000 7f95f6200640 1 Processor -- start 2024-02-26T06:27:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.106+0000 7f95f6200640 1 -- start start 2024-02-26T06:27:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.106+0000 7f95f6200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95f00fffa0 0x7f95f0113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.106+0000 7f95f6200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f01008c0 0x7f95f0114470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.106+0000 7f95f6200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f01053a0 0x7f95f010d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.106+0000 7f95f6200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f010d7e0 con 0x7f95f00fffa0 2024-02-26T06:27:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.106+0000 7f95f6200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f010d950 con 0x7f95f01008c0 2024-02-26T06:27:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.106+0000 7f95f6200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f010db10 con 0x7f95f01053a0 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.107+0000 7f95efe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f01008c0 0x7f95f0114470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.107+0000 7f95efe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f01008c0 0x7f95f0114470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47328/0 (socket says 172.21.15.49:47328) 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.107+0000 7f95efe00640 1 -- 172.21.15.49:0/1279673486 learned_addr learned my addr 172.21.15.49:0/1279673486 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.107+0000 7f95f4e00640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95f00fffa0 0x7f95f0113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.107+0000 7f95efe00640 1 -- 172.21.15.49:0/1279673486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f01053a0 msgr2=0x7f95f010d050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.107+0000 7f95efe00640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f01053a0 0x7f95f010d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.107+0000 7f95efe00640 1 -- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95f00fffa0 msgr2=0x7f95f0113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.108+0000 7f95efe00640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95f00fffa0 0x7f95f0113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.108+0000 7f95efe00640 1 -- 172.21.15.49:0/1279673486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e8009560 con 0x7f95f01008c0 2024-02-26T06:27:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.109+0000 7f95f4e00640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95f00fffa0 0x7f95f0113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.110+0000 7f95efe00640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f01008c0 0x7f95f0114470 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f95e000c940 tx=0x7f95e000ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.111+0000 7f95ed600640 1 -- 172.21.15.49:0/1279673486 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0004220 con 0x7f95f01008c0 2024-02-26T06:27:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.111+0000 7f95ed600640 1 -- 172.21.15.49:0/1279673486 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95e0007e90 con 0x7f95f01008c0 2024-02-26T06:27:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.111+0000 7f95f6200640 1 -- 172.21.15.49:0/1279673486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f010ddf0 con 0x7f95f01008c0 2024-02-26T06:27:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.111+0000 7f95f6200640 1 -- 172.21.15.49:0/1279673486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f0111d40 con 0x7f95f01008c0 2024-02-26T06:27:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.112+0000 7f95ed600640 1 -- 172.21.15.49:0/1279673486 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0005690 con 0x7f95f01008c0 2024-02-26T06:27:42.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.118+0000 7f95fa827740 1 Processor -- start 2024-02-26T06:27:42.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.118+0000 7f95ed600640 1 -- 172.21.15.49:0/1279673486 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f95e0004a00 con 0x7f95f01008c0 2024-02-26T06:27:42.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.119+0000 7f95fa827740 1 -- start start 2024-02-26T06:27:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.120+0000 7f95fa827740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55757323a140 0x557573238e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.121+0000 7f95fa827740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 0x5575732398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.121+0000 7f95fa827740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575732427d0 0x557573244bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.121+0000 7f95fa827740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575731a8120 con 0x557573239460 2024-02-26T06:27:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.121+0000 7f95fa827740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575731a5650 con 0x55757323a140 2024-02-26T06:27:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.121+0000 7f95fa827740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575731a0e30 con 0x5575732427d0 2024-02-26T06:27:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.121+0000 7f95f5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55757323a140 0x557573238e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.121+0000 7f95f5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55757323a140 0x557573238e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47338/0 (socket says 172.21.15.49:47338) 2024-02-26T06:27:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.121+0000 7f95f5800640 1 -- 172.21.15.49:0/1450987449 learned_addr learned my addr 172.21.15.49:0/1450987449 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.122+0000 7f95f4e00640 1 --2- 172.21.15.49:0/1450987449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 0x5575732398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.122+0000 7f95f5800640 1 -- 172.21.15.49:0/1450987449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575732427d0 msgr2=0x557573244bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:27:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.122+0000 7f95f5800640 1 --2- 172.21.15.49:0/1450987449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575732427d0 0x557573244bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.122+0000 7f95f5800640 1 -- 172.21.15.49:0/1450987449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 msgr2=0x5575732398e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.122+0000 7f95f5800640 1 --2- 172.21.15.49:0/1450987449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 0x5575732398e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.122+0000 7f95f5800640 1 -- 172.21.15.49:0/1450987449 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55757315a030 con 0x55757323a140 2024-02-26T06:27:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.122+0000 7f95f5800640 1 --2- 172.21.15.49:0/1450987449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55757323a140 0x557573238e90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009d00 tx=0x7f95d800b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=169e1ef8738e762f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.123+0000 7f95e6000640 1 -- 172.21.15.49:0/1450987449 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d800e070 con 0x55757323a140 2024-02-26T06:27:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.123+0000 7f95e6000640 1 -- 172.21.15.49:0/1450987449 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95d8009830 con 0x55757323a140 2024-02-26T06:27:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.123+0000 7f95e6000640 1 -- 172.21.15.49:0/1450987449 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8011690 con 0x55757323a140 2024-02-26T06:27:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.124+0000 7f95fa827740 1 -- 172.21.15.49:0/1450987449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55757323a140 msgr2=0x557573238e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.124+0000 7f95fa827740 1 --2- 172.21.15.49:0/1450987449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55757323a140 0x557573238e90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009d00 tx=0x7f95d800b420 comp rx=0 tx=0).stop 2024-02-26T06:27:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.125+0000 7f95fa827740 1 -- 172.21.15.49:0/1450987449 shutdown_connections 2024-02-26T06:27:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.125+0000 7f95fa827740 1 --2- 172.21.15.49:0/1450987449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575732427d0 0x557573244bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.125+0000 7f95fa827740 1 --2- 172.21.15.49:0/1450987449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 0x5575732398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.125+0000 7f95fa827740 1 --2- 172.21.15.49:0/1450987449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55757323a140 0x557573238e90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.125+0000 7f95fa827740 1 -- 172.21.15.49:0/1450987449 >> 172.21.15.49:0/1450987449 conn(0x557573200570 msgr2=0x557573233eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.125+0000 7f95ed600640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f95d40a2370 0x7f95d40a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.126+0000 7f95fa827740 1 -- 172.21.15.49:0/1450987449 shutdown_connections 2024-02-26T06:27:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.126+0000 7f95ed600640 1 -- 172.21.15.49:0/1279673486 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f95e00ce060 con 0x7f95f01008c0 2024-02-26T06:27:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.128+0000 7f95fa827740 1 -- 172.21.15.49:0/1450987449 wait complete. 2024-02-26T06:27:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.128+0000 7f95f4e00640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f95d40a2370 0x7f95d40a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.128+0000 7f95fa827740 1 -- start start 2024-02-26T06:27:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.128+0000 7f95fa827740 1 Processor -- start 2024-02-26T06:27:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.129+0000 7f95fa827740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 0x55757323f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.129+0000 7f95fa827740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55757323a140 0x55757323fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.129+0000 7f95fa827740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x557573240200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.130+0000 7f95efe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55757323a140 0x55757323fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.130+0000 7f95efe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55757323a140 0x55757323fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43216/0 (socket says 172.21.15.49:43216) 2024-02-26T06:27:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.130+0000 7f95efe00640 1 -- 172.21.15.49:0/2987973747 learned_addr learned my addr 172.21.15.49:0/2987973747 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.130+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557573145e90 con 0x557573239460 2024-02-26T06:27:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.130+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557573145b30 con 0x5575732427d0 2024-02-26T06:27:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.130+0000 7f95f5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x557573240200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.130+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557572f0f1c0 con 0x55757323a140 2024-02-26T06:27:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.130+0000 7f95fa827740 10 client.0 _ll_register_callbacks cb 0x557573159930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.131+0000 7f95f5800640 1 -- 172.21.15.49:0/2987973747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55757323a140 msgr2=0x55757323fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.131+0000 7f95f5800640 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55757323a140 0x55757323fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.131+0000 7f95f5800640 1 -- 172.21.15.49:0/2987973747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 msgr2=0x55757323f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.131+0000 7f95f5800640 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 0x55757323f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.131+0000 7f95f5800640 1 -- 172.21.15.49:0/2987973747 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d80099b0 con 0x5575732427d0 2024-02-26T06:27:42.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.132+0000 7f95f4e00640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f95d40a2370 0x7f95d40a4830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95e8000c00 tx=0x7f95e8004970 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.133+0000 7f95f5800640 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x557573240200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f95d8004b90 tx=0x7f95d8012c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb2095c46432474 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.133+0000 7f95dea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.133+0000 7f95dea00640 1 -- 172.21.15.49:0/2987973747 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d800e040 con 0x5575732427d0 2024-02-26T06:27:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.134+0000 7f95dea00640 1 -- 172.21.15.49:0/2987973747 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f95d8025640 con 0x5575732427d0 2024-02-26T06:27:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.134+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557572edd720 con 0x5575732427d0 2024-02-26T06:27:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.134+0000 7f95dea00640 1 -- 172.21.15.49:0/2987973747 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95d8025800 con 0x5575732427d0 2024-02-26T06:27:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.134+0000 7f95fa827740 10 client.26483 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.134+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55757319f4b0 con 0x5575732427d0 2024-02-26T06:27:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.134+0000 7f95dea00640 1 -- 172.21.15.49:0/2987973747 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557572edd720 con 0x5575732427d0 2024-02-26T06:27:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.135+0000 7f95dea00640 1 -- 172.21.15.49:0/2987973747 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95d802ecf0 con 0x5575732427d0 2024-02-26T06:27:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.135+0000 7f95fa827740 10 client.26483 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.135+0000 7f95fa827740 10 client.26483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.135+0000 7f95fa827740 20 client.26483 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.135+0000 7f95fa827740 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5575732528a0 0x557573254cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.136+0000 7f95fa827740 4 client.26483 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.136+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557572f2c890 con 0x5575732528a0 2024-02-26T06:27:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.136+0000 7f95f4e00640 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5575732528a0 0x557573254cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.137+0000 7f95f4e00640 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5575732528a0 0x557573254cc0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e07c633e8d52e7 server_cookie=67e41c49426b3ae9 in_seq=0 out_seq=0 2024-02-26T06:27:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.137+0000 7f95dea00640 10 client.26483 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.137+0000 7f95dea00640 1 -- 172.21.15.49:0/2987973747 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557572f2c890 con 0x5575732528a0 2024-02-26T06:27:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.138+0000 7f95dea00640 10 client.26483 handle_command_reply: tid=0 2024-02-26T06:27:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.138+0000 7f95fa827740 1 client.26483 shutdown 2024-02-26T06:27:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.138+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 msgr2=0x557573240200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.138+0000 7f95fa827740 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x557573240200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f95d8004b90 tx=0x7f95d8012c70 comp rx=0 tx=0).stop 2024-02-26T06:27:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 shutdown_connections 2024-02-26T06:27:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95fa827740 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5575732528a0 0x557573254cc0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95fa827740 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x557573240200 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95fa827740 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55757323a140 0x55757323fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95fa827740 1 --2- 172.21.15.49:0/2987973747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557573239460 0x55757323f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 >> 172.21.15.49:0/2987973747 conn(0x557573200570 msgr2=0x557573235c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95dea00640 0 client.26483 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 shutdown_connections 2024-02-26T06:27:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.139+0000 7f95fa827740 1 -- 172.21.15.49:0/2987973747 wait complete. 2024-02-26T06:27:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.140+0000 7f95fa827740 20 client.26483 trim_cache size 0 max 16384 2024-02-26T06:27:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.167+0000 7f95fa827740 1 Processor -- start 2024-02-26T06:27:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.167+0000 7f95fa827740 1 -- start start 2024-02-26T06:27:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.168+0000 7f95fa827740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x557573241eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.168+0000 7f95fa827740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573240040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.168+0000 7f95fa827740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x5575732497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.168+0000 7f95fa827740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757324f4c0 con 0x55757323a140 2024-02-26T06:27:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.168+0000 7f95fa827740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757324e6e0 con 0x5575732427d0 2024-02-26T06:27:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.168+0000 7f95fa827740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557573239da0 con 0x557573239460 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95f4e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x557573241eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95f4e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x557573241eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43228/0 (socket says 172.21.15.49:43228) 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95f4e00640 1 -- 172.21.15.49:0/3795447837 learned_addr learned my addr 172.21.15.49:0/3795447837 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95f5800640 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x5575732497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95efe00640 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573240040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95f4e00640 1 -- 172.21.15.49:0/3795447837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 msgr2=0x5575732497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95f4e00640 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x5575732497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95f4e00640 1 -- 172.21.15.49:0/3795447837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 msgr2=0x557573240040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.169+0000 7f95f4e00640 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573240040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.170+0000 7f95f4e00640 1 -- 172.21.15.49:0/3795447837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55757322f5a0 con 0x557573239460 2024-02-26T06:27:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.170+0000 7f95f4e00640 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x557573241eb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009900 tx=0x7f95e8041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7e1d2b3964d3ba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.171+0000 7f95e5600640 1 -- 172.21.15.49:0/3795447837 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004390 con 0x557573239460 2024-02-26T06:27:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.171+0000 7f95e5600640 1 -- 172.21.15.49:0/3795447837 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95e80044f0 con 0x557573239460 2024-02-26T06:27:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.171+0000 7f95e5600640 1 -- 172.21.15.49:0/3795447837 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004df0 con 0x557573239460 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 -- 172.21.15.49:0/3795447837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 msgr2=0x557573241eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x557573241eb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009900 tx=0x7f95e8041000 comp rx=0 tx=0).stop 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 -- 172.21.15.49:0/3795447837 shutdown_connections 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x5575732497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573240040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 --2- 172.21.15.49:0/3795447837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x557573241eb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 -- 172.21.15.49:0/3795447837 >> 172.21.15.49:0/3795447837 conn(0x557573200570 msgr2=0x5575732339e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 -- 172.21.15.49:0/3795447837 shutdown_connections 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.172+0000 7f95fa827740 1 -- 172.21.15.49:0/3795447837 wait complete. 2024-02-26T06:27:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.173+0000 7f95fa827740 1 -- start start 2024-02-26T06:27:42.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.173+0000 7f95fa827740 1 Processor -- start 2024-02-26T06:27:42.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.174+0000 7f95fa827740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x5575732497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.175+0000 7f95fa827740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573249d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.175+0000 7f95fa827740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x55757323c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.175+0000 7f95fa827740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757323c710 con 0x55757323a140 2024-02-26T06:27:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.175+0000 7f95fa827740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757323c880 con 0x5575732427d0 2024-02-26T06:27:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.175+0000 7f95fa827740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757323ca40 con 0x557573239460 2024-02-26T06:27:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.176+0000 7f95f5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x55757323c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.176+0000 7f95f5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x55757323c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47372/0 (socket says 172.21.15.49:47372) 2024-02-26T06:27:42.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.176+0000 7f95f5800640 1 -- 172.21.15.49:0/2364492543 learned_addr learned my addr 172.21.15.49:0/2364492543 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:42.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.176+0000 7f95f4e00640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x5575732497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.176+0000 7f95fa827740 10 client.0 _ll_register_callbacks cb 0x7f95d8009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:42.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.176+0000 7f95efe00640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573249d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.176+0000 7f95f5800640 1 -- 172.21.15.49:0/2364492543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 msgr2=0x5575732497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.177+0000 7f95f5800640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x5575732497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.177+0000 7f95f5800640 1 -- 172.21.15.49:0/2364492543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 msgr2=0x557573249d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.177+0000 7f95f5800640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573249d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.177+0000 7f95f5800640 1 -- 172.21.15.49:0/2364492543 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f0114d00 con 0x5575732427d0 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.177+0000 7f95f4e00640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x5575732497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.177+0000 7f95efe00640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573249d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.177+0000 7f95f5800640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x55757323c040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f95d80096d0 tx=0x7f95d8012f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc8a68b5139a97ad server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.177+0000 7f95dea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:42.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.178+0000 7f95dea00640 1 -- 172.21.15.49:0/2364492543 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d800e070 con 0x5575732427d0 2024-02-26T06:27:42.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.178+0000 7f95fa827740 1 -- 172.21.15.49:0/2364492543 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557572ea07e0 con 0x5575732427d0 2024-02-26T06:27:42.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.178+0000 7f95dea00640 1 -- 172.21.15.49:0/2364492543 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f95d80127f0 con 0x5575732427d0 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.178+0000 7f95dea00640 1 -- 172.21.15.49:0/2364492543 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95d8007000 con 0x5575732427d0 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.178+0000 7f95fa827740 10 client.26489 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.178+0000 7f95fa827740 1 -- 172.21.15.49:0/2364492543 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55757313b920 con 0x5575732427d0 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95dea00640 1 -- 172.21.15.49:0/2364492543 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557572ea07e0 con 0x5575732427d0 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95dea00640 1 -- 172.21.15.49:0/2364492543 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95d803e070 con 0x5575732427d0 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95fa827740 10 client.26489 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95fa827740 10 client.26489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95fa827740 20 client.26489 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95fa827740 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5575732a9e10 0x5575732ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95fa827740 4 client.26489 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:27:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95fa827740 1 -- 172.21.15.49:0/2364492543 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557572e4c700 con 0x5575732a9e10 2024-02-26T06:27:42.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.179+0000 7f95f4e00640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5575732a9e10 0x5575732ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.180+0000 7f95dea00640 10 client.26489 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:42.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.180+0000 7f95f4e00640 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5575732a9e10 0x5575732ac230 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d845def5454d81cc server_cookie=ebd75f9c62327cca in_seq=0 out_seq=0 2024-02-26T06:27:42.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95dea00640 1 -- 172.21.15.49:0/2364492543 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557572e4c700 con 0x5575732a9e10 2024-02-26T06:27:42.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95dea00640 10 client.26489 handle_command_reply: tid=0 2024-02-26T06:27:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 client.26489 shutdown 2024-02-26T06:27:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 -- 172.21.15.49:0/2364492543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 msgr2=0x55757323c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x55757323c040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f95d80096d0 tx=0x7f95d8012f10 comp rx=0 tx=0).stop 2024-02-26T06:27:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 -- 172.21.15.49:0/2364492543 shutdown_connections 2024-02-26T06:27:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5575732a9e10 0x5575732ac230 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575732427d0 0x55757323c040 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55757323a140 0x557573249d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 --2- 172.21.15.49:0/2364492543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557573239460 0x5575732497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95fa827740 1 -- 172.21.15.49:0/2364492543 >> 172.21.15.49:0/2364492543 conn(0x557573200570 msgr2=0x557573235300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.181+0000 7f95dea00640 0 client.26489 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.182+0000 7f95fa827740 1 -- 172.21.15.49:0/2364492543 shutdown_connections 2024-02-26T06:27:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.182+0000 7f95fa827740 1 -- 172.21.15.49:0/2364492543 wait complete. 2024-02-26T06:27:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.182+0000 7f95fa827740 20 client.26489 trim_cache size 0 max 16384 2024-02-26T06:27:42.183 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:27:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:27:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:27:42.184 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:27:42.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.182+0000 7f95f6200640 1 -- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f95d40a2370 msgr2=0x7f95d40a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.182+0000 7f95f6200640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f95d40a2370 0x7f95d40a4830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95e8000c00 tx=0x7f95e8004970 comp rx=0 tx=0).stop 2024-02-26T06:27:42.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.182+0000 7f95f6200640 1 -- 172.21.15.49:0/1279673486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f01008c0 msgr2=0x7f95f0114470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.182+0000 7f95f6200640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f01008c0 0x7f95f0114470 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f95e000c940 tx=0x7f95e000ce00 comp rx=0 tx=0).stop 2024-02-26T06:27:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.183+0000 7f95f6200640 1 -- 172.21.15.49:0/1279673486 shutdown_connections 2024-02-26T06:27:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.183+0000 7f95f6200640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f95d40a2370 0x7f95d40a4830 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.183+0000 7f95f6200640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f01008c0 0x7f95f0114470 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.183+0000 7f95f6200640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95f01053a0 0x7f95f010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.183+0000 7f95f6200640 1 --2- 172.21.15.49:0/1279673486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95f00fffa0 0x7f95f0113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.183+0000 7f95f6200640 1 -- 172.21.15.49:0/1279673486 >> 172.21.15.49:0/1279673486 conn(0x7f95f00fb9d0 msgr2=0x7f95f0102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.183+0000 7f95f6200640 1 -- 172.21.15.49:0/1279673486 shutdown_connections 2024-02-26T06:27:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.183+0000 7f95f6200640 1 -- 172.21.15.49:0/1279673486 wait complete. 2024-02-26T06:27:42.211 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:27:42.211 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:27:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.501+0000 7fab31200640 1 -- 172.21.15.49:0/3347633194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c103f30 msgr2=0x7fab2c107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.501+0000 7fab31200640 1 --2- 172.21.15.49:0/3347633194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c103f30 0x7fab2c107f70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab18009860 tx=0x7fab1802bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.502+0000 7fab31200640 1 -- 172.21.15.49:0/3347633194 shutdown_connections 2024-02-26T06:27:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.502+0000 7fab31200640 1 --2- 172.21.15.49:0/3347633194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab2c108650 0x7fab2c110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.502+0000 7fab31200640 1 --2- 172.21.15.49:0/3347633194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c103f30 0x7fab2c107f70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.502+0000 7fab31200640 1 --2- 172.21.15.49:0/3347633194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab2c103580 0x7fab2c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.503+0000 7fab31200640 1 -- 172.21.15.49:0/3347633194 >> 172.21.15.49:0/3347633194 conn(0x7fab2c0fede0 msgr2=0x7fab2c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:42.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.503+0000 7fab31200640 1 -- 172.21.15.49:0/3347633194 shutdown_connections 2024-02-26T06:27:42.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.503+0000 7fab31200640 1 -- 172.21.15.49:0/3347633194 wait complete. 2024-02-26T06:27:42.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.505+0000 7fab31200640 1 Processor -- start 2024-02-26T06:27:42.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.505+0000 7fab31200640 1 -- start start 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.505+0000 7fab31200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab2c103580 0x7fab2c10ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.505+0000 7fab31200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab2c108650 0x7fab2c10f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.506+0000 7fab29e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab2c103580 0x7fab2c10ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.506+0000 7fab29e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab2c103580 0x7fab2c10ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43702/0 (socket says 172.21.15.49:43702) 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.505+0000 7fab31200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c10f730 0x7fab2c1b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.506+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c11a8a0 con 0x7fab2c103580 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.506+0000 7fab29e00640 1 -- 172.21.15.49:0/4029414216 learned_addr learned my addr 172.21.15.49:0/4029414216 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.506+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1b2cf0 con 0x7fab2c108650 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.506+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1b2e60 con 0x7fab2c10f730 2024-02-26T06:27:42.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.506+0000 7fab29400640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab2c108650 0x7fab2c10f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.507+0000 7fab29e00640 1 -- 172.21.15.49:0/4029414216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c10f730 msgr2=0x7fab2c1b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.507+0000 7fab2a800640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c10f730 0x7fab2c1b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.507+0000 7fab29e00640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c10f730 0x7fab2c1b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.507+0000 7fab29e00640 1 -- 172.21.15.49:0/4029414216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab2c108650 msgr2=0x7fab2c10f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.507+0000 7fab29e00640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab2c108650 0x7fab2c10f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.507+0000 7fab29e00640 1 -- 172.21.15.49:0/4029414216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18009490 con 0x7fab2c103580 2024-02-26T06:27:42.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.507+0000 7fab29400640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab2c108650 0x7fab2c10f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:42.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.507+0000 7fab2a800640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c10f730 0x7fab2c1b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:42.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.508+0000 7fab29e00640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab2c103580 0x7fab2c10ecb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00e9c0 tx=0x7fab1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:42.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.508+0000 7fab22a00640 1 -- 172.21.15.49:0/4029414216 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00ccc0 con 0x7fab2c103580 2024-02-26T06:27:42.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.508+0000 7fab22a00640 1 -- 172.21.15.49:0/4029414216 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab1c00ce20 con 0x7fab2c103580 2024-02-26T06:27:42.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.508+0000 7fab22a00640 1 -- 172.21.15.49:0/4029414216 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c010430 con 0x7fab2c103580 2024-02-26T06:27:42.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.508+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c1b3140 con 0x7fab2c103580 2024-02-26T06:27:42.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.509+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1b3600 con 0x7fab2c103580 2024-02-26T06:27:42.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.511+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c0683c0 con 0x7fab2c103580 2024-02-26T06:27:42.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.514+0000 7fab22a00640 1 -- 172.21.15.49:0/4029414216 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fab1c004f80 con 0x7fab2c103580 2024-02-26T06:27:42.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.516+0000 7fab22a00640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fab100a22b0 0x7fab100a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:42.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.516+0000 7fab22a00640 1 -- 172.21.15.49:0/4029414216 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fab1c014070 con 0x7fab2c103580 2024-02-26T06:27:42.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.517+0000 7fab29400640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fab100a22b0 0x7fab100a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:42.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.518+0000 7fab29400640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fab100a22b0 0x7fab100a4770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab18000c00 tx=0x7fab18005a90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:42.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.523+0000 7fab22a00640 1 -- 172.21.15.49:0/4029414216 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab1c0970e0 con 0x7fab2c103580 2024-02-26T06:27:42.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:42 smithi080 ceph-mon[36206]: pgmap v132: 97 pgs: 97 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 694 op/s 2024-02-26T06:27:42.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:42 smithi115 ceph-mon[36426]: pgmap v132: 97 pgs: 97 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 694 op/s 2024-02-26T06:27:42.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:42 smithi049 ceph-mon[31696]: pgmap v132: 97 pgs: 97 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 694 op/s 2024-02-26T06:27:42.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.962+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab2c110190 con 0x7fab2c103580 2024-02-26T06:27:42.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.966+0000 7fab22a00640 1 -- 172.21.15.49:0/4029414216 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11334 (secure 0 0 0) 0x7fab1c09a7a0 con 0x7fab2c103580 2024-02-26T06:27:42.970 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:27:42.970 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:27:29.193881+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:27:42.970 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 35 2024-02-26T06:27:42.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fab100a22b0 msgr2=0x7fab100a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fab100a22b0 0x7fab100a4770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab18000c00 tx=0x7fab18005a90 comp rx=0 tx=0).stop 2024-02-26T06:27:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab2c103580 msgr2=0x7fab2c10ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab2c103580 0x7fab2c10ecb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00e9c0 tx=0x7fab1c00ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 shutdown_connections 2024-02-26T06:27:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fab100a22b0 0x7fab100a4770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab2c108650 0x7fab2c10f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c10f730 0x7fab2c1b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 --2- 172.21.15.49:0/4029414216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab2c103580 0x7fab2c10ecb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.971+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 >> 172.21.15.49:0/4029414216 conn(0x7fab2c0fede0 msgr2=0x7fab2c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.972+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 shutdown_connections 2024-02-26T06:27:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:42.972+0000 7fab31200640 1 -- 172.21.15.49:0/4029414216 wait complete. 2024-02-26T06:27:43.004 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:27:43.004 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:27:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.298+0000 7f2d06a00640 1 -- 172.21.15.49:0/88785933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d00106430 msgr2=0x7f2d0010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.298+0000 7f2d06a00640 1 --2- 172.21.15.49:0/88785933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d00106430 0x7f2d0010e000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009cd0 tx=0x7f2cf802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.298+0000 7f2d06a00640 1 -- 172.21.15.49:0/88785933 shutdown_connections 2024-02-26T06:27:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.298+0000 7f2d06a00640 1 --2- 172.21.15.49:0/88785933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d00106430 0x7f2d0010e000 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.298+0000 7f2d06a00640 1 --2- 172.21.15.49:0/88785933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d00101d10 0x7f2d00105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.298+0000 7f2d06a00640 1 --2- 172.21.15.49:0/88785933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d00101360 0x7f2d00101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.298+0000 7f2d06a00640 1 -- 172.21.15.49:0/88785933 >> 172.21.15.49:0/88785933 conn(0x7f2d000fcbe0 msgr2=0x7f2d000ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.299+0000 7f2d06a00640 1 -- 172.21.15.49:0/88785933 shutdown_connections 2024-02-26T06:27:43.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.299+0000 7f2d06a00640 1 -- 172.21.15.49:0/88785933 wait complete. 2024-02-26T06:27:43.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.300+0000 7f2d06a00640 1 Processor -- start 2024-02-26T06:27:43.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.301+0000 7f2d06a00640 1 -- start start 2024-02-26T06:27:43.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.301+0000 7f2d06a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d00101360 0x7f2d001a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.301+0000 7f2d06a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d00106430 0x7f2d001a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.301+0000 7f2d06a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d0010d030 0x7f2d0010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.301+0000 7f2d06a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0010dab0 con 0x7f2d00106430 2024-02-26T06:27:43.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.301+0000 7f2d06a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0010dc20 con 0x7f2d0010d030 2024-02-26T06:27:43.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.301+0000 7f2d06a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0010dde0 con 0x7f2d00101360 2024-02-26T06:27:43.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d00101360 0x7f2d001a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2d04a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d0010d030 0x7f2d0010d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d00101360 0x7f2d001a3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43284/0 (socket says 172.21.15.49:43284) 2024-02-26T06:27:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2cffe00640 1 -- 172.21.15.49:0/4294196334 learned_addr learned my addr 172.21.15.49:0/4294196334 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2cff400640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d00106430 0x7f2d001a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2d04a00640 1 -- 172.21.15.49:0/4294196334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d00101360 msgr2=0x7f2d001a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2d04a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d00101360 0x7f2d001a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2d04a00640 1 -- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d00106430 msgr2=0x7f2d001a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2d04a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d00106430 0x7f2d001a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.302+0000 7f2d04a00640 1 -- 172.21.15.49:0/4294196334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf8009900 con 0x7f2d0010d030 2024-02-26T06:27:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.303+0000 7f2d04a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d0010d030 0x7f2d0010d4b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8037040 tx=0x7f2cf8034a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.304+0000 7f2cfcc00640 1 -- 172.21.15.49:0/4294196334 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80328b0 con 0x7f2d0010d030 2024-02-26T06:27:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.304+0000 7f2cfcc00640 1 -- 172.21.15.49:0/4294196334 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cf8003e50 con 0x7f2d0010d030 2024-02-26T06:27:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.304+0000 7f2cfcc00640 1 -- 172.21.15.49:0/4294196334 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8004780 con 0x7f2d0010d030 2024-02-26T06:27:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.304+0000 7f2d06a00640 1 -- 172.21.15.49:0/4294196334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d001aa8c0 con 0x7f2d0010d030 2024-02-26T06:27:43.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.304+0000 7f2d06a00640 1 -- 172.21.15.49:0/4294196334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d001aad50 con 0x7f2d0010d030 2024-02-26T06:27:43.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.310+0000 7f2cfcc00640 1 -- 172.21.15.49:0/4294196334 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2cf800a040 con 0x7f2d0010d030 2024-02-26T06:27:43.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.313+0000 7f2cfcc00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ce80a2400 0x7f2ce80a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.313+0000 7f2cfcc00640 1 -- 172.21.15.49:0/4294196334 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2cf80f08a0 con 0x7f2d0010d030 2024-02-26T06:27:43.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.313+0000 7f2d09969740 1 Processor -- start 2024-02-26T06:27:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.314+0000 7f2cffe00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ce80a2400 0x7f2ce80a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.315+0000 7f2d09969740 1 -- start start 2024-02-26T06:27:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.316+0000 7f2cffe00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ce80a2400 0x7f2ce80a48c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009cd0 tx=0x7f2cf4005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d09969740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d09969740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d09969740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912ae500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d09969740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b912111e0 con 0x557b912a2c10 2024-02-26T06:27:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d09969740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9120e710 con 0x557b912ac110 2024-02-26T06:27:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d09969740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b91209ef0 con 0x557b912a3530 2024-02-26T06:27:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d04a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d04a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47422/0 (socket says 172.21.15.49:47422) 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.317+0000 7f2d04a00640 1 -- 172.21.15.49:0/824775563 learned_addr learned my addr 172.21.15.49:0/824775563 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2d04a00640 1 -- 172.21.15.49:0/824775563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 msgr2=0x557b912abbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2cffe00640 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2cff400640 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2d04a00640 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2d04a00640 1 -- 172.21.15.49:0/824775563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 msgr2=0x557b912a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2d04a00640 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912a2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2d04a00640 1 -- 172.21.15.49:0/824775563 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b911cbdc0 con 0x557b912ac110 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2cff400640 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2d04a00640 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912ae500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80c5640 tx=0x7f2cf80bcdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=918bc766997377f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2cf1200640 1 -- 172.21.15.49:0/824775563 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80c5b70 con 0x557b912ac110 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2cf1200640 1 -- 172.21.15.49:0/824775563 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cf8032a10 con 0x557b912ac110 2024-02-26T06:27:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.318+0000 7f2cf1200640 1 -- 172.21.15.49:0/824775563 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80c1360 con 0x557b912ac110 2024-02-26T06:27:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.319+0000 7f2d09969740 1 -- 172.21.15.49:0/824775563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 msgr2=0x557b912ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.319+0000 7f2d09969740 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912ae500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80c5640 tx=0x7f2cf80bcdb0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.320+0000 7f2d09969740 1 -- 172.21.15.49:0/824775563 shutdown_connections 2024-02-26T06:27:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.320+0000 7f2d09969740 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912ae500 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.320+0000 7f2d09969740 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.320+0000 7f2d09969740 1 --2- 172.21.15.49:0/824775563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.320+0000 7f2d09969740 1 -- 172.21.15.49:0/824775563 >> 172.21.15.49:0/824775563 conn(0x557b9129d340 msgr2=0x557b9129d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.320+0000 7f2d09969740 1 -- 172.21.15.49:0/824775563 shutdown_connections 2024-02-26T06:27:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.321+0000 7f2d09969740 1 -- 172.21.15.49:0/824775563 wait complete. 2024-02-26T06:27:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.321+0000 7f2d09969740 1 -- start start 2024-02-26T06:27:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.322+0000 7f2d09969740 1 Processor -- start 2024-02-26T06:27:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2d09969740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a2c10 0x557b912a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2d09969740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b912a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2d09969740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2d09969740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b911af120 con 0x557b912a3530 2024-02-26T06:27:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2d09969740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b911aedc0 con 0x557b912ac110 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2d09969740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b90f781c0 con 0x557b912a2c10 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2d09969740 10 client.0 _ll_register_callbacks cb 0x557b911fe280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a2c10 0x557b912a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a2c10 0x557b912a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43294/0 (socket says 172.21.15.49:43294) 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.323+0000 7f2cffe00640 1 -- 172.21.15.49:0/517926492 learned_addr learned my addr 172.21.15.49:0/517926492 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2d04a00640 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2cff400640 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b912a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2cffe00640 1 -- 172.21.15.49:0/517926492 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 msgr2=0x557b912a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2cffe00640 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912a9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2cffe00640 1 -- 172.21.15.49:0/517926492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 msgr2=0x557b912a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2cffe00640 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b912a9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2cffe00640 1 -- 172.21.15.49:0/517926492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d001aad50 con 0x557b912a2c10 2024-02-26T06:27:43.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2cffe00640 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a2c10 0x557b912a91d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2cf401b9e0 tx=0x7f2cf4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31734bffcb38ae06 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.324+0000 7f2ce5600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:43.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.325+0000 7f2ce5600640 1 -- 172.21.15.49:0/517926492 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4020070 con 0x557b912a2c10 2024-02-26T06:27:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.325+0000 7f2ce5600640 1 -- 172.21.15.49:0/517926492 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2cf401a890 con 0x557b912a2c10 2024-02-26T06:27:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.325+0000 7f2d09969740 1 -- 172.21.15.49:0/517926492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b90f097e0 con 0x557b912a2c10 2024-02-26T06:27:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.326+0000 7f2ce5600640 1 -- 172.21.15.49:0/517926492 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf4009e80 con 0x557b912a2c10 2024-02-26T06:27:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.327+0000 7f2d09969740 10 client.26509 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.327+0000 7f2d09969740 1 -- 172.21.15.49:0/517926492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557b91208570 con 0x557b912a2c10 2024-02-26T06:27:43.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2ce5600640 1 -- 172.21.15.49:0/517926492 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cf4033020 con 0x557b912a2c10 2024-02-26T06:27:43.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2ce5600640 1 -- 172.21.15.49:0/517926492 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2cf4026d20 con 0x557b912a2c10 2024-02-26T06:27:43.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2d09969740 10 client.26509 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:43.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2d09969740 10 client.26509 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:27:43.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2d09969740 20 client.26509 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:43.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2d09969740 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x557b912bbf50 0x557b912be360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2d09969740 4 client.26509 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:43.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2d09969740 1 -- 172.21.15.49:0/517926492 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b90f46720 con 0x557b912bbf50 2024-02-26T06:27:43.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.328+0000 7f2cff400640 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x557b912bbf50 0x557b912be360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.329+0000 7f2cff400640 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x557b912bbf50 0x557b912be360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=241439be4765933a server_cookie=f1e2fd66934c7d61 in_seq=0 out_seq=0 2024-02-26T06:27:43.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.329+0000 7f2ce5600640 10 client.26509 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:27:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.330+0000 7f2ce5600640 1 -- 172.21.15.49:0/517926492 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b90f46720 con 0x557b912bbf50 2024-02-26T06:27:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.330+0000 7f2ce5600640 10 client.26509 handle_command_reply: tid=0 2024-02-26T06:27:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.330+0000 7f2d09969740 1 client.26509 shutdown 2024-02-26T06:27:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.330+0000 7f2d09969740 1 -- 172.21.15.49:0/517926492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a2c10 msgr2=0x557b912a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.330+0000 7f2d09969740 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a2c10 0x557b912a91d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2cf401b9e0 tx=0x7f2cf4004730 comp rx=0 tx=0).stop 2024-02-26T06:27:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.331+0000 7f2d09969740 1 -- 172.21.15.49:0/517926492 shutdown_connections 2024-02-26T06:27:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.331+0000 7f2d09969740 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x557b912bbf50 0x557b912be360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.331+0000 7f2d09969740 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.331+0000 7f2d09969740 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b912a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.331+0000 7f2d09969740 1 --2- 172.21.15.49:0/517926492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a2c10 0x557b912a91d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.331+0000 7f2d09969740 1 -- 172.21.15.49:0/517926492 >> 172.21.15.49:0/517926492 conn(0x557b9129d340 msgr2=0x557b9129f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.331+0000 7f2ce5600640 0 client.26509 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:27:43.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.332+0000 7f2d09969740 1 -- 172.21.15.49:0/517926492 shutdown_connections 2024-02-26T06:27:43.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.332+0000 7f2d09969740 1 -- 172.21.15.49:0/517926492 wait complete. 2024-02-26T06:27:43.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.332+0000 7f2d09969740 20 client.26509 trim_cache size 0 max 16384 2024-02-26T06:27:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.365+0000 7f2d09969740 1 Processor -- start 2024-02-26T06:27:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.365+0000 7f2d09969740 1 -- start start 2024-02-26T06:27:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.366+0000 7f2d09969740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.366+0000 7f2d09969740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912a6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.366+0000 7f2d09969740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.366+0000 7f2d09969740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b90f46750 con 0x557b912a2c10 2024-02-26T06:27:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.366+0000 7f2d09969740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9129e170 con 0x557b912ac110 2024-02-26T06:27:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.366+0000 7f2d09969740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b912a4de0 con 0x557b912a3530 2024-02-26T06:27:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.367+0000 7f2cff400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.367+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.367+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912ab900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43758/0 (socket says 172.21.15.49:43758) 2024-02-26T06:27:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.367+0000 7f2cffe00640 1 -- 172.21.15.49:0/1420731762 learned_addr learned my addr 172.21.15.49:0/1420731762 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.368+0000 7f2cff400640 1 -- 172.21.15.49:0/1420731762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 msgr2=0x557b912a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.368+0000 7f2cff400640 1 --2- 172.21.15.49:0/1420731762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.368+0000 7f2cff400640 1 -- 172.21.15.49:0/1420731762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 msgr2=0x557b912ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.368+0000 7f2cff400640 1 --2- 172.21.15.49:0/1420731762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.368+0000 7f2cff400640 1 -- 172.21.15.49:0/1420731762 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b911a4a80 con 0x557b912a3530 2024-02-26T06:27:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.368+0000 7f2cffe00640 1 --2- 172.21.15.49:0/1420731762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:43.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.371+0000 7f2cff400640 1 --2- 172.21.15.49:0/1420731762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912a6450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557b912a8a30 tx=0x7f2cec005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5a1e71cda3e403 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.373+0000 7f2ce7e00640 1 -- 172.21.15.49:0/1420731762 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec01bb80 con 0x557b912a3530 2024-02-26T06:27:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.373+0000 7f2ce7e00640 1 -- 172.21.15.49:0/1420731762 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cec019040 con 0x557b912a3530 2024-02-26T06:27:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.373+0000 7f2ce7e00640 1 -- 172.21.15.49:0/1420731762 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec005020 con 0x557b912a3530 2024-02-26T06:27:43.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.374+0000 7f2d09969740 1 -- 172.21.15.49:0/1420731762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 msgr2=0x557b912a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.374+0000 7f2d09969740 1 --2- 172.21.15.49:0/1420731762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912a6450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557b912a8a30 tx=0x7f2cec005ce0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.375+0000 7f2d09969740 1 -- 172.21.15.49:0/1420731762 shutdown_connections 2024-02-26T06:27:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.375+0000 7f2d09969740 1 --2- 172.21.15.49:0/1420731762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912ac110 0x557b912a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.375+0000 7f2d09969740 1 --2- 172.21.15.49:0/1420731762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912a3530 0x557b912a6450 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.375+0000 7f2d09969740 1 --2- 172.21.15.49:0/1420731762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a2c10 0x557b912ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.375+0000 7f2d09969740 1 -- 172.21.15.49:0/1420731762 >> 172.21.15.49:0/1420731762 conn(0x557b9129d340 msgr2=0x557b912c2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.376+0000 7f2d09969740 1 -- 172.21.15.49:0/1420731762 shutdown_connections 2024-02-26T06:27:43.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.376+0000 7f2d09969740 1 -- 172.21.15.49:0/1420731762 wait complete. 2024-02-26T06:27:43.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.376+0000 7f2d09969740 1 -- start start 2024-02-26T06:27:43.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.377+0000 7f2d09969740 1 Processor -- start 2024-02-26T06:27:43.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.380+0000 7f2d09969740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912a2c10 0x557b9130ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.380+0000 7f2d09969740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b9130cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.380+0000 7f2d09969740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912ac110 0x557b912b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.381+0000 7f2d04a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912ac110 0x557b912b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.381+0000 7f2d04a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912ac110 0x557b912b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43308/0 (socket says 172.21.15.49:43308) 2024-02-26T06:27:43.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.381+0000 7f2d04a00640 1 -- 172.21.15.49:0/4200173237 learned_addr learned my addr 172.21.15.49:0/4200173237 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.381+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b912b2840 con 0x557b912a3530 2024-02-26T06:27:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.382+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b912b29b0 con 0x557b912a2c10 2024-02-26T06:27:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.382+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b912b2b70 con 0x557b912ac110 2024-02-26T06:27:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.382+0000 7f2cff400640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b9130cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.382+0000 7f2d04a00640 1 -- 172.21.15.49:0/4200173237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912a2c10 msgr2=0x557b9130ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.382+0000 7f2d09969740 10 client.0 _ll_register_callbacks cb 0x7f2cf80b9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.382+0000 7f2cffe00640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912a2c10 0x557b9130ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.385+0000 7f2d04a00640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912a2c10 0x557b9130ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.385+0000 7f2d04a00640 1 -- 172.21.15.49:0/4200173237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 msgr2=0x557b9130cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.385+0000 7f2d04a00640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b9130cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.385+0000 7f2d04a00640 1 -- 172.21.15.49:0/4200173237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec00fc90 con 0x557b912ac110 2024-02-26T06:27:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.385+0000 7f2cff400640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b9130cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.385+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cf8032a10 con 0x557b912ac110 2024-02-26T06:27:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.385+0000 7f2cffe00640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912a2c10 0x557b9130ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.385+0000 7f2d04a00640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912ac110 0x557b912b2170 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80bcea0 tx=0x7f2cf80bb8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aea17bed1d36089 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.386+0000 7f2ce5600640 10 client.26515 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:43.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.387+0000 7f2ce5600640 1 -- 172.21.15.49:0/4200173237 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80c1330 con 0x557b912ac110 2024-02-26T06:27:43.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.387+0000 7f2ce5600640 1 -- 172.21.15.49:0/4200173237 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2cf80c5b70 con 0x557b912ac110 2024-02-26T06:27:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.388+0000 7f2ce5600640 1 -- 172.21.15.49:0/4200173237 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf80c1520 con 0x557b912ac110 2024-02-26T06:27:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.388+0000 7f2d09969740 10 client.26515 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.388+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x557b911ca620 con 0x557b912ac110 2024-02-26T06:27:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.389+0000 7f2ce5600640 1 -- 172.21.15.49:0/4200173237 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2cf8050020 con 0x557b912ac110 2024-02-26T06:27:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.389+0000 7f2d09969740 10 client.26515 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.389+0000 7f2d09969740 10 client.26515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:27:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.389+0000 7f2d09969740 20 client.26515 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.389+0000 7f2d09969740 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x557b91314230 0x557b91316620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.389+0000 7f2d09969740 4 client.26515 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:27:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.389+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557b90f95890 con 0x557b91314230 2024-02-26T06:27:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.390+0000 7f2cffe00640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x557b91314230 0x557b91316620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.391+0000 7f2cffe00640 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x557b91314230 0x557b91316620 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3be08775b4394ad4 server_cookie=aaa6b7c2a43c61f5 in_seq=0 out_seq=0 2024-02-26T06:27:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.391+0000 7f2ce5600640 10 client.26515 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:27:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.391+0000 7f2ce5600640 1 -- 172.21.15.49:0/4200173237 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557b90f95890 con 0x557b91314230 2024-02-26T06:27:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.391+0000 7f2ce5600640 10 client.26515 handle_command_reply: tid=0 2024-02-26T06:27:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.391+0000 7f2d09969740 1 client.26515 shutdown 2024-02-26T06:27:43.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.392+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912ac110 msgr2=0x557b912b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.392+0000 7f2d09969740 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912ac110 0x557b912b2170 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80bcea0 tx=0x7f2cf80bb8e0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.392+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 shutdown_connections 2024-02-26T06:27:43.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.392+0000 7f2d09969740 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x557b91314230 0x557b91316620 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.394+0000 7f2d09969740 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b912ac110 0x557b912b2170 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.394+0000 7f2ce5600640 0 client.26515 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:27:43.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.394+0000 7f2d09969740 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b912a3530 0x557b9130cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.394+0000 7f2d09969740 1 --2- 172.21.15.49:0/4200173237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b912a2c10 0x557b9130ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.394+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 >> 172.21.15.49:0/4200173237 conn(0x557b9129d340 msgr2=0x557b9129e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.394+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 shutdown_connections 2024-02-26T06:27:43.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.394+0000 7f2d09969740 1 -- 172.21.15.49:0/4200173237 wait complete. 2024-02-26T06:27:43.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.394+0000 7f2d09969740 20 client.26515 trim_cache size 0 max 16384 2024-02-26T06:27:43.397 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 -- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ce80a2400 msgr2=0x7f2ce80a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ce80a2400 0x7f2ce80a48c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009cd0 tx=0x7f2cf4005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 -- 172.21.15.49:0/4294196334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d0010d030 msgr2=0x7f2d0010d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d0010d030 0x7f2d0010d4b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8037040 tx=0x7f2cf8034a90 comp rx=0 tx=0).stop 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 -- 172.21.15.49:0/4294196334 shutdown_connections 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2ce80a2400 0x7f2ce80a48c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d0010d030 0x7f2d0010d4b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d00106430 0x7f2d001a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 --2- 172.21.15.49:0/4294196334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d00101360 0x7f2d001a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 -- 172.21.15.49:0/4294196334 >> 172.21.15.49:0/4294196334 conn(0x7f2d000fcbe0 msgr2=0x7f2d000fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 -- 172.21.15.49:0/4294196334 shutdown_connections 2024-02-26T06:27:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.397+0000 7f2d06a00640 1 -- 172.21.15.49:0/4294196334 wait complete. 2024-02-26T06:27:43.429 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:27:43.429 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:27:43.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.736+0000 7f73a2800640 1 -- 172.21.15.49:0/2974282784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c0ff5d0 msgr2=0x7f739c0ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.736+0000 7f73a2800640 1 --2- 172.21.15.49:0/2974282784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c0ff5d0 0x7f739c0ff9b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f738c0074e0 tx=0x7f738c02c8f0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.737+0000 7f73a2800640 1 -- 172.21.15.49:0/2974282784 shutdown_connections 2024-02-26T06:27:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.737+0000 7f73a2800640 1 --2- 172.21.15.49:0/2974282784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f739c1046a0 0x7f739c10c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.737+0000 7f73a2800640 1 --2- 172.21.15.49:0/2974282784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f739c0fff80 0x7f739c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.737+0000 7f73a2800640 1 --2- 172.21.15.49:0/2974282784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c0ff5d0 0x7f739c0ff9b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.737+0000 7f73a2800640 1 -- 172.21.15.49:0/2974282784 >> 172.21.15.49:0/2974282784 conn(0x7f739c0761d0 msgr2=0x7f739c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.737+0000 7f73a2800640 1 -- 172.21.15.49:0/2974282784 shutdown_connections 2024-02-26T06:27:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.737+0000 7f73a2800640 1 -- 172.21.15.49:0/2974282784 wait complete. 2024-02-26T06:27:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.739+0000 7f73a2800640 1 Processor -- start 2024-02-26T06:27:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.739+0000 7f73a2800640 1 -- start start 2024-02-26T06:27:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f73a2800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f739c0ff5d0 0x7f739c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f73a2800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c1046a0 0x7f739c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f73a2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f739c197610 0x7f739c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f7393e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c1046a0 0x7f739c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f7393e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c1046a0 0x7f739c19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43318/0 (socket says 172.21.15.49:43318) 2024-02-26T06:27:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f73a2800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c198000 con 0x7f739c0ff5d0 2024-02-26T06:27:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f73a2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c198170 con 0x7f739c197610 2024-02-26T06:27:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f73a2800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c198330 con 0x7f739c1046a0 2024-02-26T06:27:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f7393e00640 1 -- 172.21.15.49:0/2081728960 learned_addr learned my addr 172.21.15.49:0/2081728960 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.740+0000 7f739be00640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f739c197610 0x7f739c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.741+0000 7f739b400640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f739c0ff5d0 0x7f739c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.741+0000 7f7393e00640 1 -- 172.21.15.49:0/2081728960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f739c197610 msgr2=0x7f739c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.741+0000 7f7393e00640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f739c197610 0x7f739c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.741+0000 7f7393e00640 1 -- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f739c0ff5d0 msgr2=0x7f739c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.741+0000 7f7393e00640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f739c0ff5d0 0x7f739c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.741+0000 7f7393e00640 1 -- 172.21.15.49:0/2081728960 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c0071c0 con 0x7f739c1046a0 2024-02-26T06:27:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.741+0000 7f7393e00640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c1046a0 0x7f739c19d9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f738400d770 tx=0x7f738400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.742+0000 7f7398c00640 1 -- 172.21.15.49:0/2081728960 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384009c30 con 0x7f739c1046a0 2024-02-26T06:27:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.742+0000 7f7398c00640 1 -- 172.21.15.49:0/2081728960 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f738400de80 con 0x7f739c1046a0 2024-02-26T06:27:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.742+0000 7f73a2800640 1 -- 172.21.15.49:0/2081728960 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c106240 con 0x7f739c1046a0 2024-02-26T06:27:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.742+0000 7f7398c00640 1 -- 172.21.15.49:0/2081728960 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400f6b0 con 0x7f739c1046a0 2024-02-26T06:27:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.742+0000 7f73a2800640 1 -- 172.21.15.49:0/2081728960 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c1067a0 con 0x7f739c1046a0 2024-02-26T06:27:43.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.747+0000 7f73a57bd740 1 Processor -- start 2024-02-26T06:27:43.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.748+0000 7f73a57bd740 1 -- start start 2024-02-26T06:27:43.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.748+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734febf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.749+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.749+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fecad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.749+0000 7f73a57bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734fe251e0 con 0x55734febf420 2024-02-26T06:27:43.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.749+0000 7f739be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734febf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734febf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43792/0 (socket says 172.21.15.49:43792) 2024-02-26T06:27:43.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739be00640 1 -- 172.21.15.49:0/1161132436 learned_addr learned my addr 172.21.15.49:0/1161132436 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f73a57bd740 1 -- 172.21.15.49:0/1161132436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734fe22710 con 0x55734febfd40 2024-02-26T06:27:43.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f73a57bd740 1 -- 172.21.15.49:0/1161132436 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734fe1def0 con 0x55734fec8920 2024-02-26T06:27:43.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739b400640 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739be00640 1 -- 172.21.15.49:0/1161132436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 msgr2=0x55734fecad10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739be00640 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fecad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739be00640 1 -- 172.21.15.49:0/1161132436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 msgr2=0x55734fec83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739be00640 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739be00640 1 -- 172.21.15.49:0/1161132436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55734fddfdc0 con 0x55734febf420 2024-02-26T06:27:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.750+0000 7f739b400640 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.751+0000 7f7398c00640 1 -- 172.21.15.49:0/2081728960 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7384010460 con 0x7f739c1046a0 2024-02-26T06:27:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.751+0000 7f739be00640 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734febf800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7394009fc0 tx=0x7f7394010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700b0c260a6ab028 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.752+0000 7f7390c00640 1 -- 172.21.15.49:0/1161132436 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394004d20 con 0x55734febf420 2024-02-26T06:27:43.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.752+0000 7f7390c00640 1 -- 172.21.15.49:0/1161132436 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7394004e80 con 0x55734febf420 2024-02-26T06:27:43.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.752+0000 7f7390c00640 1 -- 172.21.15.49:0/1161132436 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940055a0 con 0x55734febf420 2024-02-26T06:27:43.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.753+0000 7f73a57bd740 1 -- 172.21.15.49:0/1161132436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 msgr2=0x55734febf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.753+0000 7f73a57bd740 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734febf800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7394009fc0 tx=0x7f7394010800 comp rx=0 tx=0).stop 2024-02-26T06:27:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.753+0000 7f73a57bd740 1 -- 172.21.15.49:0/1161132436 shutdown_connections 2024-02-26T06:27:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.753+0000 7f73a57bd740 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.753+0000 7f73a57bd740 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fecad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.754+0000 7f73a57bd740 1 --2- 172.21.15.49:0/1161132436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734febf800 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.754+0000 7f73a57bd740 1 -- 172.21.15.49:0/1161132436 >> 172.21.15.49:0/1161132436 conn(0x55734feb9b50 msgr2=0x55734feb9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.754+0000 7f73a57bd740 1 -- 172.21.15.49:0/1161132436 shutdown_connections 2024-02-26T06:27:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.755+0000 7f73a57bd740 1 -- 172.21.15.49:0/1161132436 wait complete. 2024-02-26T06:27:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.755+0000 7f73a57bd740 1 -- start start 2024-02-26T06:27:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.756+0000 7f73a57bd740 1 Processor -- start 2024-02-26T06:27:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.756+0000 7f7398c00640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f73800a21c0 0x7f73800a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.756+0000 7f7398c00640 1 -- 172.21.15.49:0/2081728960 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f73840cddb0 con 0x7f739c1046a0 2024-02-26T06:27:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.756+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734fec5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.756+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.756+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fec6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.757+0000 7f73a57bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734fdc3120 con 0x55734febf420 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.757+0000 7f73a57bd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734fdc2dc0 con 0x55734febfd40 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.757+0000 7f73a57bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734fb8c1c0 con 0x55734fec8920 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.757+0000 7f73a57bd740 10 client.0 _ll_register_callbacks cb 0x55734fe12280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.757+0000 7f739be00640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f73800a21c0 0x7f73800a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.757+0000 7f739b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734fec5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.757+0000 7f739be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fec6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.758+0000 7f7393e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.758+0000 7f739be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fec6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43322/0 (socket says 172.21.15.49:43322) 2024-02-26T06:27:43.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.758+0000 7f739be00640 1 -- 172.21.15.49:0/3733703254 learned_addr learned my addr 172.21.15.49:0/3733703254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.759+0000 7f7393e00640 1 -- 172.21.15.49:0/3733703254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 msgr2=0x55734fec6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.759+0000 7f7393e00640 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fec6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.759+0000 7f7393e00640 1 -- 172.21.15.49:0/3733703254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 msgr2=0x55734fec5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.759+0000 7f7393e00640 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734fec5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.759+0000 7f7393e00640 1 -- 172.21.15.49:0/3733703254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7394009bd0 con 0x55734febfd40 2024-02-26T06:27:43.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.759+0000 7f739be00640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f73800a21c0 0x7f73800a4680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7394003a50 tx=0x7f73940050e0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.759+0000 7f7393e00640 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec5fd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f73840cb510 tx=0x7f738409af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751c8357f175bfcf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.760+0000 7f7389600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:43.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.760+0000 7f7389600640 1 -- 172.21.15.49:0/3733703254 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73840a2f50 con 0x55734febfd40 2024-02-26T06:27:43.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.760+0000 7f7389600640 1 -- 172.21.15.49:0/3733703254 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f73840cb790 con 0x55734febfd40 2024-02-26T06:27:43.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.760+0000 7f73a57bd740 1 -- 172.21.15.49:0/3733703254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55734fb5a720 con 0x55734febfd40 2024-02-26T06:27:43.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f7389600640 1 -- 172.21.15.49:0/3733703254 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73840cb950 con 0x55734febfd40 2024-02-26T06:27:43.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f73a57bd740 10 client.26516 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:43.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f73a57bd740 1 -- 172.21.15.49:0/3733703254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55734fe1c570 con 0x55734febfd40 2024-02-26T06:27:43.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f7389600640 1 -- 172.21.15.49:0/3733703254 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55734fb5a720 con 0x55734febfd40 2024-02-26T06:27:43.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f7389600640 1 -- 172.21.15.49:0/3733703254 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f73840ba240 con 0x55734febfd40 2024-02-26T06:27:43.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f73a57bd740 10 client.26516 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:43.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f73a57bd740 10 client.26516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:27:43.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f73a57bd740 20 client.26516 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:43.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f73a57bd740 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55734fed8ab0 0x55734fedaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f73a57bd740 4 client.26516 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:43.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.761+0000 7f73a57bd740 1 -- 172.21.15.49:0/3733703254 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55734fba9890 con 0x55734fed8ab0 2024-02-26T06:27:43.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.762+0000 7f739b400640 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55734fed8ab0 0x55734fedaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.763+0000 7f739b400640 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55734fed8ab0 0x55734fedaed0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a84c7b82c952350b server_cookie=e98a9d2a7081ef7a in_seq=0 out_seq=0 2024-02-26T06:27:43.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.763+0000 7f7389600640 10 client.26516 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:27:43.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.763+0000 7f7389600640 1 -- 172.21.15.49:0/3733703254 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55734fba9890 con 0x55734fed8ab0 2024-02-26T06:27:43.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.764+0000 7f7389600640 10 client.26516 handle_command_reply: tid=0 2024-02-26T06:27:43.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.764+0000 7f73a57bd740 1 client.26516 shutdown 2024-02-26T06:27:43.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.764+0000 7f73a57bd740 1 -- 172.21.15.49:0/3733703254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 msgr2=0x55734fec5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.764+0000 7f73a57bd740 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec5fd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f73840cb510 tx=0x7f738409af70 comp rx=0 tx=0).stop 2024-02-26T06:27:43.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.765+0000 7f73a57bd740 1 -- 172.21.15.49:0/3733703254 shutdown_connections 2024-02-26T06:27:43.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.765+0000 7f73a57bd740 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55734fed8ab0 0x55734fedaed0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.765+0000 7f73a57bd740 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febfd40 0x55734fec5fd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.765+0000 7f73a57bd740 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fec6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.765+0000 7f73a57bd740 1 --2- 172.21.15.49:0/3733703254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febf420 0x55734fec5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.765+0000 7f73a57bd740 1 -- 172.21.15.49:0/3733703254 >> 172.21.15.49:0/3733703254 conn(0x55734feb9b50 msgr2=0x55734febb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.765+0000 7f7389600640 0 client.26516 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:27:43.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.765+0000 7f73a57bd740 1 -- 172.21.15.49:0/3733703254 shutdown_connections 2024-02-26T06:27:43.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.766+0000 7f73a57bd740 1 -- 172.21.15.49:0/3733703254 wait complete. 2024-02-26T06:27:43.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.766+0000 7f73a57bd740 20 client.26516 trim_cache size 0 max 16384 2024-02-26T06:27:43.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.792+0000 7f73a57bd740 1 Processor -- start 2024-02-26T06:27:43.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.792+0000 7f73a57bd740 1 -- start start 2024-02-26T06:27:43.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.792+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fec81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.792+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734febfd40 0x55734fec6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734fec8920 0x55734fec2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f73a57bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734fb5a720 con 0x55734fec8920 2024-02-26T06:27:43.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f73a57bd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734febe510 con 0x55734febf420 2024-02-26T06:27:43.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f73a57bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734feaca70 con 0x55734febfd40 2024-02-26T06:27:43.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f7393e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734febfd40 0x55734fec6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f7393e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734febfd40 0x55734fec6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43326/0 (socket says 172.21.15.49:43326) 2024-02-26T06:27:43.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f7393e00640 1 -- 172.21.15.49:0/4096658862 learned_addr learned my addr 172.21.15.49:0/4096658862 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f739b400640 1 --2- 172.21.15.49:0/4096658862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fec81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f739b400640 1 -- 172.21.15.49:0/4096658862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734febfd40 msgr2=0x55734fec6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f739b400640 1 --2- 172.21.15.49:0/4096658862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734febfd40 0x55734fec6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f739b400640 1 -- 172.21.15.49:0/4096658862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734fec8920 msgr2=0x55734fec2d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:43.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f739b400640 1 --2- 172.21.15.49:0/4096658862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734fec8920 0x55734fec2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.793+0000 7f739b400640 1 -- 172.21.15.49:0/4096658862 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55734fdb8a80 con 0x55734febf420 2024-02-26T06:27:43.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.794+0000 7f739b400640 1 --2- 172.21.15.49:0/4096658862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fec81c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f738c000c00 tx=0x7f738c00d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf1c853976950d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.794+0000 7f738be00640 1 -- 172.21.15.49:0/4096658862 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c04c030 con 0x55734febf420 2024-02-26T06:27:43.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.794+0000 7f738be00640 1 -- 172.21.15.49:0/4096658862 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f738c00a8d0 con 0x55734febf420 2024-02-26T06:27:43.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.794+0000 7f738be00640 1 -- 172.21.15.49:0/4096658862 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c043040 con 0x55734febf420 2024-02-26T06:27:43.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.795+0000 7f73a57bd740 1 -- 172.21.15.49:0/4096658862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 msgr2=0x55734fec81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.795+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4096658862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fec81c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f738c000c00 tx=0x7f738c00d000 comp rx=0 tx=0).stop 2024-02-26T06:27:43.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.796+0000 7f73a57bd740 1 -- 172.21.15.49:0/4096658862 shutdown_connections 2024-02-26T06:27:43.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.796+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4096658862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734fec8920 0x55734fec2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.796+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4096658862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734febfd40 0x55734fec6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.796+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4096658862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fec81c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.796+0000 7f73a57bd740 1 -- 172.21.15.49:0/4096658862 >> 172.21.15.49:0/4096658862 conn(0x55734feb9b50 msgr2=0x55734fedf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.797+0000 7f73a57bd740 1 -- 172.21.15.49:0/4096658862 shutdown_connections 2024-02-26T06:27:43.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.797+0000 7f73a57bd740 1 -- 172.21.15.49:0/4096658862 wait complete. 2024-02-26T06:27:43.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.797+0000 7f73a57bd740 1 -- start start 2024-02-26T06:27:43.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.798+0000 7f73a57bd740 1 Processor -- start 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.799+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fece390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.799+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febfd40 0x55734fece8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.799+0000 7f73a57bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fecee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.799+0000 7f73a57bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734ff2f0e0 con 0x55734febfd40 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.799+0000 7f73a57bd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734ff2f250 con 0x55734febf420 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.799+0000 7f73a57bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734ff2f410 con 0x55734fec8920 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.799+0000 7f73a57bd740 10 client.0 _ll_register_callbacks cb 0x7f7394009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.799+0000 7f739b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fece390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.800+0000 7f739b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fece390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47518/0 (socket says 172.21.15.49:47518) 2024-02-26T06:27:43.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.800+0000 7f739b400640 1 -- 172.21.15.49:0/4143611091 learned_addr learned my addr 172.21.15.49:0/4143611091 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:43.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.800+0000 7f739b400640 1 -- 172.21.15.49:0/4143611091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 msgr2=0x55734fecee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.800+0000 7f739b400640 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fecee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.800+0000 7f739b400640 1 -- 172.21.15.49:0/4143611091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febfd40 msgr2=0x55734fece8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:43.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.800+0000 7f739b400640 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febfd40 0x55734fece8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.800+0000 7f739b400640 1 -- 172.21.15.49:0/4143611091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c00a0d0 con 0x55734febf420 2024-02-26T06:27:43.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.801+0000 7f739b400640 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fece390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f738c009010 tx=0x7f738c02c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc0a1bbec72b6fc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:43.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.801+0000 7f7389600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.803+0000 7f7389600640 1 -- 172.21.15.49:0/4143611091 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c043050 con 0x55734febf420 2024-02-26T06:27:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.803+0000 7f7389600640 1 -- 172.21.15.49:0/4143611091 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f738c04d070 con 0x55734febf420 2024-02-26T06:27:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.803+0000 7f73a57bd740 1 -- 172.21.15.49:0/4143611091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55734fb1d7e0 con 0x55734febf420 2024-02-26T06:27:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.803+0000 7f7389600640 1 -- 172.21.15.49:0/4143611091 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f738c04c040 con 0x55734febf420 2024-02-26T06:27:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.803+0000 7f73a57bd740 10 client.26522 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.803+0000 7f73a57bd740 1 -- 172.21.15.49:0/4143611091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55734fdde620 con 0x55734febf420 2024-02-26T06:27:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.804+0000 7f7389600640 1 -- 172.21.15.49:0/4143611091 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55734fb1d7e0 con 0x55734febf420 2024-02-26T06:27:43.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.805+0000 7f7389600640 1 -- 172.21.15.49:0/4143611091 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f738c032b00 con 0x55734febf420 2024-02-26T06:27:43.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.805+0000 7f73a57bd740 10 client.26522 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:43.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.805+0000 7f73a57bd740 10 client.26522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:27:43.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.805+0000 7f73a57bd740 20 client.26522 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:43.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.805+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55734ff30850 0x55734ff32c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:43.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.806+0000 7f73a57bd740 4 client.26522 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:27:43.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.806+0000 7f73a57bd740 1 -- 172.21.15.49:0/4143611091 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55734fb0c4e0 con 0x55734ff30850 2024-02-26T06:27:43.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.807+0000 7f7393e00640 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55734ff30850 0x55734ff32c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:43.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.811+0000 7f7393e00640 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55734ff30850 0x55734ff32c90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87178809176c43b7 server_cookie=25e2931f64c80e1e in_seq=0 out_seq=0 2024-02-26T06:27:43.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.812+0000 7f7389600640 10 client.26522 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:27:43.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.812+0000 7f7389600640 1 -- 172.21.15.49:0/4143611091 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55734fb0c4e0 con 0x55734ff30850 2024-02-26T06:27:43.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.812+0000 7f7389600640 10 client.26522 handle_command_reply: tid=0 2024-02-26T06:27:43.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.812+0000 7f73a57bd740 1 client.26522 shutdown 2024-02-26T06:27:43.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.814+0000 7f73a57bd740 1 -- 172.21.15.49:0/4143611091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 msgr2=0x55734fece390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.814+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fece390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f738c009010 tx=0x7f738c02c8f0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.814+0000 7f73a57bd740 1 -- 172.21.15.49:0/4143611091 shutdown_connections 2024-02-26T06:27:43.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.814+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55734ff30850 0x55734ff32c90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.815+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55734fec8920 0x55734fecee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.815+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55734febfd40 0x55734fece8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.815+0000 7f73a57bd740 1 --2- 172.21.15.49:0/4143611091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55734febf420 0x55734fece390 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.815+0000 7f7389600640 0 client.26522 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:27:43.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.815+0000 7f73a57bd740 1 -- 172.21.15.49:0/4143611091 >> 172.21.15.49:0/4143611091 conn(0x55734feb9b50 msgr2=0x55734feafbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.816+0000 7f73a57bd740 1 -- 172.21.15.49:0/4143611091 shutdown_connections 2024-02-26T06:27:43.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.817+0000 7f73a57bd740 1 -- 172.21.15.49:0/4143611091 wait complete. 2024-02-26T06:27:43.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.817+0000 7f73a57bd740 20 client.26522 trim_cache size 0 max 16384 2024-02-26T06:27:43.818 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:27:43.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.818+0000 7f73a2800640 1 -- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f73800a21c0 msgr2=0x7f73800a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.818+0000 7f73a2800640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f73800a21c0 0x7f73800a4680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7394003a50 tx=0x7f73940050e0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.818+0000 7f73a2800640 1 -- 172.21.15.49:0/2081728960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c1046a0 msgr2=0x7f739c19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:43.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.818+0000 7f73a2800640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c1046a0 0x7f739c19d9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f738400d770 tx=0x7f738400dc30 comp rx=0 tx=0).stop 2024-02-26T06:27:43.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.819+0000 7f73a2800640 1 -- 172.21.15.49:0/2081728960 shutdown_connections 2024-02-26T06:27:43.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.819+0000 7f73a2800640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f73800a21c0 0x7f73800a4680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.819+0000 7f73a2800640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f739c197610 0x7f739c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.819+0000 7f73a2800640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f739c1046a0 0x7f739c19d9a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.819+0000 7f73a2800640 1 --2- 172.21.15.49:0/2081728960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f739c0ff5d0 0x7f739c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:43.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.819+0000 7f73a2800640 1 -- 172.21.15.49:0/2081728960 >> 172.21.15.49:0/2081728960 conn(0x7f739c0761d0 msgr2=0x7f739c1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:43.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.819+0000 7f73a2800640 1 -- 172.21.15.49:0/2081728960 shutdown_connections 2024-02-26T06:27:43.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:43.820+0000 7f73a2800640 1 -- 172.21.15.49:0/2081728960 wait complete. 2024-02-26T06:27:43.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:43 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4029414216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:43.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:43 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4029414216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:43.852 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:43 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4029414216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:43.855 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:27:43.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:27:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.157+0000 7f07b6e00640 1 -- 172.21.15.49:0/1985323401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 msgr2=0x7f07b0104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.157+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1985323401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 0x7f07b0104960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f079c00d940 tx=0x7f079c02dbb0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.158+0000 7f07b6e00640 1 -- 172.21.15.49:0/1985323401 shutdown_connections 2024-02-26T06:27:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.158+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1985323401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07b010a260 0x7f07b010c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.158+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1985323401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 0x7f07b0104960 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.158+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1985323401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b00ffbf0 0x7f07b0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.158+0000 7f07b6e00640 1 -- 172.21.15.49:0/1985323401 >> 172.21.15.49:0/1985323401 conn(0x7f07b00761d0 msgr2=0x7f07b00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.158+0000 7f07b6e00640 1 -- 172.21.15.49:0/1985323401 shutdown_connections 2024-02-26T06:27:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.159+0000 7f07b6e00640 1 -- 172.21.15.49:0/1985323401 wait complete. 2024-02-26T06:27:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.160+0000 7f07b6e00640 1 Processor -- start 2024-02-26T06:27:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07b6e00640 1 -- start start 2024-02-26T06:27:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07b6e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07b00ffbf0 0x7f07b019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07b6e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 0x7f07b019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07b6e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b010a260 0x7f07b01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07b6e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0199f30 con 0x7f07b010a260 2024-02-26T06:27:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07b6e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b019a0a0 con 0x7f07b00ffbf0 2024-02-26T06:27:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07b6e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b019a260 con 0x7f07b0104500 2024-02-26T06:27:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07afe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07b00ffbf0 0x7f07b019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.161+0000 7f07af400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 0x7f07b019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07af400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 0x7f07b019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43346/0 (socket says 172.21.15.49:43346) 2024-02-26T06:27:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07afe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07b00ffbf0 0x7f07b019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47544/0 (socket says 172.21.15.49:47544) 2024-02-26T06:27:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07af400640 1 -- 172.21.15.49:0/1064012954 learned_addr learned my addr 172.21.15.49:0/1064012954 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07b4e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b010a260 0x7f07b01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07afe00640 1 -- 172.21.15.49:0/1064012954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 msgr2=0x7f07b019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07afe00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 0x7f07b019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07afe00640 1 -- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b010a260 msgr2=0x7f07b01998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07afe00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b010a260 0x7f07b01998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07afe00640 1 -- 172.21.15.49:0/1064012954 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c00d570 con 0x7f07b00ffbf0 2024-02-26T06:27:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.162+0000 7f07af400640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 0x7f07b019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.164+0000 7f07afe00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07b00ffbf0 0x7f07b019f6e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f07a000b3f0 tx=0x7f07a000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.164+0000 7f07acc00640 1 -- 172.21.15.49:0/1064012954 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0011830 con 0x7f07b00ffbf0 2024-02-26T06:27:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.164+0000 7f07acc00640 1 -- 172.21.15.49:0/1064012954 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07a0011e70 con 0x7f07b00ffbf0 2024-02-26T06:27:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.165+0000 7f07b6e00640 1 -- 172.21.15.49:0/1064012954 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b019a540 con 0x7f07b00ffbf0 2024-02-26T06:27:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.165+0000 7f07b6e00640 1 -- 172.21.15.49:0/1064012954 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b01a6780 con 0x7f07b00ffbf0 2024-02-26T06:27:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.165+0000 7f07acc00640 1 -- 172.21.15.49:0/1064012954 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000f2e0 con 0x7f07b00ffbf0 2024-02-26T06:27:44.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.170+0000 7f07b9c52740 1 Processor -- start 2024-02-26T06:27:44.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.171+0000 7f07b9c52740 1 -- start start 2024-02-26T06:27:44.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.173+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 0x55b817ea5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.173+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eaebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.173+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eb1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.173+0000 7f07b9c52740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e141e0 con 0x55b817ea5c10 2024-02-26T06:27:44.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.173+0000 7f07b9c52740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e11710 con 0x55b817ea6530 2024-02-26T06:27:44.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.173+0000 7f07b9c52740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e0cef0 con 0x55b817eaf110 2024-02-26T06:27:44.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.173+0000 7f07acc00640 1 -- 172.21.15.49:0/1064012954 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f07a0011990 con 0x7f07b00ffbf0 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.174+0000 7f07af400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eb1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.174+0000 7f07af400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eb1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43360/0 (socket says 172.21.15.49:43360) 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.174+0000 7f07af400640 1 -- 172.21.15.49:0/1730808597 learned_addr learned my addr 172.21.15.49:0/1730808597 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.174+0000 7f07af400640 1 -- 172.21.15.49:0/1730808597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 msgr2=0x55b817eaebd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.174+0000 7f07af400640 1 --2- 172.21.15.49:0/1730808597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eaebd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.174+0000 7f07af400640 1 -- 172.21.15.49:0/1730808597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 msgr2=0x55b817ea5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.174+0000 7f07af400640 1 --2- 172.21.15.49:0/1730808597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 0x55b817ea5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.174+0000 7f07af400640 1 -- 172.21.15.49:0/1730808597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b817dcedc0 con 0x55b817eaf110 2024-02-26T06:27:44.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.175+0000 7f07af400640 1 --2- 172.21.15.49:0/1730808597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eb1500 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f079c000c00 tx=0x7f079c004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0e823decd77ec0e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:44.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.175+0000 7f07a5600640 1 -- 172.21.15.49:0/1730808597 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c004960 con 0x55b817eaf110 2024-02-26T06:27:44.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.175+0000 7f07a5600640 1 -- 172.21.15.49:0/1730808597 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f079c0052d0 con 0x55b817eaf110 2024-02-26T06:27:44.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.175+0000 7f07a5600640 1 -- 172.21.15.49:0/1730808597 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c02eda0 con 0x55b817eaf110 2024-02-26T06:27:44.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.175+0000 7f07acc00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f07940a2440 0x7f07940a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.176+0000 7f07b9c52740 1 -- 172.21.15.49:0/1730808597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 msgr2=0x55b817eb1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.176+0000 7f07b9c52740 1 --2- 172.21.15.49:0/1730808597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eb1500 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f079c000c00 tx=0x7f079c004870 comp rx=0 tx=0).stop 2024-02-26T06:27:44.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.177+0000 7f07b4e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f07940a2440 0x7f07940a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.177+0000 7f07acc00640 1 -- 172.21.15.49:0/1064012954 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f07a00cd930 con 0x7f07b00ffbf0 2024-02-26T06:27:44.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.177+0000 7f07b9c52740 1 -- 172.21.15.49:0/1730808597 shutdown_connections 2024-02-26T06:27:44.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.177+0000 7f07b9c52740 1 --2- 172.21.15.49:0/1730808597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eaebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.177+0000 7f07b9c52740 1 --2- 172.21.15.49:0/1730808597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eb1500 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.177+0000 7f07b9c52740 1 --2- 172.21.15.49:0/1730808597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 0x55b817ea5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.177+0000 7f07b9c52740 1 -- 172.21.15.49:0/1730808597 >> 172.21.15.49:0/1730808597 conn(0x55b817ea0340 msgr2=0x55b817ea0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:44.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.178+0000 7f07b9c52740 1 -- 172.21.15.49:0/1730808597 shutdown_connections 2024-02-26T06:27:44.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.178+0000 7f07b4e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f07940a2440 0x7f07940a4900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f07a800a830 tx=0x7f07a800a780 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:44.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.178+0000 7f07b9c52740 1 -- 172.21.15.49:0/1730808597 wait complete. 2024-02-26T06:27:44.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.179+0000 7f07b9c52740 1 -- start start 2024-02-26T06:27:44.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.179+0000 7f07b9c52740 1 Processor -- start 2024-02-26T06:27:44.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.180+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 0x55b817eac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.180+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.180+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eacb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.180+0000 7f07b9c52740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817db2120 con 0x55b817ea5c10 2024-02-26T06:27:44.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.180+0000 7f07b9c52740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817db1dc0 con 0x55b817ea6530 2024-02-26T06:27:44.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.180+0000 7f07b9c52740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817b7b1c0 con 0x55b817eaf110 2024-02-26T06:27:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.181+0000 7f07afe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 0x55b817eac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.181+0000 7f07af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.181+0000 7f07b4e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eacb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.181+0000 7f07b4e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eacb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43364/0 (socket says 172.21.15.49:43364) 2024-02-26T06:27:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.181+0000 7f07af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47574/0 (socket says 172.21.15.49:47574) 2024-02-26T06:27:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.181+0000 7f07b4e00640 1 -- 172.21.15.49:0/129893744 learned_addr learned my addr 172.21.15.49:0/129893744 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:44.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.181+0000 7f07b9c52740 10 client.0 _ll_register_callbacks cb 0x55b817e01280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.182+0000 7f07b4e00640 1 -- 172.21.15.49:0/129893744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 msgr2=0x55b817eac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.182+0000 7f07b4e00640 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.182+0000 7f07b4e00640 1 -- 172.21.15.49:0/129893744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 msgr2=0x55b817eac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.182+0000 7f07b4e00640 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 0x55b817eac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.182+0000 7f07b4e00640 1 -- 172.21.15.49:0/129893744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c03a040 con 0x55b817eaf110 2024-02-26T06:27:44.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.184+0000 7f079a000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:44.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.184+0000 7f07b4e00640 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eacb50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f07a800d770 tx=0x7f07a800a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6fa7f9473b59d7d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:44.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.184+0000 7f079a000640 1 -- 172.21.15.49:0/129893744 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8013070 con 0x55b817eaf110 2024-02-26T06:27:44.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.185+0000 7f079a000640 1 -- 172.21.15.49:0/129893744 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f07a8004b20 con 0x55b817eaf110 2024-02-26T06:27:44.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.185+0000 7f07b9c52740 1 -- 172.21.15.49:0/129893744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b817b0c7e0 con 0x55b817eaf110 2024-02-26T06:27:44.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.186+0000 7f079a000640 1 -- 172.21.15.49:0/129893744 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a8004ce0 con 0x55b817eaf110 2024-02-26T06:27:44.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.187+0000 7f07b9c52740 10 client.26539 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:44.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.187+0000 7f07b9c52740 1 -- 172.21.15.49:0/129893744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b817e0b570 con 0x55b817eaf110 2024-02-26T06:27:44.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.187+0000 7f079a000640 1 -- 172.21.15.49:0/129893744 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b817b0c7e0 con 0x55b817eaf110 2024-02-26T06:27:44.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.188+0000 7f079a000640 1 -- 172.21.15.49:0/129893744 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07a8024b40 con 0x55b817eaf110 2024-02-26T06:27:44.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.188+0000 7f07b9c52740 10 client.26539 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.188+0000 7f07b9c52740 10 client.26539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.188+0000 7f07b9c52740 20 client.26539 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.188+0000 7f07b9c52740 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b817ec7b30 0x55b817ec9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.188+0000 7f07b9c52740 4 client.26539 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.188+0000 7f07b9c52740 1 -- 172.21.15.49:0/129893744 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b817b49720 con 0x55b817ec7b30 2024-02-26T06:27:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.189+0000 7f07afe00640 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b817ec7b30 0x55b817ec9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.190+0000 7f07afe00640 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b817ec7b30 0x55b817ec9f50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def9ba6a3e98193 server_cookie=2cd2dc6f5bc17802 in_seq=0 out_seq=0 2024-02-26T06:27:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.190+0000 7f079a000640 10 client.26539 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.191+0000 7f079a000640 1 -- 172.21.15.49:0/129893744 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b817b49720 con 0x55b817ec7b30 2024-02-26T06:27:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.191+0000 7f079a000640 10 client.26539 handle_command_reply: tid=0 2024-02-26T06:27:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.191+0000 7f07b9c52740 1 client.26539 shutdown 2024-02-26T06:27:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 -- 172.21.15.49:0/129893744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 msgr2=0x55b817eacb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eacb50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f07a800d770 tx=0x7f07a800a560 comp rx=0 tx=0).stop 2024-02-26T06:27:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 -- 172.21.15.49:0/129893744 shutdown_connections 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b817ec7b30 0x55b817ec9f50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817eaf110 0x55b817eacb50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 --2- 172.21.15.49:0/129893744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817ea5c10 0x55b817eac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 -- 172.21.15.49:0/129893744 >> 172.21.15.49:0/129893744 conn(0x55b817ea0340 msgr2=0x55b817ea2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f079a000640 0 client.26539 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.192+0000 7f07b9c52740 1 -- 172.21.15.49:0/129893744 shutdown_connections 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.193+0000 7f07b9c52740 1 -- 172.21.15.49:0/129893744 wait complete. 2024-02-26T06:27:44.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.193+0000 7f07b9c52740 20 client.26539 trim_cache size 0 max 16384 2024-02-26T06:27:44.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.215+0000 7f07b9c52740 1 Processor -- start 2024-02-26T06:27:44.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.215+0000 7f07b9c52740 1 -- start start 2024-02-26T06:27:44.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.216+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817eae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.216+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.216+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ea9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.216+0000 7f07afe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817eae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.216+0000 7f07af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.216+0000 7f07af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47590/0 (socket says 172.21.15.49:47590) 2024-02-26T06:27:44.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07b4e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ea9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07afe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817eae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43366/0 (socket says 172.21.15.49:43366) 2024-02-26T06:27:44.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07b9c52740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817ea1140 con 0x55b817eaf110 2024-02-26T06:27:44.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07b9c52740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e9bc80 con 0x55b817ea6530 2024-02-26T06:27:44.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07b9c52740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817ea1340 con 0x55b817ea5c10 2024-02-26T06:27:44.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07af400640 1 -- 172.21.15.49:0/1664361082 learned_addr learned my addr 172.21.15.49:0/1664361082 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:44.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07af400640 1 -- 172.21.15.49:0/1664361082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 msgr2=0x55b817eae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07af400640 1 --2- 172.21.15.49:0/1664361082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817eae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07af400640 1 -- 172.21.15.49:0/1664361082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 msgr2=0x55b817ea9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07af400640 1 --2- 172.21.15.49:0/1664361082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ea9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07af400640 1 -- 172.21.15.49:0/1664361082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b817da7a80 con 0x55b817ea6530 2024-02-26T06:27:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.217+0000 7f07b4e00640 1 --2- 172.21.15.49:0/1664361082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ea9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.218+0000 7f07af400640 1 --2- 172.21.15.49:0/1664361082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f079c02e1c0 tx=0x7f079c02e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa525fb883012ae server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.219+0000 7f07a4c00640 1 -- 172.21.15.49:0/1664361082 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c004960 con 0x55b817ea6530 2024-02-26T06:27:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.219+0000 7f07a4c00640 1 -- 172.21.15.49:0/1664361082 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f079c004ac0 con 0x55b817ea6530 2024-02-26T06:27:44.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.219+0000 7f07a4c00640 1 -- 172.21.15.49:0/1664361082 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c004340 con 0x55b817ea6530 2024-02-26T06:27:44.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.220+0000 7f07b9c52740 1 -- 172.21.15.49:0/1664361082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 msgr2=0x55b817eac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.220+0000 7f07b9c52740 1 --2- 172.21.15.49:0/1664361082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f079c02e1c0 tx=0x7f079c02e8e0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.220+0000 7f07b9c52740 1 -- 172.21.15.49:0/1664361082 shutdown_connections 2024-02-26T06:27:44.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.220+0000 7f07b9c52740 1 --2- 172.21.15.49:0/1664361082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817eac970 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.220+0000 7f07b9c52740 1 --2- 172.21.15.49:0/1664361082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ea9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.220+0000 7f07b9c52740 1 --2- 172.21.15.49:0/1664361082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817eae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.221+0000 7f07b9c52740 1 -- 172.21.15.49:0/1664361082 >> 172.21.15.49:0/1664361082 conn(0x55b817ea0340 msgr2=0x55b817eaa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:44.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.221+0000 7f07b9c52740 1 -- 172.21.15.49:0/1664361082 shutdown_connections 2024-02-26T06:27:44.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.221+0000 7f07b9c52740 1 -- 172.21.15.49:0/1664361082 wait complete. 2024-02-26T06:27:44.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.221+0000 7f07b9c52740 1 -- start start 2024-02-26T06:27:44.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.222+0000 7f07b9c52740 1 Processor -- start 2024-02-26T06:27:44.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817ebeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817ebcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07b9c52740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ebd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07afe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817ebeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07afe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817ebeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43368/0 (socket says 172.21.15.49:43368) 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817ebcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817ebcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47602/0 (socket says 172.21.15.49:47602) 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07b9c52740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817ebdaf0 con 0x55b817eaf110 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07afe00640 1 -- 172.21.15.49:0/2270815557 learned_addr learned my addr 172.21.15.49:0/2270815557 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817ebdc60 con 0x55b817ea6530 2024-02-26T06:27:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.223+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817f1e000 con 0x55b817ea5c10 2024-02-26T06:27:44.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.224+0000 7f07b4e00640 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ebd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.224+0000 7f07b9c52740 10 client.0 _ll_register_callbacks cb 0x7f079c00d790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:44.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.224+0000 7f07afe00640 1 -- 172.21.15.49:0/2270815557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 msgr2=0x55b817ebcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:44.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.224+0000 7f07afe00640 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817ebcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.224+0000 7f07afe00640 1 -- 172.21.15.49:0/2270815557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 msgr2=0x55b817ebd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:44.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.224+0000 7f07afe00640 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ebd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.224+0000 7f07afe00640 1 -- 172.21.15.49:0/2270815557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a8017e70 con 0x55b817ea5c10 2024-02-26T06:27:44.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.224+0000 7f07b4e00640 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ebd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:44.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.225+0000 7f07afe00640 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817ebeb40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f07a00960c0 tx=0x7f07a00ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6761c28668cec0ab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:44.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.225+0000 7f079a000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:44.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.225+0000 7f079a000640 1 -- 172.21.15.49:0/2270815557 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a00a2b80 con 0x55b817ea5c10 2024-02-26T06:27:44.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.225+0000 7f079a000640 1 -- 172.21.15.49:0/2270815557 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f07a00b5070 con 0x55b817ea5c10 2024-02-26T06:27:44.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.225+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f079c0052d0 con 0x55b817ea5c10 2024-02-26T06:27:44.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.226+0000 7f079a000640 1 -- 172.21.15.49:0/2270815557 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a0096a50 con 0x55b817ea5c10 2024-02-26T06:27:44.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.227+0000 7f07b9c52740 10 client.26545 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:44.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.227+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b817dcedc0 con 0x55b817ea5c10 2024-02-26T06:27:44.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.227+0000 7f079a000640 1 -- 172.21.15.49:0/2270815557 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f079c0052d0 con 0x55b817ea5c10 2024-02-26T06:27:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.229+0000 7f079a000640 1 -- 172.21.15.49:0/2270815557 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07a009e1b0 con 0x55b817ea5c10 2024-02-26T06:27:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.229+0000 7f07b9c52740 10 client.26545 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.229+0000 7f07b9c52740 10 client.26545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:44.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.229+0000 7f07b9c52740 20 client.26545 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:44.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.229+0000 7f07b9c52740 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b817f1f5b0 0x55b817f219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:44.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.229+0000 7f07b9c52740 4 client.26545 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:27:44.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.229+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b817b98890 con 0x55b817f1f5b0 2024-02-26T06:27:44.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.230+0000 7f07af400640 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b817f1f5b0 0x55b817f219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.233+0000 7f07af400640 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b817f1f5b0 0x55b817f219d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7adc5c5846f5c15a server_cookie=112268af0f8422d0 in_seq=0 out_seq=0 2024-02-26T06:27:44.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.233+0000 7f079a000640 10 client.26545 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:44.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.233+0000 7f079a000640 1 -- 172.21.15.49:0/2270815557 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b817b98890 con 0x55b817f1f5b0 2024-02-26T06:27:44.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.234+0000 7f079a000640 10 client.26545 handle_command_reply: tid=0 2024-02-26T06:27:44.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.234+0000 7f07b9c52740 1 client.26545 shutdown 2024-02-26T06:27:44.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.237+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 msgr2=0x55b817ebeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.237+0000 7f07b9c52740 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817ebeb40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f07a00960c0 tx=0x7f07a00ba040 comp rx=0 tx=0).stop 2024-02-26T06:27:44.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.237+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 shutdown_connections 2024-02-26T06:27:44.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.237+0000 7f07b9c52740 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b817f1f5b0 0x55b817f219d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.238+0000 7f07b9c52740 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b817ea6530 0x55b817ebcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.238+0000 7f079a000640 0 client.26545 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.238+0000 7f07b9c52740 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b817eaf110 0x55b817ebd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.238+0000 7f07b9c52740 1 --2- 172.21.15.49:0/2270815557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b817ea5c10 0x55b817ebeb40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.238+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 >> 172.21.15.49:0/2270815557 conn(0x55b817ea0340 msgr2=0x55b817ea1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.238+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 shutdown_connections 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.238+0000 7f07b9c52740 1 -- 172.21.15.49:0/2270815557 wait complete. 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.238+0000 7f07b9c52740 20 client.26545 trim_cache size 0 max 16384 2024-02-26T06:27:44.241 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:27:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 -- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f07940a2440 msgr2=0x7f07940a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f07940a2440 0x7f07940a4900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f07a800a830 tx=0x7f07a800a780 comp rx=0 tx=0).stop 2024-02-26T06:27:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 -- 172.21.15.49:0/1064012954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07b00ffbf0 msgr2=0x7f07b019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07b00ffbf0 0x7f07b019f6e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f07a000b3f0 tx=0x7f07a000b8b0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 -- 172.21.15.49:0/1064012954 shutdown_connections 2024-02-26T06:27:44.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f07940a2440 0x7f07940a4900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b010a260 0x7f07b01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07b0104500 0x7f07b019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 --2- 172.21.15.49:0/1064012954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07b00ffbf0 0x7f07b019f6e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:44.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 -- 172.21.15.49:0/1064012954 >> 172.21.15.49:0/1064012954 conn(0x7f07b00761d0 msgr2=0x7f07b010bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:44.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 -- 172.21.15.49:0/1064012954 shutdown_connections 2024-02-26T06:27:44.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:44.241+0000 7f07b6e00640 1 -- 172.21.15.49:0/1064012954 wait complete. 2024-02-26T06:27:44.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:44 smithi080 ceph-mon[36206]: pgmap v133: 97 pgs: 97 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 589 op/s 2024-02-26T06:27:44.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:44 smithi115 ceph-mon[36426]: pgmap v133: 97 pgs: 97 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 589 op/s 2024-02-26T06:27:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:44 smithi049 ceph-mon[31696]: pgmap v133: 97 pgs: 97 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 589 op/s 2024-02-26T06:27:45.272 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:27:45.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:45.272 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:27:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.585+0000 7fbd52e00640 1 -- 172.21.15.49:0/3945123994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 msgr2=0x7fbd4c111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.585+0000 7fbd52e00640 1 --2- 172.21.15.49:0/3945123994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 0x7fbd4c111a80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009930 tx=0x7fbd3802bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.585+0000 7fbd52e00640 1 -- 172.21.15.49:0/3945123994 shutdown_connections 2024-02-26T06:27:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.585+0000 7fbd52e00640 1 --2- 172.21.15.49:0/3945123994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd4c111fc0 0x7fbd4c1143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.585+0000 7fbd52e00640 1 --2- 172.21.15.49:0/3945123994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 0x7fbd4c111a80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.585+0000 7fbd52e00640 1 --2- 172.21.15.49:0/3945123994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd4c0756b0 0x7fbd4c075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.585+0000 7fbd52e00640 1 -- 172.21.15.49:0/3945123994 >> 172.21.15.49:0/3945123994 conn(0x7fbd4c0fedf0 msgr2=0x7fbd4c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:45.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.587+0000 7fbd52e00640 1 -- 172.21.15.49:0/3945123994 shutdown_connections 2024-02-26T06:27:45.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.587+0000 7fbd52e00640 1 -- 172.21.15.49:0/3945123994 wait complete. 2024-02-26T06:27:45.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.588+0000 7fbd52e00640 1 Processor -- start 2024-02-26T06:27:45.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.589+0000 7fbd52e00640 1 -- start start 2024-02-26T06:27:45.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.589+0000 7fbd52e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd4c0756b0 0x7fbd4c06b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.589+0000 7fbd52e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 0x7fbd4c06b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.589+0000 7fbd52e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd4c111fc0 0x7fbd4c06bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.589+0000 7fbd52e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c1b2890 con 0x7fbd4c111fc0 2024-02-26T06:27:45.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.589+0000 7fbd52e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c1b2a00 con 0x7fbd4c0756b0 2024-02-26T06:27:45.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.589+0000 7fbd52e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c1b2bc0 con 0x7fbd4c075fd0 2024-02-26T06:27:45.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.590+0000 7fbd4b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 0x7fbd4c06b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.590+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd4c0756b0 0x7fbd4c06b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.590+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd4c0756b0 0x7fbd4c06b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47610/0 (socket says 172.21.15.49:47610) 2024-02-26T06:27:45.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.590+0000 7fbd4be00640 1 -- 172.21.15.49:0/1113389682 learned_addr learned my addr 172.21.15.49:0/1113389682 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:45.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.590+0000 7fbd50e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd4c111fc0 0x7fbd4c06bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.591+0000 7fbd4b400640 1 -- 172.21.15.49:0/1113389682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd4c0756b0 msgr2=0x7fbd4c06b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.591+0000 7fbd4b400640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd4c0756b0 0x7fbd4c06b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.591+0000 7fbd4b400640 1 -- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd4c111fc0 msgr2=0x7fbd4c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.591+0000 7fbd4b400640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd4c111fc0 0x7fbd4c06bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.591+0000 7fbd4b400640 1 -- 172.21.15.49:0/1113389682 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd38009560 con 0x7fbd4c075fd0 2024-02-26T06:27:45.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.591+0000 7fbd50e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd4c111fc0 0x7fbd4c06bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:45.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.591+0000 7fbd4b400640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 0x7fbd4c06b780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd38036040 tx=0x7fbd38004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:45.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.592+0000 7fbd48c00640 1 -- 172.21.15.49:0/1113389682 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38004440 con 0x7fbd4c075fd0 2024-02-26T06:27:45.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.592+0000 7fbd48c00640 1 -- 172.21.15.49:0/1113389682 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd380052d0 con 0x7fbd4c075fd0 2024-02-26T06:27:45.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.592+0000 7fbd48c00640 1 -- 172.21.15.49:0/1113389682 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38003d00 con 0x7fbd4c075fd0 2024-02-26T06:27:45.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.592+0000 7fbd52e00640 1 -- 172.21.15.49:0/1113389682 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c1b2e40 con 0x7fbd4c075fd0 2024-02-26T06:27:45.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.592+0000 7fbd52e00640 1 -- 172.21.15.49:0/1113389682 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c1b3270 con 0x7fbd4c075fd0 2024-02-26T06:27:45.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.598+0000 7fbd55d90740 1 Processor -- start 2024-02-26T06:27:45.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.599+0000 7fbd55d90740 1 -- start start 2024-02-26T06:27:45.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.599+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.599+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.600+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceeb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.600+0000 7fbd55d90740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ace9b2d0 con 0x55b5acee0830 2024-02-26T06:27:45.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.600+0000 7fbd55d90740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ace4de20 con 0x55b5acedff10 2024-02-26T06:27:45.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.600+0000 7fbd55d90740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ace4b350 con 0x55b5acee9410 2024-02-26T06:27:45.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.600+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.601+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43906/0 (socket says 172.21.15.49:43906) 2024-02-26T06:27:45.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.601+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.601+0000 7fbd4be00640 1 -- 172.21.15.49:0/117446913 learned_addr learned my addr 172.21.15.49:0/117446913 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:45.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.601+0000 7fbd48c00640 1 -- 172.21.15.49:0/1113389682 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fbd38004df0 con 0x7fbd4c075fd0 2024-02-26T06:27:45.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.602+0000 7fbd4be00640 1 -- 172.21.15.49:0/117446913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 msgr2=0x55b5aceeb800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:27:45.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.602+0000 7fbd4be00640 1 --2- 172.21.15.49:0/117446913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceeb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.602+0000 7fbd4be00640 1 -- 172.21.15.49:0/117446913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 msgr2=0x55b5acee02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:45.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.602+0000 7fbd4be00640 1 --2- 172.21.15.49:0/117446913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.602+0000 7fbd4be00640 1 -- 172.21.15.49:0/117446913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5ace0fc50 con 0x55b5acee0830 2024-02-26T06:27:45.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.603+0000 7fbd4be00640 1 --2- 172.21.15.49:0/117446913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee8ed0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009b10 tx=0x7fbd3c00e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0487f5dce25514f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:45.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.603+0000 7fbd48c00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fbd300a2330 0x7fbd300a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.603+0000 7fbd41600640 1 -- 172.21.15.49:0/117446913 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c009610 con 0x55b5acee0830 2024-02-26T06:27:45.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.603+0000 7fbd48c00640 1 -- 172.21.15.49:0/1113389682 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fbd380f06a0 con 0x7fbd4c075fd0 2024-02-26T06:27:45.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.603+0000 7fbd41600640 1 -- 172.21.15.49:0/117446913 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c00ebe0 con 0x55b5acee0830 2024-02-26T06:27:45.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.603+0000 7fbd41600640 1 -- 172.21.15.49:0/117446913 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c009610 con 0x55b5acee0830 2024-02-26T06:27:45.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.604+0000 7fbd50e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fbd300a2330 0x7fbd300a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.604+0000 7fbd55d90740 1 -- 172.21.15.49:0/117446913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 msgr2=0x55b5acee8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.604+0000 7fbd55d90740 1 --2- 172.21.15.49:0/117446913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee8ed0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009b10 tx=0x7fbd3c00e1d0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.605+0000 7fbd55d90740 1 -- 172.21.15.49:0/117446913 shutdown_connections 2024-02-26T06:27:45.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.605+0000 7fbd55d90740 1 --2- 172.21.15.49:0/117446913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceeb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.605+0000 7fbd50e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fbd300a2330 0x7fbd300a47f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c06a580 tx=0x7fbd440094a0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:45.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.605+0000 7fbd55d90740 1 --2- 172.21.15.49:0/117446913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee8ed0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.605+0000 7fbd55d90740 1 --2- 172.21.15.49:0/117446913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.605+0000 7fbd55d90740 1 -- 172.21.15.49:0/117446913 >> 172.21.15.49:0/117446913 conn(0x55b5aceda640 msgr2=0x55b5acedaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:45.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.606+0000 7fbd55d90740 1 -- 172.21.15.49:0/117446913 shutdown_connections 2024-02-26T06:27:45.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.606+0000 7fbd55d90740 1 -- 172.21.15.49:0/117446913 wait complete. 2024-02-26T06:27:45.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.606+0000 7fbd55d90740 1 -- start start 2024-02-26T06:27:45.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.607+0000 7fbd55d90740 1 Processor -- start 2024-02-26T06:27:45.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.608+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acedff10 0x55b5acee63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.608+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.608+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acee9410 0x55b5acee6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.609+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acedff10 0x55b5acee63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.609+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acedff10 0x55b5acee63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43412/0 (socket says 172.21.15.49:43412) 2024-02-26T06:27:45.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.609+0000 7fbd4b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.609+0000 7fbd4b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43912/0 (socket says 172.21.15.49:43912) 2024-02-26T06:27:45.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.609+0000 7fbd4be00640 1 -- 172.21.15.49:0/57909899 learned_addr learned my addr 172.21.15.49:0/57909899 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.609+0000 7fbd50e00640 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acee9410 0x55b5acee6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.609+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ace46b30 con 0x55b5acee0830 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5acdebfa0 con 0x55b5acee9410 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5acdebc40 con 0x55b5acedff10 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd55d90740 10 client.0 _ll_register_callbacks cb 0x55b5acb57c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd4be00640 1 -- 172.21.15.49:0/57909899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acee9410 msgr2=0x55b5acee6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd4be00640 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acee9410 0x55b5acee6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd4be00640 1 -- 172.21.15.49:0/57909899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 msgr2=0x55b5acee6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd4be00640 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd4be00640 1 -- 172.21.15.49:0/57909899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c009770 con 0x55b5acedff10 2024-02-26T06:27:45.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.610+0000 7fbd4b400640 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:45.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.611+0000 7fbd4be00640 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acedff10 0x55b5acee63d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c000c00 tx=0x7fbd3c006ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375d46c63c2ec4f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:45.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.611+0000 7fbd36000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:45.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.612+0000 7fbd36000640 1 -- 172.21.15.49:0/57909899 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c009610 con 0x55b5acedff10 2024-02-26T06:27:45.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.612+0000 7fbd36000640 1 -- 172.21.15.49:0/57909899 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fbd3c0127e0 con 0x55b5acedff10 2024-02-26T06:27:45.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.612+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5acb46860 con 0x55b5acedff10 2024-02-26T06:27:45.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.612+0000 7fbd36000640 1 -- 172.21.15.49:0/57909899 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd3c005970 con 0x55b5acedff10 2024-02-26T06:27:45.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.612+0000 7fbd55d90740 10 client.26554 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:45.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.612+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b5ace451b0 con 0x55b5acedff10 2024-02-26T06:27:45.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.613+0000 7fbd36000640 1 -- 172.21.15.49:0/57909899 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c031020 con 0x55b5acedff10 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.613+0000 7fbd36000640 1 -- 172.21.15.49:0/57909899 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd3c011070 con 0x55b5acedff10 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.613+0000 7fbd55d90740 10 client.26554 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.613+0000 7fbd55d90740 10 client.26554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.613+0000 7fbd55d90740 20 client.26554 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.614+0000 7fbd55d90740 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b5acef9780 0x55b5acefbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.614+0000 7fbd55d90740 4 client.26554 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.614+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5acb97410 con 0x55b5acef9780 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.614+0000 7fbd4b400640 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b5acef9780 0x55b5acefbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.615+0000 7fbd36000640 10 client.26554 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:45.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.615+0000 7fbd4b400640 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b5acef9780 0x55b5acefbbb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85989c9f3644f78a server_cookie=d6e78b959b5f46a4 in_seq=0 out_seq=0 2024-02-26T06:27:45.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.616+0000 7fbd36000640 1 -- 172.21.15.49:0/57909899 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5acb97410 con 0x55b5acef9780 2024-02-26T06:27:45.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.616+0000 7fbd36000640 10 client.26554 handle_command_reply: tid=0 2024-02-26T06:27:45.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.616+0000 7fbd55d90740 1 client.26554 shutdown 2024-02-26T06:27:45.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acedff10 msgr2=0x55b5acee63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acedff10 0x55b5acee63d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c000c00 tx=0x7fbd3c006ef0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 shutdown_connections 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b5acef9780 0x55b5acefbbb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acee9410 0x55b5acee6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5acee6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 --2- 172.21.15.49:0/57909899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acedff10 0x55b5acee63d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 >> 172.21.15.49:0/57909899 conn(0x55b5aceda640 msgr2=0x55b5acedc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd36000640 0 client.26554 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.617+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 shutdown_connections 2024-02-26T06:27:45.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.618+0000 7fbd55d90740 1 -- 172.21.15.49:0/57909899 wait complete. 2024-02-26T06:27:45.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.618+0000 7fbd55d90740 20 client.26554 trim_cache size 0 max 16384 2024-02-26T06:27:45.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.654+0000 7fbd55d90740 1 Processor -- start 2024-02-26T06:27:45.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.654+0000 7fbd55d90740 1 -- start start 2024-02-26T06:27:45.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.654+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.654+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee0830 0x55b5acee3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.654+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee9410 0x55b5acee3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.654+0000 7fbd55d90740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5acb8f0c0 con 0x55b5acee9410 2024-02-26T06:27:45.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.654+0000 7fbd55d90740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5aced5f30 con 0x55b5acedff10 2024-02-26T06:27:45.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.654+0000 7fbd55d90740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5acedf2e0 con 0x55b5acee0830 2024-02-26T06:27:45.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd4be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47628/0 (socket says 172.21.15.49:47628) 2024-02-26T06:27:45.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd4be00640 1 -- 172.21.15.49:0/148837034 learned_addr learned my addr 172.21.15.49:0/148837034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:45.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd50e00640 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee9410 0x55b5acee3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd4b400640 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee0830 0x55b5acee3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd4be00640 1 -- 172.21.15.49:0/148837034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee0830 msgr2=0x55b5acee3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:45.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd4be00640 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee0830 0x55b5acee3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd4be00640 1 -- 172.21.15.49:0/148837034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee9410 msgr2=0x55b5acee3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:45.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.655+0000 7fbd4be00640 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee9410 0x55b5acee3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.656+0000 7fbd4be00640 1 -- 172.21.15.49:0/148837034 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5acde1a60 con 0x55b5acedff10 2024-02-26T06:27:45.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.656+0000 7fbd50e00640 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee9410 0x55b5acee3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:45.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.657+0000 7fbd4b400640 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee0830 0x55b5acee3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:45.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.657+0000 7fbd4be00640 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee8b00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006da0 tx=0x7fbd3c006d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f18d50b3271ad4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:45.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.658+0000 7fbd40c00640 1 -- 172.21.15.49:0/148837034 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0095f0 con 0x55b5acedff10 2024-02-26T06:27:45.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.658+0000 7fbd40c00640 1 -- 172.21.15.49:0/148837034 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c031030 con 0x55b5acedff10 2024-02-26T06:27:45.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.658+0000 7fbd40c00640 1 -- 172.21.15.49:0/148837034 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c006090 con 0x55b5acedff10 2024-02-26T06:27:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.659+0000 7fbd55d90740 1 -- 172.21.15.49:0/148837034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 msgr2=0x55b5acee8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.659+0000 7fbd55d90740 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee8b00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006da0 tx=0x7fbd3c006d10 comp rx=0 tx=0).stop 2024-02-26T06:27:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.659+0000 7fbd55d90740 1 -- 172.21.15.49:0/148837034 shutdown_connections 2024-02-26T06:27:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.659+0000 7fbd55d90740 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee9410 0x55b5acee3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.659+0000 7fbd55d90740 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee0830 0x55b5acee3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.659+0000 7fbd55d90740 1 --2- 172.21.15.49:0/148837034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5acee8b00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.659+0000 7fbd55d90740 1 -- 172.21.15.49:0/148837034 >> 172.21.15.49:0/148837034 conn(0x55b5aceda640 msgr2=0x55b5acee4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:45.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.660+0000 7fbd55d90740 1 -- 172.21.15.49:0/148837034 shutdown_connections 2024-02-26T06:27:45.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.660+0000 7fbd55d90740 1 -- 172.21.15.49:0/148837034 wait complete. 2024-02-26T06:27:45.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.660+0000 7fbd55d90740 1 -- start start 2024-02-26T06:27:45.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.660+0000 7fbd55d90740 1 Processor -- start 2024-02-26T06:27:45.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.664+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5aceeee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.665+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5aceef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.665+0000 7fbd55d90740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.665+0000 7fbd55d90740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5acf4fd30 con 0x55b5acee0830 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.665+0000 7fbd55d90740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5acf4fea0 con 0x55b5acedff10 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.665+0000 7fbd55d90740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5acf50060 con 0x55b5acee9410 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.665+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.665+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceef890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43442/0 (socket says 172.21.15.49:43442) 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.665+0000 7fbd50e00640 1 -- 172.21.15.49:0/1114356054 learned_addr learned my addr 172.21.15.49:0/1114356054 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.666+0000 7fbd4be00640 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5aceeee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.666+0000 7fbd50e00640 1 -- 172.21.15.49:0/1114356054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 msgr2=0x55b5aceeee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.666+0000 7fbd50e00640 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5aceeee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.666+0000 7fbd50e00640 1 -- 172.21.15.49:0/1114356054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 msgr2=0x55b5aceef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.666+0000 7fbd50e00640 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5aceef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.666+0000 7fbd50e00640 1 -- 172.21.15.49:0/1114356054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5ace451b0 con 0x55b5acee9410 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.666+0000 7fbd55d90740 10 client.0 _ll_register_callbacks cb 0x7fbd3c009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.666+0000 7fbd55d90740 1 -- 172.21.15.49:0/1114356054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd3c00ebe0 con 0x55b5acee9410 2024-02-26T06:27:45.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.667+0000 7fbd50e00640 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceef890 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd44012d70 tx=0x7fbd44004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b63b91152acae5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:45.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.667+0000 7fbd36000640 10 client.26557 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:45.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.667+0000 7fbd36000640 1 -- 172.21.15.49:0/1114356054 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44025070 con 0x55b5acee9410 2024-02-26T06:27:45.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd36000640 1 -- 172.21.15.49:0/1114356054 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fbd44018070 con 0x55b5acee9410 2024-02-26T06:27:45.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd36000640 1 -- 172.21.15.49:0/1114356054 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd44025070 con 0x55b5acee9410 2024-02-26T06:27:45.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd55d90740 10 client.26557 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:45.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd55d90740 1 -- 172.21.15.49:0/1114356054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x55b5ace07670 con 0x55b5acee9410 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd36000640 1 -- 172.21.15.49:0/1114356054 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd4401e5c0 con 0x55b5acee9410 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd55d90740 10 client.26557 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd55d90740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd55d90740 20 client.26557 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd55d90740 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b5acf516a0 0x55b5acf53b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd55d90740 4 client.26557 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.668+0000 7fbd55d90740 1 -- 172.21.15.49:0/1114356054 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b5acb837b0 con 0x55b5acf516a0 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.669+0000 7fbd4be00640 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b5acf516a0 0x55b5acf53b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.669+0000 7fbd4be00640 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b5acf516a0 0x55b5acf53b00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76281c2eae41b42 server_cookie=e5ee2cbf57c26a32 in_seq=0 out_seq=0 2024-02-26T06:27:45.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.669+0000 7fbd36000640 10 client.26557 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:45.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.670+0000 7fbd36000640 1 -- 172.21.15.49:0/1114356054 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b5acb837b0 con 0x55b5acf516a0 2024-02-26T06:27:45.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.670+0000 7fbd36000640 10 client.26557 handle_command_reply: tid=0 2024-02-26T06:27:45.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.670+0000 7fbd55d90740 1 client.26557 shutdown 2024-02-26T06:27:45.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 -- 172.21.15.49:0/1114356054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 msgr2=0x55b5aceef890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceef890 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd44012d70 tx=0x7fbd44004a40 comp rx=0 tx=0).stop 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 -- 172.21.15.49:0/1114356054 shutdown_connections 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b5acf516a0 0x55b5acf53b00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5acee9410 0x55b5aceef890 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5acee0830 0x55b5aceef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 --2- 172.21.15.49:0/1114356054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b5acedff10 0x55b5aceeee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 -- 172.21.15.49:0/1114356054 >> 172.21.15.49:0/1114356054 conn(0x55b5aceda640 msgr2=0x55b5aced96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd36000640 0 client.26557 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 -- 172.21.15.49:0/1114356054 shutdown_connections 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.671+0000 7fbd55d90740 1 -- 172.21.15.49:0/1114356054 wait complete. 2024-02-26T06:27:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.672+0000 7fbd55d90740 20 client.26557 trim_cache size 0 max 16384 2024-02-26T06:27:45.674 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:27:45.674 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:27:45.674 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "f96c8708-d50f-42f4-8708-0eca50c71740", 2024-02-26T06:27:45.674 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:27:45.674 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:27:45.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 -- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fbd300a2330 msgr2=0x7fbd300a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fbd300a2330 0x7fbd300a47f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c06a580 tx=0x7fbd440094a0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 -- 172.21.15.49:0/1113389682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 msgr2=0x7fbd4c06b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 0x7fbd4c06b780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd38036040 tx=0x7fbd38004040 comp rx=0 tx=0).stop 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 -- 172.21.15.49:0/1113389682 shutdown_connections 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fbd300a2330 0x7fbd300a47f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd4c111fc0 0x7fbd4c06bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd4c075fd0 0x7fbd4c06b780 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 --2- 172.21.15.49:0/1113389682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd4c0756b0 0x7fbd4c06b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.674+0000 7fbd52e00640 1 -- 172.21.15.49:0/1113389682 >> 172.21.15.49:0/1113389682 conn(0x7fbd4c0fedf0 msgr2=0x7fbd4c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:45.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.675+0000 7fbd52e00640 1 -- 172.21.15.49:0/1113389682 shutdown_connections 2024-02-26T06:27:45.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:45.675+0000 7fbd52e00640 1 -- 172.21.15.49:0/1113389682 wait complete. 2024-02-26T06:27:45.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:27:45.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:27:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.020+0000 7fb8cae00640 1 -- 172.21.15.49:0/1347382844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ff5d0 msgr2=0x7fb8c40ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.020+0000 7fb8cae00640 1 --2- 172.21.15.49:0/1347382844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ff5d0 0x7fb8c40ff9b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009930 tx=0x7fb8b402bc20 comp rx=0 tx=0).stop 2024-02-26T06:27:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.020+0000 7fb8cae00640 1 -- 172.21.15.49:0/1347382844 shutdown_connections 2024-02-26T06:27:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.020+0000 7fb8cae00640 1 --2- 172.21.15.49:0/1347382844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8c40ffef0 0x7fb8c4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.020+0000 7fb8cae00640 1 --2- 172.21.15.49:0/1347382844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8c4104500 0x7fb8c41049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.020+0000 7fb8cae00640 1 --2- 172.21.15.49:0/1347382844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ff5d0 0x7fb8c40ff9b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.020+0000 7fb8cae00640 1 -- 172.21.15.49:0/1347382844 >> 172.21.15.49:0/1347382844 conn(0x7fb8c40761d0 msgr2=0x7fb8c40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.020+0000 7fb8cae00640 1 -- 172.21.15.49:0/1347382844 shutdown_connections 2024-02-26T06:27:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.021+0000 7fb8cae00640 1 -- 172.21.15.49:0/1347382844 wait complete. 2024-02-26T06:27:46.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.022+0000 7fb8cae00640 1 Processor -- start 2024-02-26T06:27:46.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.022+0000 7fb8cae00640 1 -- start start 2024-02-26T06:27:46.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.022+0000 7fb8cae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8c40ff5d0 0x7fb8c419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.022+0000 7fb8cae00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ffef0 0x7fb8c419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.022+0000 7fb8cae00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8c4104500 0x7fb8c4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.022+0000 7fb8cae00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4199e80 con 0x7fb8c4104500 2024-02-26T06:27:46.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.022+0000 7fb8cae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4199ff0 con 0x7fb8c40ff5d0 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.022+0000 7fb8cae00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c419a1b0 con 0x7fb8c40ffef0 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.023+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ffef0 0x7fb8c419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.023+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ffef0 0x7fb8c419fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43456/0 (socket says 172.21.15.49:43456) 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.023+0000 7fb8c3400640 1 -- 172.21.15.49:0/820780458 learned_addr learned my addr 172.21.15.49:0/820780458 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.023+0000 7fb8c8e00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8c4104500 0x7fb8c4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.023+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8c40ff5d0 0x7fb8c419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.023+0000 7fb8c3400640 1 -- 172.21.15.49:0/820780458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8c40ff5d0 msgr2=0x7fb8c419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.024+0000 7fb8c3400640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8c40ff5d0 0x7fb8c419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.024+0000 7fb8c3400640 1 -- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8c4104500 msgr2=0x7fb8c4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.024+0000 7fb8c3400640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8c4104500 0x7fb8c4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.024+0000 7fb8c3400640 1 -- 172.21.15.49:0/820780458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4009560 con 0x7fb8c40ffef0 2024-02-26T06:27:46.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.025+0000 7fb8c3400640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ffef0 0x7fb8c419fbb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d950 tx=0x7fb8ac00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:46.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.026+0000 7fb8c0c00640 1 -- 172.21.15.49:0/820780458 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac009a30 con 0x7fb8c40ffef0 2024-02-26T06:27:46.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.026+0000 7fb8c0c00640 1 -- 172.21.15.49:0/820780458 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac009b90 con 0x7fb8c40ffef0 2024-02-26T06:27:46.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.026+0000 7fb8c0c00640 1 -- 172.21.15.49:0/820780458 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac004a00 con 0x7fb8c40ffef0 2024-02-26T06:27:46.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.026+0000 7fb8cae00640 1 -- 172.21.15.49:0/820780458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c419a490 con 0x7fb8c40ffef0 2024-02-26T06:27:46.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.026+0000 7fb8cae00640 1 -- 172.21.15.49:0/820780458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c41a66a0 con 0x7fb8c40ffef0 2024-02-26T06:27:46.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.031+0000 7fb8cd9fd740 1 Processor -- start 2024-02-26T06:27:46.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.031+0000 7fb8c0c00640 1 -- 172.21.15.49:0/820780458 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb8ac009d00 con 0x7fb8c40ffef0 2024-02-26T06:27:46.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.032+0000 7fb8cd9fd740 1 -- start start 2024-02-26T06:27:46.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.033+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x5628402f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.033+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1290 0x5628402f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.033+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840305200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.033+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562840260120 con 0x5628402f1f70 2024-02-26T06:27:46.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.033+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284025d650 con 0x562840302e10 2024-02-26T06:27:46.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.033+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562840258e30 con 0x5628402f1290 2024-02-26T06:27:46.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.033+0000 7fb8c8e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x5628402f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.034+0000 7fb8c0c00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8a80a22b0 0x7fb8a80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.034+0000 7fb8c3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1290 0x5628402f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.034+0000 7fb8c8e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x5628402f0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43954/0 (socket says 172.21.15.49:43954) 2024-02-26T06:27:46.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.034+0000 7fb8c8e00640 1 -- 172.21.15.49:0/1437311998 learned_addr learned my addr 172.21.15.49:0/1437311998 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:46.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.034+0000 7fb8c0c00640 1 -- 172.21.15.49:0/820780458 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb8ac0cdae0 con 0x7fb8c40ffef0 2024-02-26T06:27:46.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.034+0000 7fb8c3400640 1 --2- 172.21.15.49:0/1437311998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840305200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.034+0000 7fb8c8e00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8a80a22b0 0x7fb8a80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.035+0000 7fb8c3400640 1 -- 172.21.15.49:0/1437311998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1290 msgr2=0x5628402f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:46.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.035+0000 7fb8c3400640 1 --2- 172.21.15.49:0/1437311998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1290 0x5628402f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.035+0000 7fb8c3400640 1 -- 172.21.15.49:0/1437311998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 msgr2=0x5628402f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:46.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.035+0000 7fb8c3400640 1 --2- 172.21.15.49:0/1437311998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x5628402f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.035+0000 7fb8c3400640 1 -- 172.21.15.49:0/1437311998 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562840212030 con 0x562840302e10 2024-02-26T06:27:46.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.035+0000 7fb8c3400640 1 --2- 172.21.15.49:0/1437311998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840305200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0cb240 tx=0x7fb8ac0967f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=447100de9c1d689e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:46.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.036+0000 7fb8b9600640 1 -- 172.21.15.49:0/1437311998 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac09abe0 con 0x562840302e10 2024-02-26T06:27:46.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.036+0000 7fb8c8e00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8a80a22b0 0x7fb8a80a4770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009370 tx=0x7fb8bc009aa0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:46.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.036+0000 7fb8b9600640 1 -- 172.21.15.49:0/1437311998 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac09a4e0 con 0x562840302e10 2024-02-26T06:27:46.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.036+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1437311998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 msgr2=0x562840305200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.036+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1437311998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840305200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0cb240 tx=0x7fb8ac0967f0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.037+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1437311998 shutdown_connections 2024-02-26T06:27:46.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.037+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1437311998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840305200 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.037+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1437311998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1290 0x5628402f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.037+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1437311998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x5628402f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.037+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1437311998 >> 172.21.15.49:0/1437311998 conn(0x5628402b8570 msgr2=0x5628402ebce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.038+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1437311998 shutdown_connections 2024-02-26T06:27:46.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.039+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1437311998 wait complete. 2024-02-26T06:27:46.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.039+0000 7fb8cd9fd740 1 -- start start 2024-02-26T06:27:46.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.040+0000 7fb8cd9fd740 1 Processor -- start 2024-02-26T06:27:46.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.041+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 0x5628402ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1f70 0x562840300290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628401fde90 con 0x5628402f1290 2024-02-26T06:27:46.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628401fdb30 con 0x5628402f1f70 2024-02-26T06:27:46.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283ffc71c0 con 0x562840302e10 2024-02-26T06:27:46.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8cd9fd740 10 client.0 _ll_register_callbacks cb 0x562840211930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:46.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c8e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c8e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43474/0 (socket says 172.21.15.49:43474) 2024-02-26T06:27:46.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c8e00640 1 -- 172.21.15.49:0/68544267 learned_addr learned my addr 172.21.15.49:0/68544267 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:46.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c3400640 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1f70 0x562840300290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c8e00640 1 -- 172.21.15.49:0/68544267 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1f70 msgr2=0x562840300290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:46.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c8e00640 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1f70 0x562840300290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c8e00640 1 -- 172.21.15.49:0/68544267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 msgr2=0x5628402ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:46.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c8e00640 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 0x5628402ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.044+0000 7fb8c8e00640 1 -- 172.21.15.49:0/68544267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c41a66a0 con 0x562840302e10 2024-02-26T06:27:46.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.046+0000 7fb8c8e00640 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403007d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc011bb0 tx=0x7fb8bc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2381219557839bc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:46.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.046+0000 7fb8b2000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:46.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.046+0000 7fb8b2000640 1 -- 172.21.15.49:0/68544267 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc019070 con 0x562840302e10 2024-02-26T06:27:46.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.047+0000 7fb8b2000640 1 -- 172.21.15.49:0/68544267 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb8bc014050 con 0x562840302e10 2024-02-26T06:27:46.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.047+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/68544267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56283ff587e0 con 0x562840302e10 2024-02-26T06:27:46.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8b2000640 1 -- 172.21.15.49:0/68544267 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8bc014210 con 0x562840302e10 2024-02-26T06:27:46.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8cd9fd740 10 client.26569 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:46.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/68544267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5628402574b0 con 0x562840302e10 2024-02-26T06:27:46.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8b2000640 1 -- 172.21.15.49:0/68544267 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56283ff587e0 con 0x562840302e10 2024-02-26T06:27:46.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8b2000640 1 -- 172.21.15.49:0/68544267 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb8bc028860 con 0x562840302e10 2024-02-26T06:27:46.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8cd9fd740 10 client.26569 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:46.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8cd9fd740 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:46.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8cd9fd740 20 client.26569 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:46.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562840313020 0x562840315410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8cd9fd740 4 client.26569 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:46.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.048+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/68544267 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56283ff95720 con 0x562840313020 2024-02-26T06:27:46.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.049+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562840313020 0x562840315410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.050+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562840313020 0x562840315410 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167f7d103dd9f100 server_cookie=e3726ac44bb8e418 in_seq=0 out_seq=0 2024-02-26T06:27:46.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.050+0000 7fb8b2000640 10 client.26569 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:46.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.051+0000 7fb8b2000640 1 -- 172.21.15.49:0/68544267 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56283ff95720 con 0x562840313020 2024-02-26T06:27:46.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.051+0000 7fb8b2000640 10 client.26569 handle_command_reply: tid=0 2024-02-26T06:27:46.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.051+0000 7fb8cd9fd740 1 client.26569 shutdown 2024-02-26T06:27:46.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.052+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/68544267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 msgr2=0x5628403007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.052+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403007d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc011bb0 tx=0x7fb8bc009780 comp rx=0 tx=0).stop 2024-02-26T06:27:46.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/68544267 shutdown_connections 2024-02-26T06:27:46.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562840313020 0x562840315410 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1f70 0x562840300290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403007d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/68544267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 0x5628402ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8b2000640 0 client.26569 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/68544267 >> 172.21.15.49:0/68544267 conn(0x5628402b8570 msgr2=0x5628402eda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/68544267 shutdown_connections 2024-02-26T06:27:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.053+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/68544267 wait complete. 2024-02-26T06:27:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.054+0000 7fb8cd9fd740 20 client.26569 trim_cache size 0 max 16384 2024-02-26T06:27:46.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.089+0000 7fb8cd9fd740 1 Processor -- start 2024-02-26T06:27:46.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.089+0000 7fb8cd9fd740 1 -- start start 2024-02-26T06:27:46.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.090+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 0x562840302480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.090+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1f70 0x5628403005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.090+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840309e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.090+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284030f420 con 0x5628402f1290 2024-02-26T06:27:46.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.090+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628402eb580 con 0x562840302e10 2024-02-26T06:27:46.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.090+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628402f0510 con 0x5628402f1f70 2024-02-26T06:27:46.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c8e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840309e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1f70 0x5628403005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1f70 0x5628403005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43480/0 (socket says 172.21.15.49:43480) 2024-02-26T06:27:46.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c3400640 1 -- 172.21.15.49:0/2519279164 learned_addr learned my addr 172.21.15.49:0/2519279164 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:46.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 0x562840302480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c3400640 1 -- 172.21.15.49:0/2519279164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 msgr2=0x562840309e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:46.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c3400640 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840309e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c3400640 1 -- 172.21.15.49:0/2519279164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 msgr2=0x562840302480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:46.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.091+0000 7fb8c3400640 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 0x562840302480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.092+0000 7fb8c3400640 1 -- 172.21.15.49:0/2519279164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628401f3920 con 0x5628402f1f70 2024-02-26T06:27:46.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.092+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 0x562840302480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:46.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.092+0000 7fb8c3400640 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1f70 0x5628403005f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac096940 tx=0x7fb8ac0987e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454b876b2e22339b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:46.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.092+0000 7fb8b8c00640 1 -- 172.21.15.49:0/2519279164 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac005cd0 con 0x5628402f1f70 2024-02-26T06:27:46.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.092+0000 7fb8b8c00640 1 -- 172.21.15.49:0/2519279164 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac09abe0 con 0x5628402f1f70 2024-02-26T06:27:46.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.092+0000 7fb8b8c00640 1 -- 172.21.15.49:0/2519279164 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac09e160 con 0x5628402f1f70 2024-02-26T06:27:46.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.094+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/2519279164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1f70 msgr2=0x5628403005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.094+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1f70 0x5628403005f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac096940 tx=0x7fb8ac0987e0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.094+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/2519279164 shutdown_connections 2024-02-26T06:27:46.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.094+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562840302e10 0x562840309e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.094+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628402f1f70 0x5628403005f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.094+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/2519279164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1290 0x562840302480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.094+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/2519279164 >> 172.21.15.49:0/2519279164 conn(0x5628402b8570 msgr2=0x5628402eb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.094+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/2519279164 shutdown_connections 2024-02-26T06:27:46.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.095+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/2519279164 wait complete. 2024-02-26T06:27:46.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.095+0000 7fb8cd9fd740 1 -- start start 2024-02-26T06:27:46.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.096+0000 7fb8cd9fd740 1 Processor -- start 2024-02-26T06:27:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.097+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1290 0x562840309e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.097+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x56284030a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.097+0000 7fb8cd9fd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403063c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.097+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562840306a90 con 0x5628402f1f70 2024-02-26T06:27:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.097+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562840306c00 con 0x5628402f1290 2024-02-26T06:27:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.097+0000 7fb8cd9fd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562840306dc0 con 0x562840302e10 2024-02-26T06:27:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.097+0000 7fb8cd9fd740 10 client.0 _ll_register_callbacks cb 0x7fb8bc0146a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x56284030a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x56284030a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43990/0 (socket says 172.21.15.49:43990) 2024-02-26T06:27:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3400640 1 -- 172.21.15.49:0/1793104709 learned_addr learned my addr 172.21.15.49:0/1793104709 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1290 0x562840309e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c8e00640 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3400640 1 -- 172.21.15.49:0/1793104709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 msgr2=0x5628403063c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3400640 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403063c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3400640 1 -- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1290 msgr2=0x562840309e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3400640 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1290 0x562840309e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3400640 1 -- 172.21.15.49:0/1793104709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562840212030 con 0x5628402f1f70 2024-02-26T06:27:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.098+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1290 0x562840309e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.100+0000 7fb8c3400640 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x56284030a360 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0b0040 tx=0x7fb8ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0edabc363d2234 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.100+0000 7fb8b2000640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:27:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.100+0000 7fb8b2000640 1 -- 172.21.15.49:0/1793104709 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac005cd0 con 0x5628402f1f70 2024-02-26T06:27:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.100+0000 7fb8b2000640 1 -- 172.21.15.49:0/1793104709 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fb8ac0a2c80 con 0x5628402f1f70 2024-02-26T06:27:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.101+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1793104709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8ac09a4e0 con 0x5628402f1f70 2024-02-26T06:27:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.101+0000 7fb8b2000640 1 -- 172.21.15.49:0/1793104709 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8ac0cdda0 con 0x5628402f1f70 2024-02-26T06:27:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.101+0000 7fb8cd9fd740 10 client.16173 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1793104709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x562840219400 con 0x5628402f1f70 2024-02-26T06:27:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8b2000640 1 -- 172.21.15.49:0/1793104709 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac0a6080 con 0x5628402f1f70 2024-02-26T06:27:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8b2000640 1 -- 172.21.15.49:0/1793104709 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb8ac082070 con 0x5628402f1f70 2024-02-26T06:27:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8cd9fd740 10 client.16173 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8cd9fd740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8cd9fd740 20 client.16173 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562840313020 0x56284036c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8cd9fd740 4 client.16173 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:27:46.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.102+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1793104709 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56283ffe4890 con 0x562840313020 2024-02-26T06:27:46.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.103+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562840313020 0x56284036c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.103+0000 7fb8c3e00640 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562840313020 0x56284036c2e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736d470e48f4b456 server_cookie=3da2979c8495ab42 in_seq=0 out_seq=0 2024-02-26T06:27:46.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.104+0000 7fb8b2000640 10 client.16173 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:46.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.104+0000 7fb8b2000640 1 -- 172.21.15.49:0/1793104709 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56283ffe4890 con 0x562840313020 2024-02-26T06:27:46.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.104+0000 7fb8b2000640 10 client.16173 handle_command_reply: tid=0 2024-02-26T06:27:46.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.104+0000 7fb8cd9fd740 1 client.16173 shutdown 2024-02-26T06:27:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1793104709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 msgr2=0x56284030a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x56284030a360 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0b0040 tx=0x7fb8ac000bc0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1793104709 shutdown_connections 2024-02-26T06:27:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x562840313020 0x56284036c2e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562840302e10 0x5628403063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628402f1f70 0x56284030a360 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 --2- 172.21.15.49:0/1793104709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5628402f1290 0x562840309e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1793104709 >> 172.21.15.49:0/1793104709 conn(0x5628402b8570 msgr2=0x5628402ea570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8b2000640 0 client.16173 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1793104709 shutdown_connections 2024-02-26T06:27:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.106+0000 7fb8cd9fd740 1 -- 172.21.15.49:0/1793104709 wait complete. 2024-02-26T06:27:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.107+0000 7fb8cd9fd740 20 client.16173 trim_cache size 0 max 16384 2024-02-26T06:27:46.108 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:27:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:27:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:27:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "f96c8708-d50f-42f4-8708-0eca50c71740": { 2024-02-26T06:27:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:27:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "f96c8708-d50f-42f4-8708-0eca50c71740", 2024-02-26T06:27:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:27:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:27:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:27:46.109 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:27:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 -- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8a80a22b0 msgr2=0x7fb8a80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8a80a22b0 0x7fb8a80a4770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009370 tx=0x7fb8bc009aa0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 -- 172.21.15.49:0/820780458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ffef0 msgr2=0x7fb8c419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ffef0 0x7fb8c419fbb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d950 tx=0x7fb8ac00de10 comp rx=0 tx=0).stop 2024-02-26T06:27:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 -- 172.21.15.49:0/820780458 shutdown_connections 2024-02-26T06:27:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8a80a22b0 0x7fb8a80a4770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8c4104500 0x7fb8c4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c40ffef0 0x7fb8c419fbb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 --2- 172.21.15.49:0/820780458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8c40ff5d0 0x7fb8c419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 -- 172.21.15.49:0/820780458 >> 172.21.15.49:0/820780458 conn(0x7fb8c40761d0 msgr2=0x7fb8c41012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 -- 172.21.15.49:0/820780458 shutdown_connections 2024-02-26T06:27:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:27:46.108+0000 7fb8cae00640 1 -- 172.21.15.49:0/820780458 wait complete. 2024-02-26T06:27:46.136 INFO:tasks.cephfs.filesystem:scrub status for tag:f96c8708-d50f-42f4-8708-0eca50c71740 - {'path': '/', 'tag': 'f96c8708-d50f-42f4-8708-0eca50c71740', 'options': 'recursive,force'} 2024-02-26T06:27:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 235.0 s, 2381.8 tps, lat 42.017 ms stddev 49.701 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: pgmap v134: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.7 MiB/s wr, 882 op/s 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:46.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:46.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:27:46.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:46 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: pgmap v134: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.7 MiB/s wr, 882 op/s 2024-02-26T06:27:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:27:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:46 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: pgmap v134: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.7 MiB/s wr, 882 op/s 2024-02-26T06:27:46.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:27:46.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:46.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:46.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:46.921 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:46 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:48.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:48 smithi080 ceph-mon[36206]: pgmap v135: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-02-26T06:27:48.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:48 smithi115 ceph-mon[36426]: pgmap v135: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-02-26T06:27:48.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:48 smithi049 ceph-mon[31696]: pgmap v135: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-02-26T06:27:50.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:50 smithi080 ceph-mon[36206]: pgmap v136: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.9 MiB/s wr, 837 op/s 2024-02-26T06:27:50.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:50 smithi115 ceph-mon[36426]: pgmap v136: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.9 MiB/s wr, 837 op/s 2024-02-26T06:27:50.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:50 smithi049 ceph-mon[31696]: pgmap v136: 97 pgs: 97 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.9 MiB/s wr, 837 op/s 2024-02-26T06:27:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 240.0 s, 2273.6 tps, lat 43.874 ms stddev 49.097 2024-02-26T06:27:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.112+0000 7f8d64200640 1 -- 172.21.15.49:0/639457050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d5c103b90 msgr2=0x7f8d5c104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.112+0000 7f8d64200640 1 --2- 172.21.15.49:0/639457050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d5c103b90 0x7f8d5c104010 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009be0 tx=0x7f8d5402bef0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.112+0000 7f8d64200640 1 -- 172.21.15.49:0/639457050 shutdown_connections 2024-02-26T06:27:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.112+0000 7f8d64200640 1 --2- 172.21.15.49:0/639457050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d5c069e40 0x7f8d5c06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.112+0000 7f8d64200640 1 --2- 172.21.15.49:0/639457050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d5c103b90 0x7f8d5c104010 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.112+0000 7f8d64200640 1 --2- 172.21.15.49:0/639457050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c106100 0x7f8d5c069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.112+0000 7f8d64200640 1 -- 172.21.15.49:0/639457050 >> 172.21.15.49:0/639457050 conn(0x7f8d5c076e70 msgr2=0x7f8d5c077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.113+0000 7f8d64200640 1 -- 172.21.15.49:0/639457050 shutdown_connections 2024-02-26T06:27:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.113+0000 7f8d64200640 1 -- 172.21.15.49:0/639457050 wait complete. 2024-02-26T06:27:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.114+0000 7f8d64200640 1 Processor -- start 2024-02-26T06:27:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.115+0000 7f8d64200640 1 -- start start 2024-02-26T06:27:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.115+0000 7f8d64200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c069e40 0x7f8d5c1a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d64200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d5c103b90 0x7f8d5c1a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d64200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d5c106100 0x7f8d5c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d64200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c19dfe0 con 0x7f8d5c069e40 2024-02-26T06:27:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d64200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c19e150 con 0x7f8d5c106100 2024-02-26T06:27:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d64200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c19e310 con 0x7f8d5c103b90 2024-02-26T06:27:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d62200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d5c106100 0x7f8d5c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d61800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c069e40 0x7f8d5c1a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d62200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d5c106100 0x7f8d5c19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54090/0 (socket says 172.21.15.49:54090) 2024-02-26T06:27:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d62200640 1 -- 172.21.15.49:0/3782800047 learned_addr learned my addr 172.21.15.49:0/3782800047 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.116+0000 7f8d60e00640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d5c103b90 0x7f8d5c1a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.117+0000 7f8d62200640 1 -- 172.21.15.49:0/3782800047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d5c103b90 msgr2=0x7f8d5c1a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.117+0000 7f8d62200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d5c103b90 0x7f8d5c1a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.117+0000 7f8d62200640 1 -- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c069e40 msgr2=0x7f8d5c1a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.117+0000 7f8d62200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c069e40 0x7f8d5c1a3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.117+0000 7f8d62200640 1 -- 172.21.15.49:0/3782800047 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009560 con 0x7f8d5c106100 2024-02-26T06:27:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.117+0000 7f8d62200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d5c106100 0x7f8d5c19d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d54033610 tx=0x7f8d54033640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.118+0000 7f8d5a000640 1 -- 172.21.15.49:0/3782800047 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54003b80 con 0x7f8d5c106100 2024-02-26T06:27:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.118+0000 7f8d5a000640 1 -- 172.21.15.49:0/3782800047 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d54003ce0 con 0x7f8d5c106100 2024-02-26T06:27:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.118+0000 7f8d5a000640 1 -- 172.21.15.49:0/3782800047 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5403d990 con 0x7f8d5c106100 2024-02-26T06:27:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.118+0000 7f8d64200640 1 -- 172.21.15.49:0/3782800047 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d54009840 con 0x7f8d5c106100 2024-02-26T06:27:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.118+0000 7f8d64200640 1 -- 172.21.15.49:0/3782800047 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c1aa650 con 0x7f8d5c106100 2024-02-26T06:27:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.122+0000 7f8d5a000640 1 -- 172.21.15.49:0/3782800047 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8d54004040 con 0x7f8d5c106100 2024-02-26T06:27:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.123+0000 7f8d6706e740 1 Processor -- start 2024-02-26T06:27:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.124+0000 7f8d6706e740 1 -- start start 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.125+0000 7f8d5a000640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8d440a2350 0x7f8d440a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.125+0000 7f8d5a000640 1 -- 172.21.15.49:0/3782800047 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f8d540f0620 con 0x7f8d5c106100 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.126+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.126+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f52200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.126+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f5d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.126+0000 7f8d6706e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4f0d5f0 con 0x557fb4f5b300 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.126+0000 7f8d6706e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4ebf890 con 0x557fb4f51d80 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.126+0000 7f8d6706e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4ebcdc0 con 0x557fb4f52af0 2024-02-26T06:27:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.126+0000 7f8d60e00640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8d440a2350 0x7f8d440a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.127+0000 7f8d60e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f52200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.127+0000 7f8d61800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.127+0000 7f8d61800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f51840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33602/0 (socket says 172.21.15.49:33602) 2024-02-26T06:27:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.127+0000 7f8d61800640 1 -- 172.21.15.49:0/423667824 learned_addr learned my addr 172.21.15.49:0/423667824 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.128+0000 7f8d61800640 1 -- 172.21.15.49:0/423667824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 msgr2=0x557fb4f52200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.128+0000 7f8d61800640 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f52200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.128+0000 7f8d61800640 1 -- 172.21.15.49:0/423667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 msgr2=0x557fb4f5d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.128+0000 7f8d62200640 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f5d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.129+0000 7f8d61800640 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f5d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.129+0000 7f8d61800640 1 -- 172.21.15.49:0/423667824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fb4e39ad0 con 0x557fb4f52af0 2024-02-26T06:27:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.129+0000 7f8d62200640 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f5d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.129+0000 7f8d60e00640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8d440a2350 0x7f8d440a4810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009fd0 tx=0x7f8d48009cc0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.129+0000 7f8d61800640 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f51840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000bc30 tx=0x7f8d500107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71b0667a2198904 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.129+0000 7f8d4ea00640 1 -- 172.21.15.49:0/423667824 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5001c480 con 0x557fb4f52af0 2024-02-26T06:27:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.130+0000 7f8d4ea00640 1 -- 172.21.15.49:0/423667824 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d50015070 con 0x557fb4f52af0 2024-02-26T06:27:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.130+0000 7f8d4ea00640 1 -- 172.21.15.49:0/423667824 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50011a80 con 0x557fb4f52af0 2024-02-26T06:27:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.131+0000 7f8d6706e740 1 -- 172.21.15.49:0/423667824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 msgr2=0x557fb4f51840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.131+0000 7f8d6706e740 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f51840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000bc30 tx=0x7f8d500107a0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.132+0000 7f8d6706e740 1 -- 172.21.15.49:0/423667824 shutdown_connections 2024-02-26T06:27:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.132+0000 7f8d6706e740 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f52200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.132+0000 7f8d6706e740 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f5d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.132+0000 7f8d6706e740 1 --2- 172.21.15.49:0/423667824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f51840 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.132+0000 7f8d6706e740 1 -- 172.21.15.49:0/423667824 >> 172.21.15.49:0/423667824 conn(0x557fb4f19480 msgr2=0x557fb4f4ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.132+0000 7f8d6706e740 1 -- 172.21.15.49:0/423667824 shutdown_connections 2024-02-26T06:27:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.132+0000 7f8d6706e740 1 -- 172.21.15.49:0/423667824 wait complete. 2024-02-26T06:27:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.133+0000 7f8d6706e740 1 -- start start 2024-02-26T06:27:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.133+0000 7f8d6706e740 1 Processor -- start 2024-02-26T06:27:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f58390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f52af0 0x557fb4f588d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f5b300 0x557fb4f58e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d6706e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4eb85a0 con 0x557fb4f52af0 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d6706e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4e5db20 con 0x557fb4f51d80 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d6706e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4e5d7c0 con 0x557fb4f5b300 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d6706e740 10 client.0 _ll_register_callbacks cb 0x557fb4e71520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d61800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f58390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.134+0000 7f8d61800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f58390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54114/0 (socket says 172.21.15.49:54114) 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d61800640 1 -- 172.21.15.49:0/2316520948 learned_addr learned my addr 172.21.15.49:0/2316520948 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d60e00640 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f52af0 0x557fb4f588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d62200640 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f5b300 0x557fb4f58e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d61800640 1 -- 172.21.15.49:0/2316520948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f5b300 msgr2=0x557fb4f58e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d61800640 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f5b300 0x557fb4f58e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d61800640 1 -- 172.21.15.49:0/2316520948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f52af0 msgr2=0x557fb4f588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d61800640 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f52af0 0x557fb4f588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d61800640 1 -- 172.21.15.49:0/2316520948 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5000b8e0 con 0x557fb4f51d80 2024-02-26T06:27:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d60e00640 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f52af0 0x557fb4f588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d61800640 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f58390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000bf50 tx=0x7f8d5000bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5db06b445be47600 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.135+0000 7f8d43400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:27:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.136+0000 7f8d43400640 1 -- 172.21.15.49:0/2316520948 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50020070 con 0x557fb4f51d80 2024-02-26T06:27:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.136+0000 7f8d43400640 1 -- 172.21.15.49:0/2316520948 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f8d50017070 con 0x557fb4f51d80 2024-02-26T06:27:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.136+0000 7f8d6706e740 1 -- 172.21.15.49:0/2316520948 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fb4e8aca0 con 0x557fb4f51d80 2024-02-26T06:27:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.138+0000 7f8d43400640 1 -- 172.21.15.49:0/2316520948 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d50004020 con 0x557fb4f51d80 2024-02-26T06:27:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.138+0000 7f8d6706e740 10 client.26576 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.138+0000 7f8d6706e740 1 -- 172.21.15.49:0/2316520948 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557fb4eb6c20 con 0x557fb4f51d80 2024-02-26T06:27:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.139+0000 7f8d43400640 1 -- 172.21.15.49:0/2316520948 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d50032080 con 0x557fb4f51d80 2024-02-26T06:27:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.139+0000 7f8d43400640 1 -- 172.21.15.49:0/2316520948 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d50011a50 con 0x557fb4f51d80 2024-02-26T06:27:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.139+0000 7f8d6706e740 10 client.26576 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.139+0000 7f8d6706e740 10 client.26576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.139+0000 7f8d6706e740 20 client.26576 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.139+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557fb4f6b240 0x557fb4f6d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.140+0000 7f8d6706e740 4 client.26576 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.140+0000 7f8d6706e740 1 -- 172.21.15.49:0/2316520948 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fb4eaecb0 con 0x557fb4f6b240 2024-02-26T06:27:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.140+0000 7f8d60e00640 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557fb4f6b240 0x557fb4f6d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.141+0000 7f8d43400640 10 client.26576 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.141+0000 7f8d60e00640 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557fb4f6b240 0x557fb4f6d660 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=477538e1a3ff2594 server_cookie=db43161ab5b7a634 in_seq=0 out_seq=0 2024-02-26T06:27:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.142+0000 7f8d43400640 1 -- 172.21.15.49:0/2316520948 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fb4eaecb0 con 0x557fb4f6b240 2024-02-26T06:27:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.142+0000 7f8d43400640 10 client.26576 handle_command_reply: tid=0 2024-02-26T06:27:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.142+0000 7f8d6706e740 1 client.26576 shutdown 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 -- 172.21.15.49:0/2316520948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 msgr2=0x557fb4f58390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f58390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000bf50 tx=0x7f8d5000bf80 comp rx=0 tx=0).stop 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 -- 172.21.15.49:0/2316520948 shutdown_connections 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557fb4f6b240 0x557fb4f6d660 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f5b300 0x557fb4f58e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f52af0 0x557fb4f588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2316520948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f58390 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 -- 172.21.15.49:0/2316520948 >> 172.21.15.49:0/2316520948 conn(0x557fb4f19480 msgr2=0x557fb4f4e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d43400640 0 client.26576 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 -- 172.21.15.49:0/2316520948 shutdown_connections 2024-02-26T06:27:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.143+0000 7f8d6706e740 1 -- 172.21.15.49:0/2316520948 wait complete. 2024-02-26T06:27:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.144+0000 7f8d6706e740 20 client.26576 trim_cache size 0 max 16384 2024-02-26T06:27:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.171+0000 7f8d6706e740 1 Processor -- start 2024-02-26T06:27:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.171+0000 7f8d6706e740 1 -- start start 2024-02-26T06:27:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.171+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f5aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.171+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f58c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.171+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f62120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.171+0000 7f8d6706e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4c00e60 con 0x557fb4f5b300 2024-02-26T06:27:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.171+0000 7f8d6706e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4f512a0 con 0x557fb4f51d80 2024-02-26T06:27:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.171+0000 7f8d6706e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4f4da70 con 0x557fb4f52af0 2024-02-26T06:27:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f5aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f5aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54118/0 (socket says 172.21.15.49:54118) 2024-02-26T06:27:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 -- 172.21.15.49:0/2042265508 learned_addr learned my addr 172.21.15.49:0/2042265508 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d60e00640 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f58c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d62200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f62120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 -- 172.21.15.49:0/2042265508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 msgr2=0x557fb4f58c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f58c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 -- 172.21.15.49:0/2042265508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 msgr2=0x557fb4f62120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f62120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 -- 172.21.15.49:0/2042265508 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fb4e53500 con 0x557fb4f51d80 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d62200640 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.172+0000 7f8d61800640 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f5aac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8d50011480 tx=0x7f8d500107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb5873edeae71a28 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.173+0000 7f8d4e000640 1 -- 172.21.15.49:0/2042265508 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50020070 con 0x557fb4f51d80 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.173+0000 7f8d4e000640 1 -- 172.21.15.49:0/2042265508 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557fb4e8aca0 con 0x557fb4f51d80 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.173+0000 7f8d4e000640 1 -- 172.21.15.49:0/2042265508 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50004360 con 0x557fb4f51d80 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.174+0000 7f8d6706e740 1 -- 172.21.15.49:0/2042265508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 msgr2=0x557fb4f5aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.174+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f5aac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8d50011480 tx=0x7f8d500107a0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.174+0000 7f8d6706e740 1 -- 172.21.15.49:0/2042265508 shutdown_connections 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.174+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.174+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f52af0 0x557fb4f58c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.174+0000 7f8d6706e740 1 --2- 172.21.15.49:0/2042265508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f51d80 0x557fb4f5aac0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.174+0000 7f8d6706e740 1 -- 172.21.15.49:0/2042265508 >> 172.21.15.49:0/2042265508 conn(0x557fb4f19480 msgr2=0x557fb4f4c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.175+0000 7f8d6706e740 1 -- 172.21.15.49:0/2042265508 shutdown_connections 2024-02-26T06:27:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.175+0000 7f8d6706e740 1 -- 172.21.15.49:0/2042265508 wait complete. 2024-02-26T06:27:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.175+0000 7f8d6706e740 1 -- start start 2024-02-26T06:27:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.175+0000 7f8d6706e740 1 Processor -- start 2024-02-26T06:27:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.176+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f51d80 0x557fb4f62130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.176+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f52af0 0x557fb4f62670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.176+0000 7f8d61800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f51d80 0x557fb4f62130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.176+0000 7f8d61800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f51d80 0x557fb4f62130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33642/0 (socket says 172.21.15.49:33642) 2024-02-26T06:27:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.177+0000 7f8d6706e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f54720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.178+0000 7f8d6706e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4f54df0 con 0x557fb4f5b300 2024-02-26T06:27:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.178+0000 7f8d6706e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4f54f60 con 0x557fb4f52af0 2024-02-26T06:27:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.178+0000 7f8d6706e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4f55120 con 0x557fb4f51d80 2024-02-26T06:27:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.178+0000 7f8d6706e740 10 client.0 _ll_register_callbacks cb 0x7f8d48013680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.178+0000 7f8d61800640 1 -- 172.21.15.49:0/1292073693 learned_addr learned my addr 172.21.15.49:0/1292073693 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:27:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.179+0000 7f8d61800640 1 -- 172.21.15.49:0/1292073693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f52af0 msgr2=0x557fb4f62670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:27:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.179+0000 7f8d61800640 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f52af0 0x557fb4f62670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.179+0000 7f8d61800640 1 -- 172.21.15.49:0/1292073693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 msgr2=0x557fb4f54720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.179+0000 7f8d61800640 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f54720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.179+0000 7f8d61800640 1 -- 172.21.15.49:0/1292073693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fb4eb6c20 con 0x557fb4f51d80 2024-02-26T06:27:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.180+0000 7f8d61800640 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f51d80 0x557fb4f62130 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8d50010e80 tx=0x7f8d500116e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=807f34debcd5732a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.180+0000 7f8d43400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:27:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.180+0000 7f8d43400640 1 -- 172.21.15.49:0/1292073693 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50020040 con 0x557fb4f51d80 2024-02-26T06:27:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.181+0000 7f8d43400640 1 -- 172.21.15.49:0/1292073693 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f8d50017070 con 0x557fb4f51d80 2024-02-26T06:27:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.183+0000 7f8d6706e740 1 -- 172.21.15.49:0/1292073693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fb4eddd00 con 0x557fb4f51d80 2024-02-26T06:27:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.186+0000 7f8d43400640 1 -- 172.21.15.49:0/1292073693 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d5000bb00 con 0x557fb4f51d80 2024-02-26T06:27:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.186+0000 7f8d6706e740 10 client.26593 fetch_fsmap learned FSMap version 35 2024-02-26T06:27:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.186+0000 7f8d6706e740 1 -- 172.21.15.49:0/1292073693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557fb4e79290 con 0x557fb4f51d80 2024-02-26T06:27:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.187+0000 7f8d43400640 1 -- 172.21.15.49:0/1292073693 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557fb4eddd00 con 0x557fb4f51d80 2024-02-26T06:27:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.187+0000 7f8d43400640 1 -- 172.21.15.49:0/1292073693 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d50041070 con 0x557fb4f51d80 2024-02-26T06:27:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.188+0000 7f8d6706e740 10 client.26593 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:27:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.188+0000 7f8d6706e740 10 client.26593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.188+0000 7f8d6706e740 20 client.26593 populate_metadata read hostname 'smithi049' 2024-02-26T06:27:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.188+0000 7f8d6706e740 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557fb4fc2370 0x557fb4fc4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.188+0000 7f8d6706e740 4 client.26593 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:27:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.188+0000 7f8d6706e740 1 -- 172.21.15.49:0/1292073693 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557fb4bb8600 con 0x557fb4fc2370 2024-02-26T06:27:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.191+0000 7f8d60e00640 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557fb4fc2370 0x557fb4fc4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.192+0000 7f8d60e00640 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557fb4fc2370 0x557fb4fc4760 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d00a4cc568600ab server_cookie=fdd2ea014cba3f6e in_seq=0 out_seq=0 2024-02-26T06:27:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.192+0000 7f8d43400640 10 client.26593 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.216+0000 7f8d43400640 1 -- 172.21.15.49:0/1292073693 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557fb4bb8600 con 0x557fb4fc2370 2024-02-26T06:27:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.216+0000 7f8d43400640 10 client.26593 handle_command_reply: tid=0 2024-02-26T06:27:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.216+0000 7f8d6706e740 1 client.26593 shutdown 2024-02-26T06:27:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.217+0000 7f8d6706e740 1 -- 172.21.15.49:0/1292073693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f51d80 msgr2=0x557fb4f62130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.217+0000 7f8d6706e740 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f51d80 0x557fb4f62130 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8d50010e80 tx=0x7f8d500116e0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 1 -- 172.21.15.49:0/1292073693 shutdown_connections 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557fb4fc2370 0x557fb4fc4760 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557fb4f52af0 0x557fb4f62670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557fb4f5b300 0x557fb4f54720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d43400640 0 client.26593 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 1 --2- 172.21.15.49:0/1292073693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557fb4f51d80 0x557fb4f62130 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 1 -- 172.21.15.49:0/1292073693 >> 172.21.15.49:0/1292073693 conn(0x557fb4f19480 msgr2=0x557fb4f56b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 1 -- 172.21.15.49:0/1292073693 shutdown_connections 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 1 -- 172.21.15.49:0/1292073693 wait complete. 2024-02-26T06:27:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.218+0000 7f8d6706e740 20 client.26593 trim_cache size 0 max 16384 2024-02-26T06:27:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:27:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:27:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:27:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.219+0000 7f8d64200640 1 -- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8d440a2350 msgr2=0x7f8d440a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.219+0000 7f8d64200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8d440a2350 0x7f8d440a4810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009fd0 tx=0x7f8d48009cc0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.219+0000 7f8d64200640 1 -- 172.21.15.49:0/3782800047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d5c106100 msgr2=0x7f8d5c19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.219+0000 7f8d64200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d5c106100 0x7f8d5c19d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d54033610 tx=0x7f8d54033640 comp rx=0 tx=0).stop 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.219+0000 7f8d64200640 1 -- 172.21.15.49:0/3782800047 shutdown_connections 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.220+0000 7f8d64200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8d440a2350 0x7f8d440a4810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.220+0000 7f8d64200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d5c106100 0x7f8d5c19d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.220+0000 7f8d64200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d5c103b90 0x7f8d5c1a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.220+0000 7f8d64200640 1 --2- 172.21.15.49:0/3782800047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c069e40 0x7f8d5c1a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.220+0000 7f8d64200640 1 -- 172.21.15.49:0/3782800047 >> 172.21.15.49:0/3782800047 conn(0x7f8d5c076e70 msgr2=0x7f8d5c102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.220+0000 7f8d64200640 1 -- 172.21.15.49:0/3782800047 shutdown_connections 2024-02-26T06:27:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:27:52.220+0000 7f8d64200640 1 -- 172.21.15.49:0/3782800047 wait complete. 2024-02-26T06:27:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:52 smithi080 ceph-mon[36206]: pgmap v137: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.3 MiB/s wr, 800 op/s 2024-02-26T06:27:52.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:52 smithi115 ceph-mon[36426]: pgmap v137: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.3 MiB/s wr, 800 op/s 2024-02-26T06:27:52.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:52 smithi049 ceph-mon[31696]: pgmap v137: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.3 MiB/s wr, 800 op/s 2024-02-26T06:27:54.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:54 smithi115 ceph-mon[36426]: pgmap v138: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-02-26T06:27:54.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:54 smithi049 ceph-mon[31696]: pgmap v138: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-02-26T06:27:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:54 smithi080 ceph-mon[36206]: pgmap v138: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-02-26T06:27:55.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:55 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:27:55.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:55 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:27:56.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:55 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:27:56.304 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 245.0 s, 1972.8 tps, lat 50.657 ms stddev 62.482 2024-02-26T06:27:56.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:56 smithi049 ceph-mon[31696]: pgmap v139: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-02-26T06:27:56.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:56 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:27:56.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:56 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:27:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:56 smithi080 ceph-mon[36206]: pgmap v139: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-02-26T06:27:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:56 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:27:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:56 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:27:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:56 smithi115 ceph-mon[36426]: pgmap v139: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-02-26T06:27:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:56 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:27:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:56 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:27:58.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:27:58 smithi049 ceph-mon[31696]: pgmap v140: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 649 op/s 2024-02-26T06:27:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:27:58 smithi080 ceph-mon[36206]: pgmap v140: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 649 op/s 2024-02-26T06:27:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:27:58 smithi115 ceph-mon[36426]: pgmap v140: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 649 op/s 2024-02-26T06:28:00.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:00 smithi049 ceph-mon[31696]: pgmap v141: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 795 op/s 2024-02-26T06:28:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:00 smithi080 ceph-mon[36206]: pgmap v141: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 795 op/s 2024-02-26T06:28:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:00 smithi115 ceph-mon[36426]: pgmap v141: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 795 op/s 2024-02-26T06:28:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 250.0 s, 1695.5 tps, lat 55.670 ms stddev 73.221 2024-02-26T06:28:02.881 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:02 smithi080 ceph-mon[36206]: pgmap v142: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.7 MiB/s wr, 684 op/s 2024-02-26T06:28:02.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:02 smithi049 ceph-mon[31696]: pgmap v142: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.7 MiB/s wr, 684 op/s 2024-02-26T06:28:03.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:02 smithi115 ceph-mon[36426]: pgmap v142: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.7 MiB/s wr, 684 op/s 2024-02-26T06:28:04.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:04 smithi049 ceph-mon[31696]: pgmap v143: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-02-26T06:28:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:04 smithi080 ceph-mon[36206]: pgmap v143: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-02-26T06:28:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:04 smithi115 ceph-mon[36426]: pgmap v143: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-02-26T06:28:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.380+0000 7f0d99400640 1 -- 172.21.15.49:0/4146202633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d94101d50 msgr2=0x7f0d94105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.380+0000 7f0d99400640 1 --2- 172.21.15.49:0/4146202633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d94101d50 0x7f0d94105d90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009860 tx=0x7f0d8002bb70 comp rx=0 tx=0).stop 2024-02-26T06:28:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.381+0000 7f0d99400640 1 -- 172.21.15.49:0/4146202633 shutdown_connections 2024-02-26T06:28:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.381+0000 7f0d99400640 1 --2- 172.21.15.49:0/4146202633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d94106470 0x7f0d9410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.381+0000 7f0d99400640 1 --2- 172.21.15.49:0/4146202633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d94101d50 0x7f0d94105d90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.381+0000 7f0d99400640 1 --2- 172.21.15.49:0/4146202633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d941013a0 0x7f0d94101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.381+0000 7f0d99400640 1 -- 172.21.15.49:0/4146202633 >> 172.21.15.49:0/4146202633 conn(0x7f0d940fcc40 msgr2=0x7f0d940ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.381+0000 7f0d99400640 1 -- 172.21.15.49:0/4146202633 shutdown_connections 2024-02-26T06:28:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.381+0000 7f0d99400640 1 -- 172.21.15.49:0/4146202633 wait complete. 2024-02-26T06:28:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.382+0000 7f0d99400640 1 Processor -- start 2024-02-26T06:28:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.382+0000 7f0d99400640 1 -- start start 2024-02-26T06:28:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d99400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d941013a0 0x7f0d94197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d99400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d94106470 0x7f0d94197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d99400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9419bfd0 0x7f0d9419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d99400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9419c9c0 con 0x7f0d9419bfd0 2024-02-26T06:28:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d99400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9419cb30 con 0x7f0d94106470 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d99400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9419cca0 con 0x7f0d941013a0 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d92800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d941013a0 0x7f0d94197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d94106470 0x7f0d94197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d92800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d941013a0 0x7f0d94197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:59928/0 (socket says 172.21.15.49:59928) 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d94106470 0x7f0d94197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54754/0 (socket says 172.21.15.49:54754) 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.383+0000 7f0d92800640 1 -- 172.21.15.49:0/2420308866 learned_addr learned my addr 172.21.15.49:0/2420308866 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.384+0000 7f0d93200640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9419bfd0 0x7f0d9419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.384+0000 7f0d92800640 1 -- 172.21.15.49:0/2420308866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d94106470 msgr2=0x7f0d94197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.384+0000 7f0d92800640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d94106470 0x7f0d94197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.384+0000 7f0d92800640 1 -- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9419bfd0 msgr2=0x7f0d9419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.384+0000 7f0d92800640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9419bfd0 0x7f0d9419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.384+0000 7f0d92800640 1 -- 172.21.15.49:0/2420308866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80009490 con 0x7f0d941013a0 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.384+0000 7f0d91e00640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d94106470 0x7f0d94197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.384+0000 7f0d92800640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d941013a0 0x7f0d94197710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400e9c0 tx=0x7f0d8400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.385+0000 7f0d8b400640 1 -- 172.21.15.49:0/2420308866 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8400ccc0 con 0x7f0d941013a0 2024-02-26T06:28:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.385+0000 7f0d8b400640 1 -- 172.21.15.49:0/2420308866 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d8400ce20 con 0x7f0d941013a0 2024-02-26T06:28:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.385+0000 7f0d8b400640 1 -- 172.21.15.49:0/2420308866 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84010430 con 0x7f0d941013a0 2024-02-26T06:28:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.385+0000 7f0d99400640 1 -- 172.21.15.49:0/2420308866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d941a4280 con 0x7f0d941013a0 2024-02-26T06:28:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.385+0000 7f0d99400640 1 -- 172.21.15.49:0/2420308866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d941a46e0 con 0x7f0d941013a0 2024-02-26T06:28:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.389+0000 7f0d8b400640 1 -- 172.21.15.49:0/2420308866 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0d84004f80 con 0x7f0d941013a0 2024-02-26T06:28:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.389+0000 7f0d9c22a740 1 Processor -- start 2024-02-26T06:28:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.389+0000 7f0d9c22a740 1 -- start start 2024-02-26T06:28:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902e9b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e9675f0 con 0x55902e9abd80 2024-02-26T06:28:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e919890 con 0x55902e9acaf0 2024-02-26T06:28:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e916dc0 con 0x55902e9b5300 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d93200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d93200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9ab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54764/0 (socket says 172.21.15.49:54764) 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d93200640 1 -- 172.21.15.49:0/2302216214 learned_addr learned my addr 172.21.15.49:0/2302216214 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d91e00640 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902e9b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d92800640 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d93200640 1 -- 172.21.15.49:0/2302216214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 msgr2=0x55902e9b76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d93200640 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902e9b76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d93200640 1 -- 172.21.15.49:0/2302216214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 msgr2=0x55902e9ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d93200640 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.390+0000 7f0d93200640 1 -- 172.21.15.49:0/2302216214 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55902e893ad0 con 0x55902e9acaf0 2024-02-26T06:28:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.391+0000 7f0d8b400640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f0d780a2350 0x7f0d780a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.391+0000 7f0d93200640 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9ab840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009f20 tx=0x7f0d8c00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a524934f7400771 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.391+0000 7f0d8b400640 1 -- 172.21.15.49:0/2420308866 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f0d84014070 con 0x7f0d941013a0 2024-02-26T06:28:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.391+0000 7f0d7f400640 1 -- 172.21.15.49:0/2302216214 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c01a660 con 0x55902e9acaf0 2024-02-26T06:28:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.391+0000 7f0d7f400640 1 -- 172.21.15.49:0/2302216214 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d8c01aca0 con 0x55902e9acaf0 2024-02-26T06:28:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.391+0000 7f0d7f400640 1 -- 172.21.15.49:0/2302216214 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c011da0 con 0x55902e9acaf0 2024-02-26T06:28:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.391+0000 7f0d92800640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f0d780a2350 0x7f0d780a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2302216214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 msgr2=0x55902e9ab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d92800640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f0d780a2350 0x7f0d780a4810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d840cbc80 tx=0x7f0d84005880 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9ab840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009f20 tx=0x7f0d8c00f7f0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2302216214 shutdown_connections 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902e9b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2302216214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9ab840 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2302216214 >> 172.21.15.49:0/2302216214 conn(0x55902e973480 msgr2=0x55902e9a6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.392+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2302216214 shutdown_connections 2024-02-26T06:28:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.393+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2302216214 wait complete. 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.393+0000 7f0d9c22a740 1 -- start start 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.393+0000 7f0d9c22a740 1 Processor -- start 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9acaf0 0x55902e9b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902e9b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e9125a0 con 0x55902e9acaf0 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e8b7b20 con 0x55902e9abd80 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e8b77c0 con 0x55902e9b5300 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d9c22a740 10 client.0 _ll_register_callbacks cb 0x55902e8cb520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d93200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9acaf0 0x55902e9b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d93200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54780/0 (socket says 172.21.15.49:54780) 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d93200640 1 -- 172.21.15.49:0/1354823657 learned_addr learned my addr 172.21.15.49:0/1354823657 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.394+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9acaf0 0x55902e9b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35636/0 (socket says 172.21.15.49:35636) 2024-02-26T06:28:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.395+0000 7f0d93200640 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902e9b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.395+0000 7f0d91e00640 1 -- 172.21.15.49:0/1354823657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 msgr2=0x55902e9b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.395+0000 7f0d91e00640 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902e9b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.395+0000 7f0d91e00640 1 -- 172.21.15.49:0/1354823657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 msgr2=0x55902e9b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.395+0000 7f0d91e00640 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.395+0000 7f0d91e00640 1 -- 172.21.15.49:0/1354823657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c009b80 con 0x55902e9acaf0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.395+0000 7f0d91e00640 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9acaf0 0x55902e9b2870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009830 tx=0x7f0d80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b502f146d457718 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.395+0000 7f0d73e00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.396+0000 7f0d73e00640 1 -- 172.21.15.49:0/1354823657 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80042070 con 0x55902e9acaf0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.396+0000 7f0d73e00640 1 -- 172.21.15.49:0/1354823657 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f0d80034440 con 0x55902e9acaf0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.396+0000 7f0d9c22a740 1 -- 172.21.15.49:0/1354823657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55902e8e4ca0 con 0x55902e9acaf0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.396+0000 7f0d73e00640 1 -- 172.21.15.49:0/1354823657 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d80034600 con 0x55902e9acaf0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.396+0000 7f0d9c22a740 10 client.16194 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.396+0000 7f0d9c22a740 1 -- 172.21.15.49:0/1354823657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55902e910c20 con 0x55902e9acaf0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.396+0000 7f0d73e00640 1 -- 172.21.15.49:0/1354823657 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d80053020 con 0x55902e9acaf0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.397+0000 7f0d73e00640 1 -- 172.21.15.49:0/1354823657 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0d8002c630 con 0x55902e9acaf0 2024-02-26T06:28:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.397+0000 7f0d9c22a740 10 client.16194 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.397+0000 7f0d9c22a740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.397+0000 7f0d9c22a740 20 client.16194 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.397+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55902e9c5290 0x55902e9c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.397+0000 7f0d9c22a740 4 client.16194 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.397+0000 7f0d9c22a740 1 -- 172.21.15.49:0/1354823657 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55902e908cb0 con 0x55902e9c5290 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.397+0000 7f0d93200640 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55902e9c5290 0x55902e9c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.398+0000 7f0d93200640 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55902e9c5290 0x55902e9c76b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28da2a10f297891e server_cookie=c7d07dbb729ea826 in_seq=0 out_seq=0 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.398+0000 7f0d73e00640 10 client.16194 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.398+0000 7f0d73e00640 1 -- 172.21.15.49:0/1354823657 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55902e908cb0 con 0x55902e9c5290 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.398+0000 7f0d73e00640 10 client.16194 handle_command_reply: tid=0 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.398+0000 7f0d9c22a740 1 client.16194 shutdown 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 -- 172.21.15.49:0/1354823657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9acaf0 msgr2=0x55902e9b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9acaf0 0x55902e9b2870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009830 tx=0x7f0d80004270 comp rx=0 tx=0).stop 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 -- 172.21.15.49:0/1354823657 shutdown_connections 2024-02-26T06:28:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55902e9c5290 0x55902e9c76b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902e9b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9acaf0 0x55902e9b2870 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/1354823657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 -- 172.21.15.49:0/1354823657 >> 172.21.15.49:0/1354823657 conn(0x55902e973480 msgr2=0x55902e9a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d73e00640 0 client.16194 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 -- 172.21.15.49:0/1354823657 shutdown_connections 2024-02-26T06:28:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 1 -- 172.21.15.49:0/1354823657 wait complete. 2024-02-26T06:28:05.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.399+0000 7f0d9c22a740 20 client.16194 trim_cache size 0 max 16384 2024-02-26T06:28:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.419+0000 7f0d9c22a740 1 Processor -- start 2024-02-26T06:28:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.419+0000 7f0d9c22a740 1 -- start start 2024-02-26T06:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.419+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.419+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9acaf0 0x55902e9b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.419+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9b5300 0x55902e9bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.419+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e65ae60 con 0x55902e9b5300 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.419+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e9cc900 con 0x55902e9abd80 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.419+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e9ab050 con 0x55902e9acaf0 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9acaf0 0x55902e9b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d93200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9acaf0 0x55902e9b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:59968/0 (socket says 172.21.15.49:59968) 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 -- 172.21.15.49:0/3287493282 learned_addr learned my addr 172.21.15.49:0/3287493282 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d93200640 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9b5300 0x55902e9bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 -- 172.21.15.49:0/3287493282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 msgr2=0x55902e9b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 -- 172.21.15.49:0/3287493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9b5300 msgr2=0x55902e9bc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9b5300 0x55902e9bc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 -- 172.21.15.49:0/3287493282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55902e8ad500 con 0x55902e9acaf0 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d93200640 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9b5300 0x55902e9bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.420+0000 7f0d91e00640 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9acaf0 0x55902e9b2bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d8002ca10 tx=0x7f0d8002bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b35ab23b417f681 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.421+0000 7f0d7ea00640 1 -- 172.21.15.49:0/3287493282 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80042040 con 0x55902e9acaf0 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.421+0000 7f0d7ea00640 1 -- 172.21.15.49:0/3287493282 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d80053030 con 0x55902e9acaf0 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.421+0000 7f0d7ea00640 1 -- 172.21.15.49:0/3287493282 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8003d620 con 0x55902e9acaf0 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 -- 172.21.15.49:0/3287493282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9acaf0 msgr2=0x55902e9b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9acaf0 0x55902e9b2bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d8002ca10 tx=0x7f0d8002bb70 comp rx=0 tx=0).stop 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 -- 172.21.15.49:0/3287493282 shutdown_connections 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9b5300 0x55902e9bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9acaf0 0x55902e9b2bd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/3287493282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9abd80 0x55902e9b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 -- 172.21.15.49:0/3287493282 >> 172.21.15.49:0/3287493282 conn(0x55902e973480 msgr2=0x55902e9b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 -- 172.21.15.49:0/3287493282 shutdown_connections 2024-02-26T06:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 -- 172.21.15.49:0/3287493282 wait complete. 2024-02-26T06:28:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.422+0000 7f0d9c22a740 1 -- start start 2024-02-26T06:28:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.423+0000 7f0d9c22a740 1 Processor -- start 2024-02-26T06:28:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.423+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.423+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.423+0000 7f0d9c22a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902ea1b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.423+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e9afff0 con 0x55902e9abd80 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.423+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902e9b0160 con 0x55902e9acaf0 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.423+0000 7f0d9c22a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902ea1bcf0 con 0x55902e9b5300 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.423+0000 7f0d9c22a740 10 client.0 _ll_register_callbacks cb 0x7f0d80037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.424+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.424+0000 7f0d91e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9af850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54800/0 (socket says 172.21.15.49:54800) 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.424+0000 7f0d91e00640 1 -- 172.21.15.49:0/2728120231 learned_addr learned my addr 172.21.15.49:0/2728120231 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.424+0000 7f0d93200640 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.424+0000 7f0d93200640 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902ea1b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.425+0000 7f0d91e00640 1 -- 172.21.15.49:0/2728120231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 msgr2=0x55902ea1b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.425+0000 7f0d91e00640 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902ea1b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.425+0000 7f0d91e00640 1 -- 172.21.15.49:0/2728120231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 msgr2=0x55902e9af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.426+0000 7f0d91e00640 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.426+0000 7f0d91e00640 1 -- 172.21.15.49:0/2728120231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55902e910c20 con 0x55902e9acaf0 2024-02-26T06:28:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.426+0000 7f0d93200640 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.427+0000 7f0d91e00640 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9af850 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0d8002cb30 tx=0x7f0d800349e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e30468c660e7563b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.427+0000 7f0d73e00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.427+0000 7f0d73e00640 1 -- 172.21.15.49:0/2728120231 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80042040 con 0x55902e9acaf0 2024-02-26T06:28:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.427+0000 7f0d73e00640 1 -- 172.21.15.49:0/2728120231 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f0d80000e40 con 0x55902e9acaf0 2024-02-26T06:28:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.427+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2728120231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55902e937d00 con 0x55902e9acaf0 2024-02-26T06:28:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d73e00640 1 -- 172.21.15.49:0/2728120231 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d80005040 con 0x55902e9acaf0 2024-02-26T06:28:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d9c22a740 10 client.26594 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2728120231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55902e8d3290 con 0x55902e9acaf0 2024-02-26T06:28:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d73e00640 1 -- 172.21.15.49:0/2728120231 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55902e937d00 con 0x55902e9acaf0 2024-02-26T06:28:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d73e00640 1 -- 172.21.15.49:0/2728120231 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0d80004a00 con 0x55902e9acaf0 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d9c22a740 10 client.26594 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d9c22a740 10 client.26594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d9c22a740 20 client.26594 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55902ea1d0b0 0x55902ea1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d9c22a740 4 client.26594 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.428+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2728120231 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55902e612600 con 0x55902ea1d0b0 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.429+0000 7f0d93200640 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55902ea1d0b0 0x55902ea1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.430+0000 7f0d93200640 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55902ea1d0b0 0x55902ea1f4a0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464bfbf4b3e640e1 server_cookie=5fce8661e0786733 in_seq=0 out_seq=0 2024-02-26T06:28:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.430+0000 7f0d73e00640 10 client.26594 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.443+0000 7f0d73e00640 1 -- 172.21.15.49:0/2728120231 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55902e612600 con 0x55902ea1d0b0 2024-02-26T06:28:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.443+0000 7f0d73e00640 10 client.26594 handle_command_reply: tid=0 2024-02-26T06:28:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.443+0000 7f0d9c22a740 1 client.26594 shutdown 2024-02-26T06:28:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.443+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2728120231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 msgr2=0x55902e9af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.443+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9af850 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0d8002cb30 tx=0x7f0d800349e0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.443+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2728120231 shutdown_connections 2024-02-26T06:28:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.444+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55902ea1d0b0 0x55902ea1f4a0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.444+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55902e9acaf0 0x55902e9af850 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.444+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55902e9b5300 0x55902ea1b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.444+0000 7f0d9c22a740 1 --2- 172.21.15.49:0/2728120231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55902e9abd80 0x55902e9af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.444+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2728120231 >> 172.21.15.49:0/2728120231 conn(0x55902e973480 msgr2=0x55902e9a5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.444+0000 7f0d73e00640 0 client.26594 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.444+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2728120231 shutdown_connections 2024-02-26T06:28:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.444+0000 7f0d9c22a740 1 -- 172.21.15.49:0/2728120231 wait complete. 2024-02-26T06:28:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.445+0000 7f0d9c22a740 20 client.26594 trim_cache size 0 max 16384 2024-02-26T06:28:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:28:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:28:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:28:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:28:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 -- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f0d780a2350 msgr2=0x7f0d780a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f0d780a2350 0x7f0d780a4810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d840cbc80 tx=0x7f0d84005880 comp rx=0 tx=0).stop 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 -- 172.21.15.49:0/2420308866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d941013a0 msgr2=0x7f0d94197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d941013a0 0x7f0d94197710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400e9c0 tx=0x7f0d8400ee80 comp rx=0 tx=0).stop 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 -- 172.21.15.49:0/2420308866 shutdown_connections 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f0d780a2350 0x7f0d780a4810 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d94106470 0x7f0d94197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9419bfd0 0x7f0d9419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 --2- 172.21.15.49:0/2420308866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d941013a0 0x7f0d94197710 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 -- 172.21.15.49:0/2420308866 >> 172.21.15.49:0/2420308866 conn(0x7f0d940fcc40 msgr2=0x7f0d940feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 -- 172.21.15.49:0/2420308866 shutdown_connections 2024-02-26T06:28:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:05.446+0000 7f0d99400640 1 -- 172.21.15.49:0/2420308866 wait complete. 2024-02-26T06:28:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 255.0 s, 190.4 tps, lat 521.868 ms stddev 316.656 2024-02-26T06:28:06.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:06 smithi049 ceph-mon[31696]: pgmap v144: 97 pgs: 97 active+clean; 2.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 784 op/s 2024-02-26T06:28:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:06 smithi080 ceph-mon[36206]: pgmap v144: 97 pgs: 97 active+clean; 2.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 784 op/s 2024-02-26T06:28:07.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:06 smithi115 ceph-mon[36426]: pgmap v144: 97 pgs: 97 active+clean; 2.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 784 op/s 2024-02-26T06:28:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:08 smithi049 ceph-mon[31696]: pgmap v145: 97 pgs: 97 active+clean; 2.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 514 op/s 2024-02-26T06:28:09.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:08 smithi080 ceph-mon[36206]: pgmap v145: 97 pgs: 97 active+clean; 2.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 514 op/s 2024-02-26T06:28:09.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:08 smithi115 ceph-mon[36426]: pgmap v145: 97 pgs: 97 active+clean; 2.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 514 op/s 2024-02-26T06:28:10.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:10 smithi049 ceph-mon[31696]: pgmap v146: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-02-26T06:28:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:10 smithi080 ceph-mon[36206]: pgmap v146: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-02-26T06:28:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:10 smithi115 ceph-mon[36426]: pgmap v146: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-02-26T06:28:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 260.0 s, 153.8 tps, lat 648.238 ms stddev 490.440 2024-02-26T06:28:12.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:12 smithi049 ceph-mon[31696]: pgmap v147: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.6 MiB/s wr, 492 op/s 2024-02-26T06:28:13.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:12 smithi080 ceph-mon[36206]: pgmap v147: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.6 MiB/s wr, 492 op/s 2024-02-26T06:28:13.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:12 smithi115 ceph-mon[36426]: pgmap v147: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.6 MiB/s wr, 492 op/s 2024-02-26T06:28:14.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:14 smithi049 ceph-mon[31696]: pgmap v148: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.4 MiB/s wr, 340 op/s 2024-02-26T06:28:15.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:14 smithi080 ceph-mon[36206]: pgmap v148: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.4 MiB/s wr, 340 op/s 2024-02-26T06:28:15.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:14 smithi115 ceph-mon[36426]: pgmap v148: 97 pgs: 97 active+clean; 2.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.4 MiB/s wr, 340 op/s 2024-02-26T06:28:16.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:28:16.137 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:28:16.304 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 265.0 s, 1623.6 tps, lat 64.375 ms stddev 105.113 2024-02-26T06:28:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 -- 172.21.15.49:0/2464205837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a01142d0 msgr2=0x7f45a01166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 --2- 172.21.15.49:0/2464205837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a01142d0 0x7f45a01166c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4598009d00 tx=0x7f459802c170 comp rx=0 tx=0).stop 2024-02-26T06:28:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 -- 172.21.15.49:0/2464205837 shutdown_connections 2024-02-26T06:28:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 --2- 172.21.15.49:0/2464205837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a01142d0 0x7f45a01166c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 --2- 172.21.15.49:0/2464205837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45a0108aa0 0x7f45a0113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 --2- 172.21.15.49:0/2464205837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a0108180 0x7f45a0108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 -- 172.21.15.49:0/2464205837 >> 172.21.15.49:0/2464205837 conn(0x7f45a006c6c0 msgr2=0x7f45a006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 -- 172.21.15.49:0/2464205837 shutdown_connections 2024-02-26T06:28:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.420+0000 7f45a5800640 1 -- 172.21.15.49:0/2464205837 wait complete. 2024-02-26T06:28:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.422+0000 7f45a5800640 1 Processor -- start 2024-02-26T06:28:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.422+0000 7f45a5800640 1 -- start start 2024-02-26T06:28:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.422+0000 7f45a5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a0108180 0x7f45a01a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.422+0000 7f45a5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45a0108aa0 0x7f45a01a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f45a5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a01142d0 0x7f45a01a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f45a5800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01a4ed0 con 0x7f45a0108aa0 2024-02-26T06:28:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f45a5800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01a5040 con 0x7f45a0108180 2024-02-26T06:28:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f45a5800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01a5200 con 0x7f45a01142d0 2024-02-26T06:28:16.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f459f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a01142d0 0x7f45a01a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f459e000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45a0108aa0 0x7f45a01a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f459e000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45a0108aa0 0x7f45a01a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41390/0 (socket says 172.21.15.49:41390) 2024-02-26T06:28:16.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f459e000640 1 -- 172.21.15.49:0/3957365739 learned_addr learned my addr 172.21.15.49:0/3957365739 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:16.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f459ea00640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a0108180 0x7f45a01a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.424+0000 7f459e000640 1 -- 172.21.15.49:0/3957365739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a01142d0 msgr2=0x7f45a01a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.423+0000 7f459f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a01142d0 0x7f45a01a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43544/0 (socket says 172.21.15.49:43544) 2024-02-26T06:28:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.424+0000 7f459e000640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a01142d0 0x7f45a01a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.424+0000 7f459e000640 1 -- 172.21.15.49:0/3957365739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a0108180 msgr2=0x7f45a01a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.424+0000 7f459e000640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a0108180 0x7f45a01a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.424+0000 7f459e000640 1 -- 172.21.15.49:0/3957365739 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598009900 con 0x7f45a0108aa0 2024-02-26T06:28:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.424+0000 7f459f400640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a01142d0 0x7f45a01a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.424+0000 7f459ea00640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a0108180 0x7f45a01a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.425+0000 7f459e000640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45a0108aa0 0x7f45a01a0550 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f458c00d950 tx=0x7f458c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.425+0000 7f4597400640 1 -- 172.21.15.49:0/3957365739 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c009a30 con 0x7f45a0108aa0 2024-02-26T06:28:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.425+0000 7f4597400640 1 -- 172.21.15.49:0/3957365739 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f458c009b90 con 0x7f45a0108aa0 2024-02-26T06:28:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.425+0000 7f4597400640 1 -- 172.21.15.49:0/3957365739 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c009a30 con 0x7f45a0108aa0 2024-02-26T06:28:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.426+0000 7f45a5800640 1 -- 172.21.15.49:0/3957365739 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a01a54e0 con 0x7f45a0108aa0 2024-02-26T06:28:16.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.426+0000 7f45a5800640 1 -- 172.21.15.49:0/3957365739 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a0109d00 con 0x7f45a0108aa0 2024-02-26T06:28:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.433+0000 7f4597400640 1 -- 172.21.15.49:0/3957365739 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f458c0046c0 con 0x7f45a0108aa0 2024-02-26T06:28:16.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.437+0000 7f4597400640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f45840a2310 0x7f45840a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.438+0000 7f45a8627740 1 Processor -- start 2024-02-26T06:28:16.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.438+0000 7f45a8627740 1 -- start start 2024-02-26T06:28:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.439+0000 7f45a8627740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727f70 0x563029726cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.439+0000 7f45a8627740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727290 0x563029727710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.439+0000 7f45a8627740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x5630297329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.439+0000 7f45a8627740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029696120 con 0x563029730600 2024-02-26T06:28:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.439+0000 7f45a8627740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029693650 con 0x563029727290 2024-02-26T06:28:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.439+0000 7f45a8627740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302968ee30 con 0x563029727f70 2024-02-26T06:28:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.441+0000 7f459e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727290 0x563029727710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.441+0000 7f4597400640 1 -- 172.21.15.49:0/3957365739 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f458c0cd6c0 con 0x7f45a0108aa0 2024-02-26T06:28:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.441+0000 7f459f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x5630297329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.441+0000 7f459e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727290 0x563029727710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44714/0 (socket says 172.21.15.49:44714) 2024-02-26T06:28:16.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.441+0000 7f459e000640 1 -- 172.21.15.49:0/3056768668 learned_addr learned my addr 172.21.15.49:0/3056768668 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:16.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.442+0000 7f459ea00640 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727f70 0x563029726cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.442+0000 7f459ea00640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f45840a2310 0x7f45840a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.443+0000 7f459e000640 1 -- 172.21.15.49:0/3056768668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727f70 msgr2=0x563029726cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.443+0000 7f459e000640 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727f70 0x563029726cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.443+0000 7f459e000640 1 -- 172.21.15.49:0/3056768668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 msgr2=0x5630297329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.443+0000 7f459e000640 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x5630297329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.443+0000 7f459e000640 1 -- 172.21.15.49:0/3056768668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563029648030 con 0x563029727290 2024-02-26T06:28:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.443+0000 7f459f400640 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x5630297329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.443+0000 7f459e000640 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727290 0x563029727710 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f458c099dc0 tx=0x7f458c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f698f1e18e6d38 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.445+0000 7f458be00640 1 -- 172.21.15.49:0/3056768668 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c09a7c0 con 0x563029727290 2024-02-26T06:28:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.445+0000 7f458be00640 1 -- 172.21.15.49:0/3056768668 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f458c019310 con 0x563029727290 2024-02-26T06:28:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.445+0000 7f458be00640 1 -- 172.21.15.49:0/3056768668 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c009d00 con 0x563029727290 2024-02-26T06:28:16.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.446+0000 7f45a8627740 1 -- 172.21.15.49:0/3056768668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727290 msgr2=0x563029727710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.446+0000 7f45a8627740 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727290 0x563029727710 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f458c099dc0 tx=0x7f458c000bc0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.447+0000 7f459ea00640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f45840a2310 0x7f45840a47d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4590005fd0 tx=0x7f4590004730 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:16.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.447+0000 7f45a8627740 1 -- 172.21.15.49:0/3056768668 shutdown_connections 2024-02-26T06:28:16.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.447+0000 7f45a8627740 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727290 0x563029727710 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.447+0000 7f45a8627740 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x5630297329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.447+0000 7f45a8627740 1 --2- 172.21.15.49:0/3056768668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727f70 0x563029726cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.447+0000 7f45a8627740 1 -- 172.21.15.49:0/3056768668 >> 172.21.15.49:0/3056768668 conn(0x5630296ee570 msgr2=0x563029721ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:16.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.447+0000 7f45a8627740 1 -- 172.21.15.49:0/3056768668 shutdown_connections 2024-02-26T06:28:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.447+0000 7f45a8627740 1 -- 172.21.15.49:0/3056768668 wait complete. 2024-02-26T06:28:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.448+0000 7f45a8627740 1 -- start start 2024-02-26T06:28:16.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.448+0000 7f45a8627740 1 Processor -- start 2024-02-26T06:28:16.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f45a8627740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727290 0x56302972d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f45a8627740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x56302972daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f45a8627740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029730600 0x56302972e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f45a8627740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029633e90 con 0x563029727290 2024-02-26T06:28:16.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f45a8627740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029633b30 con 0x563029727f70 2024-02-26T06:28:16.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f45a8627740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630293fd1c0 con 0x563029730600 2024-02-26T06:28:16.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f459e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x56302972daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f459e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x56302972daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44716/0 (socket says 172.21.15.49:44716) 2024-02-26T06:28:16.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f45a8627740 10 client.0 _ll_register_callbacks cb 0x563029647930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:16.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f459e000640 1 -- 172.21.15.49:0/4279949063 learned_addr learned my addr 172.21.15.49:0/4279949063 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:16.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.451+0000 7f459f400640 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029730600 0x56302972e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.452+0000 7f459ea00640 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727290 0x56302972d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.453+0000 7f459e000640 1 -- 172.21.15.49:0/4279949063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029730600 msgr2=0x56302972e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:16.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.453+0000 7f459e000640 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029730600 0x56302972e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.453+0000 7f459e000640 1 -- 172.21.15.49:0/4279949063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727290 msgr2=0x56302972d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:16.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.453+0000 7f459e000640 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727290 0x56302972d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.453+0000 7f459e000640 1 -- 172.21.15.49:0/4279949063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a0109d00 con 0x563029727f70 2024-02-26T06:28:16.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.454+0000 7f459e000640 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x56302972daf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f458c0747b0 tx=0x7f458c09a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d69f514690a7fe84 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.454+0000 7f4588c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.454+0000 7f4588c00640 1 -- 172.21.15.49:0/4279949063 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c09e210 con 0x563029727f70 2024-02-26T06:28:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.454+0000 7f4588c00640 1 -- 172.21.15.49:0/4279949063 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f458c096730 con 0x563029727f70 2024-02-26T06:28:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.454+0000 7f45a8627740 1 -- 172.21.15.49:0/4279949063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56302938e7e0 con 0x563029727f70 2024-02-26T06:28:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.455+0000 7f4588c00640 1 -- 172.21.15.49:0/4279949063 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f458c0968f0 con 0x563029727f70 2024-02-26T06:28:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.455+0000 7f45a8627740 10 client.26603 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.455+0000 7f45a8627740 1 -- 172.21.15.49:0/4279949063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56302968d4b0 con 0x563029727f70 2024-02-26T06:28:16.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.455+0000 7f4588c00640 1 -- 172.21.15.49:0/4279949063 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f458c0b2080 con 0x563029727f70 2024-02-26T06:28:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.455+0000 7f4588c00640 1 -- 172.21.15.49:0/4279949063 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f458c099510 con 0x563029727f70 2024-02-26T06:28:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.455+0000 7f45a8627740 10 client.26603 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.456+0000 7f45a8627740 10 client.26603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.456+0000 7f45a8627740 20 client.26603 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.456+0000 7f45a8627740 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563029740290 0x5630297426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.456+0000 7f45a8627740 4 client.26603 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:16.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.456+0000 7f45a8627740 1 -- 172.21.15.49:0/4279949063 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630293cb720 con 0x563029740290 2024-02-26T06:28:16.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.456+0000 7f459ea00640 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563029740290 0x5630297426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.458+0000 7f4588c00640 10 client.26603 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:16.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.459+0000 7f459ea00640 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563029740290 0x5630297426b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e894344d11a8464 server_cookie=923b7a2f34b9c020 in_seq=0 out_seq=0 2024-02-26T06:28:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.460+0000 7f4588c00640 1 -- 172.21.15.49:0/4279949063 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5630293cb720 con 0x563029740290 2024-02-26T06:28:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.460+0000 7f4588c00640 10 client.26603 handle_command_reply: tid=0 2024-02-26T06:28:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.461+0000 7f45a8627740 1 client.26603 shutdown 2024-02-26T06:28:16.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.461+0000 7f45a8627740 1 -- 172.21.15.49:0/4279949063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 msgr2=0x56302972daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.461+0000 7f45a8627740 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x56302972daf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f458c0747b0 tx=0x7f458c09a890 comp rx=0 tx=0).stop 2024-02-26T06:28:16.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.462+0000 7f45a8627740 1 -- 172.21.15.49:0/4279949063 shutdown_connections 2024-02-26T06:28:16.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.462+0000 7f45a8627740 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563029740290 0x5630297426b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.462+0000 7f45a8627740 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x56302972daf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.462+0000 7f45a8627740 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029730600 0x56302972e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.462+0000 7f45a8627740 1 --2- 172.21.15.49:0/4279949063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727290 0x56302972d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.462+0000 7f45a8627740 1 -- 172.21.15.49:0/4279949063 >> 172.21.15.49:0/4279949063 conn(0x5630296ee570 msgr2=0x563029723a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:16.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.462+0000 7f4588c00640 0 client.26603 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:16.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.462+0000 7f45a8627740 1 -- 172.21.15.49:0/4279949063 shutdown_connections 2024-02-26T06:28:16.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.463+0000 7f45a8627740 1 -- 172.21.15.49:0/4279949063 wait complete. 2024-02-26T06:28:16.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.463+0000 7f45a8627740 20 client.26603 trim_cache size 0 max 16384 2024-02-26T06:28:16.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.506+0000 7f45a8627740 1 Processor -- start 2024-02-26T06:28:16.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.508+0000 7f45a8627740 1 -- start start 2024-02-26T06:28:16.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.508+0000 7f45a8627740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x56302972fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.508+0000 7f45a8627740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727f70 0x56302972de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.508+0000 7f45a8627740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029730600 0x56302972c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.508+0000 7f45a8627740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563029726b30 con 0x563029727f70 2024-02-26T06:28:16.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.508+0000 7f45a8627740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302971c2f0 con 0x563029730600 2024-02-26T06:28:16.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.508+0000 7f45a8627740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302973c4d0 con 0x563029727290 2024-02-26T06:28:16.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.509+0000 7f459ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x56302972fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.509+0000 7f459ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x56302972fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43586/0 (socket says 172.21.15.49:43586) 2024-02-26T06:28:16.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.509+0000 7f459ea00640 1 -- 172.21.15.49:0/2785660603 learned_addr learned my addr 172.21.15.49:0/2785660603 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:16.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.509+0000 7f459e000640 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727f70 0x56302972de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.509+0000 7f459ea00640 1 -- 172.21.15.49:0/2785660603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029730600 msgr2=0x56302972c590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:16.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.509+0000 7f459ea00640 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029730600 0x56302972c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.510+0000 7f459ea00640 1 -- 172.21.15.49:0/2785660603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727f70 msgr2=0x56302972de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:16.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.510+0000 7f459ea00640 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727f70 0x56302972de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.510+0000 7f459ea00640 1 -- 172.21.15.49:0/2785660603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563029629920 con 0x563029727290 2024-02-26T06:28:16.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.510+0000 7f459e000640 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727f70 0x56302972de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:16.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.510+0000 7f459ea00640 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x56302972fce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f459000b6b0 tx=0x7f459000d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bba05515c7e5a802 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:16.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.511+0000 7f458b400640 1 -- 172.21.15.49:0/2785660603 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590018070 con 0x563029727290 2024-02-26T06:28:16.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.511+0000 7f458b400640 1 -- 172.21.15.49:0/2785660603 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4590025040 con 0x563029727290 2024-02-26T06:28:16.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.511+0000 7f458b400640 1 -- 172.21.15.49:0/2785660603 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459001f7e0 con 0x563029727290 2024-02-26T06:28:16.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.513+0000 7f45a8627740 1 -- 172.21.15.49:0/2785660603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 msgr2=0x56302972fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.513+0000 7f45a8627740 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x56302972fce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f459000b6b0 tx=0x7f459000d000 comp rx=0 tx=0).stop 2024-02-26T06:28:16.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.513+0000 7f45a8627740 1 -- 172.21.15.49:0/2785660603 shutdown_connections 2024-02-26T06:28:16.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.513+0000 7f45a8627740 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029730600 0x56302972c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.513+0000 7f45a8627740 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029727f70 0x56302972de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.513+0000 7f45a8627740 1 --2- 172.21.15.49:0/2785660603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x56302972fce0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.513+0000 7f45a8627740 1 -- 172.21.15.49:0/2785660603 >> 172.21.15.49:0/2785660603 conn(0x5630296ee570 msgr2=0x56302972be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:16.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.514+0000 7f45a8627740 1 -- 172.21.15.49:0/2785660603 shutdown_connections 2024-02-26T06:28:16.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.514+0000 7f45a8627740 1 -- 172.21.15.49:0/2785660603 wait complete. 2024-02-26T06:28:16.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.514+0000 7f45a8627740 1 -- start start 2024-02-26T06:28:16.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.515+0000 7f45a8627740 1 Processor -- start 2024-02-26T06:28:16.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.515+0000 7f45a8627740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x563029737810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.516+0000 7f45a8627740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x563029729f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.516+0000 7f45a8627740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x56302972a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f45a8627740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302972aa50 con 0x563029730600 2024-02-26T06:28:16.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f45a8627740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302972abc0 con 0x563029727f70 2024-02-26T06:28:16.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f45a8627740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302972ad80 con 0x563029727290 2024-02-26T06:28:16.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f459e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x563029729f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f45a8627740 10 client.0 _ll_register_callbacks cb 0x7f459000b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:16.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f459e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x563029729f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44736/0 (socket says 172.21.15.49:44736) 2024-02-26T06:28:16.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f459e000640 1 -- 172.21.15.49:0/3644620186 learned_addr learned my addr 172.21.15.49:0/3644620186 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:16.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f459f400640 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x56302972a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.517+0000 7f459ea00640 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x563029737810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.518+0000 7f459f400640 1 -- 172.21.15.49:0/3644620186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 msgr2=0x563029737810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:16.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.518+0000 7f459f400640 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x563029737810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.518+0000 7f459f400640 1 -- 172.21.15.49:0/3644620186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 msgr2=0x563029729f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:16.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.518+0000 7f459f400640 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x563029729f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.518+0000 7f459f400640 1 -- 172.21.15.49:0/3644620186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56302964f400 con 0x563029730600 2024-02-26T06:28:16.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.519+0000 7f459e000640 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x563029729f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:16.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.519+0000 7f459f400640 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x56302972a440 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4598009bf0 tx=0x7f45980347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f717d0689c8c8df server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:16.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.519+0000 7f4588c00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:28:16.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.519+0000 7f4588c00640 1 -- 172.21.15.49:0/3644620186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598003b70 con 0x563029730600 2024-02-26T06:28:16.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.520+0000 7f4588c00640 1 -- 172.21.15.49:0/3644620186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f4598003cd0 con 0x563029730600 2024-02-26T06:28:16.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.520+0000 7f45a8627740 1 -- 172.21.15.49:0/3644620186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f458c019310 con 0x563029730600 2024-02-26T06:28:16.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.520+0000 7f4588c00640 1 -- 172.21.15.49:0/3644620186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4598003e90 con 0x563029730600 2024-02-26T06:28:16.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.520+0000 7f45a8627740 10 client.16212 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:16.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.520+0000 7f45a8627740 1 -- 172.21.15.49:0/3644620186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563029635c90 con 0x563029730600 2024-02-26T06:28:16.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.521+0000 7f4588c00640 1 -- 172.21.15.49:0/3644620186 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f458c019310 con 0x563029730600 2024-02-26T06:28:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.522+0000 7f4588c00640 1 -- 172.21.15.49:0/3644620186 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f459802cd20 con 0x563029730600 2024-02-26T06:28:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.522+0000 7f45a8627740 10 client.16212 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.522+0000 7f45a8627740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.522+0000 7f45a8627740 20 client.16212 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:16.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.522+0000 7f45a8627740 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563029797d80 0x56302979a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.522+0000 7f45a8627740 4 client.16212 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:28:16.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.522+0000 7f45a8627740 1 -- 172.21.15.49:0/3644620186 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56302941a890 con 0x563029797d80 2024-02-26T06:28:16.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.523+0000 7f459ea00640 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563029797d80 0x56302979a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.524+0000 7f459ea00640 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563029797d80 0x56302979a1a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67711772c5b210f2 server_cookie=231c74f4b613937d in_seq=0 out_seq=0 2024-02-26T06:28:16.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.524+0000 7f4588c00640 10 client.16212 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:16.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.525+0000 7f4588c00640 1 -- 172.21.15.49:0/3644620186 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56302941a890 con 0x563029797d80 2024-02-26T06:28:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.525+0000 7f4588c00640 10 client.16212 handle_command_reply: tid=0 2024-02-26T06:28:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.525+0000 7f45a8627740 1 client.16212 shutdown 2024-02-26T06:28:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.525+0000 7f45a8627740 1 -- 172.21.15.49:0/3644620186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 msgr2=0x56302972a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x56302972a440 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4598009bf0 tx=0x7f45980347b0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 -- 172.21.15.49:0/3644620186 shutdown_connections 2024-02-26T06:28:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563029797d80 0x56302979a1a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563029727f70 0x563029729f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563029730600 0x56302972a440 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 --2- 172.21.15.49:0/3644620186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563029727290 0x563029737810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 -- 172.21.15.49:0/3644620186 >> 172.21.15.49:0/3644620186 conn(0x5630296ee570 msgr2=0x563029720570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f4588c00640 0 client.16212 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 -- 172.21.15.49:0/3644620186 shutdown_connections 2024-02-26T06:28:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.526+0000 7f45a8627740 1 -- 172.21.15.49:0/3644620186 wait complete. 2024-02-26T06:28:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.527+0000 7f45a8627740 20 client.16212 trim_cache size 0 max 16384 2024-02-26T06:28:16.528 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:28:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:28:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:28:16.528 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:28:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.529+0000 7f45a5800640 1 -- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f45840a2310 msgr2=0x7f45840a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.529+0000 7f45a5800640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f45840a2310 0x7f45840a47d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4590005fd0 tx=0x7f4590004730 comp rx=0 tx=0).stop 2024-02-26T06:28:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.529+0000 7f45a5800640 1 -- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45a0108aa0 msgr2=0x7f45a01a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.529+0000 7f45a5800640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45a0108aa0 0x7f45a01a0550 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f458c00d950 tx=0x7f458c00de10 comp rx=0 tx=0).stop 2024-02-26T06:28:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.530+0000 7f45a5800640 1 -- 172.21.15.49:0/3957365739 shutdown_connections 2024-02-26T06:28:16.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.530+0000 7f45a5800640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f45840a2310 0x7f45840a47d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.530+0000 7f45a5800640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45a01142d0 0x7f45a01a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.530+0000 7f45a5800640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45a0108aa0 0x7f45a01a0550 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.530+0000 7f45a5800640 1 --2- 172.21.15.49:0/3957365739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a0108180 0x7f45a01a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.530+0000 7f45a5800640 1 -- 172.21.15.49:0/3957365739 >> 172.21.15.49:0/3957365739 conn(0x7f45a006c6c0 msgr2=0x7f45a006fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:16.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.530+0000 7f45a5800640 1 -- 172.21.15.49:0/3957365739 shutdown_connections 2024-02-26T06:28:16.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.530+0000 7f45a5800640 1 -- 172.21.15.49:0/3957365739 wait complete. 2024-02-26T06:28:16.555 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:28:16.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:28:16.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.864+0000 7f49b6200640 1 -- 172.21.15.49:0/2226306423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b01002f0 msgr2=0x7f49b01006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.864+0000 7f49b6200640 1 --2- 172.21.15.49:0/2226306423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b01002f0 0x7f49b01006d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009860 tx=0x7f49a002bb70 comp rx=0 tx=0).stop 2024-02-26T06:28:16.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.865+0000 7f49b6200640 1 -- 172.21.15.49:0/2226306423 shutdown_connections 2024-02-26T06:28:16.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.865+0000 7f49b6200640 1 --2- 172.21.15.49:0/2226306423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49b0100ca0 0x7f49b0104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.865+0000 7f49b6200640 1 --2- 172.21.15.49:0/2226306423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b01053c0 0x7f49b010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.865+0000 7f49b6200640 1 --2- 172.21.15.49:0/2226306423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b01002f0 0x7f49b01006d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.865+0000 7f49b6200640 1 -- 172.21.15.49:0/2226306423 >> 172.21.15.49:0/2226306423 conn(0x7f49b0077180 msgr2=0x7f49b0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.866+0000 7f49b6200640 1 -- 172.21.15.49:0/2226306423 shutdown_connections 2024-02-26T06:28:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.866+0000 7f49b6200640 1 -- 172.21.15.49:0/2226306423 wait complete. 2024-02-26T06:28:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.868+0000 7f49b6200640 1 Processor -- start 2024-02-26T06:28:16.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.868+0000 7f49b6200640 1 -- start start 2024-02-26T06:28:16.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49b6200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49b01002f0 0x7f49b0115650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49b6200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b01053c0 0x7f49b0115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49b6200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b0110760 0x7f49b0110c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49b6200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0111210 con 0x7f49b01053c0 2024-02-26T06:28:16.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49b6200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0111380 con 0x7f49b01002f0 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49b6200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0111540 con 0x7f49b0110760 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49b01002f0 0x7f49b0115650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49b01002f0 0x7f49b0115650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44756/0 (socket says 172.21.15.49:44756) 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49af400640 1 -- 172.21.15.49:0/1694302262 learned_addr learned my addr 172.21.15.49:0/1694302262 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49afe00640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b0110760 0x7f49b0110c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49af400640 1 -- 172.21.15.49:0/1694302262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b0110760 msgr2=0x7f49b0110c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49af400640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b0110760 0x7f49b0110c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49af400640 1 -- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b01053c0 msgr2=0x7f49b0115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.869+0000 7f49af400640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b01053c0 0x7f49b0115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.870+0000 7f49af400640 1 -- 172.21.15.49:0/1694302262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0009490 con 0x7f49b01002f0 2024-02-26T06:28:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.870+0000 7f49af400640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49b01002f0 0x7f49b0115650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49a0037040 tx=0x7f49a0034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:16.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.870+0000 7f49a7e00640 1 -- 172.21.15.49:0/1694302262 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a002c510 con 0x7f49b01002f0 2024-02-26T06:28:16.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.870+0000 7f49a7e00640 1 -- 172.21.15.49:0/1694302262 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a002cb50 con 0x7f49b01002f0 2024-02-26T06:28:16.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.870+0000 7f49a7e00640 1 -- 172.21.15.49:0/1694302262 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a00035f0 con 0x7f49b01002f0 2024-02-26T06:28:16.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.870+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b01b3220 con 0x7f49b01002f0 2024-02-26T06:28:16.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.871+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b01b35f0 con 0x7f49b01002f0 2024-02-26T06:28:16.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.874+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b01017c0 con 0x7f49b01002f0 2024-02-26T06:28:16.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.876+0000 7f49a7e00640 1 -- 172.21.15.49:0/1694302262 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f49a003b040 con 0x7f49b01002f0 2024-02-26T06:28:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.878+0000 7f49a7e00640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49940a2270 0x7f49940a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.878+0000 7f49a7e00640 1 -- 172.21.15.49:0/1694302262 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f49a00efbe0 con 0x7f49b01002f0 2024-02-26T06:28:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.879+0000 7f49aea00640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49940a2270 0x7f49940a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.884+0000 7f49aea00640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49940a2270 0x7f49940a4730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49b0111f20 tx=0x7f499c00b3d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:16.886+0000 7f49a7e00640 1 -- 172.21.15.49:0/1694302262 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49a00b8990 con 0x7f49b01002f0 2024-02-26T06:28:16.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:16 smithi049 ceph-mon[31696]: pgmap v149: 97 pgs: 97 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 MiB/s wr, 1.61k op/s 2024-02-26T06:28:17.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:16 smithi080 ceph-mon[36206]: pgmap v149: 97 pgs: 97 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 MiB/s wr, 1.61k op/s 2024-02-26T06:28:17.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:16 smithi115 ceph-mon[36426]: pgmap v149: 97 pgs: 97 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 MiB/s wr, 1.61k op/s 2024-02-26T06:28:17.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.266+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49b01085f0 con 0x7f49b01002f0 2024-02-26T06:28:17.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.267+0000 7f49a7e00640 1 -- 172.21.15.49:0/1694302262 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11334 (secure 0 0 0) 0x7f49a00dae60 con 0x7f49b01002f0 2024-02-26T06:28:17.270 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:28:17.270 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:27:29.193881+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:28:17.270 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 35 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49940a2270 msgr2=0x7f49940a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49940a2270 0x7f49940a4730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49b0111f20 tx=0x7f499c00b3d0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49b01002f0 msgr2=0x7f49b0115650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49b01002f0 0x7f49b0115650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49a0037040 tx=0x7f49a0034440 comp rx=0 tx=0).stop 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 shutdown_connections 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f49940a2270 0x7f49940a4730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b0110760 0x7f49b0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b01053c0 0x7f49b0115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 --2- 172.21.15.49:0/1694302262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49b01002f0 0x7f49b0115650 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.274+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 >> 172.21.15.49:0/1694302262 conn(0x7f49b0077180 msgr2=0x7f49b00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:17.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.275+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 shutdown_connections 2024-02-26T06:28:17.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.275+0000 7f49b6200640 1 -- 172.21.15.49:0/1694302262 wait complete. 2024-02-26T06:28:17.308 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:17.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:28:17.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.586+0000 7f87ca800640 1 -- 172.21.15.49:0/769835916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c4103f40 msgr2=0x7f87c4107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.586+0000 7f87ca800640 1 --2- 172.21.15.49:0/769835916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c4103f40 0x7f87c4107f80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009930 tx=0x7f87b002bc20 comp rx=0 tx=0).stop 2024-02-26T06:28:17.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.587+0000 7f87ca800640 1 -- 172.21.15.49:0/769835916 shutdown_connections 2024-02-26T06:28:17.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.587+0000 7f87ca800640 1 --2- 172.21.15.49:0/769835916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c4103f40 0x7f87c4107f80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.587+0000 7f87ca800640 1 --2- 172.21.15.49:0/769835916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87c4108660 0x7f87c4110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.587+0000 7f87ca800640 1 --2- 172.21.15.49:0/769835916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c4103590 0x7f87c4103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.587+0000 7f87ca800640 1 -- 172.21.15.49:0/769835916 >> 172.21.15.49:0/769835916 conn(0x7f87c40fedf0 msgr2=0x7f87c4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:17.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.587+0000 7f87ca800640 1 -- 172.21.15.49:0/769835916 shutdown_connections 2024-02-26T06:28:17.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.587+0000 7f87ca800640 1 -- 172.21.15.49:0/769835916 wait complete. 2024-02-26T06:28:17.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.588+0000 7f87ca800640 1 Processor -- start 2024-02-26T06:28:17.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.588+0000 7f87ca800640 1 -- start start 2024-02-26T06:28:17.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.589+0000 7f87ca800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87c4103590 0x7f87c419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.589+0000 7f87ca800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c4108660 0x7f87c419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.589+0000 7f87ca800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c41997f0 0x7f87c4199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.590+0000 7f87ca800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c419a2a0 con 0x7f87c4108660 2024-02-26T06:28:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.590+0000 7f87ca800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c419a410 con 0x7f87c41997f0 2024-02-26T06:28:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.590+0000 7f87c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87c4103590 0x7f87c419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.590+0000 7f87c2a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c4108660 0x7f87c419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.590+0000 7f87c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87c4103590 0x7f87c419f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43624/0 (socket says 172.21.15.49:43624) 2024-02-26T06:28:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.590+0000 7f87c3400640 1 -- 172.21.15.49:0/1745596274 learned_addr learned my addr 172.21.15.49:0/1745596274 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.590+0000 7f87ca800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c419a580 con 0x7f87c4103590 2024-02-26T06:28:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.590+0000 7f87c3e00640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c41997f0 0x7f87c4199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87c3e00640 1 -- 172.21.15.49:0/1745596274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87c4103590 msgr2=0x7f87c419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87c3e00640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87c4103590 0x7f87c419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87c3e00640 1 -- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c4108660 msgr2=0x7f87c419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87c3e00640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c4108660 0x7f87c419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87c3e00640 1 -- 172.21.15.49:0/1745596274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009560 con 0x7f87c41997f0 2024-02-26T06:28:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87c3400640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87c4103590 0x7f87c419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87c2a00640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c4108660 0x7f87c419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87c3e00640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c41997f0 0x7f87c4199ca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00cab0 tx=0x7f87bc00cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.591+0000 7f87bbe00640 1 -- 172.21.15.49:0/1745596274 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00ae20 con 0x7f87c41997f0 2024-02-26T06:28:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.592+0000 7f87ca800640 1 -- 172.21.15.49:0/1745596274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c41a6530 con 0x7f87c41997f0 2024-02-26T06:28:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.592+0000 7f87bbe00640 1 -- 172.21.15.49:0/1745596274 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87bc00f930 con 0x7f87c41997f0 2024-02-26T06:28:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.592+0000 7f87bbe00640 1 -- 172.21.15.49:0/1745596274 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc018a50 con 0x7f87c41997f0 2024-02-26T06:28:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.592+0000 7f87ca800640 1 -- 172.21.15.49:0/1745596274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c41a69c0 con 0x7f87c41997f0 2024-02-26T06:28:17.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.597+0000 7f87bbe00640 1 -- 172.21.15.49:0/1745596274 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f87bc00faa0 con 0x7f87c41997f0 2024-02-26T06:28:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.598+0000 7f87bbe00640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f87a80a22b0 0x7f87a80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.598+0000 7f87cd5d9740 1 Processor -- start 2024-02-26T06:28:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.598+0000 7f87cd5d9740 1 -- start start 2024-02-26T06:28:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.598+0000 7f87bbe00640 1 -- 172.21.15.49:0/1745596274 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f87bc0ce770 con 0x7f87c41997f0 2024-02-26T06:28:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.599+0000 7f87c3400640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f87a80a22b0 0x7f87a80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b39ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b42bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b45500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172aa81e0 con 0x558172b3a530 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172aa5710 con 0x558172b43110 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172aa0ef0 con 0x558172b39c10 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b39ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b39ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43634/0 (socket says 172.21.15.49:43634) 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87c3400640 1 -- 172.21.15.49:0/3035067166 learned_addr learned my addr 172.21.15.49:0/3035067166 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.600+0000 7f87c3e00640 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b45500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c3e00640 1 -- 172.21.15.49:0/3035067166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 msgr2=0x558172b39ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c3e00640 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b39ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c2a00640 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b42bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c3e00640 1 -- 172.21.15.49:0/3035067166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 msgr2=0x558172b42bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:17.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c3e00640 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b42bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c3e00640 1 -- 172.21.15.49:0/3035067166 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558172a62dc0 con 0x558172b43110 2024-02-26T06:28:17.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c3400640 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b39ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:17.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c3400640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f87a80a22b0 0x7f87a80a4770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f87b4011a30 tx=0x7f87b4011e30 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:17.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c2a00640 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b42bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:17.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87c3e00640 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b45500 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0cbed0 tx=0x7f87bc097540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f143f25282a80cf6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:17.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87b8c00640 1 -- 172.21.15.49:0/3035067166 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc09f2c0 con 0x558172b43110 2024-02-26T06:28:17.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87b8c00640 1 -- 172.21.15.49:0/3035067166 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87bc0a3910 con 0x558172b43110 2024-02-26T06:28:17.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.601+0000 7f87b8c00640 1 -- 172.21.15.49:0/3035067166 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0a3480 con 0x558172b43110 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.602+0000 7f87cd5d9740 1 -- 172.21.15.49:0/3035067166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 msgr2=0x558172b45500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.602+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b45500 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0cbed0 tx=0x7f87bc097540 comp rx=0 tx=0).stop 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.603+0000 7f87cd5d9740 1 -- 172.21.15.49:0/3035067166 shutdown_connections 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.603+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b45500 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.603+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b42bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.603+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/3035067166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b39ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.603+0000 7f87cd5d9740 1 -- 172.21.15.49:0/3035067166 >> 172.21.15.49:0/3035067166 conn(0x558172b34340 msgr2=0x558172b34720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.603+0000 7f87cd5d9740 1 -- 172.21.15.49:0/3035067166 shutdown_connections 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.603+0000 7f87cd5d9740 1 -- 172.21.15.49:0/3035067166 wait complete. 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.603+0000 7f87cd5d9740 1 -- start start 2024-02-26T06:28:17.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.604+0000 7f87cd5d9740 1 Processor -- start 2024-02-26T06:28:17.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b40150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b3a530 0x558172b40690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b40150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b43110 0x558172b40bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172a46120 con 0x558172b43110 2024-02-26T06:28:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172a45dc0 con 0x558172b3a530 2024-02-26T06:28:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817280f1c0 con 0x558172b39c10 2024-02-26T06:28:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b40150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43636/0 (socket says 172.21.15.49:43636) 2024-02-26T06:28:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87c3400640 1 -- 172.21.15.49:0/580941331 learned_addr learned my addr 172.21.15.49:0/580941331 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.606+0000 7f87cd5d9740 10 client.0 _ll_register_callbacks cb 0x558172a95280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.607+0000 7f87c2a00640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b3a530 0x558172b40690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.607+0000 7f87c3400640 1 -- 172.21.15.49:0/580941331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b3a530 msgr2=0x558172b40690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.607+0000 7f87c3e00640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b43110 0x558172b40bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.607+0000 7f87c3400640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b3a530 0x558172b40690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.607+0000 7f87c3400640 1 -- 172.21.15.49:0/580941331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b43110 msgr2=0x558172b40bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.608+0000 7f87c3400640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b43110 0x558172b40bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.608+0000 7f87c3400640 1 -- 172.21.15.49:0/580941331 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c41a69c0 con 0x558172b39c10 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.608+0000 7f87c2a00640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b3a530 0x558172b40690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.608+0000 7f87c3e00640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b43110 0x558172b40bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.608+0000 7f87c3400640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b40150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87b4020970 tx=0x7f87b4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36340e924e81e8bf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.608+0000 7f87ad600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.608+0000 7f87ad600640 1 -- 172.21.15.49:0/580941331 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4020e30 con 0x558172b39c10 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.609+0000 7f87ad600640 1 -- 172.21.15.49:0/580941331 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f87b401f8b0 con 0x558172b39c10 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.609+0000 7f87cd5d9740 1 -- 172.21.15.49:0/580941331 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581727a07e0 con 0x558172b39c10 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.609+0000 7f87ad600640 1 -- 172.21.15.49:0/580941331 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87b401fa70 con 0x558172b39c10 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87cd5d9740 10 client.26638 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87cd5d9740 1 -- 172.21.15.49:0/580941331 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558172a9f570 con 0x558172b39c10 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87ad600640 1 -- 172.21.15.49:0/580941331 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5581727a07e0 con 0x558172b39c10 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87ad600640 1 -- 172.21.15.49:0/580941331 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87b4026070 con 0x558172b39c10 2024-02-26T06:28:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87cd5d9740 10 client.26638 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87cd5d9740 10 client.26638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:28:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87cd5d9740 20 client.26638 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x558172b53060 0x558172b55480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87cd5d9740 4 client.26638 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.610+0000 7f87cd5d9740 1 -- 172.21.15.49:0/580941331 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581727dd720 con 0x558172b53060 2024-02-26T06:28:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.611+0000 7f87c2a00640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x558172b53060 0x558172b55480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.611+0000 7f87ad600640 10 client.26638 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:28:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.611+0000 7f87c2a00640 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x558172b53060 0x558172b55480 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c5e78dd8b29addf server_cookie=3aa28fe2a29f8010 in_seq=0 out_seq=0 2024-02-26T06:28:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87ad600640 1 -- 172.21.15.49:0/580941331 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581727dd720 con 0x558172b53060 2024-02-26T06:28:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87ad600640 10 client.26638 handle_command_reply: tid=0 2024-02-26T06:28:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 client.26638 shutdown 2024-02-26T06:28:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 -- 172.21.15.49:0/580941331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 msgr2=0x558172b40150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b40150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87b4020970 tx=0x7f87b4004730 comp rx=0 tx=0).stop 2024-02-26T06:28:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 -- 172.21.15.49:0/580941331 shutdown_connections 2024-02-26T06:28:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x558172b53060 0x558172b55480 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b3a530 0x558172b40690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b43110 0x558172b40bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87ad600640 0 client.26638 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:28:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/580941331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b40150 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.612+0000 7f87cd5d9740 1 -- 172.21.15.49:0/580941331 >> 172.21.15.49:0/580941331 conn(0x558172b34340 msgr2=0x558172b36090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.614+0000 7f87cd5d9740 1 -- 172.21.15.49:0/580941331 shutdown_connections 2024-02-26T06:28:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.614+0000 7f87cd5d9740 1 -- 172.21.15.49:0/580941331 wait complete. 2024-02-26T06:28:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.614+0000 7f87cd5d9740 20 client.26638 trim_cache size 0 max 16384 2024-02-26T06:28:17.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.639+0000 7f87cd5d9740 1 Processor -- start 2024-02-26T06:28:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.640+0000 7f87cd5d9740 1 -- start start 2024-02-26T06:28:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.640+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b42880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.640+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.640+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b3d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.640+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172b4efa0 con 0x558172b3a530 2024-02-26T06:28:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.640+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172b35370 con 0x558172b43110 2024-02-26T06:28:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.640+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172b33e70 con 0x558172b39c10 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.640+0000 7f87c3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b3d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c2a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b3d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44812/0 (socket says 172.21.15.49:44812) 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b42880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3e00640 1 -- 172.21.15.49:0/2261374395 learned_addr learned my addr 172.21.15.49:0/2261374395 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3400640 1 -- 172.21.15.49:0/2261374395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 msgr2=0x558172b3d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3400640 1 --2- 172.21.15.49:0/2261374395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b3d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3400640 1 -- 172.21.15.49:0/2261374395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 msgr2=0x558172b409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3400640 1 --2- 172.21.15.49:0/2261374395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3400640 1 -- 172.21.15.49:0/2261374395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558172a3ba80 con 0x558172b39c10 2024-02-26T06:28:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87c3400640 1 --2- 172.21.15.49:0/2261374395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b42880 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87b4004960 tx=0x7f87b40091e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7268487eef3faf4f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:17.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.641+0000 7f87afe00640 1 -- 172.21.15.49:0/2261374395 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4016070 con 0x558172b39c10 2024-02-26T06:28:17.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87afe00640 1 -- 172.21.15.49:0/2261374395 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87b4011440 con 0x558172b39c10 2024-02-26T06:28:17.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87afe00640 1 -- 172.21.15.49:0/2261374395 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4031ae0 con 0x558172b39c10 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87cd5d9740 1 -- 172.21.15.49:0/2261374395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 msgr2=0x558172b42880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/2261374395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b42880 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87b4004960 tx=0x7f87b40091e0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87cd5d9740 1 -- 172.21.15.49:0/2261374395 shutdown_connections 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/2261374395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172b3d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/2261374395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/2261374395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b42880 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.642+0000 7f87cd5d9740 1 -- 172.21.15.49:0/2261374395 >> 172.21.15.49:0/2261374395 conn(0x558172b34340 msgr2=0x558172b3b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.643+0000 7f87cd5d9740 1 -- 172.21.15.49:0/2261374395 shutdown_connections 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.643+0000 7f87cd5d9740 1 -- 172.21.15.49:0/2261374395 wait complete. 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.643+0000 7f87cd5d9740 1 -- start start 2024-02-26T06:28:17.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.643+0000 7f87cd5d9740 1 Processor -- start 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b48de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b49320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87cd5d9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172bb1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172b49bf0 con 0x558172b3a530 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172bb22e0 con 0x558172b43110 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87cd5d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172bb2420 con 0x558172b39c10 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87cd5d9740 10 client.0 _ll_register_callbacks cb 0x7f87bc018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172bb1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b48de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c2a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b49320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172bb1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44816/0 (socket says 172.21.15.49:44816) 2024-02-26T06:28:17.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3e00640 1 -- 172.21.15.49:0/1774600381 learned_addr learned my addr 172.21.15.49:0/1774600381 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3400640 1 -- 172.21.15.49:0/1774600381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 msgr2=0x558172bb1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3400640 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172bb1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3400640 1 -- 172.21.15.49:0/1774600381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 msgr2=0x558172b49320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3400640 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b49320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.644+0000 7f87c3400640 1 -- 172.21.15.49:0/1774600381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558172a9f570 con 0x558172b39c10 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87c3400640 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b48de0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87b40118f0 tx=0x7f87b401fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54cdc711d094901 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87ad600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87ad600640 1 -- 172.21.15.49:0/1774600381 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4016040 con 0x558172b39c10 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87ad600640 1 -- 172.21.15.49:0/1774600381 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f87b40319e0 con 0x558172b39c10 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87cd5d9740 1 -- 172.21.15.49:0/1774600381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87bc0a3910 con 0x558172b39c10 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87ad600640 1 -- 172.21.15.49:0/1774600381 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87b401fed0 con 0x558172b39c10 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87cd5d9740 10 client.26644 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87cd5d9740 1 -- 172.21.15.49:0/1774600381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558172a61620 con 0x558172b39c10 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.645+0000 7f87ad600640 1 -- 172.21.15.49:0/1774600381 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87bc0a3910 con 0x558172b39c10 2024-02-26T06:28:17.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87ad600640 1 -- 172.21.15.49:0/1774600381 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87b4026050 con 0x558172b39c10 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87cd5d9740 10 client.26644 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87cd5d9740 10 client.26644 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87cd5d9740 20 client.26644 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x558172bb3700 0x558172bb5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87cd5d9740 4 client.26644 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87cd5d9740 1 -- 172.21.15.49:0/1774600381 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55817282c890 con 0x558172bb3700 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87c2a00640 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x558172bb3700 0x558172bb5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87c2a00640 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x558172bb3700 0x558172bb5af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4d021a305638ce6 server_cookie=74c337c3a5f8e906 in_seq=0 out_seq=0 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.646+0000 7f87ad600640 10 client.26644 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87ad600640 1 -- 172.21.15.49:0/1774600381 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55817282c890 con 0x558172bb3700 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87ad600640 10 client.26644 handle_command_reply: tid=0 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 client.26644 shutdown 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 -- 172.21.15.49:0/1774600381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 msgr2=0x558172b48de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b48de0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87b40118f0 tx=0x7f87b401fc40 comp rx=0 tx=0).stop 2024-02-26T06:28:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 -- 172.21.15.49:0/1774600381 shutdown_connections 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x558172bb3700 0x558172bb5af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558172b43110 0x558172bb1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87ad600640 0 client.26644 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558172b3a530 0x558172b49320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 --2- 172.21.15.49:0/1774600381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558172b39c10 0x558172b48de0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 -- 172.21.15.49:0/1774600381 >> 172.21.15.49:0/1774600381 conn(0x558172b34340 msgr2=0x558172b35770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 -- 172.21.15.49:0/1774600381 shutdown_connections 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 1 -- 172.21.15.49:0/1774600381 wait complete. 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.647+0000 7f87cd5d9740 20 client.26644 trim_cache size 0 max 16384 2024-02-26T06:28:17.652 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:28:17.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 -- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f87a80a22b0 msgr2=0x7f87a80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f87a80a22b0 0x7f87a80a4770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f87b4011a30 tx=0x7f87b4011e30 comp rx=0 tx=0).stop 2024-02-26T06:28:17.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 -- 172.21.15.49:0/1745596274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c41997f0 msgr2=0x7f87c4199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c41997f0 0x7f87c4199ca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00cab0 tx=0x7f87bc00cf70 comp rx=0 tx=0).stop 2024-02-26T06:28:17.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 -- 172.21.15.49:0/1745596274 shutdown_connections 2024-02-26T06:28:17.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f87a80a22b0 0x7f87a80a4770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87c41997f0 0x7f87c4199ca0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c4108660 0x7f87c419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 --2- 172.21.15.49:0/1745596274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87c4103590 0x7f87c419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 -- 172.21.15.49:0/1745596274 >> 172.21.15.49:0/1745596274 conn(0x7f87c40fedf0 msgr2=0x7f87c4101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:17.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 -- 172.21.15.49:0/1745596274 shutdown_connections 2024-02-26T06:28:17.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.648+0000 7f87ca800640 1 -- 172.21.15.49:0/1745596274 wait complete. 2024-02-26T06:28:17.665 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:17.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:28:17.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:17 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1694302262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:17.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.981+0000 7f374bc00640 1 -- 172.21.15.49:0/3739653514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374411a960 msgr2=0x7f374411cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.981+0000 7f374bc00640 1 --2- 172.21.15.49:0/3739653514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374411a960 0x7f374411cd50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f373c009ca0 tx=0x7f373c02c100 comp rx=0 tx=0).stop 2024-02-26T06:28:17.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.982+0000 7f374bc00640 1 -- 172.21.15.49:0/3739653514 shutdown_connections 2024-02-26T06:28:17.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.982+0000 7f374bc00640 1 --2- 172.21.15.49:0/3739653514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374411a960 0x7f374411cd50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.982+0000 7f374bc00640 1 --2- 172.21.15.49:0/3739653514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374410d9d0 0x7f374410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.982+0000 7f374bc00640 1 --2- 172.21.15.49:0/3739653514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374410f9a0 0x7f374410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.982+0000 7f374bc00640 1 -- 172.21.15.49:0/3739653514 >> 172.21.15.49:0/3739653514 conn(0x7f374406c260 msgr2=0x7f374406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:17.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.983+0000 7f374bc00640 1 -- 172.21.15.49:0/3739653514 shutdown_connections 2024-02-26T06:28:17.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.984+0000 7f374bc00640 1 -- 172.21.15.49:0/3739653514 wait complete. 2024-02-26T06:28:17.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.985+0000 7f374bc00640 1 Processor -- start 2024-02-26T06:28:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.986+0000 7f374bc00640 1 -- start start 2024-02-26T06:28:17.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.987+0000 7f374bc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374410d9d0 0x7f3744117db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.987+0000 7f374bc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374410f9a0 0x7f37441182f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.987+0000 7f374bc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374411a960 0x7f3744110f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.988+0000 7f3749200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374410d9d0 0x7f3744117db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.988+0000 7f3749200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374410d9d0 0x7f3744117db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43666/0 (socket says 172.21.15.49:43666) 2024-02-26T06:28:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.988+0000 7f3743e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374410f9a0 0x7f37441182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.988+0000 7f3743e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374410f9a0 0x7f37441182f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44844/0 (socket says 172.21.15.49:44844) 2024-02-26T06:28:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.989+0000 7f374bc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37441116e0 con 0x7f374411a960 2024-02-26T06:28:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.989+0000 7f374bc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3744111850 con 0x7f374410f9a0 2024-02-26T06:28:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.989+0000 7f374bc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3744111a10 con 0x7f374410d9d0 2024-02-26T06:28:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.989+0000 7f3749200640 1 -- 172.21.15.49:0/2216417143 learned_addr learned my addr 172.21.15.49:0/2216417143 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:17.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.991+0000 7f3749200640 1 -- 172.21.15.49:0/2216417143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374410f9a0 msgr2=0x7f37441182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.991+0000 7f3749200640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374410f9a0 0x7f37441182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.991+0000 7f3749200640 1 -- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374411a960 msgr2=0x7f3744110f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:17.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.991+0000 7f3749200640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374411a960 0x7f3744110f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:17.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.991+0000 7f3749200640 1 -- 172.21.15.49:0/2216417143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c009900 con 0x7f374410d9d0 2024-02-26T06:28:17.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.992+0000 7f3749200640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374410d9d0 0x7f3744117db0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f373800c890 tx=0x7f373800cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:17.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.993+0000 7f3741600640 1 -- 172.21.15.49:0/2216417143 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738004220 con 0x7f374410d9d0 2024-02-26T06:28:17.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.994+0000 7f3741600640 1 -- 172.21.15.49:0/2216417143 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3738007e90 con 0x7f374410d9d0 2024-02-26T06:28:17.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.994+0000 7f3741600640 1 -- 172.21.15.49:0/2216417143 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738004a00 con 0x7f374410d9d0 2024-02-26T06:28:17.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.994+0000 7f374bc00640 1 -- 172.21.15.49:0/2216417143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3744111c90 con 0x7f374410d9d0 2024-02-26T06:28:17.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:17.994+0000 7f374bc00640 1 -- 172.21.15.49:0/2216417143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3744115c50 con 0x7f374410d9d0 2024-02-26T06:28:18.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.009+0000 7f3741600640 1 -- 172.21.15.49:0/2216417143 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f37380079b0 con 0x7f374410d9d0 2024-02-26T06:28:18.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.011+0000 7f374ea59740 1 Processor -- start 2024-02-26T06:28:18.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.015+0000 7f374ea59740 1 -- start start 2024-02-26T06:28:18.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.016+0000 7f3741600640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f372c0a2380 0x7f372c0a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.017+0000 7f3741600640 1 -- 172.21.15.49:0/2216417143 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f37380cd280 con 0x7f374410d9d0 2024-02-26T06:28:18.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.017+0000 7f3749c00640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f372c0a2380 0x7f372c0a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.019+0000 7f3749c00640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f372c0a2380 0x7f372c0a4840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f373c009430 tx=0x7f373c005c50 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.019+0000 7f374ea59740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c28080ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.019+0000 7f374ea59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28089bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.019+0000 7f374ea59740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c2808c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.019+0000 7f374ea59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c27fef1e0 con 0x562c28081530 2024-02-26T06:28:18.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.019+0000 7f374ea59740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c27fec710 con 0x562c28080c10 2024-02-26T06:28:18.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.019+0000 7f374ea59740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c27fe7ef0 con 0x562c2808a110 2024-02-26T06:28:18.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.020+0000 7f3749200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c28080ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.020+0000 7f3749200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c28080ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44848/0 (socket says 172.21.15.49:44848) 2024-02-26T06:28:18.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.020+0000 7f3749200640 1 -- 172.21.15.49:0/116989548 learned_addr learned my addr 172.21.15.49:0/116989548 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.020+0000 7f3749c00640 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c2808c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3743e00640 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28089bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3749c00640 1 -- 172.21.15.49:0/116989548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 msgr2=0x562c28080ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3749c00640 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c28080ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3749c00640 1 -- 172.21.15.49:0/116989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 msgr2=0x562c28089bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3749c00640 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28089bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3749c00640 1 -- 172.21.15.49:0/116989548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c27fa9dc0 con 0x562c2808a110 2024-02-26T06:28:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3749200640 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c28080ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3743e00640 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28089bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.021+0000 7f3749c00640 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c2808c500 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f373c037040 tx=0x7f373c03c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e59b7e36f0a5c1c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.022+0000 7f3736000640 1 -- 172.21.15.49:0/116989548 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c032a90 con 0x562c2808a110 2024-02-26T06:28:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.022+0000 7f3736000640 1 -- 172.21.15.49:0/116989548 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f373c043070 con 0x562c2808a110 2024-02-26T06:28:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.023+0000 7f374ea59740 1 -- 172.21.15.49:0/116989548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 msgr2=0x562c2808c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.023+0000 7f374ea59740 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c2808c500 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f373c037040 tx=0x7f373c03c000 comp rx=0 tx=0).stop 2024-02-26T06:28:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.023+0000 7f374ea59740 1 -- 172.21.15.49:0/116989548 shutdown_connections 2024-02-26T06:28:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.023+0000 7f374ea59740 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c2808c500 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.023+0000 7f374ea59740 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28089bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.023+0000 7f374ea59740 1 --2- 172.21.15.49:0/116989548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c28080ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.023+0000 7f374ea59740 1 -- 172.21.15.49:0/116989548 >> 172.21.15.49:0/116989548 conn(0x562c2807b340 msgr2=0x562c2807b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.024+0000 7f374ea59740 1 -- 172.21.15.49:0/116989548 shutdown_connections 2024-02-26T06:28:18.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.024+0000 7f374ea59740 1 -- 172.21.15.49:0/116989548 wait complete. 2024-02-26T06:28:18.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.024+0000 7f374ea59740 1 -- start start 2024-02-26T06:28:18.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.025+0000 7f374ea59740 1 Processor -- start 2024-02-26T06:28:18.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.026+0000 7f374ea59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28080c10 0x562c28087200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.026+0000 7f374ea59740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28087740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f374ea59740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c2808a110 0x562c28087c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f3743e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28087740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f3743e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28087740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43684/0 (socket says 172.21.15.49:43684) 2024-02-26T06:28:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f3743e00640 1 -- 172.21.15.49:0/1793829700 learned_addr learned my addr 172.21.15.49:0/1793829700 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c27f8d120 con 0x562c28080c10 2024-02-26T06:28:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c27f8cdc0 con 0x562c2808a110 2024-02-26T06:28:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c27d561c0 con 0x562c28081530 2024-02-26T06:28:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f3749200640 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28080c10 0x562c28087200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f3749c00640 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c2808a110 0x562c28087c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f374ea59740 10 client.0 _ll_register_callbacks cb 0x562c27fdc280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.027+0000 7f3749c00640 1 -- 172.21.15.49:0/1793829700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 msgr2=0x562c28087740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.028+0000 7f3749c00640 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28087740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.028+0000 7f3749c00640 1 -- 172.21.15.49:0/1793829700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28080c10 msgr2=0x562c28087200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.028+0000 7f3749c00640 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28080c10 0x562c28087200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.028+0000 7f3749c00640 1 -- 172.21.15.49:0/1793829700 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374411d680 con 0x562c2808a110 2024-02-26T06:28:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.028+0000 7f3749200640 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28080c10 0x562c28087200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.028+0000 7f3749c00640 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c2808a110 0x562c28087c80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f373c02c710 tx=0x7f373c004720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afbfec1003492e17 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.029+0000 7f372aa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:18.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.029+0000 7f372aa00640 1 -- 172.21.15.49:0/1793829700 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c034850 con 0x562c2808a110 2024-02-26T06:28:18.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.029+0000 7f372aa00640 1 -- 172.21.15.49:0/1793829700 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f373c003970 con 0x562c2808a110 2024-02-26T06:28:18.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.029+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c27d24720 con 0x562c2808a110 2024-02-26T06:28:18.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.030+0000 7f372aa00640 1 -- 172.21.15.49:0/1793829700 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f373c004890 con 0x562c2808a110 2024-02-26T06:28:18.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.030+0000 7f374ea59740 10 client.26639 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:18.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.030+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x562c27fe6570 con 0x562c2808a110 2024-02-26T06:28:18.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.030+0000 7f372aa00640 1 -- 172.21.15.49:0/1793829700 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562c27d24720 con 0x562c2808a110 2024-02-26T06:28:18.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.031+0000 7f372aa00640 1 -- 172.21.15.49:0/1793829700 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f373c03f440 con 0x562c2808a110 2024-02-26T06:28:18.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.031+0000 7f374ea59740 10 client.26639 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:18.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.031+0000 7f374ea59740 10 client.26639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:28:18.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.031+0000 7f374ea59740 20 client.26639 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:18.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.031+0000 7f374ea59740 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562c2809a010 0x562c2809c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.031+0000 7f374ea59740 4 client.26639 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:18.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.031+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c27d73890 con 0x562c2809a010 2024-02-26T06:28:18.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.031+0000 7f3749200640 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562c2809a010 0x562c2809c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.032+0000 7f372aa00640 10 client.26639 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:28:18.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.033+0000 7f3749200640 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562c2809a010 0x562c2809c430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20265e226d43401a server_cookie=2068818e5881c196 in_seq=0 out_seq=0 2024-02-26T06:28:18.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.033+0000 7f372aa00640 1 -- 172.21.15.49:0/1793829700 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c27d73890 con 0x562c2809a010 2024-02-26T06:28:18.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.034+0000 7f372aa00640 10 client.26639 handle_command_reply: tid=0 2024-02-26T06:28:18.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.034+0000 7f374ea59740 1 client.26639 shutdown 2024-02-26T06:28:18.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c2808a110 msgr2=0x562c28087c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c2808a110 0x562c28087c80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f373c02c710 tx=0x7f373c004720 comp rx=0 tx=0).stop 2024-02-26T06:28:18.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 shutdown_connections 2024-02-26T06:28:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562c2809a010 0x562c2809c430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c2808a110 0x562c28087c80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28087740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 --2- 172.21.15.49:0/1793829700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28080c10 0x562c28087200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 >> 172.21.15.49:0/1793829700 conn(0x562c2807b340 msgr2=0x562c2807d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f372aa00640 0 client.26639 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:28:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 shutdown_connections 2024-02-26T06:28:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 1 -- 172.21.15.49:0/1793829700 wait complete. 2024-02-26T06:28:18.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.035+0000 7f374ea59740 20 client.26639 trim_cache size 0 max 16384 2024-02-26T06:28:18.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.060+0000 7f374ea59740 1 Processor -- start 2024-02-26T06:28:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.060+0000 7f374ea59740 1 -- start start 2024-02-26T06:28:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.060+0000 7f374ea59740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.060+0000 7f374ea59740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28082a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.060+0000 7f374ea59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c2808a110 0x562c28085af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.060+0000 7f374ea59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2807c140 con 0x562c2808a110 2024-02-26T06:28:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.060+0000 7f374ea59740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c28086ff0 con 0x562c28080c10 2024-02-26T06:28:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.060+0000 7f374ea59740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2807c370 con 0x562c28081530 2024-02-26T06:28:18.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28082a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28082a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43688/0 (socket says 172.21.15.49:43688) 2024-02-26T06:28:18.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 -- 172.21.15.49:0/1880050152 learned_addr learned my addr 172.21.15.49:0/1880050152 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3749200640 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3749c00640 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c2808a110 0x562c28085af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 -- 172.21.15.49:0/1880050152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 msgr2=0x562c280824f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280824f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 -- 172.21.15.49:0/1880050152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c2808a110 msgr2=0x562c28085af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c2808a110 0x562c28085af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 -- 172.21.15.49:0/1880050152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c27f82a80 con 0x562c28081530 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.061+0000 7f3743e00640 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28082a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f373000ac50 tx=0x7f3730012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482e785625041b7f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.062+0000 7f3735600640 1 -- 172.21.15.49:0/1880050152 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730017070 con 0x562c28081530 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.062+0000 7f3735600640 1 -- 172.21.15.49:0/1880050152 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f373000f900 con 0x562c28081530 2024-02-26T06:28:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.062+0000 7f3735600640 1 -- 172.21.15.49:0/1880050152 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373001ba00 con 0x562c28081530 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 -- 172.21.15.49:0/1880050152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 msgr2=0x562c28082a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28082a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f373000ac50 tx=0x7f3730012820 comp rx=0 tx=0).stop 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 -- 172.21.15.49:0/1880050152 shutdown_connections 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c2808a110 0x562c28085af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c28081530 0x562c28082a30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 --2- 172.21.15.49:0/1880050152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280824f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 -- 172.21.15.49:0/1880050152 >> 172.21.15.49:0/1880050152 conn(0x562c2807b340 msgr2=0x562c280a0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 -- 172.21.15.49:0/1880050152 shutdown_connections 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 -- 172.21.15.49:0/1880050152 wait complete. 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.063+0000 7f374ea59740 1 -- start start 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f374ea59740 1 Processor -- start 2024-02-26T06:28:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f374ea59740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f374ea59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28090df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f374ea59740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c280ef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f374ea59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c28091510 con 0x562c28081530 2024-02-26T06:28:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f374ea59740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c280efa70 con 0x562c28080c10 2024-02-26T06:28:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f374ea59740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c280efbe0 con 0x562c2808a110 2024-02-26T06:28:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f374ea59740 10 client.0 _ll_register_callbacks cb 0x7f373c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:18.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.064+0000 7f3749c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c280ef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3749c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c280ef530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43700/0 (socket says 172.21.15.49:43700) 2024-02-26T06:28:18.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3749c00640 1 -- 172.21.15.49:0/716337299 learned_addr learned my addr 172.21.15.49:0/716337299 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3743e00640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28090df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3749200640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3749c00640 1 -- 172.21.15.49:0/716337299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 msgr2=0x562c280908b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3749c00640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280908b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3749c00640 1 -- 172.21.15.49:0/716337299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 msgr2=0x562c28090df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3749c00640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28090df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.065+0000 7f3749c00640 1 -- 172.21.15.49:0/716337299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373000a930 con 0x562c2808a110 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.066+0000 7f3743e00640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28090df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.066+0000 7f3749200640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.067+0000 7f3749c00640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c280ef530 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f373c009ca0 tx=0x7f373c032740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5becdca647626085 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.067+0000 7f372aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.067+0000 7f372aa00640 1 -- 172.21.15.49:0/716337299 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c004850 con 0x562c2808a110 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.067+0000 7f374ea59740 1 -- 172.21.15.49:0/716337299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c27ce77e0 con 0x562c2808a110 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f372aa00640 1 -- 172.21.15.49:0/716337299 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f373c003970 con 0x562c2808a110 2024-02-26T06:28:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f372aa00640 1 -- 172.21.15.49:0/716337299 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f373c004850 con 0x562c2808a110 2024-02-26T06:28:18.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f374ea59740 10 client.26662 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:18.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f374ea59740 1 -- 172.21.15.49:0/716337299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x562c27fa8620 con 0x562c2808a110 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f372aa00640 1 -- 172.21.15.49:0/716337299 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562c27ce77e0 con 0x562c2808a110 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f372aa00640 1 -- 172.21.15.49:0/716337299 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f373c04abe0 con 0x562c2808a110 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f374ea59740 10 client.26662 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f374ea59740 10 client.26662 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f374ea59740 20 client.26662 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.068+0000 7f374ea59740 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562c280f10e0 0x562c280f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.069+0000 7f374ea59740 4 client.26662 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.069+0000 7f374ea59740 1 -- 172.21.15.49:0/716337299 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c27cd64e0 con 0x562c280f10e0 2024-02-26T06:28:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.069+0000 7f3749200640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562c280f10e0 0x562c280f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f3749200640 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562c280f10e0 0x562c280f34d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99ff76aabd9437f0 server_cookie=c557c88877012bdf in_seq=0 out_seq=0 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f372aa00640 10 client.26662 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f372aa00640 1 -- 172.21.15.49:0/716337299 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c27cd64e0 con 0x562c280f10e0 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f372aa00640 10 client.26662 handle_command_reply: tid=0 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f374ea59740 1 client.26662 shutdown 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f374ea59740 1 -- 172.21.15.49:0/716337299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 msgr2=0x562c280ef530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f374ea59740 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c280ef530 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f373c009ca0 tx=0x7f373c032740 comp rx=0 tx=0).stop 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f374ea59740 1 -- 172.21.15.49:0/716337299 shutdown_connections 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.070+0000 7f374ea59740 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x562c280f10e0 0x562c280f34d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.071+0000 7f374ea59740 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c2808a110 0x562c280ef530 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.071+0000 7f372aa00640 0 client.26662 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:28:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.071+0000 7f374ea59740 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c28081530 0x562c28090df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.071+0000 7f374ea59740 1 --2- 172.21.15.49:0/716337299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562c28080c10 0x562c280908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.071+0000 7f374ea59740 1 -- 172.21.15.49:0/716337299 >> 172.21.15.49:0/716337299 conn(0x562c2807b340 msgr2=0x562c280824f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.071+0000 7f374ea59740 1 -- 172.21.15.49:0/716337299 shutdown_connections 2024-02-26T06:28:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.071+0000 7f374ea59740 1 -- 172.21.15.49:0/716337299 wait complete. 2024-02-26T06:28:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.071+0000 7f374ea59740 20 client.26662 trim_cache size 0 max 16384 2024-02-26T06:28:18.074 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:28:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 -- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f372c0a2380 msgr2=0x7f372c0a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f372c0a2380 0x7f372c0a4840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f373c009430 tx=0x7f373c005c50 comp rx=0 tx=0).stop 2024-02-26T06:28:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 -- 172.21.15.49:0/2216417143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374410d9d0 msgr2=0x7f3744117db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374410d9d0 0x7f3744117db0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f373800c890 tx=0x7f373800cd50 comp rx=0 tx=0).stop 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 -- 172.21.15.49:0/2216417143 shutdown_connections 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f372c0a2380 0x7f372c0a4840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374410f9a0 0x7f37441182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374411a960 0x7f3744110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 --2- 172.21.15.49:0/2216417143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f374410d9d0 0x7f3744117db0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.073+0000 7f374bc00640 1 -- 172.21.15.49:0/2216417143 >> 172.21.15.49:0/2216417143 conn(0x7f374406c260 msgr2=0x7f3744071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.074+0000 7f374bc00640 1 -- 172.21.15.49:0/2216417143 shutdown_connections 2024-02-26T06:28:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.074+0000 7f374bc00640 1 -- 172.21.15.49:0/2216417143 wait complete. 2024-02-26T06:28:18.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:17 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1694302262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:18.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:17 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1694302262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:18.102 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:18.102 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:28:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.410+0000 7f71fe000640 1 -- 172.21.15.49:0/994759246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81064c0 msgr2=0x7f71f810e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.410+0000 7f71fe000640 1 --2- 172.21.15.49:0/994759246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81064c0 0x7f71f810e070 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009930 tx=0x7f71e402bc60 comp rx=0 tx=0).stop 2024-02-26T06:28:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.412+0000 7f71fe000640 1 -- 172.21.15.49:0/994759246 shutdown_connections 2024-02-26T06:28:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.412+0000 7f71fe000640 1 --2- 172.21.15.49:0/994759246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81064c0 0x7f71f810e070 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.412+0000 7f71fe000640 1 --2- 172.21.15.49:0/994759246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f8068fa0 0x7f71f8105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.412+0000 7f71fe000640 1 --2- 172.21.15.49:0/994759246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f80685f0 0x7f71f80689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.412+0000 7f71fe000640 1 -- 172.21.15.49:0/994759246 >> 172.21.15.49:0/994759246 conn(0x7f71f8077180 msgr2=0x7f71f8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.412+0000 7f71fe000640 1 -- 172.21.15.49:0/994759246 shutdown_connections 2024-02-26T06:28:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.412+0000 7f71fe000640 1 -- 172.21.15.49:0/994759246 wait complete. 2024-02-26T06:28:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.413+0000 7f71fe000640 1 Processor -- start 2024-02-26T06:28:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.414+0000 7f71fe000640 1 -- start start 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71fe000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f80685f0 0x7f71f819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71fe000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81064c0 0x7f71f819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71fe000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f810bf20 0x7f71f810c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71fe000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f810cb10 con 0x7f71f81064c0 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71fe000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f810cc80 con 0x7f71f810bf20 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71fe000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f819a040 con 0x7f71f80685f0 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71f7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f810bf20 0x7f71f810c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71f7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f810bf20 0x7f71f810c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44890/0 (socket says 172.21.15.49:44890) 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71f7e00640 1 -- 172.21.15.49:0/4129377248 learned_addr learned my addr 172.21.15.49:0/4129377248 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.416+0000 7f71f7400640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f80685f0 0x7f71f819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.417+0000 7f71f7e00640 1 -- 172.21.15.49:0/4129377248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f80685f0 msgr2=0x7f71f819e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.417+0000 7f71f7e00640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f80685f0 0x7f71f819e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.417+0000 7f71f7e00640 1 -- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81064c0 msgr2=0x7f71f819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.417+0000 7f71f7e00640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81064c0 0x7f71f819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.417+0000 7f71f7e00640 1 -- 172.21.15.49:0/4129377248 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e4009560 con 0x7f71f810bf20 2024-02-26T06:28:18.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.417+0000 7f71f7e00640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f810bf20 0x7f71f810c3d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f71e402c1a0 tx=0x7f71e4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.418+0000 7f71efe00640 1 -- 172.21.15.49:0/4129377248 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4004440 con 0x7f71f810bf20 2024-02-26T06:28:18.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.418+0000 7f71efe00640 1 -- 172.21.15.49:0/4129377248 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e40052d0 con 0x7f71f810bf20 2024-02-26T06:28:18.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.418+0000 7f71efe00640 1 -- 172.21.15.49:0/4129377248 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4003d50 con 0x7f71f810bf20 2024-02-26T06:28:18.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.419+0000 7f71fe000640 1 -- 172.21.15.49:0/4129377248 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f819a180 con 0x7f71f810bf20 2024-02-26T06:28:18.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.419+0000 7f71fe000640 1 -- 172.21.15.49:0/4129377248 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f819a610 con 0x7f71f810bf20 2024-02-26T06:28:18.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.425+0000 7f7200f87740 1 Processor -- start 2024-02-26T06:28:18.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.426+0000 7f7200f87740 1 -- start start 2024-02-26T06:28:18.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.426+0000 7f7200f87740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.426+0000 7f71efe00640 1 -- 172.21.15.49:0/4129377248 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f71e4005440 con 0x7f71f810bf20 2024-02-26T06:28:18.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.426+0000 7f7200f87740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b5530 0x558af76bdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.426+0000 7f7200f87740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.426+0000 7f7200f87740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af76231e0 con 0x558af76b4c10 2024-02-26T06:28:18.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.426+0000 7f7200f87740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af7620710 con 0x558af76b5530 2024-02-26T06:28:18.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.426+0000 7f7200f87740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af761bef0 con 0x558af76be110 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.427+0000 7f71f7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b5530 0x558af76bdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.427+0000 7f71f7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b5530 0x558af76bdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44906/0 (socket says 172.21.15.49:44906) 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.427+0000 7f71f7400640 1 -- 172.21.15.49:0/2747951792 learned_addr learned my addr 172.21.15.49:0/2747951792 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.427+0000 7f71f6a00640 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.427+0000 7f71f6a00640 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.428+0000 7f71f7400640 1 -- 172.21.15.49:0/2747951792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 msgr2=0x558af76c0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.428+0000 7f71f7400640 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76c0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.428+0000 7f71f7400640 1 -- 172.21.15.49:0/2747951792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 msgr2=0x558af76b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.428+0000 7f71f7400640 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.428+0000 7f71f7400640 1 -- 172.21.15.49:0/2747951792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558af75dddc0 con 0x558af76b5530 2024-02-26T06:28:18.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.429+0000 7f71f7400640 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b5530 0x558af76bdbd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71f000a5b0 tx=0x7f71f00096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37b4a60f0b3269c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.429+0000 7f71e3e00640 1 -- 172.21.15.49:0/2747951792 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0010040 con 0x558af76b5530 2024-02-26T06:28:18.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.429+0000 7f71e3e00640 1 -- 172.21.15.49:0/2747951792 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71f0014cf0 con 0x558af76b5530 2024-02-26T06:28:18.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.429+0000 7f71e3e00640 1 -- 172.21.15.49:0/2747951792 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0013e10 con 0x558af76b5530 2024-02-26T06:28:18.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.431+0000 7f7200f87740 1 -- 172.21.15.49:0/2747951792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b5530 msgr2=0x558af76bdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.431+0000 7f7200f87740 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b5530 0x558af76bdbd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71f000a5b0 tx=0x7f71f00096d0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.433+0000 7f7200f87740 1 -- 172.21.15.49:0/2747951792 shutdown_connections 2024-02-26T06:28:18.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.433+0000 7f7200f87740 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b5530 0x558af76bdbd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.433+0000 7f7200f87740 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76c0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.433+0000 7f7200f87740 1 --2- 172.21.15.49:0/2747951792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.433+0000 7f7200f87740 1 -- 172.21.15.49:0/2747951792 >> 172.21.15.49:0/2747951792 conn(0x558af76af340 msgr2=0x558af76af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.433+0000 7f7200f87740 1 -- 172.21.15.49:0/2747951792 shutdown_connections 2024-02-26T06:28:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.433+0000 7f7200f87740 1 -- 172.21.15.49:0/2747951792 wait complete. 2024-02-26T06:28:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.434+0000 7f7200f87740 1 -- start start 2024-02-26T06:28:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.434+0000 7f7200f87740 1 Processor -- start 2024-02-26T06:28:18.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.436+0000 7f7200f87740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b4c10 0x558af76bb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.436+0000 7f7200f87740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b5530 0x558af76bb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.436+0000 7f7200f87740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.437+0000 7f7200f87740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af75c1120 con 0x558af76b5530 2024-02-26T06:28:18.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.437+0000 7f7200f87740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af75c0dc0 con 0x558af76b4c10 2024-02-26T06:28:18.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.437+0000 7f71f7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b5530 0x558af76bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.437+0000 7f71f7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b5530 0x558af76bb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41604/0 (socket says 172.21.15.49:41604) 2024-02-26T06:28:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.437+0000 7f71f7400640 1 -- 172.21.15.49:0/3467167572 learned_addr learned my addr 172.21.15.49:0/3467167572 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.437+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af738a1c0 con 0x558af76be110 2024-02-26T06:28:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.437+0000 7f7200f87740 10 client.0 _ll_register_callbacks cb 0x558af7610280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.437+0000 7f71efe00640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f71dc0a2310 0x7f71dc0a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.438+0000 7f71f6a00640 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b4c10 0x558af76bb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.438+0000 7f71f7e00640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f71dc0a2310 0x7f71dc0a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.438+0000 7f71f7400640 1 -- 172.21.15.49:0/3467167572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 msgr2=0x558af76bbb40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:28:18.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.438+0000 7f71f7400640 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bbb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.438+0000 7f71f7400640 1 -- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b4c10 msgr2=0x558af76bb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.439+0000 7f71efe00640 1 -- 172.21.15.49:0/4129377248 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f71e40f00d0 con 0x7f71f810bf20 2024-02-26T06:28:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.439+0000 7f71f7e00640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f71dc0a2310 0x7f71dc0a47d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f71e4005ab0 tx=0x7f71e402c6b0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.440+0000 7f71f7400640 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b4c10 0x558af76bb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.440+0000 7f71f7400640 1 -- 172.21.15.49:0/3467167572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f0009d80 con 0x558af76b5530 2024-02-26T06:28:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.440+0000 7f71f6a00640 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b4c10 0x558af76bb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:18.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.440+0000 7f71e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:28:18.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.440+0000 7f71f7400640 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b5530 0x558af76bb600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009490 tx=0x7f71f00094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3f8682183412fd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.441+0000 7f71e0c00640 1 -- 172.21.15.49:0/3467167572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0014610 con 0x558af76b5530 2024-02-26T06:28:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.441+0000 7f71e0c00640 1 -- 172.21.15.49:0/3467167572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f71f0014770 con 0x558af76b5530 2024-02-26T06:28:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.441+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558af731b7e0 con 0x558af76b5530 2024-02-26T06:28:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.442+0000 7f71e0c00640 1 -- 172.21.15.49:0/3467167572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f0014930 con 0x558af76b5530 2024-02-26T06:28:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.442+0000 7f7200f87740 10 client.16245 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:18.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.442+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558af761a570 con 0x558af76b5530 2024-02-26T06:28:18.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.443+0000 7f71e0c00640 1 -- 172.21.15.49:0/3467167572 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558af731b7e0 con 0x558af76b5530 2024-02-26T06:28:18.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.443+0000 7f71e0c00640 1 -- 172.21.15.49:0/3467167572 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71f0008030 con 0x558af76b5530 2024-02-26T06:28:18.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.443+0000 7f7200f87740 10 client.16245 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:18.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.443+0000 7f7200f87740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:18.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.443+0000 7f7200f87740 20 client.16245 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:18.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.443+0000 7f7200f87740 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558af76ce090 0x558af76d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.444+0000 7f7200f87740 4 client.16245 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:18.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.444+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558af7358720 con 0x558af76ce090 2024-02-26T06:28:18.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.444+0000 7f71f6a00640 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558af76ce090 0x558af76d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.445+0000 7f71f6a00640 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558af76ce090 0x558af76d04b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2863b9eab520b37 server_cookie=a7ba4fe5d2c04213 in_seq=0 out_seq=0 2024-02-26T06:28:18.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.445+0000 7f71e0c00640 10 client.16245 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.446+0000 7f71e0c00640 1 -- 172.21.15.49:0/3467167572 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558af7358720 con 0x558af76ce090 2024-02-26T06:28:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.446+0000 7f71e0c00640 10 client.16245 handle_command_reply: tid=0 2024-02-26T06:28:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.446+0000 7f7200f87740 1 client.16245 shutdown 2024-02-26T06:28:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.446+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b5530 msgr2=0x558af76bb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.446+0000 7f7200f87740 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b5530 0x558af76bb600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009490 tx=0x7f71f00094c0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.447+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 shutdown_connections 2024-02-26T06:28:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.447+0000 7f7200f87740 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558af76ce090 0x558af76d04b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.447+0000 7f7200f87740 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.447+0000 7f7200f87740 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b5530 0x558af76bb600 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009490 tx=0x7f71f00094c0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.447+0000 7f7200f87740 1 --2- 172.21.15.49:0/3467167572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b4c10 0x558af76bb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.447+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 >> 172.21.15.49:0/3467167572 conn(0x558af76af340 msgr2=0x558af76b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.447+0000 7f71e0c00640 0 client.16245 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:18.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.447+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 shutdown_connections 2024-02-26T06:28:18.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.448+0000 7f7200f87740 1 -- 172.21.15.49:0/3467167572 wait complete. 2024-02-26T06:28:18.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.448+0000 7f7200f87740 20 client.16245 trim_cache size 0 max 16384 2024-02-26T06:28:18.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.471+0000 7f7200f87740 1 Processor -- start 2024-02-26T06:28:18.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.471+0000 7f7200f87740 1 -- start start 2024-02-26T06:28:18.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.472+0000 7f7200f87740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.472+0000 7f7200f87740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.472+0000 7f7200f87740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.472+0000 7f7200f87740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af76b4ac0 con 0x558af76b4c10 2024-02-26T06:28:18.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.472+0000 7f7200f87740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af7358720 con 0x558af76b6450 2024-02-26T06:28:18.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.472+0000 7f7200f87740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af731b7e0 con 0x558af76be110 2024-02-26T06:28:18.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f7400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43738/0 (socket says 172.21.15.49:43738) 2024-02-26T06:28:18.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f7400640 1 -- 172.21.15.49:0/548780672 learned_addr learned my addr 172.21.15.49:0/548780672 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f7400640 1 -- 172.21.15.49:0/548780672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 msgr2=0x558af76b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f7400640 1 --2- 172.21.15.49:0/548780672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f7400640 1 -- 172.21.15.49:0/548780672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 msgr2=0x558af76bd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f7400640 1 --2- 172.21.15.49:0/548780672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76bd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.473+0000 7f71f7400640 1 -- 172.21.15.49:0/548780672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558af75b6a80 con 0x558af76be110 2024-02-26T06:28:18.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.474+0000 7f71f6a00640 1 --2- 172.21.15.49:0/548780672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76bd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:18.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.474+0000 7f71f7400640 1 --2- 172.21.15.49:0/548780672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bb960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f71f0027040 tx=0x7f71f001f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb195cb6afc99112 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.474+0000 7f71e3400640 1 -- 172.21.15.49:0/548780672 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f000a400 con 0x558af76be110 2024-02-26T06:28:18.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.474+0000 7f71e3400640 1 -- 172.21.15.49:0/548780672 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71f0010030 con 0x558af76be110 2024-02-26T06:28:18.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.474+0000 7f71e3400640 1 -- 172.21.15.49:0/548780672 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0014610 con 0x558af76be110 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.475+0000 7f7200f87740 1 -- 172.21.15.49:0/548780672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 msgr2=0x558af76bb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.475+0000 7f7200f87740 1 --2- 172.21.15.49:0/548780672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bb960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f71f0027040 tx=0x7f71f001f040 comp rx=0 tx=0).stop 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.475+0000 7f7200f87740 1 -- 172.21.15.49:0/548780672 shutdown_connections 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.475+0000 7f7200f87740 1 --2- 172.21.15.49:0/548780672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.475+0000 7f7200f87740 1 --2- 172.21.15.49:0/548780672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76be110 0x558af76bb960 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.475+0000 7f7200f87740 1 --2- 172.21.15.49:0/548780672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76b4c10 0x558af76bd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.475+0000 7f7200f87740 1 -- 172.21.15.49:0/548780672 >> 172.21.15.49:0/548780672 conn(0x558af76af340 msgr2=0x558af76b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.476+0000 7f7200f87740 1 -- 172.21.15.49:0/548780672 shutdown_connections 2024-02-26T06:28:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.476+0000 7f7200f87740 1 -- 172.21.15.49:0/548780672 wait complete. 2024-02-26T06:28:18.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.476+0000 7f7200f87740 1 -- start start 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.476+0000 7f7200f87740 1 Processor -- start 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f7200f87740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76b4c10 0x558af76c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f7200f87740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f7200f87740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76be110 0x558af76c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f7200f87740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af7724600 con 0x558af76be110 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f7200f87740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af7724770 con 0x558af76b6450 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f7200f87740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af7724930 con 0x558af76b4c10 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f7200f87740 10 client.0 _ll_register_callbacks cb 0x7f71f0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f71f7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f71f6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76b4c10 0x558af76c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f71f7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44930/0 (socket says 172.21.15.49:44930) 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.477+0000 7f71f7400640 1 -- 172.21.15.49:0/513165695 learned_addr learned my addr 172.21.15.49:0/513165695 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.478+0000 7f71f7400640 1 -- 172.21.15.49:0/513165695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76b4c10 msgr2=0x558af76c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.478+0000 7f71f7400640 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76b4c10 0x558af76c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.478+0000 7f71f7400640 1 -- 172.21.15.49:0/513165695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76be110 msgr2=0x558af76c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.478+0000 7f71f7400640 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76be110 0x558af76c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.478+0000 7f71f7400640 1 -- 172.21.15.49:0/513165695 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558af761a570 con 0x558af76b6450 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.479+0000 7f71f7400640 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76c40b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f71f0000f80 tx=0x7f71f0009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2068f0bb11bf5ce server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.479+0000 7f71e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.479+0000 7f71e0c00640 1 -- 172.21.15.49:0/513165695 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0003c10 con 0x558af76b6450 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.479+0000 7f71e0c00640 1 -- 172.21.15.49:0/513165695 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f71f0032070 con 0x558af76b6450 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.479+0000 7f7200f87740 1 -- 172.21.15.49:0/513165695 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71f0014cf0 con 0x558af76b6450 2024-02-26T06:28:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.479+0000 7f71e0c00640 1 -- 172.21.15.49:0/513165695 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f001d6e0 con 0x558af76b6450 2024-02-26T06:28:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.479+0000 7f7200f87740 10 client.26657 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.479+0000 7f7200f87740 1 -- 172.21.15.49:0/513165695 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558af75dc620 con 0x558af76b6450 2024-02-26T06:28:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.480+0000 7f71e0c00640 1 -- 172.21.15.49:0/513165695 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71f0014cf0 con 0x558af76b6450 2024-02-26T06:28:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.480+0000 7f71e0c00640 1 -- 172.21.15.49:0/513165695 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71f0045070 con 0x558af76b6450 2024-02-26T06:28:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.480+0000 7f7200f87740 10 client.26657 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.480+0000 7f7200f87740 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.480+0000 7f7200f87740 20 client.26657 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.480+0000 7f7200f87740 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558af7725b70 0x558af7727f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.480+0000 7f7200f87740 4 client.26657 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.480+0000 7f7200f87740 1 -- 172.21.15.49:0/513165695 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558af73a7890 con 0x558af7725b70 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.482+0000 7f71f6a00640 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558af7725b70 0x558af7727f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.482+0000 7f71f6a00640 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558af7725b70 0x558af7727f60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd11ed8f53cba7c server_cookie=494589b8439448f1 in_seq=0 out_seq=0 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.482+0000 7f71e0c00640 10 client.26657 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f71e0c00640 1 -- 172.21.15.49:0/513165695 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558af73a7890 con 0x558af7725b70 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f71e0c00640 10 client.26657 handle_command_reply: tid=0 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 client.26657 shutdown 2024-02-26T06:28:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 -- 172.21.15.49:0/513165695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 msgr2=0x558af76c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76c40b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f71f0000f80 tx=0x7f71f0009360 comp rx=0 tx=0).stop 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 -- 172.21.15.49:0/513165695 shutdown_connections 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558af7725b70 0x558af7727f60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558af76b6450 0x558af76c40b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558af76be110 0x558af76c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 --2- 172.21.15.49:0/513165695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558af76b4c10 0x558af76c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 -- 172.21.15.49:0/513165695 >> 172.21.15.49:0/513165695 conn(0x558af76af340 msgr2=0x558af76adbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f71e0c00640 0 client.26657 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.483+0000 7f7200f87740 1 -- 172.21.15.49:0/513165695 shutdown_connections 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.484+0000 7f7200f87740 1 -- 172.21.15.49:0/513165695 wait complete. 2024-02-26T06:28:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.484+0000 7f7200f87740 20 client.26657 trim_cache size 0 max 16384 2024-02-26T06:28:18.487 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:28:18.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 -- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f71dc0a2310 msgr2=0x7f71dc0a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f71dc0a2310 0x7f71dc0a47d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f71e4005ab0 tx=0x7f71e402c6b0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 -- 172.21.15.49:0/4129377248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f810bf20 msgr2=0x7f71f810c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f810bf20 0x7f71f810c3d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f71e402c1a0 tx=0x7f71e4004040 comp rx=0 tx=0).stop 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 -- 172.21.15.49:0/4129377248 shutdown_connections 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f71dc0a2310 0x7f71dc0a47d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f810bf20 0x7f71f810c3d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f81064c0 0x7f71f819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 --2- 172.21.15.49:0/4129377248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f80685f0 0x7f71f819e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 -- 172.21.15.49:0/4129377248 >> 172.21.15.49:0/4129377248 conn(0x7f71f8077180 msgr2=0x7f71f8104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.485+0000 7f71fe000640 1 -- 172.21.15.49:0/4129377248 shutdown_connections 2024-02-26T06:28:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:18.486+0000 7f71fe000640 1 -- 172.21.15.49:0/4129377248 wait complete. 2024-02-26T06:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.642+0000 7f3953a00640 1 -- 172.21.15.49:0/2448707494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c104500 msgr2=0x7f394c104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.642+0000 7f3953a00640 1 --2- 172.21.15.49:0/2448707494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c104500 0x7f394c104990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3944009ca0 tx=0x7f394402c100 comp rx=0 tx=0).stop 2024-02-26T06:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.642+0000 7f3953a00640 1 -- 172.21.15.49:0/2448707494 shutdown_connections 2024-02-26T06:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.642+0000 7f3953a00640 1 --2- 172.21.15.49:0/2448707494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c104500 0x7f394c104990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.642+0000 7f3953a00640 1 --2- 172.21.15.49:0/2448707494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f394c0ffef0 0x7f394c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.642+0000 7f3953a00640 1 --2- 172.21.15.49:0/2448707494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f394c0ff5d0 0x7f394c0ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.642+0000 7f3953a00640 1 -- 172.21.15.49:0/2448707494 >> 172.21.15.49:0/2448707494 conn(0x7f394c076210 msgr2=0x7f394c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.644+0000 7f3953a00640 1 -- 172.21.15.49:0/2448707494 shutdown_connections 2024-02-26T06:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.644+0000 7f3953a00640 1 -- 172.21.15.49:0/2448707494 wait complete. 2024-02-26T06:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.645+0000 7f3953a00640 1 Processor -- start 2024-02-26T06:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3953a00640 1 -- start start 2024-02-26T06:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3953a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c0ff5d0 0x7f394c113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3953a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f394c0ffef0 0x7f394c110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3953a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f394c104500 0x7f394c110650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3953a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c110c20 con 0x7f394c0ffef0 2024-02-26T06:28:18.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3953a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c110d90 con 0x7f394c0ff5d0 2024-02-26T06:28:18.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3953a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c110f50 con 0x7f394c104500 2024-02-26T06:28:18.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3951000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c0ff5d0 0x7f394c113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3951000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c0ff5d0 0x7f394c113070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44948/0 (socket says 172.21.15.49:44948) 2024-02-26T06:28:18.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3951000640 1 -- 172.21.15.49:0/3551087227 learned_addr learned my addr 172.21.15.49:0/3551087227 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f3951a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f394c104500 0x7f394c110650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.646+0000 7f394be00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f394c0ffef0 0x7f394c110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.647+0000 7f3951a00640 1 -- 172.21.15.49:0/3551087227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c0ff5d0 msgr2=0x7f394c113070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.647+0000 7f3951a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c0ff5d0 0x7f394c113070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.647+0000 7f3951a00640 1 -- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f394c0ffef0 msgr2=0x7f394c110110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.647+0000 7f3951a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f394c0ffef0 0x7f394c110110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.647+0000 7f3951a00640 1 -- 172.21.15.49:0/3551087227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944009900 con 0x7f394c104500 2024-02-26T06:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.647+0000 7f3951000640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c0ff5d0 0x7f394c113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.647+0000 7f394be00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f394c0ffef0 0x7f394c110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.647+0000 7f3951a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f394c104500 0x7f394c110650 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3944009bf0 tx=0x7f3944032570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.648+0000 7f3949600640 1 -- 172.21.15.49:0/3551087227 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944034640 con 0x7f394c104500 2024-02-26T06:28:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.648+0000 7f3949600640 1 -- 172.21.15.49:0/3551087227 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3944034e00 con 0x7f394c104500 2024-02-26T06:28:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.648+0000 7f3949600640 1 -- 172.21.15.49:0/3551087227 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440047d0 con 0x7f394c104500 2024-02-26T06:28:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.648+0000 7f3953a00640 1 -- 172.21.15.49:0/3551087227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c19fb60 con 0x7f394c104500 2024-02-26T06:28:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.648+0000 7f3953a00640 1 -- 172.21.15.49:0/3551087227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c19fff0 con 0x7f394c104500 2024-02-26T06:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.652+0000 7f3949600640 1 -- 172.21.15.49:0/3551087227 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f394400a040 con 0x7f394c104500 2024-02-26T06:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.654+0000 7f3949600640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f39340a2430 0x7f39340a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.654+0000 7f3949600640 1 -- 172.21.15.49:0/3551087227 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f39440f07d0 con 0x7f394c104500 2024-02-26T06:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.655+0000 7f39569a4740 1 Processor -- start 2024-02-26T06:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.655+0000 7f39569a4740 1 -- start start 2024-02-26T06:28:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.655+0000 7f3951000640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f39340a2430 0x7f39340a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f39569a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268bf7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f39569a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268bf8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f39569a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c14700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f39569a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268bb35f0 con 0x559268c12310 2024-02-26T06:28:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f39569a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268b65890 con 0x559268bf7d80 2024-02-26T06:28:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f39569a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268b62dc0 con 0x559268bf8af0 2024-02-26T06:28:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f3951a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c14700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f3951a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c14700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41630/0 (socket says 172.21.15.49:41630) 2024-02-26T06:28:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f3951a00640 1 -- 172.21.15.49:0/941721296 learned_addr learned my addr 172.21.15.49:0/941721296 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.656+0000 7f3951000640 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268bf7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.657+0000 7f394be00640 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268bf8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.657+0000 7f3951a00640 1 -- 172.21.15.49:0/941721296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 msgr2=0x559268bf7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.657+0000 7f3951a00640 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268bf7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.657+0000 7f3951a00640 1 -- 172.21.15.49:0/941721296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 msgr2=0x559268bf8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.657+0000 7f3951a00640 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268bf8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.657+0000 7f3951a00640 1 -- 172.21.15.49:0/941721296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559268adfad0 con 0x559268c12310 2024-02-26T06:28:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.657+0000 7f3951000640 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268bf7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.657+0000 7f394be00640 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268bf8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.658+0000 7f3951a00640 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c14700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39440b9d30 tx=0x7f39440bcea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4978355d400f55d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.658+0000 7f3951000640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f39340a2430 0x7f39340a48f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3940013c20 tx=0x7f39400134e0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.658+0000 7f393e000640 1 -- 172.21.15.49:0/941721296 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440bd360 con 0x559268c12310 2024-02-26T06:28:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.658+0000 7f393e000640 1 -- 172.21.15.49:0/941721296 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39440c14b0 con 0x559268c12310 2024-02-26T06:28:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.658+0000 7f393e000640 1 -- 172.21.15.49:0/941721296 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440bd5c0 con 0x559268c12310 2024-02-26T06:28:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.659+0000 7f39569a4740 1 -- 172.21.15.49:0/941721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 msgr2=0x559268c14700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.659+0000 7f39569a4740 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c14700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39440b9d30 tx=0x7f39440bcea0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.659+0000 7f39569a4740 1 -- 172.21.15.49:0/941721296 shutdown_connections 2024-02-26T06:28:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.659+0000 7f39569a4740 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268bf8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.659+0000 7f39569a4740 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c14700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.659+0000 7f39569a4740 1 --2- 172.21.15.49:0/941721296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268bf7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.659+0000 7f39569a4740 1 -- 172.21.15.49:0/941721296 >> 172.21.15.49:0/941721296 conn(0x559268bbf480 msgr2=0x559268bf2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.659+0000 7f39569a4740 1 -- 172.21.15.49:0/941721296 shutdown_connections 2024-02-26T06:28:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.660+0000 7f39569a4740 1 -- 172.21.15.49:0/941721296 wait complete. 2024-02-26T06:28:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.661+0000 7f39569a4740 1 -- start start 2024-02-26T06:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.661+0000 7f39569a4740 1 Processor -- start 2024-02-26T06:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f39569a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c0f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f39569a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268bf8af0 0x559268c0f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f39569a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268c12310 0x559268c0fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f39569a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268b5e5a0 con 0x559268bf8af0 2024-02-26T06:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f39569a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268b03b20 con 0x559268bf7d80 2024-02-26T06:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f39569a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268b037c0 con 0x559268c12310 2024-02-26T06:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f39569a4740 10 client.0 _ll_register_callbacks cb 0x559268b17520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f3951a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268c12310 0x559268c0fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f3951a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268c12310 0x559268c0fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43788/0 (socket says 172.21.15.49:43788) 2024-02-26T06:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.662+0000 7f3951a00640 1 -- 172.21.15.49:0/1558967983 learned_addr learned my addr 172.21.15.49:0/1558967983 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.663+0000 7f3951000640 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c0f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.663+0000 7f394be00640 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268bf8af0 0x559268c0f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.663+0000 7f3951a00640 1 -- 172.21.15.49:0/1558967983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 msgr2=0x559268c0f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.663+0000 7f3951a00640 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c0f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.663+0000 7f3951a00640 1 -- 172.21.15.49:0/1558967983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268bf8af0 msgr2=0x559268c0f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.663+0000 7f3951a00640 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268bf8af0 0x559268c0f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.663+0000 7f3951a00640 1 -- 172.21.15.49:0/1558967983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c19fff0 con 0x559268c12310 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.663+0000 7f394be00640 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268bf8af0 0x559268c0f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.664+0000 7f3951a00640 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268c12310 0x559268c0fe50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39440bd2f0 tx=0x7f39440bb870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58856fab5eb86b79 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.664+0000 7f3932a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.664+0000 7f3932a00640 1 -- 172.21.15.49:0/1558967983 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440d8020 con 0x559268c12310 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.664+0000 7f3932a00640 1 -- 172.21.15.49:0/1558967983 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f39440c5b00 con 0x559268c12310 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.664+0000 7f39569a4740 1 -- 172.21.15.49:0/1558967983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559268b30ca0 con 0x559268c12310 2024-02-26T06:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f3932a00640 1 -- 172.21.15.49:0/1558967983 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39440c7ea0 con 0x559268c12310 2024-02-26T06:28:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f39569a4740 10 client.26683 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f39569a4740 1 -- 172.21.15.49:0/1558967983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559268b5cc20 con 0x559268c12310 2024-02-26T06:28:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f3932a00640 1 -- 172.21.15.49:0/1558967983 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559268b30ca0 con 0x559268c12310 2024-02-26T06:28:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f3932a00640 1 -- 172.21.15.49:0/1558967983 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f39440c5670 con 0x559268c12310 2024-02-26T06:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f39569a4740 10 client.26683 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f39569a4740 10 client.26683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f39569a4740 20 client.26683 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f39569a4740 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x559268c21ca0 0x559268c24090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f39569a4740 4 client.26683 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f39569a4740 1 -- 172.21.15.49:0/1558967983 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559268b54cb0 con 0x559268c21ca0 2024-02-26T06:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.665+0000 7f3951000640 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x559268c21ca0 0x559268c24090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.666+0000 7f3951000640 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x559268c21ca0 0x559268c24090 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d06edfc4fcb5cb1 server_cookie=3fce0250dcbce329 in_seq=0 out_seq=0 2024-02-26T06:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.666+0000 7f3932a00640 10 client.26683 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.667+0000 7f3932a00640 1 -- 172.21.15.49:0/1558967983 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559268b54cb0 con 0x559268c21ca0 2024-02-26T06:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.667+0000 7f3932a00640 10 client.26683 handle_command_reply: tid=0 2024-02-26T06:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.667+0000 7f39569a4740 1 client.26683 shutdown 2024-02-26T06:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f39569a4740 1 -- 172.21.15.49:0/1558967983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268c12310 msgr2=0x559268c0fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f39569a4740 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268c12310 0x559268c0fe50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39440bd2f0 tx=0x7f39440bb870 comp rx=0 tx=0).stop 2024-02-26T06:28:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f39569a4740 1 -- 172.21.15.49:0/1558967983 shutdown_connections 2024-02-26T06:28:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f39569a4740 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x559268c21ca0 0x559268c24090 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f39569a4740 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268c12310 0x559268c0fe50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f39569a4740 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268bf8af0 0x559268c0f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f39569a4740 1 --2- 172.21.15.49:0/1558967983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c0f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f39569a4740 1 -- 172.21.15.49:0/1558967983 >> 172.21.15.49:0/1558967983 conn(0x559268bbf480 msgr2=0x559268bf47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.668+0000 7f3932a00640 0 client.26683 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.669+0000 7f39569a4740 1 -- 172.21.15.49:0/1558967983 shutdown_connections 2024-02-26T06:28:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.669+0000 7f39569a4740 1 -- 172.21.15.49:0/1558967983 wait complete. 2024-02-26T06:28:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.669+0000 7f39569a4740 20 client.26683 trim_cache size 0 max 16384 2024-02-26T06:28:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.690+0000 7f39569a4740 1 Processor -- start 2024-02-26T06:28:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f39569a4740 1 -- start start 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f39569a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf7d80 0x559268c18ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f39569a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf8af0 0x559268c19430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f39569a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c0f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f39569a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592688a6e60 con 0x559268c12310 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f39569a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268be4670 con 0x559268bf8af0 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f39569a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268bee180 con 0x559268bf7d80 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f394be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf8af0 0x559268c19430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f3951000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf7d80 0x559268c18ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f394be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf8af0 0x559268c19430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44972/0 (socket says 172.21.15.49:44972) 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f394be00640 1 -- 172.21.15.49:0/3286515119 learned_addr learned my addr 172.21.15.49:0/3286515119 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.691+0000 7f3951a00640 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c0f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f3951000640 1 -- 172.21.15.49:0/3286515119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf8af0 msgr2=0x559268c19430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f3951000640 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf8af0 0x559268c19430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f3951000640 1 -- 172.21.15.49:0/3286515119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 msgr2=0x559268c0f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f3951000640 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c0f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f3951000640 1 -- 172.21.15.49:0/3286515119 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559268af9500 con 0x559268bf7d80 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f394be00640 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf8af0 0x559268c19430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f3951000640 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf7d80 0x559268c18ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x559268c24900 tx=0x7f394001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6f70168e9227fee server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f393d600640 1 -- 172.21.15.49:0/3286515119 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940020d80 con 0x559268bf7d80 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f393d600640 1 -- 172.21.15.49:0/3286515119 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3940028810 con 0x559268bf7d80 2024-02-26T06:28:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.692+0000 7f393d600640 1 -- 172.21.15.49:0/3286515119 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940005280 con 0x559268bf7d80 2024-02-26T06:28:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.693+0000 7f39569a4740 1 -- 172.21.15.49:0/3286515119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf7d80 msgr2=0x559268c18ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.693+0000 7f39569a4740 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf7d80 0x559268c18ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x559268c24900 tx=0x7f394001f000 comp rx=0 tx=0).stop 2024-02-26T06:28:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.693+0000 7f39569a4740 1 -- 172.21.15.49:0/3286515119 shutdown_connections 2024-02-26T06:28:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.693+0000 7f39569a4740 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf8af0 0x559268c19430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.693+0000 7f39569a4740 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c0f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.693+0000 7f39569a4740 1 --2- 172.21.15.49:0/3286515119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf7d80 0x559268c18ef0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.693+0000 7f39569a4740 1 -- 172.21.15.49:0/3286515119 >> 172.21.15.49:0/3286515119 conn(0x559268bbf480 msgr2=0x559268c0a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.694+0000 7f39569a4740 1 -- 172.21.15.49:0/3286515119 shutdown_connections 2024-02-26T06:28:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.694+0000 7f39569a4740 1 -- 172.21.15.49:0/3286515119 wait complete. 2024-02-26T06:28:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.694+0000 7f39569a4740 1 -- start start 2024-02-26T06:28:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.696+0000 7f39569a4740 1 Processor -- start 2024-02-26T06:28:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f39569a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c18f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f39569a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268c0b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f39569a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c0bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f39569a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268c19490 con 0x559268c12310 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f39569a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268c19600 con 0x559268bf7d80 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f39569a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559268c197c0 con 0x559268bf8af0 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f39569a4740 10 client.0 _ll_register_callbacks cb 0x7f39440b96d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f3951000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c18f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f3951000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c18f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44974/0 (socket says 172.21.15.49:44974) 2024-02-26T06:28:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.697+0000 7f3951000640 1 -- 172.21.15.49:0/4036479685 learned_addr learned my addr 172.21.15.49:0/4036479685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f394be00640 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268c0b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f3951000640 1 -- 172.21.15.49:0/4036479685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 msgr2=0x559268c0b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f3951000640 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268c0b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f3951000640 1 -- 172.21.15.49:0/4036479685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 msgr2=0x559268c0bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f3951000640 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c0bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f3951000640 1 -- 172.21.15.49:0/4036479685 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3938009700 con 0x559268bf7d80 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f394be00640 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268c0b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f3951000640 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c18f00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3940013830 tx=0x7f3940013860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45579e77e16f984 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.698+0000 7f3932a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.699+0000 7f3932a00640 1 -- 172.21.15.49:0/4036479685 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940021e10 con 0x559268bf7d80 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.699+0000 7f3932a00640 1 -- 172.21.15.49:0/4036479685 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f3940026070 con 0x559268bf7d80 2024-02-26T06:28:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.699+0000 7f39569a4740 1 -- 172.21.15.49:0/4036479685 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559268b83d00 con 0x559268bf7d80 2024-02-26T06:28:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.699+0000 7f3932a00640 1 -- 172.21.15.49:0/4036479685 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3940028540 con 0x559268bf7d80 2024-02-26T06:28:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.699+0000 7f39569a4740 10 client.26675 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.699+0000 7f39569a4740 1 -- 172.21.15.49:0/4036479685 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f3940020400 con 0x559268bf7d80 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.699+0000 7f3932a00640 1 -- 172.21.15.49:0/4036479685 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559268b83d00 con 0x559268bf7d80 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.699+0000 7f3932a00640 1 -- 172.21.15.49:0/4036479685 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3940010c10 con 0x559268bf7d80 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.700+0000 7f39569a4740 10 client.26675 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.700+0000 7f39569a4740 10 client.26675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.700+0000 7f39569a4740 20 client.26675 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.700+0000 7f39569a4740 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x559268c79180 0x559268c7b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.700+0000 7f39569a4740 4 client.26675 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.700+0000 7f39569a4740 1 -- 172.21.15.49:0/4036479685 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55926885e600 con 0x559268c79180 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.700+0000 7f394be00640 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x559268c79180 0x559268c7b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.701+0000 7f394be00640 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x559268c79180 0x559268c7b5a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d491d3cd918ccc6f server_cookie=58031fed8a3f2407 in_seq=0 out_seq=0 2024-02-26T06:28:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.701+0000 7f3932a00640 10 client.26675 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.714+0000 7f3932a00640 1 -- 172.21.15.49:0/4036479685 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55926885e600 con 0x559268c79180 2024-02-26T06:28:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.714+0000 7f3932a00640 10 client.26675 handle_command_reply: tid=0 2024-02-26T06:28:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.714+0000 7f39569a4740 1 client.26675 shutdown 2024-02-26T06:28:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.714+0000 7f39569a4740 1 -- 172.21.15.49:0/4036479685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 msgr2=0x559268c18f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.714+0000 7f39569a4740 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c18f00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3940013830 tx=0x7f3940013860 comp rx=0 tx=0).stop 2024-02-26T06:28:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.714+0000 7f39569a4740 1 -- 172.21.15.49:0/4036479685 shutdown_connections 2024-02-26T06:28:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f39569a4740 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x559268c79180 0x559268c7b5a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f39569a4740 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559268c12310 0x559268c0bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f39569a4740 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559268bf8af0 0x559268c0b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f39569a4740 1 --2- 172.21.15.49:0/4036479685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559268bf7d80 0x559268c18f00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f39569a4740 1 -- 172.21.15.49:0/4036479685 >> 172.21.15.49:0/4036479685 conn(0x559268bbf480 msgr2=0x559268c0dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f3932a00640 0 client.26675 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f39569a4740 1 -- 172.21.15.49:0/4036479685 shutdown_connections 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f39569a4740 1 -- 172.21.15.49:0/4036479685 wait complete. 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.715+0000 7f39569a4740 20 client.26675 trim_cache size 0 max 16384 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:28:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:28:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 -- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f39340a2430 msgr2=0x7f39340a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f39340a2430 0x7f39340a48f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3940013c20 tx=0x7f39400134e0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 -- 172.21.15.49:0/3551087227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f394c104500 msgr2=0x7f394c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f394c104500 0x7f394c110650 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3944009bf0 tx=0x7f3944032570 comp rx=0 tx=0).stop 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 -- 172.21.15.49:0/3551087227 shutdown_connections 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f39340a2430 0x7f39340a48f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f394c104500 0x7f394c110650 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f394c0ffef0 0x7f394c110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 --2- 172.21.15.49:0/3551087227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f394c0ff5d0 0x7f394c113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 -- 172.21.15.49:0/3551087227 >> 172.21.15.49:0/3551087227 conn(0x7f394c076210 msgr2=0x7f394c1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 -- 172.21.15.49:0/3551087227 shutdown_connections 2024-02-26T06:28:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:18.716+0000 7f3953a00640 1 -- 172.21.15.49:0/3551087227 wait complete. 2024-02-26T06:28:18.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:18 smithi049 ceph-mon[31696]: pgmap v150: 97 pgs: 97 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 MiB/s wr, 1.40k op/s 2024-02-26T06:28:18.991 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:18 smithi115 ceph-mon[36426]: pgmap v150: 97 pgs: 97 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 MiB/s wr, 1.40k op/s 2024-02-26T06:28:19.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:18 smithi080 ceph-mon[36206]: pgmap v150: 97 pgs: 97 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 MiB/s wr, 1.40k op/s 2024-02-26T06:28:19.510 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:28:19.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:28:19.510 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:28:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.805+0000 7f42e7e00640 1 -- 172.21.15.49:0/995680096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 msgr2=0x7f42e010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.805+0000 7f42e7e00640 1 --2- 172.21.15.49:0/995680096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 0x7f42e010db10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009900 tx=0x7f42d402bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.806+0000 7f42e7e00640 1 -- 172.21.15.49:0/995680096 shutdown_connections 2024-02-26T06:28:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.806+0000 7f42e7e00640 1 --2- 172.21.15.49:0/995680096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e01080e0 0x7f42e01145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.806+0000 7f42e7e00640 1 --2- 172.21.15.49:0/995680096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e0107720 0x7f42e0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.806+0000 7f42e7e00640 1 --2- 172.21.15.49:0/995680096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 0x7f42e010db10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.806+0000 7f42e7e00640 1 -- 172.21.15.49:0/995680096 >> 172.21.15.49:0/995680096 conn(0x7f42e00756c0 msgr2=0x7f42e0077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.806+0000 7f42e7e00640 1 -- 172.21.15.49:0/995680096 shutdown_connections 2024-02-26T06:28:19.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.806+0000 7f42e7e00640 1 -- 172.21.15.49:0/995680096 wait complete. 2024-02-26T06:28:19.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.807+0000 7f42e7e00640 1 Processor -- start 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.807+0000 7f42e7e00640 1 -- start start 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e0107720 0x7f42e019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e01080e0 0x7f42e019e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 0x7f42e01a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e7e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01a2c50 con 0x7f42e0107720 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e7e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01a2dc0 con 0x7f42e010d730 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e7e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01a2f80 con 0x7f42e01080e0 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e4a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e01080e0 0x7f42e019e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e4a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e01080e0 0x7f42e019e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43814/0 (socket says 172.21.15.49:43814) 2024-02-26T06:28:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e4a00640 1 -- 172.21.15.49:0/2545215656 learned_addr learned my addr 172.21.15.49:0/2545215656 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:19.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.808+0000 7f42e5e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 0x7f42e01a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.809+0000 7f42e5400640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e0107720 0x7f42e019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.809+0000 7f42e5e00640 1 -- 172.21.15.49:0/2545215656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e01080e0 msgr2=0x7f42e019e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.809+0000 7f42e5e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e01080e0 0x7f42e019e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.809+0000 7f42e5e00640 1 -- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e0107720 msgr2=0x7f42e019dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.809+0000 7f42e5e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e0107720 0x7f42e019dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.809+0000 7f42e5e00640 1 -- 172.21.15.49:0/2545215656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d4009560 con 0x7f42e010d730 2024-02-26T06:28:19.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.809+0000 7f42e4a00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e01080e0 0x7f42e019e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:19.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.809+0000 7f42e5400640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e0107720 0x7f42e019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:19.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.810+0000 7f42e5e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 0x7f42e01a2650 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009fd0 tx=0x7f42d800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:19.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.810+0000 7f42de000640 1 -- 172.21.15.49:0/2545215656 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d800ce20 con 0x7f42e010d730 2024-02-26T06:28:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.810+0000 7f42de000640 1 -- 172.21.15.49:0/2545215656 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d8005020 con 0x7f42e010d730 2024-02-26T06:28:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.810+0000 7f42de000640 1 -- 172.21.15.49:0/2545215656 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8010690 con 0x7f42e010d730 2024-02-26T06:28:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.811+0000 7f42e7e00640 1 -- 172.21.15.49:0/2545215656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e01a3260 con 0x7f42e010d730 2024-02-26T06:28:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.811+0000 7f42e7e00640 1 -- 172.21.15.49:0/2545215656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e01aaae0 con 0x7f42e010d730 2024-02-26T06:28:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.815+0000 7f42eae2f740 1 Processor -- start 2024-02-26T06:28:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.816+0000 7f42de000640 1 -- 172.21.15.49:0/2545215656 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f42d8004b20 con 0x7f42e010d730 2024-02-26T06:28:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.816+0000 7f42eae2f740 1 -- start start 2024-02-26T06:28:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.817+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.817+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.817+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b22800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.817+0000 7f42eae2f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263ad22d0 con 0x558263b20410 2024-02-26T06:28:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.817+0000 7f42eae2f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263a84e20 con 0x558263b16f10 2024-02-26T06:28:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.817+0000 7f42eae2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263a82350 con 0x558263b17830 2024-02-26T06:28:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.817+0000 7f42e5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.817+0000 7f42e5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43822/0 (socket says 172.21.15.49:43822) 2024-02-26T06:28:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e5400640 1 -- 172.21.15.49:0/2701086364 learned_addr learned my addr 172.21.15.49:0/2701086364 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e4a00640 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b22800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e5400640 1 -- 172.21.15.49:0/2701086364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 msgr2=0x558263b172f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:28:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e5400640 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b172f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e5400640 1 -- 172.21.15.49:0/2701086364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 msgr2=0x558263b22800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e5400640 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b22800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e5400640 1 -- 172.21.15.49:0/2701086364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558263a46c50 con 0x558263b17830 2024-02-26T06:28:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e4a00640 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b22800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42e5400640 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1fed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000c00 tx=0x7f42d402c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46d624660ebe644 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.818+0000 7f42de000640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42c80a2400 0x7f42c80a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.819+0000 7f42de000640 1 -- 172.21.15.49:0/2545215656 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f42d8014070 con 0x7f42e010d730 2024-02-26T06:28:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.819+0000 7f42e4a00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42c80a2400 0x7f42c80a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.819+0000 7f42d2600640 1 -- 172.21.15.49:0/2701086364 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d402dea0 con 0x558263b17830 2024-02-26T06:28:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.819+0000 7f42d2600640 1 -- 172.21.15.49:0/2701086364 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d4004a40 con 0x558263b17830 2024-02-26T06:28:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.819+0000 7f42d2600640 1 -- 172.21.15.49:0/2701086364 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40051f0 con 0x558263b17830 2024-02-26T06:28:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.821+0000 7f42eae2f740 1 -- 172.21.15.49:0/2701086364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 msgr2=0x558263b1fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.821+0000 7f42e4a00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42c80a2400 0x7f42c80a48c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009fd0 tx=0x7f42cc005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.821+0000 7f42eae2f740 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1fed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000c00 tx=0x7f42d402c720 comp rx=0 tx=0).stop 2024-02-26T06:28:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.822+0000 7f42eae2f740 1 -- 172.21.15.49:0/2701086364 shutdown_connections 2024-02-26T06:28:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.822+0000 7f42eae2f740 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b22800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.822+0000 7f42eae2f740 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1fed0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.822+0000 7f42eae2f740 1 --2- 172.21.15.49:0/2701086364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.822+0000 7f42eae2f740 1 -- 172.21.15.49:0/2701086364 >> 172.21.15.49:0/2701086364 conn(0x558263b11640 msgr2=0x558263b11a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.822+0000 7f42eae2f740 1 -- 172.21.15.49:0/2701086364 shutdown_connections 2024-02-26T06:28:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.822+0000 7f42eae2f740 1 -- 172.21.15.49:0/2701086364 wait complete. 2024-02-26T06:28:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.823+0000 7f42eae2f740 1 -- start start 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.823+0000 7f42eae2f740 1 Processor -- start 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b16f10 0x558263b1d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b20410 0x558263b1dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42eae2f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263a7db30 con 0x558263b16f10 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42eae2f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263a22fa0 con 0x558263b20410 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42eae2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263a22c40 con 0x558263b17830 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42eae2f740 10 client.0 _ll_register_callbacks cb 0x55826378ec60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42e4a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42e4a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43832/0 (socket says 172.21.15.49:43832) 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42e4a00640 1 -- 172.21.15.49:0/27382491 learned_addr learned my addr 172.21.15.49:0/27382491 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42e5400640 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b16f10 0x558263b1d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42e5e00640 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b20410 0x558263b1dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42e4a00640 1 -- 172.21.15.49:0/27382491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b20410 msgr2=0x558263b1dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.824+0000 7f42e4a00640 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b20410 0x558263b1dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.825+0000 7f42e4a00640 1 -- 172.21.15.49:0/27382491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b16f10 msgr2=0x558263b1d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.825+0000 7f42e4a00640 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b16f10 0x558263b1d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.825+0000 7f42e4a00640 1 -- 172.21.15.49:0/27382491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e0114ef0 con 0x558263b17830 2024-02-26T06:28:19.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.825+0000 7f42e4a00640 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1d830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f42cc003760 tx=0x7f42cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25dcfd04875b1af0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:19.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.826+0000 7f42c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:19.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.826+0000 7f42c6a00640 1 -- 172.21.15.49:0/27382491 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc011af0 con 0x558263b17830 2024-02-26T06:28:19.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.826+0000 7f42c6a00640 1 -- 172.21.15.49:0/27382491 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f42cc011c50 con 0x558263b17830 2024-02-26T06:28:19.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.827+0000 7f42eae2f740 1 -- 172.21.15.49:0/27382491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55826377d860 con 0x558263b17830 2024-02-26T06:28:19.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.828+0000 7f42c6a00640 1 -- 172.21.15.49:0/27382491 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42cc011e10 con 0x558263b17830 2024-02-26T06:28:19.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.828+0000 7f42eae2f740 10 client.26698 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:19.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.828+0000 7f42eae2f740 1 -- 172.21.15.49:0/27382491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558263a7c1b0 con 0x558263b17830 2024-02-26T06:28:19.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.828+0000 7f42c6a00640 1 -- 172.21.15.49:0/27382491 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55826377d860 con 0x558263b17830 2024-02-26T06:28:19.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.828+0000 7f42c6a00640 1 -- 172.21.15.49:0/27382491 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f42cc033740 con 0x558263b17830 2024-02-26T06:28:19.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.828+0000 7f42eae2f740 10 client.26698 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:19.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.828+0000 7f42eae2f740 10 client.26698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:19.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.828+0000 7f42eae2f740 20 client.26698 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:19.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.829+0000 7f42eae2f740 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558263b41750 0x558263b43b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.829+0000 7f42eae2f740 4 client.26698 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:19.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.829+0000 7f42eae2f740 1 -- 172.21.15.49:0/27382491 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582637ce410 con 0x558263b41750 2024-02-26T06:28:19.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.829+0000 7f42e5400640 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558263b41750 0x558263b43b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.830+0000 7f42e5400640 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558263b41750 0x558263b43b70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=997163aee0cbc686 server_cookie=1ad01f47c02b3ff8 in_seq=0 out_seq=0 2024-02-26T06:28:19.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.830+0000 7f42c6a00640 10 client.26698 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:19.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.830+0000 7f42c6a00640 1 -- 172.21.15.49:0/27382491 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582637ce410 con 0x558263b41750 2024-02-26T06:28:19.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.830+0000 7f42c6a00640 10 client.26698 handle_command_reply: tid=0 2024-02-26T06:28:19.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.831+0000 7f42eae2f740 1 client.26698 shutdown 2024-02-26T06:28:19.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.832+0000 7f42eae2f740 1 -- 172.21.15.49:0/27382491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 msgr2=0x558263b1d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.832+0000 7f42eae2f740 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1d830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f42cc003760 tx=0x7f42cc0046c0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 1 -- 172.21.15.49:0/27382491 shutdown_connections 2024-02-26T06:28:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558263b41750 0x558263b43b70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b20410 0x558263b1dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b17830 0x558263b1d830 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 1 --2- 172.21.15.49:0/27382491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b16f10 0x558263b1d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42c6a00640 0 client.26698 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 1 -- 172.21.15.49:0/27382491 >> 172.21.15.49:0/27382491 conn(0x558263b11640 msgr2=0x558263b13390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:19.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 1 -- 172.21.15.49:0/27382491 shutdown_connections 2024-02-26T06:28:19.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 1 -- 172.21.15.49:0/27382491 wait complete. 2024-02-26T06:28:19.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.833+0000 7f42eae2f740 20 client.26698 trim_cache size 0 max 16384 2024-02-26T06:28:19.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.862+0000 7f42eae2f740 1 Processor -- start 2024-02-26T06:28:19.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.862+0000 7f42eae2f740 1 -- start start 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.862+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b16f10 0x558263b1c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.862+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b17830 0x558263b1db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b10540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42eae2f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582637c60c0 con 0x558263b20410 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42eae2f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263b16b70 con 0x558263b17830 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42eae2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263b0ca40 con 0x558263b16f10 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b16f10 0x558263b1c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b16f10 0x558263b1c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43834/0 (socket says 172.21.15.49:43834) 2024-02-26T06:28:19.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 -- 172.21.15.49:0/2531780747 learned_addr learned my addr 172.21.15.49:0/2531780747 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:19.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b17830 0x558263b1db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 -- 172.21.15.49:0/2531780747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b17830 msgr2=0x558263b1db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:19.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 --2- 172.21.15.49:0/2531780747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b17830 0x558263b1db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 -- 172.21.15.49:0/2531780747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 msgr2=0x558263b10540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:19.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5e00640 1 --2- 172.21.15.49:0/2531780747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b10540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 --2- 172.21.15.49:0/2531780747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b10540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 -- 172.21.15.49:0/2531780747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558263a18a60 con 0x558263b16f10 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.863+0000 7f42e5400640 1 --2- 172.21.15.49:0/2531780747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b16f10 0x558263b1c470 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x558263b169e0 tx=0x7f42d4040040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d9716e53e65e86 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.864+0000 7f42d1c00640 1 -- 172.21.15.49:0/2531780747 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d402dea0 con 0x558263b16f10 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.864+0000 7f42d1c00640 1 -- 172.21.15.49:0/2531780747 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d402cc50 con 0x558263b16f10 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.864+0000 7f42d1c00640 1 -- 172.21.15.49:0/2531780747 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4051610 con 0x558263b16f10 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 -- 172.21.15.49:0/2531780747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b16f10 msgr2=0x558263b1c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 --2- 172.21.15.49:0/2531780747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b16f10 0x558263b1c470 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x558263b169e0 tx=0x7f42d4040040 comp rx=0 tx=0).stop 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 -- 172.21.15.49:0/2531780747 shutdown_connections 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 --2- 172.21.15.49:0/2531780747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b17830 0x558263b1db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 --2- 172.21.15.49:0/2531780747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b20410 0x558263b10540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 --2- 172.21.15.49:0/2531780747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b16f10 0x558263b1c470 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 -- 172.21.15.49:0/2531780747 >> 172.21.15.49:0/2531780747 conn(0x558263b11640 msgr2=0x558263b0fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:19.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 -- 172.21.15.49:0/2531780747 shutdown_connections 2024-02-26T06:28:19.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.865+0000 7f42eae2f740 1 -- 172.21.15.49:0/2531780747 wait complete. 2024-02-26T06:28:19.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.866+0000 7f42eae2f740 1 -- start start 2024-02-26T06:28:19.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.866+0000 7f42eae2f740 1 Processor -- start 2024-02-26T06:28:19.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b38800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b17830 0x558263b36ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42eae2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b20410 0x558263b372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42eae2f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263b379e0 con 0x558263b17830 2024-02-26T06:28:19.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42eae2f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263b91bf0 con 0x558263b16f10 2024-02-26T06:28:19.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42eae2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558263b91db0 con 0x558263b20410 2024-02-26T06:28:19.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42eae2f740 10 client.0 _ll_register_callbacks cb 0x7f42cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:19.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42e5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b38800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42e5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b38800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45020/0 (socket says 172.21.15.49:45020) 2024-02-26T06:28:19.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.867+0000 7f42e5400640 1 -- 172.21.15.49:0/1567742674 learned_addr learned my addr 172.21.15.49:0/1567742674 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:19.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e4a00640 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b17830 0x558263b36ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e5e00640 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b20410 0x558263b372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e5400640 1 -- 172.21.15.49:0/1567742674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b20410 msgr2=0x558263b372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:19.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e5400640 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b20410 0x558263b372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e5400640 1 -- 172.21.15.49:0/1567742674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b17830 msgr2=0x558263b36ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:19.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e5400640 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b17830 0x558263b36ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e5400640 1 -- 172.21.15.49:0/1567742674 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558263a46c50 con 0x558263b16f10 2024-02-26T06:28:19.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e4a00640 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b17830 0x558263b36ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:19.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42e5400640 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b38800 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000c00 tx=0x7f42d40047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ffc174f1ed2203 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:19.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.868+0000 7f42c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:19.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.869+0000 7f42c6a00640 1 -- 172.21.15.49:0/1567742674 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4004240 con 0x558263b16f10 2024-02-26T06:28:19.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.869+0000 7f42c6a00640 1 -- 172.21.15.49:0/1567742674 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f42d404e070 con 0x558263b16f10 2024-02-26T06:28:19.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.869+0000 7f42eae2f740 1 -- 172.21.15.49:0/1567742674 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42d4004a40 con 0x558263b16f10 2024-02-26T06:28:19.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.869+0000 7f42c6a00640 1 -- 172.21.15.49:0/1567742674 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42d402cad0 con 0x558263b16f10 2024-02-26T06:28:19.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.869+0000 7f42eae2f740 10 client.26690 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:19.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.869+0000 7f42eae2f740 1 -- 172.21.15.49:0/1567742674 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558263a3e670 con 0x558263b16f10 2024-02-26T06:28:19.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42c6a00640 1 -- 172.21.15.49:0/1567742674 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d4004a40 con 0x558263b16f10 2024-02-26T06:28:19.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42c6a00640 1 -- 172.21.15.49:0/1567742674 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f42d403c040 con 0x558263b16f10 2024-02-26T06:28:19.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42eae2f740 10 client.26690 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:19.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42eae2f740 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:19.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42eae2f740 20 client.26690 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:19.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42eae2f740 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558263b93080 0x558263b93490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:19.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42eae2f740 4 client.26690 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:28:19.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42eae2f740 1 -- 172.21.15.49:0/1567742674 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5582637ba7b0 con 0x558263b93080 2024-02-26T06:28:19.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.870+0000 7f42e4a00640 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558263b93080 0x558263b93490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:19.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.872+0000 7f42e4a00640 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558263b93080 0x558263b93490 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d487b9b1e3452df2 server_cookie=70854ba9bc415728 in_seq=0 out_seq=0 2024-02-26T06:28:19.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.872+0000 7f42c6a00640 10 client.26690 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.873+0000 7f42c6a00640 1 -- 172.21.15.49:0/1567742674 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5582637ba7b0 con 0x558263b93080 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.873+0000 7f42c6a00640 10 client.26690 handle_command_reply: tid=0 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.873+0000 7f42eae2f740 1 client.26690 shutdown 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 -- 172.21.15.49:0/1567742674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 msgr2=0x558263b38800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b38800 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000c00 tx=0x7f42d40047d0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 -- 172.21.15.49:0/1567742674 shutdown_connections 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558263b93080 0x558263b93490 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558263b20410 0x558263b372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558263b17830 0x558263b36ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 --2- 172.21.15.49:0/1567742674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558263b16f10 0x558263b38800 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 -- 172.21.15.49:0/1567742674 >> 172.21.15.49:0/1567742674 conn(0x558263b11640 msgr2=0x558263b1c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42c6a00640 0 client.26690 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 -- 172.21.15.49:0/1567742674 shutdown_connections 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 1 -- 172.21.15.49:0/1567742674 wait complete. 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.874+0000 7f42eae2f740 20 client.26690 trim_cache size 0 max 16384 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:28:19.879 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "aa8afd5a-4467-4146-b23f-2149d065d67f", 2024-02-26T06:28:19.880 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:28:19.880 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:28:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 -- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42c80a2400 msgr2=0x7f42c80a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42c80a2400 0x7f42c80a48c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009fd0 tx=0x7f42cc005f90 comp rx=0 tx=0).stop 2024-02-26T06:28:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 -- 172.21.15.49:0/2545215656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 msgr2=0x7f42e01a2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 0x7f42e01a2650 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009fd0 tx=0x7f42d800ef10 comp rx=0 tx=0).stop 2024-02-26T06:28:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 -- 172.21.15.49:0/2545215656 shutdown_connections 2024-02-26T06:28:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42c80a2400 0x7f42c80a48c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42e010d730 0x7f42e01a2650 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e01080e0 0x7f42e019e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 --2- 172.21.15.49:0/2545215656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e0107720 0x7f42e019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:19.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 -- 172.21.15.49:0/2545215656 >> 172.21.15.49:0/2545215656 conn(0x7f42e00756c0 msgr2=0x7f42e0075e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:19.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 -- 172.21.15.49:0/2545215656 shutdown_connections 2024-02-26T06:28:19.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:19.875+0000 7f42e7e00640 1 -- 172.21.15.49:0/2545215656 wait complete. 2024-02-26T06:28:19.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:28:19.904 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:28:20.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.214+0000 7f8b0a200640 1 -- 172.21.15.49:0/1321730134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b040756b0 msgr2=0x7f8b04075a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.214+0000 7f8b0a200640 1 --2- 172.21.15.49:0/1321730134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b040756b0 0x7f8b04075a90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009930 tx=0x7f8af402bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:20.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.214+0000 7f8b0a200640 1 -- 172.21.15.49:0/1321730134 shutdown_connections 2024-02-26T06:28:20.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.214+0000 7f8b0a200640 1 --2- 172.21.15.49:0/1321730134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b04111fc0 0x7f8b041143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.214+0000 7f8b0a200640 1 --2- 172.21.15.49:0/1321730134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b04075fd0 0x7f8b04111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.214+0000 7f8b0a200640 1 --2- 172.21.15.49:0/1321730134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b040756b0 0x7f8b04075a90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.214+0000 7f8b0a200640 1 -- 172.21.15.49:0/1321730134 >> 172.21.15.49:0/1321730134 conn(0x7f8b040fedf0 msgr2=0x7f8b04101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:20.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.215+0000 7f8b0a200640 1 -- 172.21.15.49:0/1321730134 shutdown_connections 2024-02-26T06:28:20.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.215+0000 7f8b0a200640 1 -- 172.21.15.49:0/1321730134 wait complete. 2024-02-26T06:28:20.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.216+0000 7f8b0a200640 1 Processor -- start 2024-02-26T06:28:20.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.217+0000 7f8b0a200640 1 -- start start 2024-02-26T06:28:20.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.217+0000 7f8b0a200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b04075fd0 0x7f8b0419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.217+0000 7f8b0a200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b04111fc0 0x7f8b0419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.217+0000 7f8b0a200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b041a2960 0x7f8b041a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.217+0000 7f8b0a200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b041a3350 con 0x7f8b04111fc0 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.217+0000 7f8b0a200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b041a34c0 con 0x7f8b041a2960 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.217+0000 7f8b0a200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b041a3630 con 0x7f8b04075fd0 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b04075fd0 0x7f8b0419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b04075fd0 0x7f8b0419e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34440/0 (socket says 172.21.15.49:34440) 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03400640 1 -- 172.21.15.49:0/525583308 learned_addr learned my addr 172.21.15.49:0/525583308 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03e00640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b041a2960 0x7f8b041a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b02a00640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b04111fc0 0x7f8b0419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03400640 1 -- 172.21.15.49:0/525583308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b041a2960 msgr2=0x7f8b041a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03400640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b041a2960 0x7f8b041a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03400640 1 -- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b04111fc0 msgr2=0x7f8b0419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03400640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b04111fc0 0x7f8b0419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b03400640 1 -- 172.21.15.49:0/525583308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4009560 con 0x7f8b04075fd0 2024-02-26T06:28:20.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.218+0000 7f8b02a00640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b04111fc0 0x7f8b0419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:20.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.219+0000 7f8b03400640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b04075fd0 0x7f8b0419e0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009900 tx=0x7f8af4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:20.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.219+0000 7f8afbe00640 1 -- 172.21.15.49:0/525583308 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4004440 con 0x7f8b04075fd0 2024-02-26T06:28:20.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.219+0000 7f8b0a200640 1 -- 172.21.15.49:0/525583308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b041aac10 con 0x7f8b04075fd0 2024-02-26T06:28:20.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.219+0000 7f8afbe00640 1 -- 172.21.15.49:0/525583308 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8af40052d0 con 0x7f8b04075fd0 2024-02-26T06:28:20.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.219+0000 7f8afbe00640 1 -- 172.21.15.49:0/525583308 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4003d10 con 0x7f8b04075fd0 2024-02-26T06:28:20.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.219+0000 7f8b0a200640 1 -- 172.21.15.49:0/525583308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b041aafe0 con 0x7f8b04075fd0 2024-02-26T06:28:20.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.224+0000 7f8afbe00640 1 -- 172.21.15.49:0/525583308 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8af4005440 con 0x7f8b04075fd0 2024-02-26T06:28:20.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.226+0000 7f8b0cff2740 1 Processor -- start 2024-02-26T06:28:20.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.226+0000 7f8afbe00640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8ae80a2230 0x7f8ae80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.226+0000 7f8b0cff2740 1 -- start start 2024-02-26T06:28:20.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.226+0000 7f8afbe00640 1 -- 172.21.15.49:0/525583308 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f8af40f0650 con 0x7f8b04075fd0 2024-02-26T06:28:20.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b03e00640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8ae80a2230 0x7f8ae80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1df70 0x55e60ba1ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1d290 0x55e60ba1d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 0x55e60ba289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60b98c120 con 0x55e60ba26600 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60b989650 con 0x55e60ba1df70 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60b984e30 con 0x55e60ba1d290 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b03400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1df70 0x55e60ba1ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b03400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1df70 0x55e60ba1ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:59270/0 (socket says 172.21.15.49:59270) 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b03400640 1 -- 172.21.15.49:0/1067220280 learned_addr learned my addr 172.21.15.49:0/1067220280 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.227+0000 7f8b02a00640 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1d290 0x55e60ba1d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b03e00640 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 0x55e60ba289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b03400640 1 -- 172.21.15.49:0/1067220280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1d290 msgr2=0x55e60ba1d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:20.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b03400640 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1d290 0x55e60ba1d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b03400640 1 -- 172.21.15.49:0/1067220280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 msgr2=0x55e60ba289f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:20.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b03400640 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 0x55e60ba289f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b03400640 1 -- 172.21.15.49:0/1067220280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e60b93e030 con 0x55e60ba1df70 2024-02-26T06:28:20.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b02a00640 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1d290 0x55e60ba1d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:20.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b03e00640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8ae80a2230 0x7f8ae80a46f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8b0406a630 tx=0x7f8afc010740 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:20.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.228+0000 7f8b03400640 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1df70 0x55e60ba1ccc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8af40c5370 tx=0x7f8af40bca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86dbf6488a9ea055 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:20.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.229+0000 7f8aefe00640 1 -- 172.21.15.49:0/1067220280 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40c1250 con 0x55e60ba1df70 2024-02-26T06:28:20.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.229+0000 7f8aefe00640 1 -- 172.21.15.49:0/1067220280 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8af40c58a0 con 0x55e60ba1df70 2024-02-26T06:28:20.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.229+0000 7f8aefe00640 1 -- 172.21.15.49:0/1067220280 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40c5410 con 0x55e60ba1df70 2024-02-26T06:28:20.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.230+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1067220280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1df70 msgr2=0x55e60ba1ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.231+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1df70 0x55e60ba1ccc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8af40c5370 tx=0x7f8af40bca60 comp rx=0 tx=0).stop 2024-02-26T06:28:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.231+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1067220280 shutdown_connections 2024-02-26T06:28:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.231+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 0x55e60ba289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.231+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1d290 0x55e60ba1d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.231+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1067220280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1df70 0x55e60ba1ccc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.231+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1067220280 >> 172.21.15.49:0/1067220280 conn(0x55e60b9e4570 msgr2=0x55e60ba17ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:20.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.232+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1067220280 shutdown_connections 2024-02-26T06:28:20.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.232+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1067220280 wait complete. 2024-02-26T06:28:20.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.232+0000 7f8b0cff2740 1 -- start start 2024-02-26T06:28:20.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.233+0000 7f8b0cff2740 1 Processor -- start 2024-02-26T06:28:20.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.233+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.233+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1df70 0x55e60ba23b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.233+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 0x55e60ba24070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.233+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60b929e90 con 0x55e60ba26600 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.233+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60b929b30 con 0x55e60ba1d290 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.233+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60b6f31c0 con 0x55e60ba1df70 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.233+0000 7f8b0cff2740 10 client.0 _ll_register_callbacks cb 0x55e60b93d930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 0x55e60ba24070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba235f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:59274/0 (socket says 172.21.15.49:59274) 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 -- 172.21.15.49:0/4222855564 learned_addr learned my addr 172.21.15.49:0/4222855564 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 -- 172.21.15.49:0/4222855564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1df70 msgr2=0x55e60ba23b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:28:20.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1df70 0x55e60ba23b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 -- 172.21.15.49:0/4222855564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 msgr2=0x55e60ba24070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:20.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 0x55e60ba24070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 -- 172.21.15.49:0/4222855564 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b041aafe0 con 0x55e60ba1d290 2024-02-26T06:28:20.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8b03400640 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba235f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8af4004650 tx=0x7f8af4005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee47a0ed8825d45 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:20.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.234+0000 7f8aecc00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:20.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.235+0000 7f8aecc00640 1 -- 172.21.15.49:0/4222855564 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40c1250 con 0x55e60ba1d290 2024-02-26T06:28:20.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.235+0000 7f8aecc00640 1 -- 172.21.15.49:0/4222855564 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f8af40c5410 con 0x55e60ba1d290 2024-02-26T06:28:20.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.235+0000 7f8b0cff2740 1 -- 172.21.15.49:0/4222855564 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e60b6847e0 con 0x55e60ba1d290 2024-02-26T06:28:20.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.235+0000 7f8aecc00640 1 -- 172.21.15.49:0/4222855564 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8af40c55d0 con 0x55e60ba1d290 2024-02-26T06:28:20.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.235+0000 7f8b0cff2740 10 client.26702 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:20.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.236+0000 7f8b0cff2740 1 -- 172.21.15.49:0/4222855564 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e60b9834b0 con 0x55e60ba1d290 2024-02-26T06:28:20.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.236+0000 7f8aecc00640 1 -- 172.21.15.49:0/4222855564 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e60b6847e0 con 0x55e60ba1d290 2024-02-26T06:28:20.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.236+0000 7f8aecc00640 1 -- 172.21.15.49:0/4222855564 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8af404f020 con 0x55e60ba1d290 2024-02-26T06:28:20.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.237+0000 7f8b0cff2740 10 client.26702 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.237+0000 7f8b0cff2740 10 client.26702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.237+0000 7f8b0cff2740 20 client.26702 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.237+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e60ba35fd0 0x55e60ba383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.237+0000 7f8b0cff2740 4 client.26702 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.237+0000 7f8b0cff2740 1 -- 172.21.15.49:0/4222855564 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e60b6c1720 con 0x55e60ba35fd0 2024-02-26T06:28:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.237+0000 7f8b02a00640 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e60ba35fd0 0x55e60ba383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.238+0000 7f8b02a00640 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e60ba35fd0 0x55e60ba383c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c6163da8a64b73 server_cookie=dcce240ad00b7f29 in_seq=0 out_seq=0 2024-02-26T06:28:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.238+0000 7f8aecc00640 10 client.26702 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:20.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.239+0000 7f8aecc00640 1 -- 172.21.15.49:0/4222855564 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e60b6c1720 con 0x55e60ba35fd0 2024-02-26T06:28:20.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.239+0000 7f8aecc00640 10 client.26702 handle_command_reply: tid=0 2024-02-26T06:28:20.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.239+0000 7f8b0cff2740 1 client.26702 shutdown 2024-02-26T06:28:20.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.239+0000 7f8b0cff2740 1 -- 172.21.15.49:0/4222855564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 msgr2=0x55e60ba235f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba235f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8af4004650 tx=0x7f8af4005b70 comp rx=0 tx=0).stop 2024-02-26T06:28:20.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 -- 172.21.15.49:0/4222855564 shutdown_connections 2024-02-26T06:28:20.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e60ba35fd0 0x55e60ba383c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba26600 0x55e60ba24070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba1df70 0x55e60ba23b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8aecc00640 0 client.26702 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/4222855564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba235f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 -- 172.21.15.49:0/4222855564 >> 172.21.15.49:0/4222855564 conn(0x55e60b9e4570 msgr2=0x55e60ba19a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 -- 172.21.15.49:0/4222855564 shutdown_connections 2024-02-26T06:28:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.240+0000 7f8b0cff2740 1 -- 172.21.15.49:0/4222855564 wait complete. 2024-02-26T06:28:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.241+0000 7f8b0cff2740 20 client.26702 trim_cache size 0 max 16384 2024-02-26T06:28:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.267+0000 7f8b0cff2740 1 Processor -- start 2024-02-26T06:28:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.267+0000 7f8b0cff2740 1 -- start start 2024-02-26T06:28:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba25d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 0x55e60ba225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60b6847e0 con 0x55e60ba1df70 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60ba1e350 con 0x55e60ba1d290 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60ba1c510 con 0x55e60ba26600 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba25d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba25d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:59282/0 (socket says 172.21.15.49:59282) 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03400640 1 -- 172.21.15.49:0/3070216380 learned_addr learned my addr 172.21.15.49:0/3070216380 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b02a00640 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 0x55e60ba225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03e00640 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03400640 1 -- 172.21.15.49:0/3070216380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 msgr2=0x55e60ba23e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03400640 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba23e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03400640 1 -- 172.21.15.49:0/3070216380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 msgr2=0x55e60ba225d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:20.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03400640 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 0x55e60ba225d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.268+0000 7f8b03400640 1 -- 172.21.15.49:0/3070216380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e60b91f920 con 0x55e60ba1d290 2024-02-26T06:28:20.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.269+0000 7f8b03e00640 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba23e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:20.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.269+0000 7f8b02a00640 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 0x55e60ba225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:20.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.269+0000 7f8b03400640 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba25d20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8af4005ab0 tx=0x7f8af40ede80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569c0409df2ba854 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:20.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.269+0000 7f8aef400640 1 -- 172.21.15.49:0/3070216380 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40b93a0 con 0x55e60ba1d290 2024-02-26T06:28:20.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.269+0000 7f8aef400640 1 -- 172.21.15.49:0/3070216380 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8af40c1250 con 0x55e60ba1d290 2024-02-26T06:28:20.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.269+0000 7f8aef400640 1 -- 172.21.15.49:0/3070216380 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40b91f0 con 0x55e60ba1d290 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 -- 172.21.15.49:0/3070216380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 msgr2=0x55e60ba25d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba25d20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8af4005ab0 tx=0x7f8af40ede80 comp rx=0 tx=0).stop 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 -- 172.21.15.49:0/3070216380 shutdown_connections 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba23e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 0x55e60ba225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/3070216380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba25d20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 -- 172.21.15.49:0/3070216380 >> 172.21.15.49:0/3070216380 conn(0x55e60b9e4570 msgr2=0x55e60ba21ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 -- 172.21.15.49:0/3070216380 shutdown_connections 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.270+0000 7f8b0cff2740 1 -- 172.21.15.49:0/3070216380 wait complete. 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.271+0000 7f8b0cff2740 1 -- start start 2024-02-26T06:28:20.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.271+0000 7f8b0cff2740 1 Processor -- start 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.272+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba2d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.272+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 0x55e60ba29880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b0cff2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba29dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60ba2a300 con 0x55e60ba1df70 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60ba2a470 con 0x55e60ba1d290 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b0cff2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60ba2a630 con 0x55e60ba26600 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b0cff2740 10 client.0 _ll_register_callbacks cb 0x7f8af0009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba29dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba29dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34468/0 (socket says 172.21.15.49:34468) 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03e00640 1 -- 172.21.15.49:0/1469498884 learned_addr learned my addr 172.21.15.49:0/1469498884 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03400640 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba2d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03e00640 1 -- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 msgr2=0x55e60ba2d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:20.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03e00640 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba2d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03e00640 1 -- 172.21.15.49:0/1469498884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 msgr2=0x55e60ba29880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:20.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03e00640 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 0x55e60ba29880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03e00640 1 -- 172.21.15.49:0/1469498884 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b041aafe0 con 0x55e60ba26600 2024-02-26T06:28:20.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.273+0000 7f8b03400640 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba2d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:20.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.274+0000 7f8b03e00640 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba29dc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8afc013de0 tx=0x7f8afc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3569e772a23788b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:20.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.274+0000 7f8aecc00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:20.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.274+0000 7f8aecc00640 1 -- 172.21.15.49:0/1469498884 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc025070 con 0x55e60ba26600 2024-02-26T06:28:20.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.274+0000 7f8aecc00640 1 -- 172.21.15.49:0/1469498884 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f8afc019040 con 0x55e60ba26600 2024-02-26T06:28:20.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.274+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1469498884 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8af40c58a0 con 0x55e60ba26600 2024-02-26T06:28:20.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.275+0000 7f8aecc00640 1 -- 172.21.15.49:0/1469498884 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8afc003970 con 0x55e60ba26600 2024-02-26T06:28:20.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.275+0000 7f8b0cff2740 10 client.26716 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:20.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.278+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1469498884 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e60b945400 con 0x55e60ba26600 2024-02-26T06:28:20.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.279+0000 7f8aecc00640 1 -- 172.21.15.49:0/1469498884 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8af40c58a0 con 0x55e60ba26600 2024-02-26T06:28:20.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.279+0000 7f8aecc00640 1 -- 172.21.15.49:0/1469498884 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8afc0115f0 con 0x55e60ba26600 2024-02-26T06:28:20.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.279+0000 7f8b0cff2740 10 client.26716 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:20.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.279+0000 7f8b0cff2740 10 client.26716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:20.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.279+0000 7f8b0cff2740 20 client.26716 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:20.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.279+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e60ba8d960 0x55e60ba8fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:20.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.280+0000 7f8b0cff2740 4 client.26716 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:28:20.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.280+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1469498884 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e60b710890 con 0x55e60ba8d960 2024-02-26T06:28:20.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.280+0000 7f8b03400640 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e60ba8d960 0x55e60ba8fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:20.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8b03400640 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e60ba8d960 0x55e60ba8fd80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fcc9347809b475 server_cookie=593ec57d6b35b0eb in_seq=0 out_seq=0 2024-02-26T06:28:20.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8aecc00640 10 client.26716 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:20.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8aecc00640 1 -- 172.21.15.49:0/1469498884 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e60b710890 con 0x55e60ba8d960 2024-02-26T06:28:20.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8aecc00640 10 client.26716 handle_command_reply: tid=0 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8b0cff2740 1 client.26716 shutdown 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1469498884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 msgr2=0x55e60ba29dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba29dc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8afc013de0 tx=0x7f8afc004ab0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1469498884 shutdown_connections 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.281+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e60ba8d960 0x55e60ba8fd80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.282+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e60ba26600 0x55e60ba29dc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.282+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e60ba1df70 0x55e60ba29880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.282+0000 7f8b0cff2740 1 --2- 172.21.15.49:0/1469498884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e60ba1d290 0x55e60ba2d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.282+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1469498884 >> 172.21.15.49:0/1469498884 conn(0x55e60b9e4570 msgr2=0x55e60ba18fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.282+0000 7f8aecc00640 0 client.26716 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.282+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1469498884 shutdown_connections 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.282+0000 7f8b0cff2740 1 -- 172.21.15.49:0/1469498884 wait complete. 2024-02-26T06:28:20.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.282+0000 7f8b0cff2740 20 client.26716 trim_cache size 0 max 16384 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout: "aa8afd5a-4467-4146-b23f-2149d065d67f": { 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "aa8afd5a-4467-4146-b23f-2149d065d67f", 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:28:20.284 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:28:20.285 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:28:20.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 -- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8ae80a2230 msgr2=0x7f8ae80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8ae80a2230 0x7f8ae80a46f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8b0406a630 tx=0x7f8afc010740 comp rx=0 tx=0).stop 2024-02-26T06:28:20.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 -- 172.21.15.49:0/525583308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b04075fd0 msgr2=0x7f8b0419e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b04075fd0 0x7f8b0419e0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009900 tx=0x7f8af4004040 comp rx=0 tx=0).stop 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 -- 172.21.15.49:0/525583308 shutdown_connections 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f8ae80a2230 0x7f8ae80a46f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b041a2960 0x7f8b041a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b04111fc0 0x7f8b0419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 --2- 172.21.15.49:0/525583308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b04075fd0 0x7f8b0419e0a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.284+0000 7f8b0a200640 1 -- 172.21.15.49:0/525583308 >> 172.21.15.49:0/525583308 conn(0x7f8b040fedf0 msgr2=0x7f8b041137c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.285+0000 7f8b0a200640 1 -- 172.21.15.49:0/525583308 shutdown_connections 2024-02-26T06:28:20.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:20.285+0000 7f8b0a200640 1 -- 172.21.15.49:0/525583308 wait complete. 2024-02-26T06:28:20.309 INFO:tasks.cephfs.filesystem:scrub status for tag:aa8afd5a-4467-4146-b23f-2149d065d67f - {'path': '/', 'tag': 'aa8afd5a-4467-4146-b23f-2149d065d67f', 'options': 'recursive,force'} 2024-02-26T06:28:21.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: pgmap v151: 97 pgs: 97 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 39 MiB/s wr, 1.71k op/s 2024-02-26T06:28:21.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:28:21.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:21.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:20 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: pgmap v151: 97 pgs: 97 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 39 MiB/s wr, 1.71k op/s 2024-02-26T06:28:21.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:28:21.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:20 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: pgmap v151: 97 pgs: 97 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 39 MiB/s wr, 1.71k op/s 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:21.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:21.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:21.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:21.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:20 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 270.0 s, 1763.5 tps, lat 57.769 ms stddev 72.998 2024-02-26T06:28:23.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:22 smithi080 ceph-mon[36206]: pgmap v152: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 MiB/s wr, 1.83k op/s 2024-02-26T06:28:23.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:22 smithi115 ceph-mon[36426]: pgmap v152: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 MiB/s wr, 1.83k op/s 2024-02-26T06:28:23.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:22 smithi049 ceph-mon[31696]: pgmap v152: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 MiB/s wr, 1.83k op/s 2024-02-26T06:28:25.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:24 smithi080 ceph-mon[36206]: pgmap v153: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 MiB/s wr, 1.77k op/s 2024-02-26T06:28:25.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:24 smithi115 ceph-mon[36426]: pgmap v153: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 MiB/s wr, 1.77k op/s 2024-02-26T06:28:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:24 smithi049 ceph-mon[31696]: pgmap v153: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 MiB/s wr, 1.77k op/s 2024-02-26T06:28:26.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:25 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:28:26.079 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:25 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:28:26.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:25 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:28:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 275.0 s, 2166.5 tps, lat 46.073 ms stddev 60.113 2024-02-26T06:28:27.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:26 smithi080 ceph-mon[36206]: pgmap v154: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 58 MiB/s wr, 2.14k op/s 2024-02-26T06:28:27.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:26 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:28:27.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:26 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:28:27.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:26 smithi115 ceph-mon[36426]: pgmap v154: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 58 MiB/s wr, 2.14k op/s 2024-02-26T06:28:27.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:26 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:28:27.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:26 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:28:27.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:26 smithi049 ceph-mon[31696]: pgmap v154: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 58 MiB/s wr, 2.14k op/s 2024-02-26T06:28:27.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:26 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:28:27.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:26 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:28:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:28 smithi080 ceph-mon[36206]: pgmap v155: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 27 MiB/s wr, 868 op/s 2024-02-26T06:28:29.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:28 smithi115 ceph-mon[36426]: pgmap v155: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 27 MiB/s wr, 868 op/s 2024-02-26T06:28:29.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:28 smithi049 ceph-mon[31696]: pgmap v155: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 27 MiB/s wr, 868 op/s 2024-02-26T06:28:31.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:30 smithi080 ceph-mon[36206]: pgmap v156: 97 pgs: 97 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 36 MiB/s wr, 1.09k op/s 2024-02-26T06:28:31.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:30 smithi115 ceph-mon[36426]: pgmap v156: 97 pgs: 97 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 36 MiB/s wr, 1.09k op/s 2024-02-26T06:28:31.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:30 smithi049 ceph-mon[31696]: pgmap v156: 97 pgs: 97 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 36 MiB/s wr, 1.09k op/s 2024-02-26T06:28:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 280.0 s, 2277.8 tps, lat 43.940 ms stddev 58.955 2024-02-26T06:28:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.928+0000 7f7c7c000640 1 -- 172.21.15.49:0/1878858154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c7410b870 msgr2=0x7f7c74108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.928+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1878858154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c7410b870 0x7f7c74108180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009900 tx=0x7f7c6802bcd0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.930+0000 7f7c7c000640 1 -- 172.21.15.49:0/1878858154 shutdown_connections 2024-02-26T06:28:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.930+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1878858154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c74116180 0x7f7c741185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.930+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1878858154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c741086c0 0x7f7c74108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.930+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1878858154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c7410b870 0x7f7c74108180 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.930+0000 7f7c7c000640 1 -- 172.21.15.49:0/1878858154 >> 172.21.15.49:0/1878858154 conn(0x7f7c7406c6b0 msgr2=0x7f7c7406cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.930+0000 7f7c7c000640 1 -- 172.21.15.49:0/1878858154 shutdown_connections 2024-02-26T06:28:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.930+0000 7f7c7c000640 1 -- 172.21.15.49:0/1878858154 wait complete. 2024-02-26T06:28:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.931+0000 7f7c7c000640 1 Processor -- start 2024-02-26T06:28:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.932+0000 7f7c7c000640 1 -- start start 2024-02-26T06:28:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7c000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c741086c0 0x7f7c74115820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7c000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c7410b870 0x7f7c74113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7c000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c74116180 0x7f7c74113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7c000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74114420 con 0x7f7c741086c0 2024-02-26T06:28:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7c000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74114590 con 0x7f7c74116180 2024-02-26T06:28:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7c000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74114750 con 0x7f7c7410b870 2024-02-26T06:28:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c74116180 0x7f7c74113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c74116180 0x7f7c74113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38038/0 (socket says 172.21.15.49:38038) 2024-02-26T06:28:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c7a000640 1 -- 172.21.15.49:0/1170634992 learned_addr learned my addr 172.21.15.49:0/1170634992 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.933+0000 7f7c78c00640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c7410b870 0x7f7c74113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.934+0000 7f7c78c00640 1 -- 172.21.15.49:0/1170634992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c74116180 msgr2=0x7f7c74113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.934+0000 7f7c78c00640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c74116180 0x7f7c74113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.934+0000 7f7c78c00640 1 -- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c741086c0 msgr2=0x7f7c74115820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.934+0000 7f7c78c00640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c741086c0 0x7f7c74115820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.934+0000 7f7c78c00640 1 -- 172.21.15.49:0/1170634992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60009490 con 0x7f7c7410b870 2024-02-26T06:28:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.935+0000 7f7c78c00640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c7410b870 0x7f7c74113970 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000e9c0 tx=0x7f7c6000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.935+0000 7f7c72000640 1 -- 172.21.15.49:0/1170634992 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000cd80 con 0x7f7c7410b870 2024-02-26T06:28:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.935+0000 7f7c72000640 1 -- 172.21.15.49:0/1170634992 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c60010910 con 0x7f7c7410b870 2024-02-26T06:28:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.935+0000 7f7c72000640 1 -- 172.21.15.49:0/1170634992 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60018850 con 0x7f7c7410b870 2024-02-26T06:28:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.935+0000 7f7c7c000640 1 -- 172.21.15.49:0/1170634992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c68009560 con 0x7f7c7410b870 2024-02-26T06:28:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.936+0000 7f7c7c000640 1 -- 172.21.15.49:0/1170634992 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741b3080 con 0x7f7c7410b870 2024-02-26T06:28:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.942+0000 7f7c72000640 1 -- 172.21.15.49:0/1170634992 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7c60010a80 con 0x7f7c7410b870 2024-02-26T06:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.944+0000 7f7c72000640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7c5c0a2430 0x7f7c5c0a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.944+0000 7f7c72000640 1 -- 172.21.15.49:0/1170634992 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7c60014070 con 0x7f7c7410b870 2024-02-26T06:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.945+0000 7f7c7ee3c740 1 Processor -- start 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.945+0000 7f7c7ee3c740 1 -- start start 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.945+0000 7f7c79600640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7c5c0a2430 0x7f7c5c0a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.945+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 0x55b9ac9f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9aca056f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac9b55f0 con 0x55b9ac9f9d80 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac967890 con 0x55b9aca03300 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac964dc0 con 0x55b9ac9faaf0 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c78c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c78c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37390/0 (socket says 172.21.15.49:37390) 2024-02-26T06:28:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c78c00640 1 -- 172.21.15.49:0/2097174098 learned_addr learned my addr 172.21.15.49:0/2097174098 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.946+0000 7f7c79600640 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 0x55b9ac9f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.947+0000 7f7c7a000640 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9aca056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.947+0000 7f7c79600640 1 -- 172.21.15.49:0/2097174098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 msgr2=0x55b9aca056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.947+0000 7f7c79600640 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9aca056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.947+0000 7f7c79600640 1 -- 172.21.15.49:0/2097174098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 msgr2=0x55b9ac9fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.947+0000 7f7c79600640 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.947+0000 7f7c79600640 1 -- 172.21.15.49:0/2097174098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ac8e1ad0 con 0x55b9ac9faaf0 2024-02-26T06:28:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.948+0000 7f7c7a000640 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9aca056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.948+0000 7f7c78c00640 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.948+0000 7f7c79600640 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 0x55b9ac9f9840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009230 tx=0x7f7c68004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d12acdeb613769 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.948+0000 7f7c79600640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7c5c0a2430 0x7f7c5c0a48f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7c68004c50 tx=0x7f7c68039000 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.948+0000 7f7c66600640 1 -- 172.21.15.49:0/2097174098 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c680037f0 con 0x55b9ac9faaf0 2024-02-26T06:28:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.949+0000 7f7c66600640 1 -- 172.21.15.49:0/2097174098 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c68003e30 con 0x55b9ac9faaf0 2024-02-26T06:28:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.949+0000 7f7c66600640 1 -- 172.21.15.49:0/2097174098 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68005650 con 0x55b9ac9faaf0 2024-02-26T06:28:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.950+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2097174098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 msgr2=0x55b9ac9f9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.950+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 0x55b9ac9f9840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009230 tx=0x7f7c68004260 comp rx=0 tx=0).stop 2024-02-26T06:28:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.951+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2097174098 shutdown_connections 2024-02-26T06:28:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.951+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9aca056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.952+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.952+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2097174098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 0x55b9ac9f9840 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.952+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2097174098 >> 172.21.15.49:0/2097174098 conn(0x55b9ac9c1480 msgr2=0x55b9ac9f4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.952+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2097174098 shutdown_connections 2024-02-26T06:28:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.953+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2097174098 wait complete. 2024-02-26T06:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.953+0000 7f7c7ee3c740 1 -- start start 2024-02-26T06:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.954+0000 7f7c7ee3c740 1 Processor -- start 2024-02-26T06:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9aca00420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 0x55b9aca00960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca00ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac9605a0 con 0x55b9ac9f9d80 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac905b20 con 0x55b9ac9faaf0 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac9057c0 con 0x55b9aca03300 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7ee3c740 10 client.0 _ll_register_callbacks cb 0x55b9ac919520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca00ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca00ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:36500/0 (socket says 172.21.15.49:36500) 2024-02-26T06:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.955+0000 7f7c7a000640 1 -- 172.21.15.49:0/3616981710 learned_addr learned my addr 172.21.15.49:0/3616981710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.956+0000 7f7c7a000640 1 -- 172.21.15.49:0/3616981710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 msgr2=0x55b9aca00960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.956+0000 7f7c7a000640 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 0x55b9aca00960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.956+0000 7f7c7a000640 1 -- 172.21.15.49:0/3616981710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 msgr2=0x55b9aca00420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.956+0000 7f7c7a000640 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9aca00420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.956+0000 7f7c7a000640 1 -- 172.21.15.49:0/3616981710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c74119230 con 0x55b9aca03300 2024-02-26T06:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.957+0000 7f7c7a000640 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca00ea0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0109f0 tx=0x7f7c6c010df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b41bd690508012 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.957+0000 7f7c5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.958+0000 7f7c5aa00640 1 -- 172.21.15.49:0/3616981710 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00ee00 con 0x55b9aca03300 2024-02-26T06:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.958+0000 7f7c5aa00640 1 -- 172.21.15.49:0/3616981710 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7c6c017070 con 0x55b9aca03300 2024-02-26T06:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.958+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3616981710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9ac932ca0 con 0x55b9aca03300 2024-02-26T06:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.959+0000 7f7c5aa00640 1 -- 172.21.15.49:0/3616981710 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c6c01c480 con 0x55b9aca03300 2024-02-26T06:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.959+0000 7f7c7ee3c740 10 client.26728 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.959+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3616981710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b9ac95ec20 con 0x55b9aca03300 2024-02-26T06:28:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.960+0000 7f7c5aa00640 1 -- 172.21.15.49:0/3616981710 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b9ac932ca0 con 0x55b9aca03300 2024-02-26T06:28:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.960+0000 7f7c5aa00640 1 -- 172.21.15.49:0/3616981710 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c6c0118c0 con 0x55b9aca03300 2024-02-26T06:28:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.960+0000 7f7c7ee3c740 10 client.26728 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.960+0000 7f7c7ee3c740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.960+0000 7f7c7ee3c740 20 client.26728 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.960+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b9aca24350 0x55b9aca26790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.960+0000 7f7c7ee3c740 4 client.26728 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.960+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3616981710 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9ac956cb0 con 0x55b9aca24350 2024-02-26T06:28:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.962+0000 7f7c79600640 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b9aca24350 0x55b9aca26790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.962+0000 7f7c79600640 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b9aca24350 0x55b9aca26790 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8faa16df91162da server_cookie=547ab772b75a3d7f in_seq=0 out_seq=0 2024-02-26T06:28:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.963+0000 7f7c5aa00640 10 client.26728 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.963+0000 7f7c5aa00640 1 -- 172.21.15.49:0/3616981710 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9ac956cb0 con 0x55b9aca24350 2024-02-26T06:28:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.963+0000 7f7c5aa00640 10 client.26728 handle_command_reply: tid=0 2024-02-26T06:28:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.963+0000 7f7c7ee3c740 1 client.26728 shutdown 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3616981710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 msgr2=0x55b9aca00ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca00ea0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0109f0 tx=0x7f7c6c010df0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3616981710 shutdown_connections 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b9aca24350 0x55b9aca26790 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 0x55b9aca00960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca00ea0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3616981710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9aca00420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3616981710 >> 172.21.15.49:0/3616981710 conn(0x55b9ac9c1480 msgr2=0x55b9ac9f67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c5aa00640 0 client.26728 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.964+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3616981710 shutdown_connections 2024-02-26T06:28:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.965+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3616981710 wait complete. 2024-02-26T06:28:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.965+0000 7f7c7ee3c740 20 client.26728 trim_cache size 0 max 16384 2024-02-26T06:28:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.992+0000 7f7c7ee3c740 1 Processor -- start 2024-02-26T06:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.992+0000 7f7c7ee3c740 1 -- start start 2024-02-26T06:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9aca02b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 0x55b9aca1b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca1b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac6a8e60 con 0x55b9ac9f9d80 2024-02-26T06:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac956ce0 con 0x55b9ac9faaf0 2024-02-26T06:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ac9f5870 con 0x55b9aca03300 2024-02-26T06:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca1b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca1b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:36516/0 (socket says 172.21.15.49:36516) 2024-02-26T06:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.993+0000 7f7c7a000640 1 -- 172.21.15.49:0/3869849324 learned_addr learned my addr 172.21.15.49:0/3869849324 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.994+0000 7f7c78c00640 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 0x55b9aca1b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.994+0000 7f7c79600640 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9aca02b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.994+0000 7f7c7a000640 1 -- 172.21.15.49:0/3869849324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 msgr2=0x55b9aca1b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.994+0000 7f7c7a000640 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 0x55b9aca1b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.995+0000 7f7c7a000640 1 -- 172.21.15.49:0/3869849324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 msgr2=0x55b9aca02b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.995+0000 7f7c7a000640 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9aca02b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.995+0000 7f7c7a000640 1 -- 172.21.15.49:0/3869849324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ac8fb500 con 0x55b9aca03300 2024-02-26T06:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.995+0000 7f7c79600640 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9aca02b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.996+0000 7f7c7a000640 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca1b610 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c002230 tx=0x7f7c6c0072a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dc763e403c1e249 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.996+0000 7f7c65c00640 1 -- 172.21.15.49:0/3869849324 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00ee00 con 0x55b9aca03300 2024-02-26T06:28:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.996+0000 7f7c65c00640 1 -- 172.21.15.49:0/3869849324 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c6c015070 con 0x55b9aca03300 2024-02-26T06:28:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.996+0000 7f7c65c00640 1 -- 172.21.15.49:0/3869849324 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c011a30 con 0x55b9aca03300 2024-02-26T06:28:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.997+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3869849324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 msgr2=0x55b9aca1b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.997+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca1b610 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c002230 tx=0x7f7c6c0072a0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.997+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3869849324 shutdown_connections 2024-02-26T06:28:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.997+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9ac9faaf0 0x55b9aca1b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.997+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9aca03300 0x55b9aca1b610 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.997+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/3869849324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9aca02b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.997+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3869849324 >> 172.21.15.49:0/3869849324 conn(0x55b9ac9c1480 msgr2=0x55b9ac9f45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.998+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3869849324 shutdown_connections 2024-02-26T06:28:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.998+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/3869849324 wait complete. 2024-02-26T06:28:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.998+0000 7f7c7ee3c740 1 -- start start 2024-02-26T06:28:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.998+0000 7f7c7ee3c740 1 Processor -- start 2024-02-26T06:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 0x55b9ac9fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7ee3c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9ac9fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9aca7a8e0 con 0x55b9ac9f9d80 2024-02-26T06:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9aca7aa50 con 0x55b9aca03300 2024-02-26T06:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7ee3c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9aca7ac10 con 0x55b9ac9faaf0 2024-02-26T06:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7ee3c740 10 client.0 _ll_register_callbacks cb 0x7f7c6c007880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9ac9fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9ac9fdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38072/0 (socket says 172.21.15.49:38072) 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:31.999+0000 7f7c7a000640 1 -- 172.21.15.49:0/2138019659 learned_addr learned my addr 172.21.15.49:0/2138019659 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.000+0000 7f7c7a000640 1 -- 172.21.15.49:0/2138019659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 msgr2=0x55b9ac9fd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.000+0000 7f7c7a000640 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 0x55b9ac9fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.000+0000 7f7c7a000640 1 -- 172.21.15.49:0/2138019659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 msgr2=0x55b9ac9fd220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.000+0000 7f7c79600640 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.000+0000 7f7c7a000640 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.000+0000 7f7c7a000640 1 -- 172.21.15.49:0/2138019659 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9ac95ec20 con 0x55b9aca03300 2024-02-26T06:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.001+0000 7f7c7a000640 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9ac9fdca0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0118e0 tx=0x7f7c6c011910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df43afcd2c8a6f68 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.001+0000 7f7c5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.002+0000 7f7c5aa00640 1 -- 172.21.15.49:0/2138019659 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c007560 con 0x55b9aca03300 2024-02-26T06:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.002+0000 7f7c5aa00640 1 -- 172.21.15.49:0/2138019659 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f7c6c02b070 con 0x55b9aca03300 2024-02-26T06:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.002+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2138019659 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9ac985d00 con 0x55b9aca03300 2024-02-26T06:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.003+0000 7f7c5aa00640 1 -- 172.21.15.49:0/2138019659 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c6c004c10 con 0x55b9aca03300 2024-02-26T06:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.003+0000 7f7c7ee3c740 10 client.26717 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.003+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2138019659 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b9ac921290 con 0x55b9aca03300 2024-02-26T06:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.003+0000 7f7c5aa00640 1 -- 172.21.15.49:0/2138019659 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b9ac985d00 con 0x55b9aca03300 2024-02-26T06:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.004+0000 7f7c5aa00640 1 -- 172.21.15.49:0/2138019659 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c6c01c480 con 0x55b9aca03300 2024-02-26T06:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.004+0000 7f7c7ee3c740 10 client.26717 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.004+0000 7f7c7ee3c740 10 client.26717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.004+0000 7f7c7ee3c740 20 client.26717 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.004+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b9aca7bfa0 0x55b9aca7e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.004+0000 7f7c7ee3c740 4 client.26717 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.004+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2138019659 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9ac660600 con 0x55b9aca7bfa0 2024-02-26T06:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.004+0000 7f7c79600640 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b9aca7bfa0 0x55b9aca7e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.005+0000 7f7c79600640 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b9aca7bfa0 0x55b9aca7e3c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e1472e268b4e68b server_cookie=13c0614b2765d697 in_seq=0 out_seq=0 2024-02-26T06:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.005+0000 7f7c5aa00640 10 client.26717 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.033+0000 7f7c5aa00640 1 -- 172.21.15.49:0/2138019659 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9ac660600 con 0x55b9aca7bfa0 2024-02-26T06:28:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.033+0000 7f7c5aa00640 10 client.26717 handle_command_reply: tid=0 2024-02-26T06:28:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.033+0000 7f7c7ee3c740 1 client.26717 shutdown 2024-02-26T06:28:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.034+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2138019659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 msgr2=0x55b9ac9fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.034+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9ac9fdca0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0118e0 tx=0x7f7c6c011910 comp rx=0 tx=0).stop 2024-02-26T06:28:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.035+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2138019659 shutdown_connections 2024-02-26T06:28:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.035+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b9aca7bfa0 0x55b9aca7e3c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.035+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b9aca03300 0x55b9ac9fdca0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.035+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9ac9faaf0 0x55b9ac9fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.035+0000 7f7c7ee3c740 1 --2- 172.21.15.49:0/2138019659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9ac9f9d80 0x55b9ac9fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.035+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2138019659 >> 172.21.15.49:0/2138019659 conn(0x55b9ac9c1480 msgr2=0x55b9ac9fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.035+0000 7f7c5aa00640 0 client.26717 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.035+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2138019659 shutdown_connections 2024-02-26T06:28:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.036+0000 7f7c7ee3c740 1 -- 172.21.15.49:0/2138019659 wait complete. 2024-02-26T06:28:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.036+0000 7f7c7ee3c740 20 client.26717 trim_cache size 0 max 16384 2024-02-26T06:28:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:28:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:28:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:28:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:28:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.037+0000 7f7c7c000640 1 -- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7c5c0a2430 msgr2=0x7f7c5c0a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.037+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7c5c0a2430 0x7f7c5c0a48f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7c68004c50 tx=0x7f7c68039000 comp rx=0 tx=0).stop 2024-02-26T06:28:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 -- 172.21.15.49:0/1170634992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c7410b870 msgr2=0x7f7c74113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c7410b870 0x7f7c74113970 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000e9c0 tx=0x7f7c6000ee80 comp rx=0 tx=0).stop 2024-02-26T06:28:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 -- 172.21.15.49:0/1170634992 shutdown_connections 2024-02-26T06:28:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7c5c0a2430 0x7f7c5c0a48f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c74116180 0x7f7c74113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c7410b870 0x7f7c74113970 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 --2- 172.21.15.49:0/1170634992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c741086c0 0x7f7c74115820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 -- 172.21.15.49:0/1170634992 >> 172.21.15.49:0/1170634992 conn(0x7f7c7406c6b0 msgr2=0x7f7c741165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 -- 172.21.15.49:0/1170634992 shutdown_connections 2024-02-26T06:28:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:32.038+0000 7f7c7c000640 1 -- 172.21.15.49:0/1170634992 wait complete. 2024-02-26T06:28:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:32 smithi080 ceph-mon[36206]: pgmap v157: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 31 MiB/s wr, 892 op/s 2024-02-26T06:28:33.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:32 smithi115 ceph-mon[36426]: pgmap v157: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 31 MiB/s wr, 892 op/s 2024-02-26T06:28:33.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:32 smithi049 ceph-mon[31696]: pgmap v157: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 31 MiB/s wr, 892 op/s 2024-02-26T06:28:35.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:34 smithi080 ceph-mon[36206]: pgmap v158: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 25 MiB/s wr, 703 op/s 2024-02-26T06:28:35.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:34 smithi115 ceph-mon[36426]: pgmap v158: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 25 MiB/s wr, 703 op/s 2024-02-26T06:28:35.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:34 smithi049 ceph-mon[31696]: pgmap v158: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 25 MiB/s wr, 703 op/s 2024-02-26T06:28:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 285.0 s, 2347.5 tps, lat 42.592 ms stddev 52.855 2024-02-26T06:28:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:36 smithi080 ceph-mon[36206]: pgmap v159: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 36 MiB/s wr, 1.01k op/s 2024-02-26T06:28:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:36 smithi115 ceph-mon[36426]: pgmap v159: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 36 MiB/s wr, 1.01k op/s 2024-02-26T06:28:37.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:36 smithi049 ceph-mon[31696]: pgmap v159: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 36 MiB/s wr, 1.01k op/s 2024-02-26T06:28:39.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:38 smithi080 ceph-mon[36206]: pgmap v160: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 641 op/s 2024-02-26T06:28:39.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:38 smithi115 ceph-mon[36426]: pgmap v160: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 641 op/s 2024-02-26T06:28:39.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:38 smithi049 ceph-mon[31696]: pgmap v160: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 641 op/s 2024-02-26T06:28:41.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:40 smithi080 ceph-mon[36206]: pgmap v161: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 30 MiB/s wr, 840 op/s 2024-02-26T06:28:41.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:40 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:41.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:40 smithi115 ceph-mon[36426]: pgmap v161: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 30 MiB/s wr, 840 op/s 2024-02-26T06:28:41.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:40 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:41.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:40 smithi049 ceph-mon[31696]: pgmap v161: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 30 MiB/s wr, 840 op/s 2024-02-26T06:28:41.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:40 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 290.0 s, 2477.3 tps, lat 40.450 ms stddev 48.923 2024-02-26T06:28:43.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:42 smithi080 ceph-mon[36206]: pgmap v162: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 717 op/s 2024-02-26T06:28:43.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:42 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:43.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:42 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:43.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:42 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:28:43.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:42 smithi115 ceph-mon[36426]: pgmap v162: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 717 op/s 2024-02-26T06:28:43.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:42 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:43.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:42 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:43.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:42 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:28:43.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:42 smithi049 ceph-mon[31696]: pgmap v162: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 717 op/s 2024-02-26T06:28:43.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:42 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:43.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:42 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:43.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:42 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:28:45.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:44 smithi080 ceph-mon[36206]: pgmap v163: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 601 op/s 2024-02-26T06:28:45.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:44 smithi115 ceph-mon[36426]: pgmap v163: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 601 op/s 2024-02-26T06:28:45.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:44 smithi049 ceph-mon[31696]: pgmap v163: 97 pgs: 97 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 601 op/s 2024-02-26T06:28:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.245+0000 7fc60d400640 1 -- 172.21.15.49:0/2186398297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6081035c0 msgr2=0x7fc6081039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.245+0000 7fc60d400640 1 --2- 172.21.15.49:0/2186398297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6081035c0 0x7fc6081039a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009930 tx=0x7fc5f802bc50 comp rx=0 tx=0).stop 2024-02-26T06:28:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.245+0000 7fc60d400640 1 -- 172.21.15.49:0/2186398297 shutdown_connections 2024-02-26T06:28:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.245+0000 7fc60d400640 1 --2- 172.21.15.49:0/2186398297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc608108690 0x7fc608110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.245+0000 7fc60d400640 1 --2- 172.21.15.49:0/2186398297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc608103f70 0x7fc608107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.245+0000 7fc60d400640 1 --2- 172.21.15.49:0/2186398297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6081035c0 0x7fc6081039a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.246+0000 7fc60d400640 1 -- 172.21.15.49:0/2186398297 >> 172.21.15.49:0/2186398297 conn(0x7fc6080fee40 msgr2=0x7fc608101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.246+0000 7fc60d400640 1 -- 172.21.15.49:0/2186398297 shutdown_connections 2024-02-26T06:28:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.247+0000 7fc60d400640 1 -- 172.21.15.49:0/2186398297 wait complete. 2024-02-26T06:28:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.249+0000 7fc60d400640 1 Processor -- start 2024-02-26T06:28:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.252+0000 7fc60d400640 1 -- start start 2024-02-26T06:28:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.254+0000 7fc60d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc608108690 0x7fc6081a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.255+0000 7fc60d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6081a3150 0x7fc60810ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.255+0000 7fc60d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60810f260 0x7fc60810f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.256+0000 7fc60d400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6081a37c0 con 0x7fc60810f260 2024-02-26T06:28:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.256+0000 7fc60d400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6081aab30 con 0x7fc608108690 2024-02-26T06:28:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.256+0000 7fc60d400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6081aac70 con 0x7fc6081a3150 2024-02-26T06:28:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.256+0000 7fc5ffe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6081a3150 0x7fc60810ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.256+0000 7fc5ffe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6081a3150 0x7fc60810ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:50602/0 (socket says 172.21.15.49:50602) 2024-02-26T06:28:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.256+0000 7fc5ffe00640 1 -- 172.21.15.49:0/652673738 learned_addr learned my addr 172.21.15.49:0/652673738 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.257+0000 7fc606800640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc608108690 0x7fc6081a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.257+0000 7fc607200640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60810f260 0x7fc60810f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.257+0000 7fc606800640 1 -- 172.21.15.49:0/652673738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6081a3150 msgr2=0x7fc60810ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.257+0000 7fc606800640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6081a3150 0x7fc60810ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.258+0000 7fc606800640 1 -- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60810f260 msgr2=0x7fc60810f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.258+0000 7fc606800640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60810f260 0x7fc60810f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.258+0000 7fc606800640 1 -- 172.21.15.49:0/652673738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600009840 con 0x7fc608108690 2024-02-26T06:28:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.258+0000 7fc607200640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60810f260 0x7fc60810f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.259+0000 7fc606800640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc608108690 0x7fc6081a2c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc6081047e0 tx=0x7fc5f802cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.271+0000 7fc5ff400640 1 -- 172.21.15.49:0/652673738 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80042d0 con 0x7fc608108690 2024-02-26T06:28:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.271+0000 7fc5ff400640 1 -- 172.21.15.49:0/652673738 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8004430 con 0x7fc608108690 2024-02-26T06:28:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.271+0000 7fc5ff400640 1 -- 172.21.15.49:0/652673738 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8003b20 con 0x7fc608108690 2024-02-26T06:28:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.273+0000 7fc60d400640 1 -- 172.21.15.49:0/652673738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f8009560 con 0x7fc608108690 2024-02-26T06:28:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.273+0000 7fc60d400640 1 -- 172.21.15.49:0/652673738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6081ab1c0 con 0x7fc608108690 2024-02-26T06:28:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.281+0000 7fc61040b740 1 Processor -- start 2024-02-26T06:28:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.281+0000 7fc61040b740 1 -- start start 2024-02-26T06:28:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.281+0000 7fc61040b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc7b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc607200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc7b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc607200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc7b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43272/0 (socket says 172.21.15.49:43272) 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc61040b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 0x56254cc7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc607200640 1 -- 172.21.15.49:0/1921389091 learned_addr learned my addr 172.21.15.49:0/1921389091 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc61040b740 1 --2- 172.21.15.49:0/1921389091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 0x56254cc876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc61040b740 1 -- 172.21.15.49:0/1921389091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cc375f0 con 0x56254cc85300 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc61040b740 1 -- 172.21.15.49:0/1921389091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cbe9890 con 0x56254cc7caf0 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc61040b740 1 -- 172.21.15.49:0/1921389091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cbe6dc0 con 0x56254cc7bd80 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc607200640 1 -- 172.21.15.49:0/1921389091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 msgr2=0x56254cc7c200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc607200640 1 --2- 172.21.15.49:0/1921389091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 0x56254cc7c200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc607200640 1 -- 172.21.15.49:0/1921389091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 msgr2=0x56254cc876f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc607200640 1 --2- 172.21.15.49:0/1921389091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 0x56254cc876f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.283+0000 7fc607200640 1 -- 172.21.15.49:0/1921389091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254cb63ad0 con 0x56254cc7caf0 2024-02-26T06:28:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.284+0000 7fc607200640 1 --2- 172.21.15.49:0/1921389091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc7b840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc60000f460 tx=0x7fc60000f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5130222719652e26 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.284+0000 7fc5f7400640 1 -- 172.21.15.49:0/1921389091 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600011700 con 0x56254cc7caf0 2024-02-26T06:28:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.284+0000 7fc5ff400640 1 -- 172.21.15.49:0/652673738 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc5f8005460 con 0x7fc608108690 2024-02-26T06:28:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.284+0000 7fc5f7400640 1 -- 172.21.15.49:0/1921389091 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc600011d40 con 0x56254cc7caf0 2024-02-26T06:28:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.285+0000 7fc5f7400640 1 -- 172.21.15.49:0/1921389091 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600011700 con 0x56254cc7caf0 2024-02-26T06:28:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.285+0000 7fc61040b740 1 -- 172.21.15.49:0/1921389091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 msgr2=0x56254cc7b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.286+0000 7fc61040b740 1 --2- 172.21.15.49:0/1921389091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc7b840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc60000f460 tx=0x7fc60000f920 comp rx=0 tx=0).stop 2024-02-26T06:28:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.287+0000 7fc5ff400640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc5ec0a24b0 0x7fc5ec0a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.288+0000 7fc5ff400640 1 -- 172.21.15.49:0/652673738 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc5f80ef980 con 0x7fc608108690 2024-02-26T06:28:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.289+0000 7fc61040b740 1 -- 172.21.15.49:0/1921389091 shutdown_connections 2024-02-26T06:28:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.289+0000 7fc61040b740 1 --2- 172.21.15.49:0/1921389091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 0x56254cc876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.289+0000 7fc61040b740 1 --2- 172.21.15.49:0/1921389091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 0x56254cc7c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.289+0000 7fc61040b740 1 --2- 172.21.15.49:0/1921389091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc7b840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.289+0000 7fc61040b740 1 -- 172.21.15.49:0/1921389091 >> 172.21.15.49:0/1921389091 conn(0x56254cc43480 msgr2=0x56254cc76a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.289+0000 7fc606800640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc5ec0a24b0 0x7fc5ec0a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.291+0000 7fc606800640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc5ec0a24b0 0x7fc5ec0a4970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x56254cc7d250 tx=0x7fc5f8005fd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.291+0000 7fc61040b740 1 -- 172.21.15.49:0/1921389091 shutdown_connections 2024-02-26T06:28:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.292+0000 7fc61040b740 1 -- 172.21.15.49:0/1921389091 wait complete. 2024-02-26T06:28:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.293+0000 7fc61040b740 1 -- start start 2024-02-26T06:28:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.293+0000 7fc61040b740 1 Processor -- start 2024-02-26T06:28:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.294+0000 7fc61040b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 0x56254cc82280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.295+0000 7fc61040b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.295+0000 7fc61040b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 0x56254cc82d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.296+0000 7fc61040b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cbe25a0 con 0x56254cc85300 2024-02-26T06:28:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.296+0000 7fc61040b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cb87b20 con 0x56254cc7caf0 2024-02-26T06:28:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.296+0000 7fc61040b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cb877c0 con 0x56254cc7bd80 2024-02-26T06:28:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.296+0000 7fc61040b740 10 client.0 _ll_register_callbacks cb 0x56254cb9b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.296+0000 7fc5ffe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.297+0000 7fc5ffe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc827c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43280/0 (socket says 172.21.15.49:43280) 2024-02-26T06:28:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.297+0000 7fc5ffe00640 1 -- 172.21.15.49:0/4032630195 learned_addr learned my addr 172.21.15.49:0/4032630195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.297+0000 7fc607200640 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 0x56254cc82d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.298+0000 7fc607200640 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 0x56254cc82280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.298+0000 7fc5ffe00640 1 -- 172.21.15.49:0/4032630195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 msgr2=0x56254cc82280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.298+0000 7fc5ffe00640 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 0x56254cc82280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.298+0000 7fc5ffe00640 1 -- 172.21.15.49:0/4032630195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 msgr2=0x56254cc82d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.298+0000 7fc5ffe00640 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 0x56254cc82d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.298+0000 7fc5ffe00640 1 -- 172.21.15.49:0/4032630195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc600009dd0 con 0x56254cc7caf0 2024-02-26T06:28:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.299+0000 7fc5ffe00640 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc827c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000d160 tx=0x7fc5f000d620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=481878275c9000d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.299+0000 7fc5ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.302+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4032630195 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f000de50 con 0x56254cc7caf0 2024-02-26T06:28:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.302+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4032630195 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc5f00125f0 con 0x56254cc7caf0 2024-02-26T06:28:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.302+0000 7fc61040b740 1 -- 172.21.15.49:0/4032630195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56254cbb4ca0 con 0x56254cc7caf0 2024-02-26T06:28:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.303+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4032630195 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5f00127b0 con 0x56254cc7caf0 2024-02-26T06:28:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.303+0000 7fc61040b740 10 client.26729 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.303+0000 7fc61040b740 1 -- 172.21.15.49:0/4032630195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56254cbe0c20 con 0x56254cc7caf0 2024-02-26T06:28:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.304+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4032630195 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56254cbb4ca0 con 0x56254cc7caf0 2024-02-26T06:28:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.305+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4032630195 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc5f0013c90 con 0x56254cc7caf0 2024-02-26T06:28:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.305+0000 7fc61040b740 10 client.26729 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.305+0000 7fc61040b740 10 client.26729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.305+0000 7fc61040b740 20 client.26729 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.305+0000 7fc61040b740 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56254cc95400 0x56254cc97820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.305+0000 7fc61040b740 4 client.26729 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.305+0000 7fc61040b740 1 -- 172.21.15.49:0/4032630195 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56254cbd8cb0 con 0x56254cc95400 2024-02-26T06:28:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.305+0000 7fc607200640 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56254cc95400 0x56254cc97820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.306+0000 7fc5ebe00640 10 client.26729 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.306+0000 7fc607200640 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56254cc95400 0x56254cc97820 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf6c2c29b01e9f5 server_cookie=c537c077099de22f in_seq=0 out_seq=0 2024-02-26T06:28:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.308+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4032630195 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56254cbd8cb0 con 0x56254cc95400 2024-02-26T06:28:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.308+0000 7fc5ebe00640 10 client.26729 handle_command_reply: tid=0 2024-02-26T06:28:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.308+0000 7fc61040b740 1 client.26729 shutdown 2024-02-26T06:28:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.310+0000 7fc61040b740 1 -- 172.21.15.49:0/4032630195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 msgr2=0x56254cc827c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.310+0000 7fc61040b740 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc827c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000d160 tx=0x7fc5f000d620 comp rx=0 tx=0).stop 2024-02-26T06:28:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.311+0000 7fc61040b740 1 -- 172.21.15.49:0/4032630195 shutdown_connections 2024-02-26T06:28:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.311+0000 7fc61040b740 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56254cc95400 0x56254cc97820 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.311+0000 7fc61040b740 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7caf0 0x56254cc827c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.311+0000 7fc61040b740 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc85300 0x56254cc82d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.311+0000 7fc61040b740 1 --2- 172.21.15.49:0/4032630195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc7bd80 0x56254cc82280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.311+0000 7fc61040b740 1 -- 172.21.15.49:0/4032630195 >> 172.21.15.49:0/4032630195 conn(0x56254cc43480 msgr2=0x56254cc787c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.311+0000 7fc5ebe00640 0 client.26729 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.312+0000 7fc61040b740 1 -- 172.21.15.49:0/4032630195 shutdown_connections 2024-02-26T06:28:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.312+0000 7fc61040b740 1 -- 172.21.15.49:0/4032630195 wait complete. 2024-02-26T06:28:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.312+0000 7fc61040b740 20 client.26729 trim_cache size 0 max 16384 2024-02-26T06:28:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.349+0000 7fc61040b740 1 Processor -- start 2024-02-26T06:28:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.349+0000 7fc61040b740 1 -- start start 2024-02-26T06:28:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.350+0000 7fc61040b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.351+0000 7fc61040b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 0x56254cc82b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.351+0000 7fc607200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.351+0000 7fc607200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc849b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43282/0 (socket says 172.21.15.49:43282) 2024-02-26T06:28:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.351+0000 7fc61040b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 0x56254cc7f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.351+0000 7fc61040b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254c92ae60 con 0x56254cc7caf0 2024-02-26T06:28:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.351+0000 7fc61040b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cbd8ce0 con 0x56254cc7bd80 2024-02-26T06:28:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.351+0000 7fc61040b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cc7d0e0 con 0x56254cc85300 2024-02-26T06:28:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.352+0000 7fc607200640 1 -- 172.21.15.49:0/3118822376 learned_addr learned my addr 172.21.15.49:0/3118822376 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.352+0000 7fc5ffe00640 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 0x56254cc82b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.353+0000 7fc607200640 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 0x56254cc7f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.353+0000 7fc607200640 1 -- 172.21.15.49:0/3118822376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 msgr2=0x56254cc7f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.353+0000 7fc607200640 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 0x56254cc7f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.353+0000 7fc607200640 1 -- 172.21.15.49:0/3118822376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 msgr2=0x56254cc82b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.353+0000 7fc607200640 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 0x56254cc82b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.353+0000 7fc607200640 1 -- 172.21.15.49:0/3118822376 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254cb7d500 con 0x56254cc7bd80 2024-02-26T06:28:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.354+0000 7fc607200640 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc849b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc60000f430 tx=0x7fc60000f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0856273c7f39a62 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.355+0000 7fc5f6a00640 1 -- 172.21.15.49:0/3118822376 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600010e40 con 0x56254cc7bd80 2024-02-26T06:28:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.355+0000 7fc5f6a00640 1 -- 172.21.15.49:0/3118822376 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6000049c0 con 0x56254cc7bd80 2024-02-26T06:28:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.355+0000 7fc5f6a00640 1 -- 172.21.15.49:0/3118822376 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600011990 con 0x56254cc7bd80 2024-02-26T06:28:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.356+0000 7fc61040b740 1 -- 172.21.15.49:0/3118822376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 msgr2=0x56254cc849b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.356+0000 7fc61040b740 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc849b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc60000f430 tx=0x7fc60000f920 comp rx=0 tx=0).stop 2024-02-26T06:28:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.357+0000 7fc61040b740 1 -- 172.21.15.49:0/3118822376 shutdown_connections 2024-02-26T06:28:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.357+0000 7fc61040b740 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 0x56254cc7f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.357+0000 7fc61040b740 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 0x56254cc82b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.357+0000 7fc61040b740 1 --2- 172.21.15.49:0/3118822376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc849b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.357+0000 7fc61040b740 1 -- 172.21.15.49:0/3118822376 >> 172.21.15.49:0/3118822376 conn(0x56254cc43480 msgr2=0x56254cc80b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.358+0000 7fc61040b740 1 -- 172.21.15.49:0/3118822376 shutdown_connections 2024-02-26T06:28:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.358+0000 7fc61040b740 1 -- 172.21.15.49:0/3118822376 wait complete. 2024-02-26T06:28:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.358+0000 7fc61040b740 1 -- start start 2024-02-26T06:28:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.359+0000 7fc61040b740 1 Processor -- start 2024-02-26T06:28:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.360+0000 7fc61040b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc8ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.360+0000 7fc61040b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 0x56254cc8b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.361+0000 7fc61040b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 0x56254cc8b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.361+0000 7fc607200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc8ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.361+0000 7fc607200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc8ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43290/0 (socket says 172.21.15.49:43290) 2024-02-26T06:28:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.361+0000 7fc61040b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254cceba00 con 0x56254cc7caf0 2024-02-26T06:28:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.361+0000 7fc61040b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ccebb70 con 0x56254cc7bd80 2024-02-26T06:28:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.361+0000 7fc61040b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ccebd30 con 0x56254cc85300 2024-02-26T06:28:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.361+0000 7fc61040b740 10 client.0 _ll_register_callbacks cb 0x7fc5f0007160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.361+0000 7fc607200640 1 -- 172.21.15.49:0/4241066587 learned_addr learned my addr 172.21.15.49:0/4241066587 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.362+0000 7fc607200640 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 0x56254cc8b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.362+0000 7fc607200640 1 -- 172.21.15.49:0/4241066587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 msgr2=0x56254cc8b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.362+0000 7fc607200640 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 0x56254cc8b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.362+0000 7fc607200640 1 -- 172.21.15.49:0/4241066587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 msgr2=0x56254cc8b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.362+0000 7fc607200640 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 0x56254cc8b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.362+0000 7fc607200640 1 -- 172.21.15.49:0/4241066587 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60001f040 con 0x56254cc7bd80 2024-02-26T06:28:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.363+0000 7fc607200640 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc8ad80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56254cb9b560 tx=0x7fc600009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b9374ce4d678842 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.363+0000 7fc5ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.365+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4241066587 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600004cb0 con 0x56254cc7bd80 2024-02-26T06:28:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.366+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4241066587 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc600004e10 con 0x56254cc7bd80 2024-02-26T06:28:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.366+0000 7fc61040b740 1 -- 172.21.15.49:0/4241066587 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56254cc07d00 con 0x56254cc7bd80 2024-02-26T06:28:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.367+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4241066587 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc600029450 con 0x56254cc7bd80 2024-02-26T06:28:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.368+0000 7fc61040b740 10 client.26735 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.368+0000 7fc61040b740 1 -- 172.21.15.49:0/4241066587 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56254cba3290 con 0x56254cc7bd80 2024-02-26T06:28:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.368+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4241066587 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56254cc07d00 con 0x56254cc7bd80 2024-02-26T06:28:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.368+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4241066587 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc600005040 con 0x56254cc7bd80 2024-02-26T06:28:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.369+0000 7fc61040b740 10 client.26735 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.369+0000 7fc61040b740 10 client.26735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.369+0000 7fc61040b740 20 client.26735 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.369+0000 7fc61040b740 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56254cced120 0x56254ccef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.369+0000 7fc61040b740 4 client.26735 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:28:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.369+0000 7fc61040b740 1 -- 172.21.15.49:0/4241066587 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56254c8e2600 con 0x56254cced120 2024-02-26T06:28:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.369+0000 7fc5ffe00640 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56254cced120 0x56254ccef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.370+0000 7fc5ffe00640 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56254cced120 0x56254ccef540 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a494545cfa0eafe4 server_cookie=f3e497fc79e104a in_seq=0 out_seq=0 2024-02-26T06:28:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.370+0000 7fc5ebe00640 10 client.26735 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.603+0000 7fc5ebe00640 1 -- 172.21.15.49:0/4241066587 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56254c8e2600 con 0x56254cced120 2024-02-26T06:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.603+0000 7fc5ebe00640 10 client.26735 handle_command_reply: tid=0 2024-02-26T06:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.603+0000 7fc61040b740 1 client.26735 shutdown 2024-02-26T06:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.603+0000 7fc61040b740 1 -- 172.21.15.49:0/4241066587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 msgr2=0x56254cc8ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.603+0000 7fc61040b740 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc8ad80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56254cb9b560 tx=0x7fc600009500 comp rx=0 tx=0).stop 2024-02-26T06:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc61040b740 1 -- 172.21.15.49:0/4241066587 shutdown_connections 2024-02-26T06:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc61040b740 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56254cced120 0x56254ccef540 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc61040b740 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56254cc85300 0x56254cc8b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc61040b740 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56254cc7caf0 0x56254cc8b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc61040b740 1 --2- 172.21.15.49:0/4241066587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56254cc7bd80 0x56254cc8ad80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc61040b740 1 -- 172.21.15.49:0/4241066587 >> 172.21.15.49:0/4241066587 conn(0x56254cc43480 msgr2=0x56254cc75d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc5ebe00640 0 client.26735 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc61040b740 1 -- 172.21.15.49:0/4241066587 shutdown_connections 2024-02-26T06:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.604+0000 7fc61040b740 1 -- 172.21.15.49:0/4241066587 wait complete. 2024-02-26T06:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.605+0000 7fc61040b740 20 client.26735 trim_cache size 0 max 16384 2024-02-26T06:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:28:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.606+0000 7fc60d400640 1 -- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc5ec0a24b0 msgr2=0x7fc5ec0a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.606+0000 7fc60d400640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc5ec0a24b0 0x7fc5ec0a4970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x56254cc7d250 tx=0x7fc5f8005fd0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.606+0000 7fc60d400640 1 -- 172.21.15.49:0/652673738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc608108690 msgr2=0x7fc6081a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.606+0000 7fc60d400640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc608108690 0x7fc6081a2c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc6081047e0 tx=0x7fc5f802cdd0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.607+0000 7fc60d400640 1 -- 172.21.15.49:0/652673738 shutdown_connections 2024-02-26T06:28:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.607+0000 7fc60d400640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc5ec0a24b0 0x7fc5ec0a4970 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.607+0000 7fc60d400640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60810f260 0x7fc60810f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.607+0000 7fc60d400640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6081a3150 0x7fc60810ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.607+0000 7fc60d400640 1 --2- 172.21.15.49:0/652673738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc608108690 0x7fc6081a2c10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.607+0000 7fc60d400640 1 -- 172.21.15.49:0/652673738 >> 172.21.15.49:0/652673738 conn(0x7fc6080fee40 msgr2=0x7fc608100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.607+0000 7fc60d400640 1 -- 172.21.15.49:0/652673738 shutdown_connections 2024-02-26T06:28:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:45.607+0000 7fc60d400640 1 -- 172.21.15.49:0/652673738 wait complete. 2024-02-26T06:28:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 295.0 s, 1626.4 tps, lat 56.842 ms stddev 107.146 2024-02-26T06:28:47.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:46 smithi080 ceph-mon[36206]: pgmap v164: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 27 MiB/s wr, 889 op/s 2024-02-26T06:28:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:46 smithi115 ceph-mon[36426]: pgmap v164: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 27 MiB/s wr, 889 op/s 2024-02-26T06:28:47.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:46 smithi049 ceph-mon[31696]: pgmap v164: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 27 MiB/s wr, 889 op/s 2024-02-26T06:28:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:48 smithi080 ceph-mon[36206]: pgmap v165: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 584 op/s 2024-02-26T06:28:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:48 smithi115 ceph-mon[36426]: pgmap v165: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 584 op/s 2024-02-26T06:28:49.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:48 smithi049 ceph-mon[31696]: pgmap v165: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 584 op/s 2024-02-26T06:28:50.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:28:50.311 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:28:50.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.624+0000 7fc10da00640 1 -- 172.21.15.49:0/2755752211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc108108360 msgr2=0x7fc108108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.624+0000 7fc10da00640 1 --2- 172.21.15.49:0/2755752211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc108108360 0x7fc108108740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009860 tx=0x7fc0f802bb70 comp rx=0 tx=0).stop 2024-02-26T06:28:50.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.625+0000 7fc10da00640 1 -- 172.21.15.49:0/2755752211 shutdown_connections 2024-02-26T06:28:50.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.625+0000 7fc10da00640 1 --2- 172.21.15.49:0/2755752211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1081142d0 0x7fc1081166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.625+0000 7fc10da00640 1 --2- 172.21.15.49:0/2755752211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc108108c80 0x7fc108113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.625+0000 7fc10da00640 1 --2- 172.21.15.49:0/2755752211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc108108360 0x7fc108108740 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.625+0000 7fc10da00640 1 -- 172.21.15.49:0/2755752211 >> 172.21.15.49:0/2755752211 conn(0x7fc10806c260 msgr2=0x7fc10806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:50.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.625+0000 7fc10da00640 1 -- 172.21.15.49:0/2755752211 shutdown_connections 2024-02-26T06:28:50.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.626+0000 7fc10da00640 1 -- 172.21.15.49:0/2755752211 wait complete. 2024-02-26T06:28:50.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.627+0000 7fc10da00640 1 Processor -- start 2024-02-26T06:28:50.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.627+0000 7fc10da00640 1 -- start start 2024-02-26T06:28:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc10da00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc108108360 0x7fc108113010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc10da00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc108108c80 0x7fc108113550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc10da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1081142d0 0x7fc1081a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc10da00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081a7040 con 0x7fc108108360 2024-02-26T06:28:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc10da00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081a71b0 con 0x7fc1081142d0 2024-02-26T06:28:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc10da00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081a7370 con 0x7fc108108c80 2024-02-26T06:28:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc107400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1081142d0 0x7fc1081a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc107400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1081142d0 0x7fc1081a6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52596/0 (socket says 172.21.15.49:52596) 2024-02-26T06:28:50.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.628+0000 7fc107400640 1 -- 172.21.15.49:0/4183835791 learned_addr learned my addr 172.21.15.49:0/4183835791 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:50.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.629+0000 7fc106000640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc108108c80 0x7fc108113550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.629+0000 7fc107400640 1 -- 172.21.15.49:0/4183835791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc108108c80 msgr2=0x7fc108113550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.629+0000 7fc107400640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc108108c80 0x7fc108113550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.629+0000 7fc107400640 1 -- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc108108360 msgr2=0x7fc108113010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.629+0000 7fc107400640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc108108360 0x7fc108113010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.629+0000 7fc107400640 1 -- 172.21.15.49:0/4183835791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009490 con 0x7fc1081142d0 2024-02-26T06:28:50.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.630+0000 7fc107400640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1081142d0 0x7fc1081a6a40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc10000e9d0 tx=0x7fc10000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:50.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.631+0000 7fc0ff400640 1 -- 172.21.15.49:0/4183835791 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10000cd80 con 0x7fc1081142d0 2024-02-26T06:28:50.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.631+0000 7fc0ff400640 1 -- 172.21.15.49:0/4183835791 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc100005020 con 0x7fc1081142d0 2024-02-26T06:28:50.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.631+0000 7fc0ff400640 1 -- 172.21.15.49:0/4183835791 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1000116d0 con 0x7fc1081142d0 2024-02-26T06:28:50.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.631+0000 7fc10da00640 1 -- 172.21.15.49:0/4183835791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1081a7650 con 0x7fc1081142d0 2024-02-26T06:28:50.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.631+0000 7fc10da00640 1 -- 172.21.15.49:0/4183835791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10811a550 con 0x7fc1081142d0 2024-02-26T06:28:50.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.637+0000 7fc0ff400640 1 -- 172.21.15.49:0/4183835791 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc100011830 con 0x7fc1081142d0 2024-02-26T06:28:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.639+0000 7fc110990740 1 Processor -- start 2024-02-26T06:28:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.640+0000 7fc110990740 1 -- start start 2024-02-26T06:28:50.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.640+0000 7fc110990740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6ceecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.640+0000 7fc110990740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.640+0000 7fc110990740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cfa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.640+0000 7fc110990740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6c5e120 con 0x556fb6cf8600 2024-02-26T06:28:50.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.640+0000 7fc110990740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6c5b650 con 0x556fb6cef290 2024-02-26T06:28:50.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.640+0000 7fc110990740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6c56e30 con 0x556fb6ceff70 2024-02-26T06:28:50.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.641+0000 7fc106000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6ceecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.641+0000 7fc106000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cfa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.641+0000 7fc106000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6ceecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60358/0 (socket says 172.21.15.49:60358) 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.641+0000 7fc106000640 1 -- 172.21.15.49:0/566428556 learned_addr learned my addr 172.21.15.49:0/566428556 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.641+0000 7fc106a00640 1 --2- 172.21.15.49:0/566428556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.642+0000 7fc0ff400640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc0ec0a22b0 0x7fc0ec0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.642+0000 7fc106000640 1 -- 172.21.15.49:0/566428556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 msgr2=0x556fb6cef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.642+0000 7fc106000640 1 --2- 172.21.15.49:0/566428556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.642+0000 7fc0ff400640 1 -- 172.21.15.49:0/4183835791 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc1000ce320 con 0x7fc1081142d0 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.642+0000 7fc106000640 1 -- 172.21.15.49:0/566428556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 msgr2=0x556fb6cfa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.642+0000 7fc106000640 1 --2- 172.21.15.49:0/566428556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cfa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.642+0000 7fc106000640 1 -- 172.21.15.49:0/566428556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fb6c10030 con 0x556fb6ceff70 2024-02-26T06:28:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.643+0000 7fc106000640 1 --2- 172.21.15.49:0/566428556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6ceecc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009d90 tx=0x7fc0f4009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207e690094464046 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.643+0000 7fc107400640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc0ec0a22b0 0x7fc0ec0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.643+0000 7fc0f3e00640 1 -- 172.21.15.49:0/566428556 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4010660 con 0x556fb6ceff70 2024-02-26T06:28:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.643+0000 7fc0f3e00640 1 -- 172.21.15.49:0/566428556 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0f4010ca0 con 0x556fb6ceff70 2024-02-26T06:28:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.643+0000 7fc0f3e00640 1 -- 172.21.15.49:0/566428556 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f401bcb0 con 0x556fb6ceff70 2024-02-26T06:28:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.644+0000 7fc110990740 1 -- 172.21.15.49:0/566428556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 msgr2=0x556fb6ceecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.644+0000 7fc107400640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc0ec0a22b0 0x7fc0ec0a4770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc100098150 tx=0x7fc100005870 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.644+0000 7fc110990740 1 --2- 172.21.15.49:0/566428556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6ceecc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009d90 tx=0x7fc0f4009480 comp rx=0 tx=0).stop 2024-02-26T06:28:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.645+0000 7fc110990740 1 -- 172.21.15.49:0/566428556 shutdown_connections 2024-02-26T06:28:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.645+0000 7fc110990740 1 --2- 172.21.15.49:0/566428556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.645+0000 7fc110990740 1 --2- 172.21.15.49:0/566428556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cfa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.645+0000 7fc110990740 1 --2- 172.21.15.49:0/566428556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6ceecc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.645+0000 7fc110990740 1 -- 172.21.15.49:0/566428556 >> 172.21.15.49:0/566428556 conn(0x556fb6cb6570 msgr2=0x556fb6ce9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.645+0000 7fc110990740 1 -- 172.21.15.49:0/566428556 shutdown_connections 2024-02-26T06:28:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.645+0000 7fc110990740 1 -- 172.21.15.49:0/566428556 wait complete. 2024-02-26T06:28:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.646+0000 7fc110990740 1 -- start start 2024-02-26T06:28:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.647+0000 7fc110990740 1 Processor -- start 2024-02-26T06:28:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.648+0000 7fc110990740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cf55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.648+0000 7fc110990740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.648+0000 7fc110990740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cf6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.648+0000 7fc110990740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6bfbe90 con 0x556fb6cf8600 2024-02-26T06:28:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.648+0000 7fc110990740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6bfbb30 con 0x556fb6cef290 2024-02-26T06:28:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.648+0000 7fc110990740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb69c51c0 con 0x556fb6ceff70 2024-02-26T06:28:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.648+0000 7fc110990740 10 client.0 _ll_register_callbacks cb 0x556fb6c0f930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.649+0000 7fc106000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cf55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.649+0000 7fc106000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cf55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52608/0 (socket says 172.21.15.49:52608) 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.649+0000 7fc106000640 1 -- 172.21.15.49:0/2722172105 learned_addr learned my addr 172.21.15.49:0/2722172105 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.649+0000 7fc106000640 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cf6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.649+0000 7fc106a00640 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc106000640 1 -- 172.21.15.49:0/2722172105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 msgr2=0x556fb6cf5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc106000640 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc106000640 1 -- 172.21.15.49:0/2722172105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 msgr2=0x556fb6cf6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc106000640 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cf6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc106000640 1 -- 172.21.15.49:0/2722172105 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f4009a40 con 0x556fb6cef290 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc106000640 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cf55e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005940 tx=0x7fc0f401b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef53b494039db668 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc0f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc0f0c00640 1 -- 172.21.15.49:0/2722172105 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4010460 con 0x556fb6cef290 2024-02-26T06:28:50.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc110990740 1 -- 172.21.15.49:0/2722172105 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fb69567e0 con 0x556fb6cef290 2024-02-26T06:28:50.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.650+0000 7fc0f0c00640 1 -- 172.21.15.49:0/2722172105 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc0f40105c0 con 0x556fb6cef290 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.651+0000 7fc0f0c00640 1 -- 172.21.15.49:0/2722172105 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f4010780 con 0x556fb6cef290 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.651+0000 7fc110990740 10 client.26747 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.651+0000 7fc110990740 1 -- 172.21.15.49:0/2722172105 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556fb6c554b0 con 0x556fb6cef290 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.651+0000 7fc0f0c00640 1 -- 172.21.15.49:0/2722172105 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556fb69567e0 con 0x556fb6cef290 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.651+0000 7fc0f0c00640 1 -- 172.21.15.49:0/2722172105 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0f400e3b0 con 0x556fb6cef290 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc110990740 10 client.26747 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc110990740 10 client.26747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc110990740 20 client.26747 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc110990740 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556fb6d085e0 0x556fb6d0aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc110990740 4 client.26747 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc110990740 1 -- 172.21.15.49:0/2722172105 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fb6993720 con 0x556fb6d085e0 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc106a00640 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556fb6d085e0 0x556fb6d0aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc106a00640 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556fb6d085e0 0x556fb6d0aa00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99d8b4ce5d60a8e server_cookie=cb8ebaa8e0c9d124 in_seq=0 out_seq=0 2024-02-26T06:28:50.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.652+0000 7fc0f0c00640 10 client.26747 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:50.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.653+0000 7fc0f0c00640 1 -- 172.21.15.49:0/2722172105 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556fb6993720 con 0x556fb6d085e0 2024-02-26T06:28:50.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.653+0000 7fc0f0c00640 10 client.26747 handle_command_reply: tid=0 2024-02-26T06:28:50.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.653+0000 7fc110990740 1 client.26747 shutdown 2024-02-26T06:28:50.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.654+0000 7fc110990740 1 -- 172.21.15.49:0/2722172105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 msgr2=0x556fb6cf55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.654+0000 7fc110990740 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cf55e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005940 tx=0x7fc0f401b850 comp rx=0 tx=0).stop 2024-02-26T06:28:50.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.654+0000 7fc110990740 1 -- 172.21.15.49:0/2722172105 shutdown_connections 2024-02-26T06:28:50.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.654+0000 7fc110990740 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556fb6d085e0 0x556fb6d0aa00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.654+0000 7fc110990740 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cf6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.654+0000 7fc110990740 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.654+0000 7fc110990740 1 --2- 172.21.15.49:0/2722172105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6cef290 0x556fb6cf55e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.654+0000 7fc110990740 1 -- 172.21.15.49:0/2722172105 >> 172.21.15.49:0/2722172105 conn(0x556fb6cb6570 msgr2=0x556fb6ceb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:50.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.655+0000 7fc0f0c00640 0 client.26747 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:50.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.655+0000 7fc110990740 1 -- 172.21.15.49:0/2722172105 shutdown_connections 2024-02-26T06:28:50.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.655+0000 7fc110990740 1 -- 172.21.15.49:0/2722172105 wait complete. 2024-02-26T06:28:50.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.656+0000 7fc110990740 20 client.26747 trim_cache size 0 max 16384 2024-02-26T06:28:50.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.687+0000 7fc110990740 1 Processor -- start 2024-02-26T06:28:50.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.687+0000 7fc110990740 1 -- start start 2024-02-26T06:28:50.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.687+0000 7fc110990740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.687+0000 7fc110990740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cf3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.687+0000 7fc110990740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cf43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.687+0000 7fc110990740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6d0adf0 con 0x556fb6cf8600 2024-02-26T06:28:50.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.687+0000 7fc110990740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6d052c0 con 0x556fb6ce9f20 2024-02-26T06:28:50.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.687+0000 7fc110990740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6cee510 con 0x556fb6ceff70 2024-02-26T06:28:50.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cf3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60378/0 (socket says 172.21.15.49:60378) 2024-02-26T06:28:50.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 -- 172.21.15.49:0/2581363087 learned_addr learned my addr 172.21.15.49:0/2581363087 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:50.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 --2- 172.21.15.49:0/2581363087 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cf43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 -- 172.21.15.49:0/2581363087 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 msgr2=0x556fb6cf43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:50.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 --2- 172.21.15.49:0/2581363087 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cf43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 -- 172.21.15.49:0/2581363087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 msgr2=0x556fb6cf3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:50.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 --2- 172.21.15.49:0/2581363087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cf3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.688+0000 7fc106000640 1 -- 172.21.15.49:0/2581363087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fb6bf1920 con 0x556fb6ceff70 2024-02-26T06:28:50.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.690+0000 7fc106000640 1 --2- 172.21.15.49:0/2581363087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5eb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x556fb6cf83a0 tx=0x7fc0f40097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61c65fbac6836d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:50.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.690+0000 7fc0f3400640 1 -- 172.21.15.49:0/2581363087 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4013cc0 con 0x556fb6ceff70 2024-02-26T06:28:50.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.690+0000 7fc0f3400640 1 -- 172.21.15.49:0/2581363087 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0f400ed10 con 0x556fb6ceff70 2024-02-26T06:28:50.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.690+0000 7fc0f3400640 1 -- 172.21.15.49:0/2581363087 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40053b0 con 0x556fb6ceff70 2024-02-26T06:28:50.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.691+0000 7fc110990740 1 -- 172.21.15.49:0/2581363087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 msgr2=0x556fb6cf5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.692+0000 7fc110990740 1 --2- 172.21.15.49:0/2581363087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5eb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x556fb6cf83a0 tx=0x7fc0f40097b0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.692+0000 7fc110990740 1 -- 172.21.15.49:0/2581363087 shutdown_connections 2024-02-26T06:28:50.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.692+0000 7fc110990740 1 --2- 172.21.15.49:0/2581363087 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cf43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.692+0000 7fc110990740 1 --2- 172.21.15.49:0/2581363087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cf8600 0x556fb6cf3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.692+0000 7fc110990740 1 --2- 172.21.15.49:0/2581363087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6ceff70 0x556fb6cf5eb0 secure :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0x556fb6cf83a0 tx=0x7fc0f40097b0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.692+0000 7fc110990740 1 -- 172.21.15.49:0/2581363087 >> 172.21.15.49:0/2581363087 conn(0x556fb6cb6570 msgr2=0x556fb6ce9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:50.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.693+0000 7fc110990740 1 -- 172.21.15.49:0/2581363087 shutdown_connections 2024-02-26T06:28:50.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.693+0000 7fc110990740 1 -- 172.21.15.49:0/2581363087 wait complete. 2024-02-26T06:28:50.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.694+0000 7fc110990740 1 -- start start 2024-02-26T06:28:50.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.694+0000 7fc110990740 1 Processor -- start 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc110990740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc110990740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6cf8600 0x556fb6cfbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc110990740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cfc080 0x556fb6cfc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc110990740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6cffd40 con 0x556fb6cfc080 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc110990740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6cffeb0 con 0x556fb6ce9f20 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc110990740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb6d5eb20 con 0x556fb6cf8600 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc110990740 10 client.0 _ll_register_callbacks cb 0x7fc0f4009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc106a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6cf8600 0x556fb6cfbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc106a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6cf8600 0x556fb6cfbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60386/0 (socket says 172.21.15.49:60386) 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc106a00640 1 -- 172.21.15.49:0/1071481313 learned_addr learned my addr 172.21.15.49:0/1071481313 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc106000640 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.696+0000 7fc106000640 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cfc080 0x556fb6cfc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.697+0000 7fc106a00640 1 -- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 msgr2=0x556fb6cff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:50.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.697+0000 7fc106a00640 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.697+0000 7fc106a00640 1 -- 172.21.15.49:0/1071481313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cfc080 msgr2=0x556fb6cfc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:50.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.697+0000 7fc106a00640 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cfc080 0x556fb6cfc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.697+0000 7fc106a00640 1 -- 172.21.15.49:0/1071481313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fb6c554b0 con 0x556fb6cf8600 2024-02-26T06:28:50.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.697+0000 7fc106000640 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.698+0000 7fc106a00640 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6cf8600 0x556fb6cfbb40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc0f802c0b0 tx=0x7fc0f8034d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49a6999d8ac593e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.698+0000 7fc0f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.698+0000 7fc0f0c00640 1 -- 172.21.15.49:0/1071481313 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8043ea0 con 0x556fb6cf8600 2024-02-26T06:28:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.699+0000 7fc0f0c00640 1 -- 172.21.15.49:0/1071481313 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fc0f8004450 con 0x556fb6cf8600 2024-02-26T06:28:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.699+0000 7fc110990740 1 -- 172.21.15.49:0/1071481313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fb6cefc10 con 0x556fb6cf8600 2024-02-26T06:28:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.699+0000 7fc0f0c00640 1 -- 172.21.15.49:0/1071481313 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f8034440 con 0x556fb6cf8600 2024-02-26T06:28:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.700+0000 7fc110990740 10 client.26761 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.700+0000 7fc110990740 1 -- 172.21.15.49:0/1071481313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556fb6c17400 con 0x556fb6cf8600 2024-02-26T06:28:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.700+0000 7fc0f0c00640 1 -- 172.21.15.49:0/1071481313 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556fb6cefc10 con 0x556fb6cf8600 2024-02-26T06:28:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.700+0000 7fc0f0c00640 1 -- 172.21.15.49:0/1071481313 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0f8053070 con 0x556fb6cf8600 2024-02-26T06:28:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.700+0000 7fc110990740 10 client.26761 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.700+0000 7fc110990740 10 client.26761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.700+0000 7fc110990740 20 client.26761 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.701+0000 7fc110990740 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556fb6d603e0 0x556fb6d62800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.701+0000 7fc110990740 4 client.26761 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:28:50.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.701+0000 7fc110990740 1 -- 172.21.15.49:0/1071481313 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fc0f4010ca0 con 0x556fb6d603e0 2024-02-26T06:28:50.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.701+0000 7fc106000640 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556fb6d603e0 0x556fb6d62800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.703+0000 7fc106000640 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556fb6d603e0 0x556fb6d62800 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5b9e1dc868d54e server_cookie=85ae11fd3787fecc in_seq=0 out_seq=0 2024-02-26T06:28:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.703+0000 7fc0f0c00640 10 client.26761 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.703+0000 7fc0f0c00640 1 -- 172.21.15.49:0/1071481313 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fc0f4010ca0 con 0x556fb6d603e0 2024-02-26T06:28:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.703+0000 7fc0f0c00640 10 client.26761 handle_command_reply: tid=0 2024-02-26T06:28:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.703+0000 7fc110990740 1 client.26761 shutdown 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 -- 172.21.15.49:0/1071481313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6cf8600 msgr2=0x556fb6cfbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6cf8600 0x556fb6cfbb40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc0f802c0b0 tx=0x7fc0f8034d70 comp rx=0 tx=0).stop 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 -- 172.21.15.49:0/1071481313 shutdown_connections 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556fb6d603e0 0x556fb6d62800 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb6cfc080 0x556fb6cfc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fb6cf8600 0x556fb6cfbb40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 --2- 172.21.15.49:0/1071481313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556fb6ce9f20 0x556fb6cff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 -- 172.21.15.49:0/1071481313 >> 172.21.15.49:0/1071481313 conn(0x556fb6cb6570 msgr2=0x556fb6cf3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc0f0c00640 0 client.26761 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 -- 172.21.15.49:0/1071481313 shutdown_connections 2024-02-26T06:28:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 1 -- 172.21.15.49:0/1071481313 wait complete. 2024-02-26T06:28:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.704+0000 7fc110990740 20 client.26761 trim_cache size 0 max 16384 2024-02-26T06:28:50.705 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:28:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:28:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:28:50.705 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 -- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc0ec0a22b0 msgr2=0x7fc0ec0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc0ec0a22b0 0x7fc0ec0a4770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc100098150 tx=0x7fc100005870 comp rx=0 tx=0).stop 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 -- 172.21.15.49:0/4183835791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1081142d0 msgr2=0x7fc1081a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1081142d0 0x7fc1081a6a40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc10000e9d0 tx=0x7fc10000ee90 comp rx=0 tx=0).stop 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 -- 172.21.15.49:0/4183835791 shutdown_connections 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc0ec0a22b0 0x7fc0ec0a4770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1081142d0 0x7fc1081a6a40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc108108c80 0x7fc108113550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 --2- 172.21.15.49:0/4183835791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc108108360 0x7fc108113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.706+0000 7fc10da00640 1 -- 172.21.15.49:0/4183835791 >> 172.21.15.49:0/4183835791 conn(0x7fc10806c260 msgr2=0x7fc10806fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:50.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.707+0000 7fc10da00640 1 -- 172.21.15.49:0/4183835791 shutdown_connections 2024-02-26T06:28:50.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:50.707+0000 7fc10da00640 1 -- 172.21.15.49:0/4183835791 wait complete. 2024-02-26T06:28:50.730 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:28:50.731 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:28:51.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.007+0000 7f8423200640 1 -- 172.21.15.49:0/1137165044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107560 msgr2=0x7f84241079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.007+0000 7f8423200640 1 --2- 172.21.15.49:0/1137165044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107560 0x7f84241079e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8410009930 tx=0x7f841002bc20 comp rx=0 tx=0).stop 2024-02-26T06:28:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.011+0000 7f8423200640 1 -- 172.21.15.49:0/1137165044 shutdown_connections 2024-02-26T06:28:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.012+0000 7f8423200640 1 --2- 172.21.15.49:0/1137165044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107560 0x7f84241079e0 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8410009930 tx=0x7f841002bc20 comp rx=0 tx=0).stop 2024-02-26T06:28:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.012+0000 7f8423200640 1 --2- 172.21.15.49:0/1137165044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8424107f20 0x7f84241143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.012+0000 7f8423200640 1 --2- 172.21.15.49:0/1137165044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842410d620 0x7f842410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.012+0000 7f8423200640 1 -- 172.21.15.49:0/1137165044 >> 172.21.15.49:0/1137165044 conn(0x7f8424076ca0 msgr2=0x7f84240790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:51.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.015+0000 7f8423200640 1 -- 172.21.15.49:0/1137165044 shutdown_connections 2024-02-26T06:28:51.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.015+0000 7f8423200640 1 -- 172.21.15.49:0/1137165044 wait complete. 2024-02-26T06:28:51.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.016+0000 7f8423200640 1 Processor -- start 2024-02-26T06:28:51.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.017+0000 7f8423200640 1 -- start start 2024-02-26T06:28:51.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.017+0000 7f8423200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107f20 0x7f842419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.019+0000 7f8423200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842410d620 0x7f842419e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.019+0000 7f8423200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84241a2650 0x7f84241a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.019+0000 7f8423200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84241a3040 con 0x7f84241a2650 2024-02-26T06:28:51.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.019+0000 7f8423200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84241a31b0 con 0x7f8424107f20 2024-02-26T06:28:51.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.019+0000 7f8423200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84241a3320 con 0x7f842410d620 2024-02-26T06:28:51.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.019+0000 7f8421e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107f20 0x7f842419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.019+0000 7f8421e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107f20 0x7f842419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52648/0 (socket says 172.21.15.49:52648) 2024-02-26T06:28:51.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.019+0000 7f8421e00640 1 -- 172.21.15.49:0/2108956294 learned_addr learned my addr 172.21.15.49:0/2108956294 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:51.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.020+0000 7f8421e00640 1 -- 172.21.15.49:0/2108956294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842410d620 msgr2=0x7f842419e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.022+0000 7f8421e00640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842410d620 0x7f842419e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.022+0000 7f8421e00640 1 -- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84241a2650 msgr2=0x7f84241a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.022+0000 7f8421e00640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84241a2650 0x7f84241a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.022+0000 7f8421e00640 1 -- 172.21.15.49:0/2108956294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c009840 con 0x7f8424107f20 2024-02-26T06:28:51.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.022+0000 7f8421e00640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107f20 0x7f842419dd90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f841400ea60 tx=0x7f841400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:51.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.023+0000 7f841aa00640 1 -- 172.21.15.49:0/2108956294 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841400cde0 con 0x7f8424107f20 2024-02-26T06:28:51.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.023+0000 7f841aa00640 1 -- 172.21.15.49:0/2108956294 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84140035f0 con 0x7f8424107f20 2024-02-26T06:28:51.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.023+0000 7f841aa00640 1 -- 172.21.15.49:0/2108956294 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414010690 con 0x7f8424107f20 2024-02-26T06:28:51.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.024+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8414009490 con 0x7f8424107f20 2024-02-26T06:28:51.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.024+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8410009560 con 0x7f8424107f20 2024-02-26T06:28:51.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.026+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84240682e0 con 0x7f8424107f20 2024-02-26T06:28:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.031+0000 7f841aa00640 1 -- 172.21.15.49:0/2108956294 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8414003af0 con 0x7f8424107f20 2024-02-26T06:28:51.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.034+0000 7f841aa00640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f84080a2370 0x7f84080a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.035+0000 7f841aa00640 1 -- 172.21.15.49:0/2108956294 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f8414014070 con 0x7f8424107f20 2024-02-26T06:28:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.038+0000 7f8421400640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f84080a2370 0x7f84080a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.039+0000 7f841aa00640 1 -- 172.21.15.49:0/2108956294 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8414097260 con 0x7f8424107f20 2024-02-26T06:28:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.040+0000 7f8421400640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f84080a2370 0x7f84080a4830 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8410009850 tx=0x7f8410000bc0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:50 smithi080 ceph-mon[36206]: pgmap v166: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 1.46k op/s 2024-02-26T06:28:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:50 smithi115 ceph-mon[36426]: pgmap v166: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 1.46k op/s 2024-02-26T06:28:51.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:50 smithi049 ceph-mon[31696]: pgmap v166: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 1.46k op/s 2024-02-26T06:28:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 300.0 s, 2050.8 tps, lat 52.133 ms stddev 83.001 2024-02-26T06:28:51.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.437+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f842419f0e0 con 0x7f8424107f20 2024-02-26T06:28:51.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.440+0000 7f841aa00640 1 -- 172.21.15.49:0/2108956294 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11334 (secure 0 0 0) 0x7f841409a920 con 0x7f8424107f20 2024-02-26T06:28:51.443 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:28:51.443 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:27:29.193881+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 35 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f84080a2370 msgr2=0x7f84080a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f84080a2370 0x7f84080a4830 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8410009850 tx=0x7f8410000bc0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107f20 msgr2=0x7f842419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107f20 0x7f842419dd90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f841400ea60 tx=0x7f841400ef20 comp rx=0 tx=0).stop 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 shutdown_connections 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f84080a2370 0x7f84080a4830 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84241a2650 0x7f84241a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842410d620 0x7f842419e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 --2- 172.21.15.49:0/2108956294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107f20 0x7f842419dd90 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f841400ea60 tx=0x7f841400ef20 comp rx=0 tx=0).stop 2024-02-26T06:28:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.443+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 >> 172.21.15.49:0/2108956294 conn(0x7f8424076ca0 msgr2=0x7f84240786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:51.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.444+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 shutdown_connections 2024-02-26T06:28:51.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.444+0000 7f8423200640 1 -- 172.21.15.49:0/2108956294 wait complete. 2024-02-26T06:28:51.476 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:51.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:28:51.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.754+0000 7f37e0400640 1 -- 172.21.15.49:0/2331421657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8103590 msgr2=0x7f37d8103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.754+0000 7f37e0400640 1 --2- 172.21.15.49:0/2331421657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8103590 0x7f37d8103970 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009930 tx=0x7f37cc02bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:51.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.755+0000 7f37e0400640 1 -- 172.21.15.49:0/2331421657 shutdown_connections 2024-02-26T06:28:51.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.755+0000 7f37e0400640 1 --2- 172.21.15.49:0/2331421657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8108660 0x7f37d8110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.755+0000 7f37e0400640 1 --2- 172.21.15.49:0/2331421657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d8103f40 0x7f37d8107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.755+0000 7f37e0400640 1 --2- 172.21.15.49:0/2331421657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8103590 0x7f37d8103970 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.755+0000 7f37e0400640 1 -- 172.21.15.49:0/2331421657 >> 172.21.15.49:0/2331421657 conn(0x7f37d80fee30 msgr2=0x7f37d8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:51.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.755+0000 7f37e0400640 1 -- 172.21.15.49:0/2331421657 shutdown_connections 2024-02-26T06:28:51.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.755+0000 7f37e0400640 1 -- 172.21.15.49:0/2331421657 wait complete. 2024-02-26T06:28:51.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.756+0000 7f37e0400640 1 Processor -- start 2024-02-26T06:28:51.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.756+0000 7f37e0400640 1 -- start start 2024-02-26T06:28:51.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.756+0000 7f37e0400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8103590 0x7f37d8075690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.756+0000 7f37e0400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8108660 0x7f37d8075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37e0400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d8076110 0x7f37d81a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37e0400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d8079870 con 0x7f37d8103590 2024-02-26T06:28:51.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37e0400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81aa4e0 con 0x7f37d8108660 2024-02-26T06:28:51.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37e0400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81aa6a0 con 0x7f37d8076110 2024-02-26T06:28:51.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dd000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8108660 0x7f37d8075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dd000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8108660 0x7f37d8075bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52662/0 (socket says 172.21.15.49:52662) 2024-02-26T06:28:51.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dd000640 1 -- 172.21.15.49:0/1667158809 learned_addr learned my addr 172.21.15.49:0/1667158809 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:51.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dda00640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8103590 0x7f37d8075690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37de400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d8076110 0x7f37d81a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dd000640 1 -- 172.21.15.49:0/1667158809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d8076110 msgr2=0x7f37d81a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dd000640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d8076110 0x7f37d81a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dd000640 1 -- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8103590 msgr2=0x7f37d8075690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dd000640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8103590 0x7f37d8075690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.757+0000 7f37dd000640 1 -- 172.21.15.49:0/1667158809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009560 con 0x7f37d8108660 2024-02-26T06:28:51.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.758+0000 7f37dd000640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8108660 0x7f37d8075bd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f37c400ea20 tx=0x7f37c400eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:51.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.758+0000 7f37d6000640 1 -- 172.21.15.49:0/1667158809 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c40096c0 con 0x7f37d8108660 2024-02-26T06:28:51.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.758+0000 7f37d6000640 1 -- 172.21.15.49:0/1667158809 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37c4005500 con 0x7f37d8108660 2024-02-26T06:28:51.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.758+0000 7f37d6000640 1 -- 172.21.15.49:0/1667158809 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4010430 con 0x7f37d8108660 2024-02-26T06:28:51.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.758+0000 7f37e0400640 1 -- 172.21.15.49:0/1667158809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d81aa980 con 0x7f37d8108660 2024-02-26T06:28:51.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.759+0000 7f37e0400640 1 -- 172.21.15.49:0/1667158809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d81aae40 con 0x7f37d8108660 2024-02-26T06:28:51.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.763+0000 7f37e33e1740 1 Processor -- start 2024-02-26T06:28:51.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.766+0000 7f37e33e1740 1 -- start start 2024-02-26T06:28:51.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b884fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8858bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8859110 0x5575b885b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37e33e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b87be1e0 con 0x5575b8850530 2024-02-26T06:28:51.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37e33e1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b87bb710 con 0x5575b884fc10 2024-02-26T06:28:51.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37e33e1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b87b6ef0 con 0x5575b8859110 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37de400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b884fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37de400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b884fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52674/0 (socket says 172.21.15.49:52674) 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37de400640 1 -- 172.21.15.49:0/3397812867 learned_addr learned my addr 172.21.15.49:0/3397812867 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.767+0000 7f37dda00640 1 --2- 172.21.15.49:0/3397812867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8858bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.768+0000 7f37de400640 1 -- 172.21.15.49:0/3397812867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8859110 msgr2=0x5575b885b500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.768+0000 7f37de400640 1 --2- 172.21.15.49:0/3397812867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8859110 0x5575b885b500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.768+0000 7f37de400640 1 -- 172.21.15.49:0/3397812867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 msgr2=0x5575b8858bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.768+0000 7f37de400640 1 --2- 172.21.15.49:0/3397812867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8858bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.768+0000 7f37de400640 1 -- 172.21.15.49:0/3397812867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b8778dc0 con 0x5575b884fc10 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.768+0000 7f37d6000640 1 -- 172.21.15.49:0/1667158809 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f37c4005020 con 0x7f37d8108660 2024-02-26T06:28:51.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.768+0000 7f37de400640 1 --2- 172.21.15.49:0/3397812867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b884fff0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f37d0007ec0 tx=0x7f37d000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24bddd89c7400f3b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:51.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.769+0000 7f37caa00640 1 -- 172.21.15.49:0/3397812867 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0010840 con 0x5575b884fc10 2024-02-26T06:28:51.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.769+0000 7f37caa00640 1 -- 172.21.15.49:0/3397812867 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37d0010e80 con 0x5575b884fc10 2024-02-26T06:28:51.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.769+0000 7f37caa00640 1 -- 172.21.15.49:0/3397812867 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d00173a0 con 0x5575b884fc10 2024-02-26T06:28:51.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.770+0000 7f37e33e1740 1 -- 172.21.15.49:0/3397812867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 msgr2=0x5575b884fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.770+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3397812867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b884fff0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f37d0007ec0 tx=0x7f37d000f790 comp rx=0 tx=0).stop 2024-02-26T06:28:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.770+0000 7f37d6000640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37c00a2440 0x7f37c00a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.771+0000 7f37d6000640 1 -- 172.21.15.49:0/1667158809 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f37c4014070 con 0x7f37d8108660 2024-02-26T06:28:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.771+0000 7f37e33e1740 1 -- 172.21.15.49:0/3397812867 shutdown_connections 2024-02-26T06:28:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.771+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3397812867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8859110 0x5575b885b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.771+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3397812867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8858bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.771+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3397812867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b884fff0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.771+0000 7f37e33e1740 1 -- 172.21.15.49:0/3397812867 >> 172.21.15.49:0/3397812867 conn(0x5575b884a340 msgr2=0x5575b884a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:51.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.771+0000 7f37de400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37c00a2440 0x7f37c00a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.772+0000 7f37e33e1740 1 -- 172.21.15.49:0/3397812867 shutdown_connections 2024-02-26T06:28:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.772+0000 7f37e33e1740 1 -- 172.21.15.49:0/3397812867 wait complete. 2024-02-26T06:28:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.772+0000 7f37de400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37c00a2440 0x7f37c00a4900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37d0018a00 tx=0x7f37d0018a60 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.772+0000 7f37e33e1740 1 -- start start 2024-02-26T06:28:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.773+0000 7f37e33e1740 1 Processor -- start 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b88561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 0x5575b8856700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 0x5575b8856c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37e33e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b875c120 con 0x5575b8859110 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37e33e1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b875bdc0 con 0x5575b884fc10 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37e33e1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b85251c0 con 0x5575b8850530 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37e33e1740 10 client.0 _ll_register_callbacks cb 0x5575b87ab280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37dda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b88561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37dda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b88561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52686/0 (socket says 172.21.15.49:52686) 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37dda00640 1 -- 172.21.15.49:0/3996786225 learned_addr learned my addr 172.21.15.49:0/3996786225 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37dda00640 1 -- 172.21.15.49:0/3996786225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 msgr2=0x5575b8856700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37dda00640 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 0x5575b8856700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37dda00640 1 -- 172.21.15.49:0/3996786225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 msgr2=0x5575b8856c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37dda00640 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 0x5575b8856c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.774+0000 7f37dda00640 1 -- 172.21.15.49:0/3996786225 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d0007af0 con 0x5575b884fc10 2024-02-26T06:28:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.775+0000 7f37dda00640 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b88561c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f37cc005ab0 tx=0x7f37cc02cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dac2296a976ab55 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.775+0000 7f37bf400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.775+0000 7f37bf400640 1 -- 172.21.15.49:0/3996786225 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc004250 con 0x5575b884fc10 2024-02-26T06:28:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.775+0000 7f37bf400640 1 -- 172.21.15.49:0/3996786225 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f37cc046070 con 0x5575b884fc10 2024-02-26T06:28:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.775+0000 7f37e33e1740 1 -- 172.21.15.49:0/3996786225 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575b84f3720 con 0x5575b884fc10 2024-02-26T06:28:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.776+0000 7f37bf400640 1 -- 172.21.15.49:0/3996786225 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37cc0043b0 con 0x5575b884fc10 2024-02-26T06:28:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.776+0000 7f37e33e1740 10 client.26771 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.776+0000 7f37e33e1740 1 -- 172.21.15.49:0/3996786225 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5575b87b5570 con 0x5575b884fc10 2024-02-26T06:28:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.777+0000 7f37bf400640 1 -- 172.21.15.49:0/3996786225 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5575b84f3720 con 0x5575b884fc10 2024-02-26T06:28:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.777+0000 7f37bf400640 1 -- 172.21.15.49:0/3996786225 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37cc03c040 con 0x5575b884fc10 2024-02-26T06:28:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.777+0000 7f37e33e1740 10 client.26771 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.777+0000 7f37e33e1740 10 client.26771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:28:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.777+0000 7f37e33e1740 20 client.26771 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.777+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5575b887a260 0x5575b887c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.777+0000 7f37e33e1740 4 client.26771 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.777+0000 7f37e33e1740 1 -- 172.21.15.49:0/3996786225 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575b8542890 con 0x5575b887a260 2024-02-26T06:28:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.778+0000 7f37dd000640 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5575b887a260 0x5575b887c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.779+0000 7f37bf400640 10 client.26771 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:28:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.779+0000 7f37dd000640 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5575b887a260 0x5575b887c680 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c079439bb0b47a4 server_cookie=5b1bec4b9ffe9606 in_seq=0 out_seq=0 2024-02-26T06:28:51.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.780+0000 7f37bf400640 1 -- 172.21.15.49:0/3996786225 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575b8542890 con 0x5575b887a260 2024-02-26T06:28:51.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.780+0000 7f37bf400640 10 client.26771 handle_command_reply: tid=0 2024-02-26T06:28:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.780+0000 7f37e33e1740 1 client.26771 shutdown 2024-02-26T06:28:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.780+0000 7f37e33e1740 1 -- 172.21.15.49:0/3996786225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 msgr2=0x5575b88561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.780+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b88561c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f37cc005ab0 tx=0x7f37cc02cb50 comp rx=0 tx=0).stop 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 1 -- 172.21.15.49:0/3996786225 shutdown_connections 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5575b887a260 0x5575b887c680 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37bf400640 0 client.26771 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 0x5575b8856c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 0x5575b8856700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 1 --2- 172.21.15.49:0/3996786225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b88561c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 1 -- 172.21.15.49:0/3996786225 >> 172.21.15.49:0/3996786225 conn(0x5575b884a340 msgr2=0x5575b884c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 1 -- 172.21.15.49:0/3996786225 shutdown_connections 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 1 -- 172.21.15.49:0/3996786225 wait complete. 2024-02-26T06:28:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.781+0000 7f37e33e1740 20 client.26771 trim_cache size 0 max 16384 2024-02-26T06:28:51.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.803+0000 7f37e33e1740 1 Processor -- start 2024-02-26T06:28:51.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37e33e1740 1 -- start start 2024-02-26T06:28:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b884fc10 0x5575b8856710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8856c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b8859110 0x5575b8853420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37e33e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b88818d0 con 0x5575b8850530 2024-02-26T06:28:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37e33e1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b884af10 con 0x5575b8859110 2024-02-26T06:28:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37e33e1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b8845c10 con 0x5575b884fc10 2024-02-26T06:28:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37dda00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b884fc10 0x5575b8856710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37dda00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b884fc10 0x5575b8856710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60426/0 (socket says 172.21.15.49:60426) 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.804+0000 7f37dda00640 1 -- 172.21.15.49:0/4060659501 learned_addr learned my addr 172.21.15.49:0/4060659501 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.805+0000 7f37dd000640 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8856c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.805+0000 7f37de400640 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b8859110 0x5575b8853420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.805+0000 7f37dda00640 1 -- 172.21.15.49:0/4060659501 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b8859110 msgr2=0x5575b8853420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.805+0000 7f37dda00640 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b8859110 0x5575b8853420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.805+0000 7f37dda00640 1 -- 172.21.15.49:0/4060659501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 msgr2=0x5575b8856c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.805+0000 7f37dda00640 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8856c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.805+0000 7f37dda00640 1 -- 172.21.15.49:0/4060659501 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b8751a80 con 0x5575b884fc10 2024-02-26T06:28:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.805+0000 7f37dd000640 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8856c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:51.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.806+0000 7f37dda00640 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b884fc10 0x5575b8856710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009a50 tx=0x7f37cc009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde2dbdc7a183334 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:51.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.806+0000 7f37ca000640 1 -- 172.21.15.49:0/4060659501 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc02c630 con 0x5575b884fc10 2024-02-26T06:28:51.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.806+0000 7f37ca000640 1 -- 172.21.15.49:0/4060659501 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37cc044070 con 0x5575b884fc10 2024-02-26T06:28:51.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.806+0000 7f37ca000640 1 -- 172.21.15.49:0/4060659501 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc02c630 con 0x5575b884fc10 2024-02-26T06:28:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.807+0000 7f37e33e1740 1 -- 172.21.15.49:0/4060659501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b884fc10 msgr2=0x5575b8856710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.807+0000 7f37e33e1740 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b884fc10 0x5575b8856710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009a50 tx=0x7f37cc009230 comp rx=0 tx=0).stop 2024-02-26T06:28:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.807+0000 7f37e33e1740 1 -- 172.21.15.49:0/4060659501 shutdown_connections 2024-02-26T06:28:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.807+0000 7f37e33e1740 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b8859110 0x5575b8853420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.807+0000 7f37e33e1740 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8850530 0x5575b8856c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.807+0000 7f37e33e1740 1 --2- 172.21.15.49:0/4060659501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b884fc10 0x5575b8856710 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.807+0000 7f37e33e1740 1 -- 172.21.15.49:0/4060659501 >> 172.21.15.49:0/4060659501 conn(0x5575b884a340 msgr2=0x5575b88514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.808+0000 7f37e33e1740 1 -- 172.21.15.49:0/4060659501 shutdown_connections 2024-02-26T06:28:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.808+0000 7f37e33e1740 1 -- 172.21.15.49:0/4060659501 wait complete. 2024-02-26T06:28:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.808+0000 7f37e33e1740 1 -- start start 2024-02-26T06:28:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.808+0000 7f37e33e1740 1 Processor -- start 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b885e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 0x5575b885eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37e33e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 0x5575b885f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37e33e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b88d07b0 con 0x5575b8859110 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37e33e1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b88d08f0 con 0x5575b884fc10 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37e33e1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b88d0a30 con 0x5575b8850530 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37e33e1740 10 client.0 _ll_register_callbacks cb 0x7f37d00078e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37dd000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 0x5575b885eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37dda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b885e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.809+0000 7f37de400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 0x5575b885f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.810+0000 7f37dda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b885e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52708/0 (socket says 172.21.15.49:52708) 2024-02-26T06:28:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.810+0000 7f37dd000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 0x5575b885eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60430/0 (socket says 172.21.15.49:60430) 2024-02-26T06:28:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.810+0000 7f37dda00640 1 -- 172.21.15.49:0/2718218889 learned_addr learned my addr 172.21.15.49:0/2718218889 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.810+0000 7f37de400640 1 -- 172.21.15.49:0/2718218889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 msgr2=0x5575b885eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.810+0000 7f37de400640 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 0x5575b885eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.810+0000 7f37de400640 1 -- 172.21.15.49:0/2718218889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 msgr2=0x5575b885e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.810+0000 7f37de400640 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b885e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.810+0000 7f37de400640 1 -- 172.21.15.49:0/2718218889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575b87b5570 con 0x5575b8859110 2024-02-26T06:28:51.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.811+0000 7f37de400640 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 0x5575b885f410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f37d0007e90 tx=0x7f37d0018b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae017d4b8f8e9a00 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:51.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.811+0000 7f37bf400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:28:51.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.812+0000 7f37bf400640 1 -- 172.21.15.49:0/2718218889 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0010a50 con 0x5575b8859110 2024-02-26T06:28:51.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.812+0000 7f37bf400640 1 -- 172.21.15.49:0/2718218889 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f37d002a040 con 0x5575b8859110 2024-02-26T06:28:51.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.812+0000 7f37e33e1740 1 -- 172.21.15.49:0/2718218889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575b84b67e0 con 0x5575b8859110 2024-02-26T06:28:51.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.813+0000 7f37bf400640 1 -- 172.21.15.49:0/2718218889 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37d0010a50 con 0x5575b8859110 2024-02-26T06:28:51.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.813+0000 7f37e33e1740 10 client.16314 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:51.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.813+0000 7f37e33e1740 1 -- 172.21.15.49:0/2718218889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5575b8777620 con 0x5575b8859110 2024-02-26T06:28:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.813+0000 7f37bf400640 1 -- 172.21.15.49:0/2718218889 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5575b84b67e0 con 0x5575b8859110 2024-02-26T06:28:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.813+0000 7f37bf400640 1 -- 172.21.15.49:0/2718218889 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37d0020040 con 0x5575b8859110 2024-02-26T06:28:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.813+0000 7f37e33e1740 10 client.16314 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.814+0000 7f37e33e1740 10 client.16314 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:28:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.814+0000 7f37e33e1740 20 client.16314 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.814+0000 7f37e33e1740 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5575b88d2180 0x5575b88d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.814+0000 7f37e33e1740 4 client.16314 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:51.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.814+0000 7f37e33e1740 1 -- 172.21.15.49:0/2718218889 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575b84a54e0 con 0x5575b88d2180 2024-02-26T06:28:51.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.814+0000 7f37dda00640 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5575b88d2180 0x5575b88d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:51.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.815+0000 7f37bf400640 10 client.16314 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:28:51.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.815+0000 7f37dda00640 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5575b88d2180 0x5575b88d45a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc8bbfa58379e4ee server_cookie=6e0964499d0a52dc in_seq=0 out_seq=0 2024-02-26T06:28:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37bf400640 1 -- 172.21.15.49:0/2718218889 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575b84a54e0 con 0x5575b88d2180 2024-02-26T06:28:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37bf400640 10 client.16314 handle_command_reply: tid=0 2024-02-26T06:28:51.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 client.16314 shutdown 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 -- 172.21.15.49:0/2718218889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 msgr2=0x5575b885f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 0x5575b885f410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f37d0007e90 tx=0x7f37d0018b00 comp rx=0 tx=0).stop 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 -- 172.21.15.49:0/2718218889 shutdown_connections 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5575b88d2180 0x5575b88d45a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575b8859110 0x5575b885f410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575b8850530 0x5575b885eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 --2- 172.21.15.49:0/2718218889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5575b884fc10 0x5575b885e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 -- 172.21.15.49:0/2718218889 >> 172.21.15.49:0/2718218889 conn(0x5575b884a340 msgr2=0x5575b884b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37bf400640 0 client.16314 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:28:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 -- 172.21.15.49:0/2718218889 shutdown_connections 2024-02-26T06:28:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.816+0000 7f37e33e1740 1 -- 172.21.15.49:0/2718218889 wait complete. 2024-02-26T06:28:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.817+0000 7f37e33e1740 20 client.16314 trim_cache size 0 max 16384 2024-02-26T06:28:51.817 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:28:51.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.818+0000 7f37e0400640 1 -- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37c00a2440 msgr2=0x7f37c00a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.818+0000 7f37e0400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37c00a2440 0x7f37c00a4900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37d0018a00 tx=0x7f37d0018a60 comp rx=0 tx=0).stop 2024-02-26T06:28:51.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.818+0000 7f37e0400640 1 -- 172.21.15.49:0/1667158809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8108660 msgr2=0x7f37d8075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:51.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.818+0000 7f37e0400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8108660 0x7f37d8075bd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f37c400ea20 tx=0x7f37c400eee0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.819+0000 7f37e0400640 1 -- 172.21.15.49:0/1667158809 shutdown_connections 2024-02-26T06:28:51.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.819+0000 7f37e0400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37c00a2440 0x7f37c00a4900 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.819+0000 7f37e0400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37d8108660 0x7f37d8075bd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.819+0000 7f37e0400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d8076110 0x7f37d81a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.819+0000 7f37e0400640 1 --2- 172.21.15.49:0/1667158809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37d8103590 0x7f37d8075690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:51.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.819+0000 7f37e0400640 1 -- 172.21.15.49:0/1667158809 >> 172.21.15.49:0/1667158809 conn(0x7f37d80fee30 msgr2=0x7f37d8101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:51.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.819+0000 7f37e0400640 1 -- 172.21.15.49:0/1667158809 shutdown_connections 2024-02-26T06:28:51.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:51.819+0000 7f37e0400640 1 -- 172.21.15.49:0/1667158809 wait complete. 2024-02-26T06:28:51.840 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:51.840 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:28:52.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:51 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2108956294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:52.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:51 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2108956294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:52.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.132+0000 7fd92c000640 1 -- 172.21.15.49:0/2991076096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd92410d730 msgr2=0x7fd92410db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.132+0000 7fd92c000640 1 --2- 172.21.15.49:0/2991076096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd92410d730 0x7fd92410db10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd918009930 tx=0x7fd91802bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:52.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.133+0000 7fd92c000640 1 -- 172.21.15.49:0/2991076096 shutdown_connections 2024-02-26T06:28:52.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.133+0000 7fd92c000640 1 --2- 172.21.15.49:0/2991076096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9241080e0 0x7fd9241145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.133+0000 7fd92c000640 1 --2- 172.21.15.49:0/2991076096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd924107720 0x7fd924107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.133+0000 7fd92c000640 1 --2- 172.21.15.49:0/2991076096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd92410d730 0x7fd92410db10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.133+0000 7fd92c000640 1 -- 172.21.15.49:0/2991076096 >> 172.21.15.49:0/2991076096 conn(0x7fd9240756c0 msgr2=0x7fd924077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.133+0000 7fd92c000640 1 -- 172.21.15.49:0/2991076096 shutdown_connections 2024-02-26T06:28:52.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.133+0000 7fd92c000640 1 -- 172.21.15.49:0/2991076096 wait complete. 2024-02-26T06:28:52.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.136+0000 7fd92c000640 1 Processor -- start 2024-02-26T06:28:52.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.136+0000 7fd92c000640 1 -- start start 2024-02-26T06:28:52.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.137+0000 7fd92c000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd924107720 0x7fd92419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.137+0000 7fd92c000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9241080e0 0x7fd92419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd92c000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd92410d730 0x7fd9241a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd92c000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a2c90 con 0x7fd92410d730 2024-02-26T06:28:52.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd92c000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a2e00 con 0x7fd9241080e0 2024-02-26T06:28:52.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd92c000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a2fc0 con 0x7fd924107720 2024-02-26T06:28:52.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd929600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd924107720 0x7fd92419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd929600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd924107720 0x7fd92419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60442/0 (socket says 172.21.15.49:60442) 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd929600640 1 -- 172.21.15.49:0/4115187688 learned_addr learned my addr 172.21.15.49:0/4115187688 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd928c00640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9241080e0 0x7fd92419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.138+0000 7fd92a000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd92410d730 0x7fd9241a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.139+0000 7fd929600640 1 -- 172.21.15.49:0/4115187688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9241080e0 msgr2=0x7fd92419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.139+0000 7fd929600640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9241080e0 0x7fd92419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.139+0000 7fd929600640 1 -- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd92410d730 msgr2=0x7fd9241a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.139+0000 7fd929600640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd92410d730 0x7fd9241a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.139+0000 7fd929600640 1 -- 172.21.15.49:0/4115187688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918009560 con 0x7fd924107720 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.139+0000 7fd92a000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd92410d730 0x7fd9241a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:52.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.139+0000 7fd929600640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd924107720 0x7fd92419ddd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd918000c00 tx=0x7fd91802cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.140+0000 7fd922000640 1 -- 172.21.15.49:0/4115187688 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918004350 con 0x7fd924107720 2024-02-26T06:28:52.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.140+0000 7fd922000640 1 -- 172.21.15.49:0/4115187688 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9180044b0 con 0x7fd924107720 2024-02-26T06:28:52.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.140+0000 7fd922000640 1 -- 172.21.15.49:0/4115187688 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918003d30 con 0x7fd924107720 2024-02-26T06:28:52.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.140+0000 7fd92c000640 1 -- 172.21.15.49:0/4115187688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9241a3240 con 0x7fd924107720 2024-02-26T06:28:52.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.140+0000 7fd92c000640 1 -- 172.21.15.49:0/4115187688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9241aab20 con 0x7fd924107720 2024-02-26T06:28:52.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.144+0000 7fd922000640 1 -- 172.21.15.49:0/4115187688 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd918004df0 con 0x7fd924107720 2024-02-26T06:28:52.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.145+0000 7fd92efa2740 1 Processor -- start 2024-02-26T06:28:52.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.145+0000 7fd92efa2740 1 -- start start 2024-02-26T06:28:52.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.145+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d3466ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.145+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d346fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.146+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3470110 0x5573d3472500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.146+0000 7fd92efa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d33d51e0 con 0x5573d3466c10 2024-02-26T06:28:52.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.146+0000 7fd92efa2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d33d2710 con 0x5573d3470110 2024-02-26T06:28:52.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.146+0000 7fd92efa2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d33cdef0 con 0x5573d3467530 2024-02-26T06:28:52.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.146+0000 7fd929600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d346fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.146+0000 7fd92a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d3466ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.146+0000 7fd92a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d3466ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34758/0 (socket says 172.21.15.49:34758) 2024-02-26T06:28:52.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.146+0000 7fd92a000640 1 -- 172.21.15.49:0/1577404785 learned_addr learned my addr 172.21.15.49:0/1577404785 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.147+0000 7fd928c00640 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3470110 0x5573d3472500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.147+0000 7fd92a000640 1 -- 172.21.15.49:0/1577404785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 msgr2=0x5573d346fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.147+0000 7fd92a000640 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d346fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.147+0000 7fd92a000640 1 -- 172.21.15.49:0/1577404785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3470110 msgr2=0x5573d3472500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.147+0000 7fd92a000640 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3470110 0x5573d3472500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.147+0000 7fd92a000640 1 -- 172.21.15.49:0/1577404785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573d338fdc0 con 0x5573d3466c10 2024-02-26T06:28:52.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.147+0000 7fd928c00640 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3470110 0x5573d3472500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:52.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.148+0000 7fd92a000640 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d3466ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011790 tx=0x7fd91c011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95587fdf753a86e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.149+0000 7fd922000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd90c0a2330 0x7fd90c0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.149+0000 7fd922000640 1 -- 172.21.15.49:0/4115187688 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd9180f07a0 con 0x7fd924107720 2024-02-26T06:28:52.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.149+0000 7fd929600640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd90c0a2330 0x7fd90c0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.149+0000 7fd916600640 1 -- 172.21.15.49:0/1577404785 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c011ea0 con 0x5573d3466c10 2024-02-26T06:28:52.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.149+0000 7fd916600640 1 -- 172.21.15.49:0/1577404785 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd91c00fe80 con 0x5573d3466c10 2024-02-26T06:28:52.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.149+0000 7fd916600640 1 -- 172.21.15.49:0/1577404785 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c004820 con 0x5573d3466c10 2024-02-26T06:28:52.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.150+0000 7fd92efa2740 1 -- 172.21.15.49:0/1577404785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 msgr2=0x5573d3466ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.150+0000 7fd92efa2740 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d3466ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011790 tx=0x7fd91c011c50 comp rx=0 tx=0).stop 2024-02-26T06:28:52.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.150+0000 7fd929600640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd90c0a2330 0x7fd90c0a47f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5573d34666e0 tx=0x7fd91802bc30 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.151+0000 7fd92efa2740 1 -- 172.21.15.49:0/1577404785 shutdown_connections 2024-02-26T06:28:52.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.151+0000 7fd92efa2740 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3470110 0x5573d3472500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.151+0000 7fd92efa2740 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d346fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.151+0000 7fd92efa2740 1 --2- 172.21.15.49:0/1577404785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d3466ff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.151+0000 7fd92efa2740 1 -- 172.21.15.49:0/1577404785 >> 172.21.15.49:0/1577404785 conn(0x5573d3461340 msgr2=0x5573d3461720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.151+0000 7fd92efa2740 1 -- 172.21.15.49:0/1577404785 shutdown_connections 2024-02-26T06:28:52.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.151+0000 7fd92efa2740 1 -- 172.21.15.49:0/1577404785 wait complete. 2024-02-26T06:28:52.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.151+0000 7fd92efa2740 1 -- start start 2024-02-26T06:28:52.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.152+0000 7fd92efa2740 1 Processor -- start 2024-02-26T06:28:52.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.152+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d346d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.152+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.152+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3470110 0x5573d346dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.152+0000 7fd92efa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d3373120 con 0x5573d3466c10 2024-02-26T06:28:52.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.152+0000 7fd92efa2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d3372dc0 con 0x5573d3467530 2024-02-26T06:28:52.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.152+0000 7fd92efa2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d313c1c0 con 0x5573d3470110 2024-02-26T06:28:52.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.152+0000 7fd92efa2740 10 client.0 _ll_register_callbacks cb 0x5573d33c2280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:52.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.153+0000 7fd928c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.153+0000 7fd928c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52748/0 (socket says 172.21.15.49:52748) 2024-02-26T06:28:52.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.153+0000 7fd928c00640 1 -- 172.21.15.49:0/2675461603 learned_addr learned my addr 172.21.15.49:0/2675461603 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.154+0000 7fd92a000640 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3470110 0x5573d346dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.154+0000 7fd92a000640 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d346d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.154+0000 7fd928c00640 1 -- 172.21.15.49:0/2675461603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3470110 msgr2=0x5573d346dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.154+0000 7fd928c00640 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3470110 0x5573d346dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.154+0000 7fd928c00640 1 -- 172.21.15.49:0/2675461603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 msgr2=0x5573d346d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.154+0000 7fd928c00640 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d346d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.154+0000 7fd928c00640 1 -- 172.21.15.49:0/2675461603 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c011440 con 0x5573d3467530 2024-02-26T06:28:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.155+0000 7fd92a000640 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d346d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.155+0000 7fd928c00640 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd91000fe80 tx=0x7fd9100117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c09a784eb84d00a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.155+0000 7fd90aa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:52.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.155+0000 7fd90aa00640 1 -- 172.21.15.49:0/2675461603 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910005220 con 0x5573d3467530 2024-02-26T06:28:52.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.156+0000 7fd90aa00640 1 -- 172.21.15.49:0/2675461603 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd910018070 con 0x5573d3467530 2024-02-26T06:28:52.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.156+0000 7fd92efa2740 1 -- 172.21.15.49:0/2675461603 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573d30cd7e0 con 0x5573d3467530 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.156+0000 7fd90aa00640 1 -- 172.21.15.49:0/2675461603 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd910005380 con 0x5573d3467530 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.156+0000 7fd92efa2740 10 client.26786 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.156+0000 7fd92efa2740 1 -- 172.21.15.49:0/2675461603 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5573d33cc570 con 0x5573d3467530 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.157+0000 7fd90aa00640 1 -- 172.21.15.49:0/2675461603 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5573d30cd7e0 con 0x5573d3467530 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.157+0000 7fd90aa00640 1 -- 172.21.15.49:0/2675461603 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd9100128e0 con 0x5573d3467530 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.157+0000 7fd92efa2740 10 client.26786 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.157+0000 7fd92efa2740 10 client.26786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.157+0000 7fd92efa2740 20 client.26786 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.157+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5573d3480190 0x5573d34825b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.157+0000 7fd92efa2740 4 client.26786 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.157+0000 7fd92efa2740 1 -- 172.21.15.49:0/2675461603 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573d310a720 con 0x5573d3480190 2024-02-26T06:28:52.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.158+0000 7fd92a000640 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5573d3480190 0x5573d34825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.159+0000 7fd92a000640 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5573d3480190 0x5573d34825b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90cc4e36e71720f8 server_cookie=46b9659d509888e in_seq=0 out_seq=0 2024-02-26T06:28:52.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.159+0000 7fd90aa00640 10 client.26786 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:28:52.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.159+0000 7fd90aa00640 1 -- 172.21.15.49:0/2675461603 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573d310a720 con 0x5573d3480190 2024-02-26T06:28:52.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.159+0000 7fd90aa00640 10 client.26786 handle_command_reply: tid=0 2024-02-26T06:28:52.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.159+0000 7fd92efa2740 1 client.26786 shutdown 2024-02-26T06:28:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 -- 172.21.15.49:0/2675461603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 msgr2=0x5573d346d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd91000fe80 tx=0x7fd9100117b0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 -- 172.21.15.49:0/2675461603 shutdown_connections 2024-02-26T06:28:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5573d3480190 0x5573d34825b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346d6f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd90aa00640 0 client.26786 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:28:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3470110 0x5573d346dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2675461603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3466c10 0x5573d346d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 -- 172.21.15.49:0/2675461603 >> 172.21.15.49:0/2675461603 conn(0x5573d3461340 msgr2=0x5573d3463090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.160+0000 7fd92efa2740 1 -- 172.21.15.49:0/2675461603 shutdown_connections 2024-02-26T06:28:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.161+0000 7fd92efa2740 1 -- 172.21.15.49:0/2675461603 wait complete. 2024-02-26T06:28:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.161+0000 7fd92efa2740 20 client.26786 trim_cache size 0 max 16384 2024-02-26T06:28:52.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:51 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2108956294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:52.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.181+0000 7fd92efa2740 1 Processor -- start 2024-02-26T06:28:52.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.181+0000 7fd92efa2740 1 -- start start 2024-02-26T06:28:52.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.182+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3466c10 0x5573d346f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.182+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.182+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 0x5573d346bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.182+0000 7fd92efa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d347ea60 con 0x5573d3470110 2024-02-26T06:28:52.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.182+0000 7fd92efa2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d346e500 con 0x5573d3467530 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.182+0000 7fd92efa2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d3462490 con 0x5573d3466c10 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.183+0000 7fd92a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3466c10 0x5573d346f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.183+0000 7fd92a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3466c10 0x5573d346f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60468/0 (socket says 172.21.15.49:60468) 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.183+0000 7fd92a000640 1 -- 172.21.15.49:0/2242068347 learned_addr learned my addr 172.21.15.49:0/2242068347 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.183+0000 7fd928c00640 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.183+0000 7fd928c00640 1 -- 172.21.15.49:0/2242068347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3466c10 msgr2=0x5573d346f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.183+0000 7fd928c00640 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3466c10 0x5573d346f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.184+0000 7fd928c00640 1 -- 172.21.15.49:0/2242068347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 msgr2=0x5573d346bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.184+0000 7fd92a000640 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 0x5573d346bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.184+0000 7fd928c00640 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 0x5573d346bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.184+0000 7fd928c00640 1 -- 172.21.15.49:0/2242068347 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573d3368a80 con 0x5573d3467530 2024-02-26T06:28:52.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.184+0000 7fd928c00640 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346ba80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd910012fb0 tx=0x7fd910011de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62040685e481dfdf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd915c00640 1 -- 172.21.15.49:0/2242068347 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910025470 con 0x5573d3467530 2024-02-26T06:28:52.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd915c00640 1 -- 172.21.15.49:0/2242068347 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd910016070 con 0x5573d3467530 2024-02-26T06:28:52.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd915c00640 1 -- 172.21.15.49:0/2242068347 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910025910 con 0x5573d3467530 2024-02-26T06:28:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd92efa2740 1 -- 172.21.15.49:0/2242068347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 msgr2=0x5573d346ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346ba80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd910012fb0 tx=0x7fd910011de0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd92efa2740 1 -- 172.21.15.49:0/2242068347 shutdown_connections 2024-02-26T06:28:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3467530 0x5573d346ba80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 0x5573d346bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd92efa2740 1 --2- 172.21.15.49:0/2242068347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3466c10 0x5573d346f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.185+0000 7fd92efa2740 1 -- 172.21.15.49:0/2242068347 >> 172.21.15.49:0/2242068347 conn(0x5573d3461340 msgr2=0x5573d34684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.186+0000 7fd92efa2740 1 -- 172.21.15.49:0/2242068347 shutdown_connections 2024-02-26T06:28:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.186+0000 7fd92efa2740 1 -- 172.21.15.49:0/2242068347 wait complete. 2024-02-26T06:28:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.186+0000 7fd92efa2740 1 -- start start 2024-02-26T06:28:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.187+0000 7fd92efa2740 1 Processor -- start 2024-02-26T06:28:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.187+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3466c10 0x5573d3475d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.187+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d3476290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.187+0000 7fd92efa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 0x5573d34767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.187+0000 7fd92efa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d34d67c0 con 0x5573d3470110 2024-02-26T06:28:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.187+0000 7fd92efa2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d34d6930 con 0x5573d3466c10 2024-02-26T06:28:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.187+0000 7fd92efa2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d34d6af0 con 0x5573d3467530 2024-02-26T06:28:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92efa2740 10 client.0 _ll_register_callbacks cb 0x7fd91c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3466c10 0x5573d3475d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3466c10 0x5573d3475d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52762/0 (socket says 172.21.15.49:52762) 2024-02-26T06:28:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 -- 172.21.15.49:0/444550410 learned_addr learned my addr 172.21.15.49:0/444550410 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 0x5573d34767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd928c00640 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d3476290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 -- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 msgr2=0x5573d3476290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d3476290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 -- 172.21.15.49:0/444550410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 msgr2=0x5573d34767d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 0x5573d34767d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.188+0000 7fd92a000640 1 -- 172.21.15.49:0/444550410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573d33cc570 con 0x5573d3466c10 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.189+0000 7fd928c00640 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d3476290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.189+0000 7fd92a000640 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3466c10 0x5573d3475d50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011c90 tx=0x7fd91c02a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6acff467ca2f5239 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.189+0000 7fd90aa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:52.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.190+0000 7fd90aa00640 1 -- 172.21.15.49:0/444550410 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00f9a0 con 0x5573d3466c10 2024-02-26T06:28:52.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.191+0000 7fd90aa00640 1 -- 172.21.15.49:0/444550410 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd91c00fb00 con 0x5573d3466c10 2024-02-26T06:28:52.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.191+0000 7fd92efa2740 1 -- 172.21.15.49:0/444550410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd91c00fe80 con 0x5573d3466c10 2024-02-26T06:28:52.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.192+0000 7fd90aa00640 1 -- 172.21.15.49:0/444550410 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd91c005a40 con 0x5573d3466c10 2024-02-26T06:28:52.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.192+0000 7fd92efa2740 10 client.26792 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:52.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.192+0000 7fd92efa2740 1 -- 172.21.15.49:0/444550410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5573d338e620 con 0x5573d3466c10 2024-02-26T06:28:52.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.193+0000 7fd90aa00640 1 -- 172.21.15.49:0/444550410 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd91c00fe80 con 0x5573d3466c10 2024-02-26T06:28:52.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.193+0000 7fd90aa00640 1 -- 172.21.15.49:0/444550410 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd91c01a960 con 0x5573d3466c10 2024-02-26T06:28:52.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.193+0000 7fd92efa2740 10 client.26792 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:52.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.193+0000 7fd92efa2740 10 client.26792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:28:52.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.194+0000 7fd92efa2740 20 client.26792 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:52.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.194+0000 7fd92efa2740 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5573d3480190 0x5573d34da300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.194+0000 7fd92efa2740 4 client.26792 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:52.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.194+0000 7fd92efa2740 1 -- 172.21.15.49:0/444550410 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573d3159890 con 0x5573d3480190 2024-02-26T06:28:52.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.194+0000 7fd928c00640 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5573d3480190 0x5573d34da300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.195+0000 7fd928c00640 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5573d3480190 0x5573d34da300 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff149443c3d57f71 server_cookie=45e3f2d45927fb91 in_seq=0 out_seq=0 2024-02-26T06:28:52.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.195+0000 7fd90aa00640 10 client.26792 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:28:52.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.196+0000 7fd90aa00640 1 -- 172.21.15.49:0/444550410 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573d3159890 con 0x5573d3480190 2024-02-26T06:28:52.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.196+0000 7fd90aa00640 10 client.26792 handle_command_reply: tid=0 2024-02-26T06:28:52.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.196+0000 7fd92efa2740 1 client.26792 shutdown 2024-02-26T06:28:52.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.196+0000 7fd92efa2740 1 -- 172.21.15.49:0/444550410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3466c10 msgr2=0x5573d3475d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.196+0000 7fd92efa2740 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3466c10 0x5573d3475d50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011c90 tx=0x7fd91c02a040 comp rx=0 tx=0).stop 2024-02-26T06:28:52.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 1 -- 172.21.15.49:0/444550410 shutdown_connections 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5573d3480190 0x5573d34da300 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573d3470110 0x5573d34767d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d3467530 0x5573d3476290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 1 --2- 172.21.15.49:0/444550410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5573d3466c10 0x5573d3475d50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 1 -- 172.21.15.49:0/444550410 >> 172.21.15.49:0/444550410 conn(0x5573d3461340 msgr2=0x5573d34684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd90aa00640 0 client.26792 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 1 -- 172.21.15.49:0/444550410 shutdown_connections 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 1 -- 172.21.15.49:0/444550410 wait complete. 2024-02-26T06:28:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.197+0000 7fd92efa2740 20 client.26792 trim_cache size 0 max 16384 2024-02-26T06:28:52.198 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:28:52.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.199+0000 7fd92c000640 1 -- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd90c0a2330 msgr2=0x7fd90c0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.199+0000 7fd92c000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd90c0a2330 0x7fd90c0a47f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5573d34666e0 tx=0x7fd91802bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:52.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.199+0000 7fd92c000640 1 -- 172.21.15.49:0/4115187688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd924107720 msgr2=0x7fd92419ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.199+0000 7fd92c000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd924107720 0x7fd92419ddd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd918000c00 tx=0x7fd91802cd80 comp rx=0 tx=0).stop 2024-02-26T06:28:52.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.200+0000 7fd92c000640 1 -- 172.21.15.49:0/4115187688 shutdown_connections 2024-02-26T06:28:52.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.200+0000 7fd92c000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd90c0a2330 0x7fd90c0a47f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.200+0000 7fd92c000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9241080e0 0x7fd92419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.200+0000 7fd92c000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd92410d730 0x7fd9241a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.200+0000 7fd92c000640 1 --2- 172.21.15.49:0/4115187688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd924107720 0x7fd92419ddd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.200+0000 7fd92c000640 1 -- 172.21.15.49:0/4115187688 >> 172.21.15.49:0/4115187688 conn(0x7fd9240756c0 msgr2=0x7fd924075e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.200+0000 7fd92c000640 1 -- 172.21.15.49:0/4115187688 shutdown_connections 2024-02-26T06:28:52.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.200+0000 7fd92c000640 1 -- 172.21.15.49:0/4115187688 wait complete. 2024-02-26T06:28:52.222 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:52.222 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:28:52.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.519+0000 7f2fe7e00640 1 -- 172.21.15.49:0/1143768045 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc032340 con 0x7f2fe8103590 2024-02-26T06:28:52.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.520+0000 7f2ff0000640 1 -- 172.21.15.49:0/1143768045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8103590 msgr2=0x7f2fe8103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.520+0000 7f2ff0000640 1 --2- 172.21.15.49:0/1143768045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8103590 0x7f2fe8103970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009890 tx=0x7f2fdc02bb90 comp rx=0 tx=0).stop 2024-02-26T06:28:52.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.522+0000 7f2ff0000640 1 -- 172.21.15.49:0/1143768045 shutdown_connections 2024-02-26T06:28:52.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.522+0000 7f2ff0000640 1 --2- 172.21.15.49:0/1143768045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe8108660 0x7f2fe8110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.522+0000 7f2ff0000640 1 --2- 172.21.15.49:0/1143768045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe8103f40 0x7f2fe8107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.522+0000 7f2ff0000640 1 --2- 172.21.15.49:0/1143768045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8103590 0x7f2fe8103970 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.522+0000 7f2ff0000640 1 -- 172.21.15.49:0/1143768045 >> 172.21.15.49:0/1143768045 conn(0x7f2fe80fee30 msgr2=0x7f2fe8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.525+0000 7f2ff0000640 1 -- 172.21.15.49:0/1143768045 shutdown_connections 2024-02-26T06:28:52.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.525+0000 7f2ff0000640 1 -- 172.21.15.49:0/1143768045 wait complete. 2024-02-26T06:28:52.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.527+0000 7f2ff0000640 1 Processor -- start 2024-02-26T06:28:52.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.529+0000 7f2ff0000640 1 -- start start 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.529+0000 7f2ff0000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe8103590 0x7f2fe8072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.529+0000 7f2ff0000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8108660 0x7f2fe8072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.529+0000 7f2ff0000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe806c6f0 0x7f2fe806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.529+0000 7f2ff0000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe806d1a0 con 0x7f2fe8103590 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.529+0000 7f2ff0000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe806d310 con 0x7f2fe8108660 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.529+0000 7f2ff0000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe806d4d0 con 0x7f2fe806c6f0 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fecc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8108660 0x7f2fe8072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fecc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8108660 0x7f2fe8072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52778/0 (socket says 172.21.15.49:52778) 2024-02-26T06:28:52.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fecc00640 1 -- 172.21.15.49:0/3441460748 learned_addr learned my addr 172.21.15.49:0/3441460748 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fee000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe806c6f0 0x7f2fe806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fecc00640 1 -- 172.21.15.49:0/3441460748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe806c6f0 msgr2=0x7f2fe806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fecc00640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe806c6f0 0x7f2fe806cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fecc00640 1 -- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe8103590 msgr2=0x7f2fe8072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fecc00640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe8103590 0x7f2fe8072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fecc00640 1 -- 172.21.15.49:0/3441460748 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc009490 con 0x7f2fe8108660 2024-02-26T06:28:52.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.530+0000 7f2fee000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe806c6f0 0x7f2fe806cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:52.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.532+0000 7f2fecc00640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8108660 0x7f2fe8072b70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400d770 tx=0x7f2fd400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.532+0000 7f2fe6000640 1 -- 172.21.15.49:0/3441460748 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4009c30 con 0x7f2fe8108660 2024-02-26T06:28:52.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.533+0000 7f2ff0000640 1 -- 172.21.15.49:0/3441460748 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe81aedf0 con 0x7f2fe8108660 2024-02-26T06:28:52.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.533+0000 7f2ff0000640 1 -- 172.21.15.49:0/3441460748 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe81af270 con 0x7f2fe8108660 2024-02-26T06:28:52.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.533+0000 7f2fe6000640 1 -- 172.21.15.49:0/3441460748 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fd400de80 con 0x7f2fe8108660 2024-02-26T06:28:52.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.534+0000 7f2fe6000640 1 -- 172.21.15.49:0/3441460748 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400f740 con 0x7f2fe8108660 2024-02-26T06:28:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.544+0000 7f2fe6000640 1 -- 172.21.15.49:0/3441460748 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2fd4010460 con 0x7f2fe8108660 2024-02-26T06:28:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.544+0000 7f2ff2e50740 1 Processor -- start 2024-02-26T06:28:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.545+0000 7f2ff2e50740 1 -- start start 2024-02-26T06:28:52.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.545+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b49ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.545+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b52bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b55500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0ab81e0 con 0x55b4f0b4a530 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0ab5710 con 0x55b4f0b53110 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0ab0ef0 con 0x55b4f0b49c10 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2fee000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b49ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2fee000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b49ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60506/0 (socket says 172.21.15.49:60506) 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2fee000640 1 -- 172.21.15.49:0/2171187780 learned_addr learned my addr 172.21.15.49:0/2171187780 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2fecc00640 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b55500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2fee000640 1 -- 172.21.15.49:0/2171187780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 msgr2=0x55b4f0b55500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2fee000640 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b55500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.546+0000 7f2fee000640 1 -- 172.21.15.49:0/2171187780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 msgr2=0x55b4f0b52bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.547+0000 7f2fed600640 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b52bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.547+0000 7f2fee000640 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b52bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.547+0000 7f2fee000640 1 -- 172.21.15.49:0/2171187780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4f0a72dc0 con 0x55b4f0b49c10 2024-02-26T06:28:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.548+0000 7f2fee000640 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b49ff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009e40 tx=0x7f2fe000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a2d151ba2eda3f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.548+0000 7f2fe6000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fd00aab40 0x7f2fd00ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.548+0000 7f2fda600640 1 -- 172.21.15.49:0/2171187780 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0004d20 con 0x55b4f0b49c10 2024-02-26T06:28:52.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.549+0000 7f2fda600640 1 -- 172.21.15.49:0/2171187780 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fe0004e80 con 0x55b4f0b49c10 2024-02-26T06:28:52.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.549+0000 7f2fda600640 1 -- 172.21.15.49:0/2171187780 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0011860 con 0x55b4f0b49c10 2024-02-26T06:28:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.549+0000 7f2fed600640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fd00aab40 0x7f2fd00ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.548+0000 7f2fe6000640 1 -- 172.21.15.49:0/3441460748 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2fd40ce110 con 0x7f2fe8108660 2024-02-26T06:28:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.550+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2171187780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 msgr2=0x55b4f0b49ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.550+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b49ff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009e40 tx=0x7f2fe000f730 comp rx=0 tx=0).stop 2024-02-26T06:28:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.550+0000 7f2fed600640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fd00aab40 0x7f2fd00ad000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc037040 tx=0x7f2fdc0058b0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.550+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2171187780 shutdown_connections 2024-02-26T06:28:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.550+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b55500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.550+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b52bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.550+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/2171187780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b49ff0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.550+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2171187780 >> 172.21.15.49:0/2171187780 conn(0x55b4f0b44340 msgr2=0x55b4f0b44720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.551+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2171187780 shutdown_connections 2024-02-26T06:28:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.551+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2171187780 wait complete. 2024-02-26T06:28:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.551+0000 7f2ff2e50740 1 -- start start 2024-02-26T06:28:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.552+0000 7f2ff2e50740 1 Processor -- start 2024-02-26T06:28:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.553+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.553+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.553+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b50b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.553+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0a56120 con 0x55b4f0b4a530 2024-02-26T06:28:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.553+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0a55dc0 con 0x55b4f0b53110 2024-02-26T06:28:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.553+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f081f1c0 con 0x55b4f0b49c10 2024-02-26T06:28:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.553+0000 7f2ff2e50740 10 client.0 _ll_register_callbacks cb 0x55b4f0aa5280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.554+0000 7f2fed600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.554+0000 7f2fed600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60520/0 (socket says 172.21.15.49:60520) 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.554+0000 7f2fed600640 1 -- 172.21.15.49:0/1687928502 learned_addr learned my addr 172.21.15.49:0/1687928502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.554+0000 7f2fee000640 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b50b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.554+0000 7f2fee000640 1 -- 172.21.15.49:0/1687928502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 msgr2=0x55b4f0b500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.554+0000 7f2fee000640 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.554+0000 7f2fee000640 1 -- 172.21.15.49:0/1687928502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 msgr2=0x55b4f0b50600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.554+0000 7f2fecc00640 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.555+0000 7f2fee000640 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.555+0000 7f2fee000640 1 -- 172.21.15.49:0/1687928502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe0009aa0 con 0x55b4f0b53110 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.555+0000 7f2fecc00640 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.555+0000 7f2fee000640 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b50b40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009e40 tx=0x7f2fe0004c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de56e458bc0128e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.555+0000 7f2fcea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.556+0000 7f2fcea00640 1 -- 172.21.15.49:0/1687928502 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe001f070 con 0x55b4f0b53110 2024-02-26T06:28:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.556+0000 7f2fcea00640 1 -- 172.21.15.49:0/1687928502 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2fe0015070 con 0x55b4f0b53110 2024-02-26T06:28:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.556+0000 7f2ff2e50740 1 -- 172.21.15.49:0/1687928502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4f07ed720 con 0x55b4f0b53110 2024-02-26T06:28:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.556+0000 7f2fcea00640 1 -- 172.21.15.49:0/1687928502 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fe0005920 con 0x55b4f0b53110 2024-02-26T06:28:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.557+0000 7f2ff2e50740 10 client.26804 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.557+0000 7f2ff2e50740 1 -- 172.21.15.49:0/1687928502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b4f0aaf570 con 0x55b4f0b53110 2024-02-26T06:28:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.557+0000 7f2fcea00640 1 -- 172.21.15.49:0/1687928502 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fe0030020 con 0x55b4f0b53110 2024-02-26T06:28:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.557+0000 7f2fcea00640 1 -- 172.21.15.49:0/1687928502 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2fe0019430 con 0x55b4f0b53110 2024-02-26T06:28:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.557+0000 7f2ff2e50740 10 client.26804 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.557+0000 7f2ff2e50740 10 client.26804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.558+0000 7f2ff2e50740 20 client.26804 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.558+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b4f0b741e0 0x55b4f0b76600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.558+0000 7f2ff2e50740 4 client.26804 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.558+0000 7f2ff2e50740 1 -- 172.21.15.49:0/1687928502 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4f083c890 con 0x55b4f0b741e0 2024-02-26T06:28:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.558+0000 7f2fed600640 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b4f0b741e0 0x55b4f0b76600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.558+0000 7f2fed600640 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b4f0b741e0 0x55b4f0b76600 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0a8e7117f2533c server_cookie=f80379487814f8fe in_seq=0 out_seq=0 2024-02-26T06:28:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.558+0000 7f2fcea00640 10 client.26804 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.559+0000 7f2fcea00640 1 -- 172.21.15.49:0/1687928502 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4f083c890 con 0x55b4f0b741e0 2024-02-26T06:28:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.559+0000 7f2fcea00640 10 client.26804 handle_command_reply: tid=0 2024-02-26T06:28:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.559+0000 7f2ff2e50740 1 client.26804 shutdown 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 -- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 msgr2=0x55b4f0b50b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b50b40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009e40 tx=0x7f2fe0004c10 comp rx=0 tx=0).stop 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 -- 172.21.15.49:0/1687928502 shutdown_connections 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b4f0b741e0 0x55b4f0b76600 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b53110 0x55b4f0b50b40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2fcea00640 0 client.26804 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/1687928502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 -- 172.21.15.49:0/1687928502 >> 172.21.15.49:0/1687928502 conn(0x55b4f0b44340 msgr2=0x55b4f0b46090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.560+0000 7f2ff2e50740 1 -- 172.21.15.49:0/1687928502 shutdown_connections 2024-02-26T06:28:52.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.561+0000 7f2ff2e50740 1 -- 172.21.15.49:0/1687928502 wait complete. 2024-02-26T06:28:52.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.561+0000 7f2ff2e50740 20 client.26804 trim_cache size 0 max 16384 2024-02-26T06:28:52.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.575+0000 7f2ff2e50740 1 Processor -- start 2024-02-26T06:28:52.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.575+0000 7f2ff2e50740 1 -- start start 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b53110 0x55b4f0b4d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0b70d00 con 0x55b4f0b53110 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0b45340 con 0x55b4f0b4a530 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0b44f10 con 0x55b4f0b49c10 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2fed600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2fed600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60534/0 (socket says 172.21.15.49:60534) 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2fed600640 1 -- 172.21.15.49:0/2184882674 learned_addr learned my addr 172.21.15.49:0/2184882674 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2fee000640 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b53110 0x55b4f0b4d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.576+0000 7f2fecc00640 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.577+0000 7f2fed600640 1 -- 172.21.15.49:0/2184882674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b4a530 msgr2=0x55b4f0b50960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.577+0000 7f2fed600640 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.577+0000 7f2fed600640 1 -- 172.21.15.49:0/2184882674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b53110 msgr2=0x55b4f0b4d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.577+0000 7f2fed600640 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b53110 0x55b4f0b4d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.577+0000 7f2fed600640 1 -- 172.21.15.49:0/2184882674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4f0a4ba80 con 0x55b4f0b49c10 2024-02-26T06:28:52.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.577+0000 7f2fee000640 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b53110 0x55b4f0b4d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:52.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.577+0000 7f2fed600640 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b527f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc02c1a0 tx=0x7f2fdc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98ad8bbe70af48b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.578+0000 7f2fd9c00640 1 -- 172.21.15.49:0/2184882674 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc004420 con 0x55b4f0b49c10 2024-02-26T06:28:52.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.578+0000 7f2fd9c00640 1 -- 172.21.15.49:0/2184882674 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fdc055070 con 0x55b4f0b49c10 2024-02-26T06:28:52.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.578+0000 7f2fd9c00640 1 -- 172.21.15.49:0/2184882674 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc02c790 con 0x55b4f0b49c10 2024-02-26T06:28:52.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.580+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2184882674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 msgr2=0x55b4f0b527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.580+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b527f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc02c1a0 tx=0x7f2fdc005f90 comp rx=0 tx=0).stop 2024-02-26T06:28:52.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.580+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2184882674 shutdown_connections 2024-02-26T06:28:52.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.580+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b4a530 0x55b4f0b50960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.580+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b53110 0x55b4f0b4d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.580+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/2184882674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b49c10 0x55b4f0b527f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.580+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2184882674 >> 172.21.15.49:0/2184882674 conn(0x55b4f0b44340 msgr2=0x55b4f0b4e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.581+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2184882674 shutdown_connections 2024-02-26T06:28:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.581+0000 7f2ff2e50740 1 -- 172.21.15.49:0/2184882674 wait complete. 2024-02-26T06:28:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.581+0000 7f2ff2e50740 1 -- start start 2024-02-26T06:28:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.581+0000 7f2ff2e50740 1 Processor -- start 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b49c10 0x55b4f0b591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2ff2e50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b53110 0x55b4f0bca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0b59e80 con 0x55b4f0b4a530 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0bcac30 con 0x55b4f0b49c10 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2ff2e50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f0bcadf0 con 0x55b4f0b53110 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2ff2e50740 10 client.0 _ll_register_callbacks cb 0x7f2fe00095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2fed600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b49c10 0x55b4f0b591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2fed600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b49c10 0x55b4f0b591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52812/0 (socket says 172.21.15.49:52812) 2024-02-26T06:28:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.582+0000 7f2fed600640 1 -- 172.21.15.49:0/671053887 learned_addr learned my addr 172.21.15.49:0/671053887 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fee000640 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b53110 0x55b4f0bca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fecc00640 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fed600640 1 -- 172.21.15.49:0/671053887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b53110 msgr2=0x55b4f0bca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fed600640 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b53110 0x55b4f0bca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fed600640 1 -- 172.21.15.49:0/671053887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 msgr2=0x55b4f0b596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fed600640 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fed600640 1 -- 172.21.15.49:0/671053887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe8110a40 con 0x55b4f0b49c10 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fecc00640 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fed600640 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b49c10 0x55b4f0b591b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0050f0 tx=0x7f2fdc005120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0928734058e79ef server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.583+0000 7f2fcea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:28:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.584+0000 7f2fcea00640 1 -- 172.21.15.49:0/671053887 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc060070 con 0x55b4f0b49c10 2024-02-26T06:28:52.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.584+0000 7f2fcea00640 1 -- 172.21.15.49:0/671053887 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2fdc045040 con 0x55b4f0b49c10 2024-02-26T06:28:52.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.584+0000 7f2ff2e50740 1 -- 172.21.15.49:0/671053887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4f07b07e0 con 0x55b4f0b49c10 2024-02-26T06:28:52.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.584+0000 7f2fcea00640 1 -- 172.21.15.49:0/671053887 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fdc004220 con 0x55b4f0b49c10 2024-02-26T06:28:52.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.587+0000 7f2ff2e50740 10 client.26810 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:52.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.587+0000 7f2ff2e50740 1 -- 172.21.15.49:0/671053887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b4f0a71620 con 0x55b4f0b49c10 2024-02-26T06:28:52.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.587+0000 7f2fcea00640 1 -- 172.21.15.49:0/671053887 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b4f07b07e0 con 0x55b4f0b49c10 2024-02-26T06:28:52.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.587+0000 7f2fcea00640 1 -- 172.21.15.49:0/671053887 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2fdc063020 con 0x55b4f0b49c10 2024-02-26T06:28:52.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.587+0000 7f2ff2e50740 10 client.26810 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:52.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.587+0000 7f2ff2e50740 10 client.26810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:52.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.587+0000 7f2ff2e50740 20 client.26810 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:52.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.588+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b4f0bcc390 0x55b4f0bce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:52.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.588+0000 7f2ff2e50740 4 client.26810 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:52.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.588+0000 7f2ff2e50740 1 -- 172.21.15.49:0/671053887 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4f079f4e0 con 0x55b4f0bcc390 2024-02-26T06:28:52.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.588+0000 7f2fecc00640 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b4f0bcc390 0x55b4f0bce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.588+0000 7f2fecc00640 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b4f0bcc390 0x55b4f0bce7b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e9ffd7d3934ad2 server_cookie=c29ab9f46de4f4a2 in_seq=0 out_seq=0 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.588+0000 7f2fcea00640 10 client.26810 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2fcea00640 1 -- 172.21.15.49:0/671053887 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4f079f4e0 con 0x55b4f0bcc390 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2fcea00640 10 client.26810 handle_command_reply: tid=0 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 client.26810 shutdown 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 -- 172.21.15.49:0/671053887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b49c10 msgr2=0x55b4f0b591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b49c10 0x55b4f0b591b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0050f0 tx=0x7f2fdc005120 comp rx=0 tx=0).stop 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 -- 172.21.15.49:0/671053887 shutdown_connections 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b4f0bcc390 0x55b4f0bce7b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4f0b53110 0x55b4f0bca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4f0b4a530 0x55b4f0b596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 --2- 172.21.15.49:0/671053887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4f0b49c10 0x55b4f0b591b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 -- 172.21.15.49:0/671053887 >> 172.21.15.49:0/671053887 conn(0x55b4f0b44340 msgr2=0x55b4f0b42bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2fcea00640 0 client.26810 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:52.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.589+0000 7f2ff2e50740 1 -- 172.21.15.49:0/671053887 shutdown_connections 2024-02-26T06:28:52.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.590+0000 7f2ff2e50740 1 -- 172.21.15.49:0/671053887 wait complete. 2024-02-26T06:28:52.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.590+0000 7f2ff2e50740 20 client.26810 trim_cache size 0 max 16384 2024-02-26T06:28:52.591 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:28:52.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 -- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fd00aab40 msgr2=0x7f2fd00ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fd00aab40 0x7f2fd00ad000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc037040 tx=0x7f2fdc0058b0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 -- 172.21.15.49:0/3441460748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8108660 msgr2=0x7f2fe8072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8108660 0x7f2fe8072b70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400d770 tx=0x7f2fd400dc30 comp rx=0 tx=0).stop 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 -- 172.21.15.49:0/3441460748 shutdown_connections 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fd00aab40 0x7f2fd00ad000 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fe8108660 0x7f2fe8072b70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fe806c6f0 0x7f2fe806cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 --2- 172.21.15.49:0/3441460748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fe8103590 0x7f2fe8072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 -- 172.21.15.49:0/3441460748 >> 172.21.15.49:0/3441460748 conn(0x7f2fe80fee30 msgr2=0x7f2fe8100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 -- 172.21.15.49:0/3441460748 shutdown_connections 2024-02-26T06:28:52.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:52.592+0000 7f2ff0000640 1 -- 172.21.15.49:0/3441460748 wait complete. 2024-02-26T06:28:52.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:52 smithi049 ceph-mon[31696]: pgmap v167: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-02-26T06:28:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:52 smithi080 ceph-mon[36206]: pgmap v167: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-02-26T06:28:53.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:52 smithi115 ceph-mon[36426]: pgmap v167: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-02-26T06:28:53.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:28:53.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:28:53.618 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:28:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.932+0000 7ff342600640 1 -- 172.21.15.49:0/813871883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 msgr2=0x7ff33c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.932+0000 7ff342600640 1 --2- 172.21.15.49:0/813871883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 0x7ff33c1027f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009900 tx=0x7ff32c02bcc0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.932+0000 7ff342600640 1 -- 172.21.15.49:0/813871883 shutdown_connections 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.932+0000 7ff342600640 1 --2- 172.21.15.49:0/813871883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff33c10b750 0x7ff33c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.932+0000 7ff342600640 1 --2- 172.21.15.49:0/813871883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff33c102d30 0x7ff33c10b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.932+0000 7ff342600640 1 --2- 172.21.15.49:0/813871883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 0x7ff33c1027f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.933+0000 7ff342600640 1 -- 172.21.15.49:0/813871883 >> 172.21.15.49:0/813871883 conn(0x7ff33c0771e0 msgr2=0x7ff33c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.933+0000 7ff342600640 1 -- 172.21.15.49:0/813871883 shutdown_connections 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.933+0000 7ff342600640 1 -- 172.21.15.49:0/813871883 wait complete. 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.934+0000 7ff342600640 1 Processor -- start 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.934+0000 7ff342600640 1 -- start start 2024-02-26T06:28:54.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.935+0000 7ff342600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 0x7ff33c197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.935+0000 7ff342600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff33c102d30 0x7ff33c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.935+0000 7ff342600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff33c10b750 0x7ff33c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.935+0000 7ff342600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c19c5f0 con 0x7ff33c10b750 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.935+0000 7ff342600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c19c760 con 0x7ff33c102410 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.935+0000 7ff342600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c19c920 con 0x7ff33c102d30 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.935+0000 7ff33b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 0x7ff33c197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.936+0000 7ff33b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 0x7ff33c197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52824/0 (socket says 172.21.15.49:52824) 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.936+0000 7ff33b400640 1 -- 172.21.15.49:0/146103723 learned_addr learned my addr 172.21.15.49:0/146103723 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.936+0000 7ff33aa00640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff33c102d30 0x7ff33c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.936+0000 7ff33b400640 1 -- 172.21.15.49:0/146103723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff33c102d30 msgr2=0x7ff33c197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.936+0000 7ff33b400640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff33c102d30 0x7ff33c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.936+0000 7ff33b400640 1 -- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff33c10b750 msgr2=0x7ff33c19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.936+0000 7ff33be00640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff33c10b750 0x7ff33c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.937+0000 7ff33b400640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff33c10b750 0x7ff33c19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.937+0000 7ff33b400640 1 -- 172.21.15.49:0/146103723 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c009560 con 0x7ff33c102410 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.938+0000 7ff33b400640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 0x7ff33c197730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff32c000c00 tx=0x7ff32c02dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.938+0000 7ff333e00640 1 -- 172.21.15.49:0/146103723 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0044f0 con 0x7ff33c102410 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.938+0000 7ff333e00640 1 -- 172.21.15.49:0/146103723 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff32c02de30 con 0x7ff33c102410 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.938+0000 7ff333e00640 1 -- 172.21.15.49:0/146103723 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c003c60 con 0x7ff33c102410 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.940+0000 7ff342600640 1 -- 172.21.15.49:0/146103723 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c19cba0 con 0x7ff33c102410 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.940+0000 7ff342600640 1 -- 172.21.15.49:0/146103723 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c100170 con 0x7ff33c102410 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.944+0000 7ff3455c7740 1 Processor -- start 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.945+0000 7ff3455c7740 1 -- start start 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.945+0000 7ff333e00640 1 -- 172.21.15.49:0/146103723 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff32c005560 con 0x7ff33c102410 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.946+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a1f10 0x558e861a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.946+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a2830 0x558e861aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.946+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861ad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.946+0000 7ff3455c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8615d2d0 con 0x558e861ab410 2024-02-26T06:28:54.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.946+0000 7ff3455c7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8610fe20 con 0x558e861a1f10 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.946+0000 7ff3455c7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8610d350 con 0x558e861a2830 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a2830 0x558e861aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a2830 0x558e861aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60574/0 (socket says 172.21.15.49:60574) 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33b400640 1 -- 172.21.15.49:0/920644165 learned_addr learned my addr 172.21.15.49:0/920644165 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33be00640 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a1f10 0x558e861a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33aa00640 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33b400640 1 -- 172.21.15.49:0/920644165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a1f10 msgr2=0x558e861a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33b400640 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a1f10 0x558e861a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33b400640 1 -- 172.21.15.49:0/920644165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 msgr2=0x558e861ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33b400640 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.947+0000 7ff33b400640 1 -- 172.21.15.49:0/920644165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e860d1c50 con 0x558e861a2830 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.948+0000 7ff333e00640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff3200a6780 0x7ff3200a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.948+0000 7ff33b400640 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a2830 0x558e861aaed0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0edb90 tx=0x7ff32c0f84d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fac492f914513a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.948+0000 7ff333e00640 1 -- 172.21.15.49:0/146103723 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7ff32c0f0440 con 0x7ff33c102410 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.948+0000 7ff33be00640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff3200a6780 0x7ff3200a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.949+0000 7ff330c00640 1 -- 172.21.15.49:0/920644165 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0b9190 con 0x558e861a2830 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.949+0000 7ff330c00640 1 -- 172.21.15.49:0/920644165 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff32c0bccc0 con 0x558e861a2830 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.949+0000 7ff330c00640 1 -- 172.21.15.49:0/920644165 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0bca30 con 0x558e861a2830 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.950+0000 7ff3455c7740 1 -- 172.21.15.49:0/920644165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a2830 msgr2=0x558e861aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.950+0000 7ff3455c7740 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a2830 0x558e861aaed0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0edb90 tx=0x7ff32c0f84d0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.951+0000 7ff33be00640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff3200a6780 0x7ff3200a8c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff334008df0 tx=0x7ff334008540 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.951+0000 7ff3455c7740 1 -- 172.21.15.49:0/920644165 shutdown_connections 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.951+0000 7ff3455c7740 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.951+0000 7ff3455c7740 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a2830 0x558e861aaed0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.951+0000 7ff3455c7740 1 --2- 172.21.15.49:0/920644165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a1f10 0x558e861a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.951+0000 7ff3455c7740 1 -- 172.21.15.49:0/920644165 >> 172.21.15.49:0/920644165 conn(0x558e8619c640 msgr2=0x558e8619ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.951+0000 7ff3455c7740 1 -- 172.21.15.49:0/920644165 shutdown_connections 2024-02-26T06:28:54.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.952+0000 7ff3455c7740 1 -- 172.21.15.49:0/920644165 wait complete. 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.953+0000 7ff3455c7740 1 -- start start 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.953+0000 7ff3455c7740 1 Processor -- start 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.954+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861a8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.954+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff3455c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e86108b30 con 0x558e861ab410 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff3455c7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e860adfa0 con 0x558e861a2830 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff3455c7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e860adc40 con 0x558e861a1f10 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff3455c7740 10 client.0 _ll_register_callbacks cb 0x558e85e19c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff33aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff33aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52848/0 (socket says 172.21.15.49:52848) 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff33aa00640 1 -- 172.21.15.49:0/2078639556 learned_addr learned my addr 172.21.15.49:0/2078639556 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff33b400640 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.955+0000 7ff33be00640 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.957+0000 7ff33be00640 1 -- 172.21.15.49:0/2078639556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 msgr2=0x558e861a8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.957+0000 7ff33be00640 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861a8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.957+0000 7ff33be00640 1 -- 172.21.15.49:0/2078639556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 msgr2=0x558e861a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.957+0000 7ff33be00640 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.957+0000 7ff33be00640 1 -- 172.21.15.49:0/2078639556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c100170 con 0x558e861ab410 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.958+0000 7ff33be00640 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861a8e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff334010c90 tx=0x7ff334004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f82a40b95c315f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.958+0000 7ff325600640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.958+0000 7ff325600640 1 -- 172.21.15.49:0/2078639556 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334025400 con 0x558e861ab410 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.959+0000 7ff325600640 1 -- 172.21.15.49:0/2078639556 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7ff334025560 con 0x558e861ab410 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.959+0000 7ff3455c7740 1 -- 172.21.15.49:0/2078639556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e85e08860 con 0x558e861ab410 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.960+0000 7ff325600640 1 -- 172.21.15.49:0/2078639556 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff334025720 con 0x558e861ab410 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.960+0000 7ff3455c7740 10 client.16344 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.960+0000 7ff3455c7740 1 -- 172.21.15.49:0/2078639556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558e861071b0 con 0x558e861ab410 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.961+0000 7ff325600640 1 -- 172.21.15.49:0/2078639556 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558e85e08860 con 0x558e861ab410 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.961+0000 7ff325600640 1 -- 172.21.15.49:0/2078639556 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff33402c070 con 0x558e861ab410 2024-02-26T06:28:54.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.961+0000 7ff3455c7740 10 client.16344 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.961+0000 7ff3455c7740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.962+0000 7ff3455c7740 20 client.16344 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.962+0000 7ff3455c7740 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558e861bb610 0x558e861bda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.962+0000 7ff3455c7740 4 client.16344 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.962+0000 7ff3455c7740 1 -- 172.21.15.49:0/2078639556 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e85e59410 con 0x558e861bb610 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.962+0000 7ff33b400640 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558e861bb610 0x558e861bda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.963+0000 7ff33b400640 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558e861bb610 0x558e861bda30 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41604555297c721 server_cookie=8e253b88243c2aa9 in_seq=0 out_seq=0 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.964+0000 7ff325600640 10 client.16344 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.964+0000 7ff325600640 1 -- 172.21.15.49:0/2078639556 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e85e59410 con 0x558e861bb610 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.964+0000 7ff325600640 10 client.16344 handle_command_reply: tid=0 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.964+0000 7ff3455c7740 1 client.16344 shutdown 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.965+0000 7ff3455c7740 1 -- 172.21.15.49:0/2078639556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 msgr2=0x558e861a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.965+0000 7ff3455c7740 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861a8e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff334010c90 tx=0x7ff334004a40 comp rx=0 tx=0).stop 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.966+0000 7ff3455c7740 1 -- 172.21.15.49:0/2078639556 shutdown_connections 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.966+0000 7ff3455c7740 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558e861bb610 0x558e861bda30 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.966+0000 7ff3455c7740 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.966+0000 7ff3455c7740 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861ab410 0x558e861a8e90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.966+0000 7ff3455c7740 1 --2- 172.21.15.49:0/2078639556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861a8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.966+0000 7ff3455c7740 1 -- 172.21.15.49:0/2078639556 >> 172.21.15.49:0/2078639556 conn(0x558e8619c640 msgr2=0x558e8619e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.966+0000 7ff325600640 0 client.16344 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.967+0000 7ff3455c7740 1 -- 172.21.15.49:0/2078639556 shutdown_connections 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.967+0000 7ff3455c7740 1 -- 172.21.15.49:0/2078639556 wait complete. 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:53.967+0000 7ff3455c7740 20 client.16344 trim_cache size 0 max 16384 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.012+0000 7ff3455c7740 1 Processor -- start 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.012+0000 7ff3455c7740 1 -- start start 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.012+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a1f10 0x558e861aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.012+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.012+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861ab410 0x558e861a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.012+0000 7ff3455c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e85e510c0 con 0x558e861a1f10 2024-02-26T06:28:54.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.012+0000 7ff3455c7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e85e08860 con 0x558e861a2830 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.012+0000 7ff3455c7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e861b8020 con 0x558e861ab410 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861ab410 0x558e861a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a1f10 0x558e861aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861ab410 0x558e861a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60596/0 (socket says 172.21.15.49:60596) 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33be00640 1 -- 172.21.15.49:0/4091147222 learned_addr learned my addr 172.21.15.49:0/4091147222 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33aa00640 1 --2- 172.21.15.49:0/4091147222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33b400640 1 -- 172.21.15.49:0/4091147222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861ab410 msgr2=0x558e861a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33b400640 1 --2- 172.21.15.49:0/4091147222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861ab410 0x558e861a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33b400640 1 -- 172.21.15.49:0/4091147222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 msgr2=0x558e861a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.013+0000 7ff33b400640 1 --2- 172.21.15.49:0/4091147222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.014+0000 7ff33b400640 1 -- 172.21.15.49:0/4091147222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e860a3a60 con 0x558e861a1f10 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.015+0000 7ff33b400640 1 --2- 172.21.15.49:0/4091147222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a1f10 0x558e861aab40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0edb90 tx=0x7ff32c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6f12af4eaf0a4d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.016+0000 7ff327e00640 1 -- 172.21.15.49:0/4091147222 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0b9190 con 0x558e861a1f10 2024-02-26T06:28:54.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.016+0000 7ff327e00640 1 -- 172.21.15.49:0/4091147222 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff32c0f3e90 con 0x558e861a1f10 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.017+0000 7ff327e00640 1 -- 172.21.15.49:0/4091147222 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0bcef0 con 0x558e861a1f10 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.017+0000 7ff3455c7740 1 -- 172.21.15.49:0/4091147222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a1f10 msgr2=0x558e861aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.017+0000 7ff3455c7740 1 --2- 172.21.15.49:0/4091147222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a1f10 0x558e861aab40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0edb90 tx=0x7ff32c005fd0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.018+0000 7ff3455c7740 1 -- 172.21.15.49:0/4091147222 shutdown_connections 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.018+0000 7ff3455c7740 1 --2- 172.21.15.49:0/4091147222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861a2830 0x558e861a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.018+0000 7ff3455c7740 1 --2- 172.21.15.49:0/4091147222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861ab410 0x558e861a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.018+0000 7ff3455c7740 1 --2- 172.21.15.49:0/4091147222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a1f10 0x558e861aab40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.018+0000 7ff3455c7740 1 -- 172.21.15.49:0/4091147222 >> 172.21.15.49:0/4091147222 conn(0x558e8619c640 msgr2=0x558e861c2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.018+0000 7ff3455c7740 1 -- 172.21.15.49:0/4091147222 shutdown_connections 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.018+0000 7ff3455c7740 1 -- 172.21.15.49:0/4091147222 wait complete. 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.019+0000 7ff3455c7740 1 -- start start 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.019+0000 7ff3455c7740 1 Processor -- start 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.020+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.020+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a2830 0x558e861b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.020+0000 7ff3455c7740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861ab410 0x558e861b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.020+0000 7ff3455c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e861b1310 con 0x558e861a2830 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.020+0000 7ff3455c7740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e861b1480 con 0x558e861ab410 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.020+0000 7ff3455c7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8621a330 con 0x558e861a1f10 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff3455c7740 10 client.0 _ll_register_callbacks cb 0x7ff32c0f3de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60604/0 (socket says 172.21.15.49:60604) 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33b400640 1 -- 172.21.15.49:0/3301515330 learned_addr learned my addr 172.21.15.49:0/3301515330 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33be00640 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861ab410 0x558e861b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33b400640 1 -- 172.21.15.49:0/3301515330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861ab410 msgr2=0x558e861b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33b400640 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861ab410 0x558e861b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33b400640 1 -- 172.21.15.49:0/3301515330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a2830 msgr2=0x558e861b0700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33b400640 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a2830 0x558e861b0700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.021+0000 7ff33b400640 1 -- 172.21.15.49:0/3301515330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e860d1c50 con 0x558e861a1f10 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.022+0000 7ff33b400640 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861b2360 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0f81d0 tx=0x7ff32c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38902c02207299 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.022+0000 7ff325600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:54.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.022+0000 7ff325600640 1 -- 172.21.15.49:0/3301515330 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0bca30 con 0x558e861a1f10 2024-02-26T06:28:54.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.022+0000 7ff325600640 1 -- 172.21.15.49:0/3301515330 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7ff32c0bd5f0 con 0x558e861a1f10 2024-02-26T06:28:54.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.023+0000 7ff3455c7740 1 -- 172.21.15.49:0/3301515330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff32c0bccc0 con 0x558e861a1f10 2024-02-26T06:28:54.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.023+0000 7ff325600640 1 -- 172.21.15.49:0/3301515330 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff32c0b9190 con 0x558e861a1f10 2024-02-26T06:28:54.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.023+0000 7ff3455c7740 10 client.26821 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:54.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.023+0000 7ff3455c7740 1 -- 172.21.15.49:0/3301515330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558e860c9670 con 0x558e861a1f10 2024-02-26T06:28:54.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.023+0000 7ff325600640 1 -- 172.21.15.49:0/3301515330 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff32c0bccc0 con 0x558e861a1f10 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.024+0000 7ff325600640 1 -- 172.21.15.49:0/3301515330 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff32c03c070 con 0x558e861a1f10 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.024+0000 7ff3455c7740 10 client.26821 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.024+0000 7ff3455c7740 10 client.26821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.024+0000 7ff3455c7740 20 client.26821 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.024+0000 7ff3455c7740 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558e8621b520 0x558e8621d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.025+0000 7ff3455c7740 4 client.26821 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.025+0000 7ff3455c7740 1 -- 172.21.15.49:0/3301515330 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558e85e457b0 con 0x558e8621b520 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.025+0000 7ff33aa00640 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558e8621b520 0x558e8621d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.028+0000 7ff33aa00640 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558e8621b520 0x558e8621d910 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd18a57c45d15aa5 server_cookie=4db1991456ad13b8 in_seq=0 out_seq=0 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.028+0000 7ff325600640 10 client.26821 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff325600640 1 -- 172.21.15.49:0/3301515330 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558e85e457b0 con 0x558e8621b520 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff325600640 10 client.26821 handle_command_reply: tid=0 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 client.26821 shutdown 2024-02-26T06:28:54.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 -- 172.21.15.49:0/3301515330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 msgr2=0x558e861b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861b2360 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0f81d0 tx=0x7ff32c009260 comp rx=0 tx=0).stop 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 -- 172.21.15.49:0/3301515330 shutdown_connections 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x558e8621b520 0x558e8621d910 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558e861ab410 0x558e861b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e861a2830 0x558e861b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 --2- 172.21.15.49:0/3301515330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e861a1f10 0x558e861b2360 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff3455c7740 1 -- 172.21.15.49:0/3301515330 >> 172.21.15.49:0/3301515330 conn(0x558e8619c640 msgr2=0x558e8619d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.029+0000 7ff325600640 0 client.26821 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.030+0000 7ff3455c7740 1 -- 172.21.15.49:0/3301515330 shutdown_connections 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.030+0000 7ff3455c7740 1 -- 172.21.15.49:0/3301515330 wait complete. 2024-02-26T06:28:54.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.030+0000 7ff3455c7740 20 client.26821 trim_cache size 0 max 16384 2024-02-26T06:28:54.040 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:28:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:28:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "59f0e69f-5a7e-4c02-9e44-302a524aa1b3", 2024-02-26T06:28:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:28:54.040 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.031+0000 7ff342600640 1 -- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff3200a6780 msgr2=0x7ff3200a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.031+0000 7ff342600640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff3200a6780 0x7ff3200a8c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff334008df0 tx=0x7ff334008540 comp rx=0 tx=0).stop 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 -- 172.21.15.49:0/146103723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 msgr2=0x7ff33c197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 0x7ff33c197730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff32c000c00 tx=0x7ff32c02dab0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 -- 172.21.15.49:0/146103723 shutdown_connections 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff3200a6780 0x7ff3200a8c40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff33c10b750 0x7ff33c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff33c102d30 0x7ff33c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 --2- 172.21.15.49:0/146103723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff33c102410 0x7ff33c197730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 -- 172.21.15.49:0/146103723 >> 172.21.15.49:0/146103723 conn(0x7ff33c0771e0 msgr2=0x7ff33c0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 -- 172.21.15.49:0/146103723 shutdown_connections 2024-02-26T06:28:54.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.032+0000 7ff342600640 1 -- 172.21.15.49:0/146103723 wait complete. 2024-02-26T06:28:54.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:28:54.059 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 -- 172.21.15.49:0/2163931990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac103590 msgr2=0x7feaac103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 --2- 172.21.15.49:0/2163931990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac103590 0x7feaac103970 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009900 tx=0x7fea9c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 -- 172.21.15.49:0/2163931990 shutdown_connections 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 --2- 172.21.15.49:0/2163931990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feaac108660 0x7feaac110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 --2- 172.21.15.49:0/2163931990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feaac103f40 0x7feaac107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 --2- 172.21.15.49:0/2163931990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac103590 0x7feaac103970 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 -- 172.21.15.49:0/2163931990 >> 172.21.15.49:0/2163931990 conn(0x7feaac0fee30 msgr2=0x7feaac101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 -- 172.21.15.49:0/2163931990 shutdown_connections 2024-02-26T06:28:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.369+0000 7feab1000640 1 -- 172.21.15.49:0/2163931990 wait complete. 2024-02-26T06:28:54.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.370+0000 7feab1000640 1 Processor -- start 2024-02-26T06:28:54.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.371+0000 7feab1000640 1 -- start start 2024-02-26T06:28:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.371+0000 7feab1000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feaac103590 0x7feaac19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.371+0000 7feab1000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac108660 0x7feaac19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.371+0000 7feab1000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feaac199830 0x7feaac199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.371+0000 7feab1000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac19a2e0 con 0x7feaac199830 2024-02-26T06:28:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.371+0000 7feab1000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac19a450 con 0x7feaac103590 2024-02-26T06:28:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.371+0000 7feab1000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac19a5c0 con 0x7feaac108660 2024-02-26T06:28:54.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.372+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feaac103590 0x7feaac19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.372+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feaac103590 0x7feaac19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52888/0 (socket says 172.21.15.49:52888) 2024-02-26T06:28:54.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.372+0000 7feaa9e00640 1 -- 172.21.15.49:0/4152406404 learned_addr learned my addr 172.21.15.49:0/4152406404 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.372+0000 7feaa9400640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac108660 0x7feaac19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.372+0000 7feaaa800640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feaac199830 0x7feaac199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.372+0000 7feaa9e00640 1 -- 172.21.15.49:0/4152406404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac108660 msgr2=0x7feaac19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.372+0000 7feaa9e00640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac108660 0x7feaac19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.372+0000 7feaa9e00640 1 -- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feaac199830 msgr2=0x7feaac199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.373+0000 7feaa9e00640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feaac199830 0x7feaac199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.373+0000 7feaa9e00640 1 -- 172.21.15.49:0/4152406404 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c009560 con 0x7feaac103590 2024-02-26T06:28:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.373+0000 7feaaa800640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feaac199830 0x7feaac199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.373+0000 7feaa9e00640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feaac103590 0x7feaac19f6c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fea9c000c00 tx=0x7fea9c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.373+0000 7feaa2a00640 1 -- 172.21.15.49:0/4152406404 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c02dea0 con 0x7feaac103590 2024-02-26T06:28:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.374+0000 7feaa2a00640 1 -- 172.21.15.49:0/4152406404 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea9c004500 con 0x7feaac103590 2024-02-26T06:28:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.374+0000 7feaa2a00640 1 -- 172.21.15.49:0/4152406404 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c003cd0 con 0x7feaac103590 2024-02-26T06:28:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.374+0000 7feab1000640 1 -- 172.21.15.49:0/4152406404 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac1a6510 con 0x7feaac103590 2024-02-26T06:28:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.374+0000 7feab1000640 1 -- 172.21.15.49:0/4152406404 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac1a69a0 con 0x7feaac103590 2024-02-26T06:28:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.379+0000 7feab3f7c740 1 Processor -- start 2024-02-26T06:28:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.379+0000 7feab3f7c740 1 -- start start 2024-02-26T06:28:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.380+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd90cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.380+0000 7feaa2a00640 1 -- 172.21.15.49:0/4152406404 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fea9c004df0 con 0x7feaac103590 2024-02-26T06:28:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd91710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feab3f7c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfd00120 con 0x55cbdfd91f70 2024-02-26T06:28:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feab3f7c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfcfd650 con 0x55cbdfd9a600 2024-02-26T06:28:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feab3f7c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfcf8e30 con 0x55cbdfd91290 2024-02-26T06:28:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feaaa800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd90cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd91710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd91710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60616/0 (socket says 172.21.15.49:60616) 2024-02-26T06:28:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feaa9e00640 1 -- 172.21.15.49:0/308398055 learned_addr learned my addr 172.21.15.49:0/308398055 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.381+0000 7feaa9400640 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.382+0000 7feaa9400640 1 -- 172.21.15.49:0/308398055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 msgr2=0x55cbdfd91710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.382+0000 7feaa9400640 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd91710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.382+0000 7feaa9400640 1 -- 172.21.15.49:0/308398055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 msgr2=0x55cbdfd90cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.382+0000 7feaa9400640 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd90cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.382+0000 7feaa9400640 1 -- 172.21.15.49:0/308398055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbdfcb2030 con 0x55cbdfd9a600 2024-02-26T06:28:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.382+0000 7feaaa800640 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd90cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.382+0000 7feaa9400640 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9c9f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feaac19ab80 tx=0x7fea980107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d374cf79e7b28f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.382+0000 7feaa2a00640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea900a22b0 0x7fea900a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.383+0000 7feaa2a00640 1 -- 172.21.15.49:0/4152406404 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fea9c0f0610 con 0x7feaac103590 2024-02-26T06:28:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.383+0000 7fea97400640 1 -- 172.21.15.49:0/308398055 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9801c480 con 0x55cbdfd9a600 2024-02-26T06:28:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.383+0000 7fea97400640 1 -- 172.21.15.49:0/308398055 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea98015070 con 0x55cbdfd9a600 2024-02-26T06:28:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.383+0000 7fea97400640 1 -- 172.21.15.49:0/308398055 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980119f0 con 0x55cbdfd9a600 2024-02-26T06:28:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.384+0000 7feaaa800640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea900a22b0 0x7fea900a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.385+0000 7feab3f7c740 1 -- 172.21.15.49:0/308398055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 msgr2=0x55cbdfd9c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.385+0000 7feab3f7c740 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9c9f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feaac19ab80 tx=0x7fea980107a0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.386+0000 7feaaa800640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea900a22b0 0x7fea900a4770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feaa4005fd0 tx=0x7feaa4009680 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.386+0000 7feab3f7c740 1 -- 172.21.15.49:0/308398055 shutdown_connections 2024-02-26T06:28:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.386+0000 7feab3f7c740 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9c9f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.386+0000 7feab3f7c740 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd91710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.386+0000 7feab3f7c740 1 --2- 172.21.15.49:0/308398055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd90cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.386+0000 7feab3f7c740 1 -- 172.21.15.49:0/308398055 >> 172.21.15.49:0/308398055 conn(0x55cbdfd58570 msgr2=0x55cbdfd8bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.387+0000 7feab3f7c740 1 -- 172.21.15.49:0/308398055 shutdown_connections 2024-02-26T06:28:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.387+0000 7feab3f7c740 1 -- 172.21.15.49:0/308398055 wait complete. 2024-02-26T06:28:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.387+0000 7feab3f7c740 1 -- start start 2024-02-26T06:28:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.388+0000 7feab3f7c740 1 Processor -- start 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.388+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.388+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd97af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.388+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd98030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.388+0000 7feab3f7c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfc9de90 con 0x55cbdfd91f70 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.388+0000 7feab3f7c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfc9db30 con 0x55cbdfd9a600 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.388+0000 7feab3f7c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfa671c0 con 0x55cbdfd91290 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.389+0000 7feab3f7c740 10 client.0 _ll_register_callbacks cb 0x55cbdfcb1930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.389+0000 7feaa9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd97af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.389+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.389+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60628/0 (socket says 172.21.15.49:60628) 2024-02-26T06:28:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.389+0000 7feaa9e00640 1 -- 172.21.15.49:0/336174300 learned_addr learned my addr 172.21.15.49:0/336174300 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.389+0000 7feaaa800640 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd98030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7feaa9e00640 1 -- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 msgr2=0x55cbdfd98030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7feaa9e00640 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd98030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7feaa9e00640 1 -- 172.21.15.49:0/336174300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 msgr2=0x55cbdfd97af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7feaa9e00640 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd97af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7feaa9e00640 1 -- 172.21.15.49:0/336174300 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9800b8c0 con 0x55cbdfd91290 2024-02-26T06:28:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7feaa9400640 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd97af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7feaaa800640 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd98030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7feaa9e00640 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd975b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0bce20 tx=0x7fea9c0b93c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa546dca70cb197 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.390+0000 7fea8be00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.391+0000 7fea8be00640 1 -- 172.21.15.49:0/336174300 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0bca80 con 0x55cbdfd91290 2024-02-26T06:28:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.391+0000 7fea8be00640 1 -- 172.21.15.49:0/336174300 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fea9c0035f0 con 0x55cbdfd91290 2024-02-26T06:28:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.391+0000 7feab3f7c740 1 -- 172.21.15.49:0/336174300 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbdf9f87e0 con 0x55cbdfd91290 2024-02-26T06:28:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.391+0000 7fea8be00640 1 -- 172.21.15.49:0/336174300 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea9c003e30 con 0x55cbdfd91290 2024-02-26T06:28:54.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.392+0000 7feab3f7c740 10 client.26833 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:54.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.392+0000 7feab3f7c740 1 -- 172.21.15.49:0/336174300 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55cbdfcf74b0 con 0x55cbdfd91290 2024-02-26T06:28:54.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.393+0000 7fea8be00640 1 -- 172.21.15.49:0/336174300 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cbdf9f87e0 con 0x55cbdfd91290 2024-02-26T06:28:54.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.393+0000 7fea8be00640 1 -- 172.21.15.49:0/336174300 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea9c0f8070 con 0x55cbdfd91290 2024-02-26T06:28:54.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.394+0000 7feab3f7c740 10 client.26833 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:54.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.394+0000 7feab3f7c740 10 client.26833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:54.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.394+0000 7feab3f7c740 20 client.26833 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:54.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.394+0000 7feab3f7c740 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cbdfdb2bd0 0x55cbdfdb4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.394+0000 7feab3f7c740 4 client.26833 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:54.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.394+0000 7feab3f7c740 1 -- 172.21.15.49:0/336174300 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbdfa35720 con 0x55cbdfdb2bd0 2024-02-26T06:28:54.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.394+0000 7feaa9400640 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cbdfdb2bd0 0x55cbdfdb4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.397+0000 7feaa9400640 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cbdfdb2bd0 0x55cbdfdb4ff0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9327b49bdae86b95 server_cookie=83011b676f5cc4e in_seq=0 out_seq=0 2024-02-26T06:28:54.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.397+0000 7fea8be00640 10 client.26833 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:54.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.397+0000 7fea8be00640 1 -- 172.21.15.49:0/336174300 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbdfa35720 con 0x55cbdfdb2bd0 2024-02-26T06:28:54.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.397+0000 7fea8be00640 10 client.26833 handle_command_reply: tid=0 2024-02-26T06:28:54.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 client.26833 shutdown 2024-02-26T06:28:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 -- 172.21.15.49:0/336174300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 msgr2=0x55cbdfd975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd975b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0bce20 tx=0x7fea9c0b93c0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 -- 172.21.15.49:0/336174300 shutdown_connections 2024-02-26T06:28:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cbdfdb2bd0 0x55cbdfdb4ff0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd98030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7fea8be00640 0 client.26833 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91f70 0x55cbdfd97af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 --2- 172.21.15.49:0/336174300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91290 0x55cbdfd975b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.398+0000 7feab3f7c740 1 -- 172.21.15.49:0/336174300 >> 172.21.15.49:0/336174300 conn(0x55cbdfd58570 msgr2=0x55cbdfd8da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.399+0000 7feab3f7c740 1 -- 172.21.15.49:0/336174300 shutdown_connections 2024-02-26T06:28:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.399+0000 7feab3f7c740 1 -- 172.21.15.49:0/336174300 wait complete. 2024-02-26T06:28:54.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.399+0000 7feab3f7c740 20 client.26833 trim_cache size 0 max 16384 2024-02-26T06:28:54.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.435+0000 7feab3f7c740 1 Processor -- start 2024-02-26T06:28:54.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.436+0000 7feab3f7c740 1 -- start start 2024-02-26T06:28:54.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.436+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91290 0x55cbdfd99ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.436+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfd96590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.436+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd97e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.436+0000 7feab3f7c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdf9f87e0 con 0x55cbdfd91290 2024-02-26T06:28:54.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.436+0000 7feab3f7c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfd86cb0 con 0x55cbdfd9a600 2024-02-26T06:28:54.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.436+0000 7feab3f7c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfd90b00 con 0x55cbdfd91f70 2024-02-26T06:28:54.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.437+0000 7feaaa800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd97e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.437+0000 7feaaa800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd97e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52914/0 (socket says 172.21.15.49:52914) 2024-02-26T06:28:54.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.439+0000 7feaaa800640 1 -- 172.21.15.49:0/4291475043 learned_addr learned my addr 172.21.15.49:0/4291475043 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.438+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91290 0x55cbdfd99ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.438+0000 7feaa9400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfd96590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.440+0000 7feaa9e00640 1 -- 172.21.15.49:0/4291475043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 msgr2=0x55cbdfd96590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.440+0000 7feaa9e00640 1 --2- 172.21.15.49:0/4291475043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfd96590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.440+0000 7feaa9e00640 1 -- 172.21.15.49:0/4291475043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 msgr2=0x55cbdfd97e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.440+0000 7feaa9e00640 1 --2- 172.21.15.49:0/4291475043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd97e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.440+0000 7feaa9e00640 1 -- 172.21.15.49:0/4291475043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbdfc93920 con 0x55cbdfd91290 2024-02-26T06:28:54.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.441+0000 7feaa9e00640 1 --2- 172.21.15.49:0/4291475043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91290 0x55cbdfd99ce0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0f25a0 tx=0x7fea9c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2368c47edc3b43 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.443+0000 7fea96a00640 1 -- 172.21.15.49:0/4291475043 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c003e30 con 0x55cbdfd91290 2024-02-26T06:28:54.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.443+0000 7fea96a00640 1 -- 172.21.15.49:0/4291475043 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0f2110 con 0x55cbdfd91290 2024-02-26T06:28:54.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.443+0000 7fea96a00640 1 -- 172.21.15.49:0/4291475043 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0bd120 con 0x55cbdfd91290 2024-02-26T06:28:54.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.444+0000 7feab3f7c740 1 -- 172.21.15.49:0/4291475043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91290 msgr2=0x55cbdfd99ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.444+0000 7feab3f7c740 1 --2- 172.21.15.49:0/4291475043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91290 0x55cbdfd99ce0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0f25a0 tx=0x7fea9c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:54.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.444+0000 7feab3f7c740 1 -- 172.21.15.49:0/4291475043 shutdown_connections 2024-02-26T06:28:54.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.444+0000 7feab3f7c740 1 --2- 172.21.15.49:0/4291475043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd97e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.444+0000 7feab3f7c740 1 --2- 172.21.15.49:0/4291475043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfd96590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.444+0000 7feab3f7c740 1 --2- 172.21.15.49:0/4291475043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfd91290 0x55cbdfd99ce0 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0f25a0 tx=0x7fea9c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:54.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.444+0000 7feab3f7c740 1 -- 172.21.15.49:0/4291475043 >> 172.21.15.49:0/4291475043 conn(0x55cbdfd58570 msgr2=0x55cbdfd95e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.444+0000 7feab3f7c740 1 -- 172.21.15.49:0/4291475043 shutdown_connections 2024-02-26T06:28:54.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.446+0000 7feab3f7c740 1 -- 172.21.15.49:0/4291475043 wait complete. 2024-02-26T06:28:54.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.446+0000 7feab3f7c740 1 -- start start 2024-02-26T06:28:54.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.446+0000 7feab3f7c740 1 Processor -- start 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfda1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feab3f7c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfda03e0 0x55cbdfe090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfda1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feaa9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfda1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60636/0 (socket says 172.21.15.49:60636) 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfda09f0 con 0x55cbdfda03e0 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfda0b60 con 0x55cbdfd9a600 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbdfe09620 con 0x55cbdfd91f70 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feaa9e00640 1 -- 172.21.15.49:0/3297542287 learned_addr learned my addr 172.21.15.49:0/3297542287 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:54.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.448+0000 7feab3f7c740 10 client.0 _ll_register_callbacks cb 0x7fea9c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:54.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.449+0000 7feaa9400640 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.449+0000 7feaa9e00640 1 -- 172.21.15.49:0/3297542287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 msgr2=0x55cbdfd9fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.449+0000 7feaa9e00640 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.449+0000 7feaa9e00640 1 -- 172.21.15.49:0/3297542287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfda03e0 msgr2=0x55cbdfe090e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:54.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.449+0000 7feaa9e00640 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfda03e0 0x55cbdfe090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.449+0000 7feaa9e00640 1 -- 172.21.15.49:0/3297542287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9800b8c0 con 0x55cbdfd91f70 2024-02-26T06:28:54.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.450+0000 7fea8be00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:54.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.450+0000 7feaa9e00640 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfda1a40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0b9af0 tx=0x7fea9c0bb630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f94b0c557780be server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:54.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.450+0000 7fea8be00640 1 -- 172.21.15.49:0/3297542287 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0f62b0 con 0x55cbdfd91f70 2024-02-26T06:28:54.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.450+0000 7fea8be00640 1 -- 172.21.15.49:0/3297542287 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fea9c0035f0 con 0x55cbdfd91f70 2024-02-26T06:28:54.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.451+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea98015070 con 0x55cbdfd91f70 2024-02-26T06:28:54.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.453+0000 7fea8be00640 1 -- 172.21.15.49:0/3297542287 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea9c0bca80 con 0x55cbdfd91f70 2024-02-26T06:28:54.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.453+0000 7feab3f7c740 10 client.26839 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:54.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.453+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55cbdfcb9400 con 0x55cbdfd91f70 2024-02-26T06:28:54.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.457+0000 7fea8be00640 1 -- 172.21.15.49:0/3297542287 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea98015070 con 0x55cbdfd91f70 2024-02-26T06:28:54.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.457+0000 7fea8be00640 1 -- 172.21.15.49:0/3297542287 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea9c0f8070 con 0x55cbdfd91f70 2024-02-26T06:28:54.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.458+0000 7feab3f7c740 10 client.26839 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:54.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.458+0000 7feab3f7c740 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:54.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.458+0000 7feab3f7c740 20 client.26839 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:54.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.458+0000 7feab3f7c740 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cbdfe0a7f0 0x55cbdfe0cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:54.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.458+0000 7feab3f7c740 4 client.26839 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:28:54.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.458+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cbdfa84890 con 0x55cbdfe0a7f0 2024-02-26T06:28:54.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.458+0000 7feaa9400640 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cbdfe0a7f0 0x55cbdfe0cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:54.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.459+0000 7feaa9400640 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cbdfe0a7f0 0x55cbdfe0cbe0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ca807ca24b3da7 server_cookie=4ca6c52d7f1c9ce0 in_seq=0 out_seq=0 2024-02-26T06:28:54.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.459+0000 7fea8be00640 10 client.26839 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:54.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.465+0000 7fea8be00640 1 -- 172.21.15.49:0/3297542287 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cbdfa84890 con 0x55cbdfe0a7f0 2024-02-26T06:28:54.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.465+0000 7fea8be00640 10 client.26839 handle_command_reply: tid=0 2024-02-26T06:28:54.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.465+0000 7feab3f7c740 1 client.26839 shutdown 2024-02-26T06:28:54.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.465+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 msgr2=0x55cbdfda1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.465+0000 7feab3f7c740 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfda1a40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0b9af0 tx=0x7fea9c0bb630 comp rx=0 tx=0).stop 2024-02-26T06:28:54.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.466+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 shutdown_connections 2024-02-26T06:28:54.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.466+0000 7feab3f7c740 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cbdfe0a7f0 0x55cbdfe0cbe0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.466+0000 7feab3f7c740 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cbdfd9a600 0x55cbdfd9fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.466+0000 7feab3f7c740 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbdfda03e0 0x55cbdfe090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.466+0000 7feab3f7c740 1 --2- 172.21.15.49:0/3297542287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbdfd91f70 0x55cbdfda1a40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.466+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 >> 172.21.15.49:0/3297542287 conn(0x55cbdfd58570 msgr2=0x55cbdfd8a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.466+0000 7fea8be00640 0 client.26839 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:54.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.468+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 shutdown_connections 2024-02-26T06:28:54.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.468+0000 7feab3f7c740 1 -- 172.21.15.49:0/3297542287 wait complete. 2024-02-26T06:28:54.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.471+0000 7feab3f7c740 20 client.26839 trim_cache size 0 max 16384 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout: "59f0e69f-5a7e-4c02-9e44-302a524aa1b3": { 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "59f0e69f-5a7e-4c02-9e44-302a524aa1b3", 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:28:54.472 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:28:54.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.472+0000 7feab1000640 1 -- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea900a22b0 msgr2=0x7fea900a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.472+0000 7feab1000640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea900a22b0 0x7fea900a4770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feaa4005fd0 tx=0x7feaa4009680 comp rx=0 tx=0).stop 2024-02-26T06:28:54.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.472+0000 7feab1000640 1 -- 172.21.15.49:0/4152406404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feaac103590 msgr2=0x7feaac19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:54.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.472+0000 7feab1000640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feaac103590 0x7feaac19f6c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fea9c000c00 tx=0x7fea9c004040 comp rx=0 tx=0).stop 2024-02-26T06:28:54.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.473+0000 7feab1000640 1 -- 172.21.15.49:0/4152406404 shutdown_connections 2024-02-26T06:28:54.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.473+0000 7feab1000640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fea900a22b0 0x7fea900a4770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.473+0000 7feab1000640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feaac199830 0x7feaac199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.473+0000 7feab1000640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaac108660 0x7feaac19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.473+0000 7feab1000640 1 --2- 172.21.15.49:0/4152406404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feaac103590 0x7feaac19f6c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:54.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.473+0000 7feab1000640 1 -- 172.21.15.49:0/4152406404 >> 172.21.15.49:0/4152406404 conn(0x7feaac0fee30 msgr2=0x7feaac101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:54.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.473+0000 7feab1000640 1 -- 172.21.15.49:0/4152406404 shutdown_connections 2024-02-26T06:28:54.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:28:54.473+0000 7feab1000640 1 -- 172.21.15.49:0/4152406404 wait complete. 2024-02-26T06:28:54.507 INFO:tasks.cephfs.filesystem:scrub status for tag:59f0e69f-5a7e-4c02-9e44-302a524aa1b3 - {'path': '/', 'tag': '59f0e69f-5a7e-4c02-9e44-302a524aa1b3', 'options': 'recursive,force'} 2024-02-26T06:28:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: pgmap v168: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 32 MiB/s wr, 1.71k op/s 2024-02-26T06:28:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:28:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:54 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: pgmap v168: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 32 MiB/s wr, 1.71k op/s 2024-02-26T06:28:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:28:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:55.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:55.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:55.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:55.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:28:55.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:54 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: pgmap v168: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 32 MiB/s wr, 1.71k op/s 2024-02-26T06:28:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:28:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:55.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:54 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 305.0 s, 2465.8 tps, lat 40.667 ms stddev 49.128 2024-02-26T06:28:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:56 smithi080 ceph-mon[36206]: pgmap v169: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 38 MiB/s wr, 2.27k op/s 2024-02-26T06:28:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:56 smithi115 ceph-mon[36426]: pgmap v169: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 38 MiB/s wr, 2.27k op/s 2024-02-26T06:28:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:56 smithi049 ceph-mon[31696]: pgmap v169: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 38 MiB/s wr, 2.27k op/s 2024-02-26T06:28:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.820+0000 7f1d1f200640 1 -- 172.21.15.49:0/3375925974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d18103f80 msgr2=0x7f1d18107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.820+0000 7f1d1f200640 1 --2- 172.21.15.49:0/3375925974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d18103f80 0x7f1d18107fc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009930 tx=0x7f1d0402bc20 comp rx=0 tx=0).stop 2024-02-26T06:28:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.820+0000 7f1d1f200640 1 -- 172.21.15.49:0/3375925974 shutdown_connections 2024-02-26T06:28:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.820+0000 7f1d1f200640 1 --2- 172.21.15.49:0/3375925974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d18103f80 0x7f1d18107fc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.820+0000 7f1d1f200640 1 --2- 172.21.15.49:0/3375925974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d181086a0 0x7f1d18110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.820+0000 7f1d1f200640 1 --2- 172.21.15.49:0/3375925974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d181035d0 0x7f1d181039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.820+0000 7f1d1f200640 1 -- 172.21.15.49:0/3375925974 >> 172.21.15.49:0/3375925974 conn(0x7f1d180fee30 msgr2=0x7f1d18101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.821+0000 7f1d1f200640 1 -- 172.21.15.49:0/3375925974 shutdown_connections 2024-02-26T06:28:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.821+0000 7f1d1f200640 1 -- 172.21.15.49:0/3375925974 wait complete. 2024-02-26T06:28:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.822+0000 7f1d1f200640 1 Processor -- start 2024-02-26T06:28:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.822+0000 7f1d1f200640 1 -- start start 2024-02-26T06:28:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.822+0000 7f1d1f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d181035d0 0x7f1d1819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.822+0000 7f1d1f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d181086a0 0x7f1d1819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.822+0000 7f1d1f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d181997d0 0x7f1d18199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.822+0000 7f1d1f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1819a280 con 0x7f1d181086a0 2024-02-26T06:28:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.822+0000 7f1d1f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1819a3f0 con 0x7f1d181997d0 2024-02-26T06:28:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.822+0000 7f1d1f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1819a560 con 0x7f1d181035d0 2024-02-26T06:28:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d181035d0 0x7f1d1819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d181035d0 0x7f1d1819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60654/0 (socket says 172.21.15.49:60654) 2024-02-26T06:28:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d17e00640 1 -- 172.21.15.49:0/1529817652 learned_addr learned my addr 172.21.15.49:0/1529817652 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d1d200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d181997d0 0x7f1d18199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d17400640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d181086a0 0x7f1d1819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d1d200640 1 -- 172.21.15.49:0/1529817652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d181035d0 msgr2=0x7f1d1819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d1d200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d181035d0 0x7f1d1819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d1d200640 1 -- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d181086a0 msgr2=0x7f1d1819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d1d200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d181086a0 0x7f1d1819fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d1d200640 1 -- 172.21.15.49:0/1529817652 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04009560 con 0x7f1d181997d0 2024-02-26T06:28:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d17e00640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d181035d0 0x7f1d1819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.823+0000 7f1d1d200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d181997d0 0x7f1d18199c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000d680 tx=0x7f1d1000db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.828+0000 7f1d14c00640 1 -- 172.21.15.49:0/1529817652 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10005220 con 0x7f1d181997d0 2024-02-26T06:28:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.828+0000 7f1d14c00640 1 -- 172.21.15.49:0/1529817652 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d10009d00 con 0x7f1d181997d0 2024-02-26T06:28:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.828+0000 7f1d14c00640 1 -- 172.21.15.49:0/1529817652 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10004ea0 con 0x7f1d181997d0 2024-02-26T06:28:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.828+0000 7f1d1f200640 1 -- 172.21.15.49:0/1529817652 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d181a6510 con 0x7f1d181997d0 2024-02-26T06:28:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.828+0000 7f1d1f200640 1 -- 172.21.15.49:0/1529817652 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d181a69a0 con 0x7f1d181997d0 2024-02-26T06:28:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.831+0000 7f1d14c00640 1 -- 172.21.15.49:0/1529817652 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1d10009820 con 0x7f1d181997d0 2024-02-26T06:28:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.831+0000 7f1d2217a740 1 Processor -- start 2024-02-26T06:28:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.831+0000 7f1d2217a740 1 -- start start 2024-02-26T06:28:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.831+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d79840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.831+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d7a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d2217a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0d355f0 con 0x564be0d7aaf0 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d2217a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0ce7890 con 0x564be0d83300 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d2217a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0ce4dc0 con 0x564be0d79d80 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d79840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d7a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d7a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60666/0 (socket says 172.21.15.49:60666) 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17e00640 1 -- 172.21.15.49:0/3072853811 learned_addr learned my addr 172.21.15.49:0/3072853811 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d14c00640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1cfc0a23a0 0x7f1cfc0a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17400640 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17e00640 1 -- 172.21.15.49:0/3072853811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 msgr2=0x564be0d856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17e00640 1 -- 172.21.15.49:0/3072853811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 msgr2=0x564be0d79840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d79840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17e00640 1 -- 172.21.15.49:0/3072853811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564be0c61ad0 con 0x564be0d79d80 2024-02-26T06:28:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d17400640 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d79840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d14c00640 1 -- 172.21.15.49:0/1529817652 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1d100cd9a0 con 0x7f1d181997d0 2024-02-26T06:28:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.832+0000 7f1d1d200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1cfc0a23a0 0x7f1cfc0a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.833+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d7a200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800f8d0 tx=0x7f1d0800fd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875ed1102665f506 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.833+0000 7f1d0d600640 1 -- 172.21.15.49:0/3072853811 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08011c20 con 0x564be0d79d80 2024-02-26T06:28:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.833+0000 7f1d0d600640 1 -- 172.21.15.49:0/3072853811 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d08011d80 con 0x564be0d79d80 2024-02-26T06:28:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.833+0000 7f1d0d600640 1 -- 172.21.15.49:0/3072853811 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080187b0 con 0x564be0d79d80 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d1d200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1cfc0a23a0 0x7f1cfc0a4860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d100967a0 tx=0x7f1d100ba040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 -- 172.21.15.49:0/3072853811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 msgr2=0x564be0d7a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d7a200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800f8d0 tx=0x7f1d0800fd90 comp rx=0 tx=0).stop 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 -- 172.21.15.49:0/3072853811 shutdown_connections 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d7a200 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3072853811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d79840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 -- 172.21.15.49:0/3072853811 >> 172.21.15.49:0/3072853811 conn(0x564be0d41480 msgr2=0x564be0d74a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 -- 172.21.15.49:0/3072853811 shutdown_connections 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 -- 172.21.15.49:0/3072853811 wait complete. 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.834+0000 7f1d2217a740 1 -- start start 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d2217a740 1 Processor -- start 2024-02-26T06:28:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d80370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d80df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d2217a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0ce05a0 con 0x564be0d7aaf0 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d2217a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0c85b20 con 0x564be0d83300 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d2217a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0c857c0 con 0x564be0d79d80 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d2217a740 10 client.0 _ll_register_callbacks cb 0x564be0c99520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d17400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d80df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d17e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d17400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d80df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52964/0 (socket says 172.21.15.49:52964) 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.835+0000 7f1d17400640 1 -- 172.21.15.49:0/3451504867 learned_addr learned my addr 172.21.15.49:0/3451504867 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.836+0000 7f1d17400640 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d80370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.836+0000 7f1d17e00640 1 -- 172.21.15.49:0/3451504867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 msgr2=0x564be0d80370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.836+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d80370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.836+0000 7f1d17e00640 1 -- 172.21.15.49:0/3451504867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 msgr2=0x564be0d80df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.837+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d80df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.837+0000 7f1d17e00640 1 -- 172.21.15.49:0/3451504867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0800f530 con 0x564be0d7aaf0 2024-02-26T06:28:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.837+0000 7f1d17400640 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d80df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.837+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d808b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d08011810 tx=0x7f1d08003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c48f8196432413 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.837+0000 7f1d02000640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:28:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.838+0000 7f1d02000640 1 -- 172.21.15.49:0/3451504867 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08004c40 con 0x564be0d7aaf0 2024-02-26T06:28:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.838+0000 7f1d02000640 1 -- 172.21.15.49:0/3451504867 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1d08004da0 con 0x564be0d7aaf0 2024-02-26T06:28:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.839+0000 7f1d2217a740 1 -- 172.21.15.49:0/3451504867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564be0cb2ca0 con 0x564be0d7aaf0 2024-02-26T06:28:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.840+0000 7f1d02000640 1 -- 172.21.15.49:0/3451504867 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d08004c40 con 0x564be0d7aaf0 2024-02-26T06:28:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.841+0000 7f1d2217a740 10 client.16368 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.841+0000 7f1d2217a740 1 -- 172.21.15.49:0/3451504867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564be0cdec20 con 0x564be0d7aaf0 2024-02-26T06:28:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.842+0000 7f1d02000640 1 -- 172.21.15.49:0/3451504867 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564be0cb2ca0 con 0x564be0d7aaf0 2024-02-26T06:28:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.843+0000 7f1d02000640 1 -- 172.21.15.49:0/3451504867 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1d08005350 con 0x564be0d7aaf0 2024-02-26T06:28:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.843+0000 7f1d2217a740 10 client.16368 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.844+0000 7f1d2217a740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.844+0000 7f1d2217a740 20 client.16368 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.844+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564be0d93260 0x564be0d95680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.844+0000 7f1d2217a740 4 client.16368 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.844+0000 7f1d2217a740 1 -- 172.21.15.49:0/3451504867 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564be0cd6cb0 con 0x564be0d93260 2024-02-26T06:28:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.846+0000 7f1d17400640 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564be0d93260 0x564be0d95680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.850+0000 7f1d17400640 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564be0d93260 0x564be0d95680 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2829c508fbe73c19 server_cookie=4b8a11757cd9c24 in_seq=0 out_seq=0 2024-02-26T06:28:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.850+0000 7f1d02000640 10 client.16368 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.855+0000 7f1d02000640 1 -- 172.21.15.49:0/3451504867 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564be0cd6cb0 con 0x564be0d93260 2024-02-26T06:28:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.855+0000 7f1d02000640 10 client.16368 handle_command_reply: tid=0 2024-02-26T06:28:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.855+0000 7f1d2217a740 1 client.16368 shutdown 2024-02-26T06:28:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.856+0000 7f1d2217a740 1 -- 172.21.15.49:0/3451504867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 msgr2=0x564be0d808b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.856+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d808b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d08011810 tx=0x7f1d08003ba0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.856+0000 7f1d2217a740 1 -- 172.21.15.49:0/3451504867 shutdown_connections 2024-02-26T06:28:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.856+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564be0d93260 0x564be0d95680 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.856+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d83300 0x564be0d80df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.856+0000 7f1d02000640 0 client.16368 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.857+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d808b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.857+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3451504867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d79d80 0x564be0d80370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.857+0000 7f1d2217a740 1 -- 172.21.15.49:0/3451504867 >> 172.21.15.49:0/3451504867 conn(0x564be0d41480 msgr2=0x564be0d767c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.858+0000 7f1d2217a740 1 -- 172.21.15.49:0/3451504867 shutdown_connections 2024-02-26T06:28:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.858+0000 7f1d2217a740 1 -- 172.21.15.49:0/3451504867 wait complete. 2024-02-26T06:28:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.859+0000 7f1d2217a740 20 client.16368 trim_cache size 0 max 16384 2024-02-26T06:28:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.889+0000 7f1d2217a740 1 Processor -- start 2024-02-26T06:28:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.890+0000 7f1d2217a740 1 -- start start 2024-02-26T06:28:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.890+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d79d80 0x564be0d82aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.890+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d80c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.890+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0d8a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.890+0000 7f1d2217a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0a28e60 con 0x564be0d7aaf0 2024-02-26T06:28:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.890+0000 7f1d2217a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0d7bf70 con 0x564be0d79d80 2024-02-26T06:28:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.890+0000 7f1d2217a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0d75ca0 con 0x564be0d83300 2024-02-26T06:28:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.891+0000 7f1d17400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d79d80 0x564be0d82aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.891+0000 7f1d17400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d79d80 0x564be0d82aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52966/0 (socket says 172.21.15.49:52966) 2024-02-26T06:28:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.891+0000 7f1d17400640 1 -- 172.21.15.49:0/3764926413 learned_addr learned my addr 172.21.15.49:0/3764926413 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.891+0000 7f1d17400640 1 --2- 172.21.15.49:0/3764926413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0d8a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d17400640 1 -- 172.21.15.49:0/3764926413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 msgr2=0x564be0d8a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d17400640 1 --2- 172.21.15.49:0/3764926413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0d8a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d17400640 1 -- 172.21.15.49:0/3764926413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 msgr2=0x564be0d80c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d17400640 1 --2- 172.21.15.49:0/3764926413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d80c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d17400640 1 -- 172.21.15.49:0/3764926413 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564be0c7b500 con 0x564be0d79d80 2024-02-26T06:28:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d17400640 1 --2- 172.21.15.49:0/3764926413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d79d80 0x564be0d82aa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1d0402c830 tx=0x7f1d04005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0e026cbb4a88ee server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d0cc00640 1 -- 172.21.15.49:0/3764926413 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0404cb80 con 0x564be0d79d80 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d0cc00640 1 -- 172.21.15.49:0/3764926413 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d0402ce90 con 0x564be0d79d80 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.892+0000 7f1d0cc00640 1 -- 172.21.15.49:0/3764926413 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0404c660 con 0x564be0d79d80 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.893+0000 7f1d2217a740 1 -- 172.21.15.49:0/3764926413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d79d80 msgr2=0x564be0d82aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.893+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3764926413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d79d80 0x564be0d82aa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1d0402c830 tx=0x7f1d04005fd0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.893+0000 7f1d2217a740 1 -- 172.21.15.49:0/3764926413 shutdown_connections 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.893+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3764926413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0d8a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.893+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3764926413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d7aaf0 0x564be0d80c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.893+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3764926413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d79d80 0x564be0d82aa0 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1d0402c830 tx=0x7f1d04005fd0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.893+0000 7f1d2217a740 1 -- 172.21.15.49:0/3764926413 >> 172.21.15.49:0/3764926413 conn(0x564be0d41480 msgr2=0x564be0d745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.894+0000 7f1d2217a740 1 -- 172.21.15.49:0/3764926413 shutdown_connections 2024-02-26T06:28:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.894+0000 7f1d2217a740 1 -- 172.21.15.49:0/3764926413 wait complete. 2024-02-26T06:28:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.895+0000 7f1d2217a740 1 -- start start 2024-02-26T06:28:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.895+0000 7f1d2217a740 1 Processor -- start 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.896+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d7aaf0 0x564be0d8a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.896+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0de4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.896+0000 7f1d2217a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d93260 0x564be0de4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.897+0000 7f1d2217a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0de5510 con 0x564be0d93260 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.897+0000 7f1d2217a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0de1f60 con 0x564be0d7aaf0 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.897+0000 7f1d2217a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0de2120 con 0x564be0d83300 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.897+0000 7f1d17400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d7aaf0 0x564be0d8a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.897+0000 7f1d2217a740 10 client.0 _ll_register_callbacks cb 0x7f1d0800f750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.897+0000 7f1d17400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d7aaf0 0x564be0d8a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:52978/0 (socket says 172.21.15.49:52978) 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.897+0000 7f1d17400640 1 -- 172.21.15.49:0/3156151674 learned_addr learned my addr 172.21.15.49:0/3156151674 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:28:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.897+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0de4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.898+0000 7f1d17e00640 1 -- 172.21.15.49:0/3156151674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d7aaf0 msgr2=0x564be0d8a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.898+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d7aaf0 0x564be0d8a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.898+0000 7f1d17e00640 1 -- 172.21.15.49:0/3156151674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d93260 msgr2=0x564be0de4f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.899+0000 7f1d17400640 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d93260 0x564be0de4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.899+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d93260 0x564be0de4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.899+0000 7f1d17e00640 1 -- 172.21.15.49:0/3156151674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0404c040 con 0x564be0d83300 2024-02-26T06:28:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.899+0000 7f1d17400640 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d93260 0x564be0de4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.900+0000 7f1d17e00640 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0de4a10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x564be0d930f0 tx=0x7f1d080096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57858c3ea39b22c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.900+0000 7f1d02000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:28:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.900+0000 7f1d02000640 1 -- 172.21.15.49:0/3156151674 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08011840 con 0x564be0d83300 2024-02-26T06:28:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.901+0000 7f1d02000640 1 -- 172.21.15.49:0/3156151674 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f1d08019770 con 0x564be0d83300 2024-02-26T06:28:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.901+0000 7f1d2217a740 1 -- 172.21.15.49:0/3156151674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564be0d05d00 con 0x564be0d83300 2024-02-26T06:28:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.902+0000 7f1d02000640 1 -- 172.21.15.49:0/3156151674 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d08005880 con 0x564be0d83300 2024-02-26T06:28:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.902+0000 7f1d2217a740 10 client.26857 fetch_fsmap learned FSMap version 35 2024-02-26T06:28:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.902+0000 7f1d2217a740 1 -- 172.21.15.49:0/3156151674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564be0ca1290 con 0x564be0d83300 2024-02-26T06:28:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.902+0000 7f1d02000640 1 -- 172.21.15.49:0/3156151674 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564be0d05d00 con 0x564be0d83300 2024-02-26T06:28:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.903+0000 7f1d02000640 1 -- 172.21.15.49:0/3156151674 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1d0803e070 con 0x564be0d83300 2024-02-26T06:28:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.903+0000 7f1d2217a740 10 client.26857 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:28:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.903+0000 7f1d2217a740 10 client.26857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.903+0000 7f1d2217a740 20 client.26857 populate_metadata read hostname 'smithi049' 2024-02-26T06:28:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.903+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564be0d894f0 0x564be0d898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.903+0000 7f1d2217a740 4 client.26857 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:28:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.903+0000 7f1d2217a740 1 -- 172.21.15.49:0/3156151674 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564be09e0600 con 0x564be0d894f0 2024-02-26T06:28:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.903+0000 7f1d17400640 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564be0d894f0 0x564be0d898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.904+0000 7f1d17400640 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564be0d894f0 0x564be0d898d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a4885add576b30 server_cookie=c81a160d30d66480 in_seq=0 out_seq=0 2024-02-26T06:28:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.904+0000 7f1d02000640 10 client.26857 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.935+0000 7f1d02000640 1 -- 172.21.15.49:0/3156151674 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564be09e0600 con 0x564be0d894f0 2024-02-26T06:28:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.935+0000 7f1d02000640 10 client.26857 handle_command_reply: tid=0 2024-02-26T06:28:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.935+0000 7f1d2217a740 1 client.26857 shutdown 2024-02-26T06:28:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 -- 172.21.15.49:0/3156151674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 msgr2=0x564be0de4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0de4a10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x564be0d930f0 tx=0x7f1d080096c0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 -- 172.21.15.49:0/3156151674 shutdown_connections 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x564be0d894f0 0x564be0d898d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564be0d93260 0x564be0de4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564be0d83300 0x564be0de4a10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 --2- 172.21.15.49:0/3156151674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564be0d7aaf0 0x564be0d8a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d02000640 0 client.26857 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 -- 172.21.15.49:0/3156151674 >> 172.21.15.49:0/3156151674 conn(0x564be0d41480 msgr2=0x564be0d7eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 -- 172.21.15.49:0/3156151674 shutdown_connections 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 1 -- 172.21.15.49:0/3156151674 wait complete. 2024-02-26T06:28:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.936+0000 7f1d2217a740 20 client.26857 trim_cache size 0 max 16384 2024-02-26T06:28:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:28:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:28:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:28:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:28:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 -- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1cfc0a23a0 msgr2=0x7f1cfc0a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1cfc0a23a0 0x7f1cfc0a4860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d100967a0 tx=0x7f1d100ba040 comp rx=0 tx=0).stop 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 -- 172.21.15.49:0/1529817652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d181997d0 msgr2=0x7f1d18199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d181997d0 0x7f1d18199c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000d680 tx=0x7f1d1000db40 comp rx=0 tx=0).stop 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 -- 172.21.15.49:0/1529817652 shutdown_connections 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f1cfc0a23a0 0x7f1cfc0a4860 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d181997d0 0x7f1d18199c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d181086a0 0x7f1d1819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 --2- 172.21.15.49:0/1529817652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d181035d0 0x7f1d1819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.937+0000 7f1d1f200640 1 -- 172.21.15.49:0/1529817652 >> 172.21.15.49:0/1529817652 conn(0x7f1d180fee30 msgr2=0x7f1d180ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.938+0000 7f1d1f200640 1 -- 172.21.15.49:0/1529817652 shutdown_connections 2024-02-26T06:28:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:28:58.938+0000 7f1d1f200640 1 -- 172.21.15.49:0/1529817652 wait complete. 2024-02-26T06:28:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:28:58 smithi080 ceph-mon[36206]: pgmap v170: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 31 MiB/s wr, 1.98k op/s 2024-02-26T06:28:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:28:58 smithi115 ceph-mon[36426]: pgmap v170: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 31 MiB/s wr, 1.98k op/s 2024-02-26T06:28:59.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:28:58 smithi049 ceph-mon[31696]: pgmap v170: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 31 MiB/s wr, 1.98k op/s 2024-02-26T06:29:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:00 smithi080 ceph-mon[36206]: pgmap v171: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 33 MiB/s wr, 2.18k op/s 2024-02-26T06:29:01.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:00 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:29:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:00 smithi115 ceph-mon[36426]: pgmap v171: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 33 MiB/s wr, 2.18k op/s 2024-02-26T06:29:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:00 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:29:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:00 smithi049 ceph-mon[31696]: pgmap v171: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 33 MiB/s wr, 2.18k op/s 2024-02-26T06:29:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:00 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:29:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 310.0 s, 2414.0 tps, lat 40.105 ms stddev 48.230 2024-02-26T06:29:02.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:01 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:29:02.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:01 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:29:02.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:01 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:29:02.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:01 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:29:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:01 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:29:02.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:01 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:29:03.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:02 smithi049 ceph-mon[31696]: pgmap v172: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-02-26T06:29:03.230 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:02 smithi080 ceph-mon[36206]: pgmap v172: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-02-26T06:29:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:02 smithi115 ceph-mon[36426]: pgmap v172: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-02-26T06:29:05.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:04 smithi049 ceph-mon[31696]: pgmap v173: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 10 MiB/s wr, 863 op/s 2024-02-26T06:29:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:04 smithi080 ceph-mon[36206]: pgmap v173: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 10 MiB/s wr, 863 op/s 2024-02-26T06:29:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:04 smithi115 ceph-mon[36426]: pgmap v173: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 10 MiB/s wr, 863 op/s 2024-02-26T06:29:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 315.0 s, 1669.2 tps, lat 61.622 ms stddev 126.345 2024-02-26T06:29:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:06 smithi049 ceph-mon[31696]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 25 MiB/s wr, 2.09k op/s 2024-02-26T06:29:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:06 smithi080 ceph-mon[36206]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 25 MiB/s wr, 2.09k op/s 2024-02-26T06:29:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:06 smithi115 ceph-mon[36426]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 25 MiB/s wr, 2.09k op/s 2024-02-26T06:29:09.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:08 smithi049 ceph-mon[31696]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-02-26T06:29:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:08 smithi080 ceph-mon[36206]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-02-26T06:29:09.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:08 smithi115 ceph-mon[36426]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-02-26T06:29:11.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:10 smithi049 ceph-mon[31696]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-02-26T06:29:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 320.0 s, 2249.4 tps, lat 44.230 ms stddev 51.870 2024-02-26T06:29:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:10 smithi080 ceph-mon[36206]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-02-26T06:29:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:10 smithi115 ceph-mon[36426]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-02-26T06:29:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 -- 172.21.15.49:0/3197619768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5204108180 msgr2=0x7f52041085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 --2- 172.21.15.49:0/3197619768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5204108180 0x7f52041085e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f51ec007740 tx=0x7f51ec02cb80 comp rx=0 tx=0).stop 2024-02-26T06:29:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 -- 172.21.15.49:0/3197619768 shutdown_connections 2024-02-26T06:29:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 --2- 172.21.15.49:0/3197619768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5204108b20 0x7f52041149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 --2- 172.21.15.49:0/3197619768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5204108180 0x7f52041085e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 --2- 172.21.15.49:0/3197619768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f520410cb60 0x7f520410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 -- 172.21.15.49:0/3197619768 >> 172.21.15.49:0/3197619768 conn(0x7f520406c220 msgr2=0x7f520406c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 -- 172.21.15.49:0/3197619768 shutdown_connections 2024-02-26T06:29:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.140+0000 7f520ac00640 1 -- 172.21.15.49:0/3197619768 wait complete. 2024-02-26T06:29:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.142+0000 7f520ac00640 1 Processor -- start 2024-02-26T06:29:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.143+0000 7f520ac00640 1 -- start start 2024-02-26T06:29:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.144+0000 7f520ac00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5204108180 0x7f520419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.144+0000 7f520ac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5204108b20 0x7f520419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.144+0000 7f520ac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f520410cb60 0x7f52041a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.144+0000 7f520ac00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52041a2d00 con 0x7f5204108180 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.144+0000 7f520ac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52041a2e40 con 0x7f520410cb60 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.144+0000 7f520ac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52041a3000 con 0x7f5204108b20 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.144+0000 7f5208c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f520410cb60 0x7f52041a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5208c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f520410cb60 0x7f52041a2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54844/0 (socket says 172.21.15.49:54844) 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5208c00640 1 -- 172.21.15.49:0/3934446849 learned_addr learned my addr 172.21.15.49:0/3934446849 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f51fbe00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5204108b20 0x7f520419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5203e00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5204108180 0x7f520419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5208c00640 1 -- 172.21.15.49:0/3934446849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5204108b20 msgr2=0x7f520419e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5208c00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5204108b20 0x7f520419e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5208c00640 1 -- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5204108180 msgr2=0x7f520419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5208c00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5204108180 0x7f520419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5208c00640 1 -- 172.21.15.49:0/3934446849 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0071c0 con 0x7f520410cb60 2024-02-26T06:29:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.145+0000 7f5208c00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f520410cb60 0x7f52041a2700 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00db20 tx=0x7f51fc00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.146+0000 7f5201600640 1 -- 172.21.15.49:0/3934446849 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00bea0 con 0x7f520410cb60 2024-02-26T06:29:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.146+0000 7f5201600640 1 -- 172.21.15.49:0/3934446849 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51fc009940 con 0x7f520410cb60 2024-02-26T06:29:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.146+0000 7f5201600640 1 -- 172.21.15.49:0/3934446849 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0106c0 con 0x7f520410cb60 2024-02-26T06:29:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.146+0000 7f520ac00640 1 -- 172.21.15.49:0/3934446849 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52041a32e0 con 0x7f520410cb60 2024-02-26T06:29:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.146+0000 7f520ac00640 1 -- 172.21.15.49:0/3934446849 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52041aab90 con 0x7f520410cb60 2024-02-26T06:29:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.155+0000 7f5201600640 1 -- 172.21.15.49:0/3934446849 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f51fc004240 con 0x7f520410cb60 2024-02-26T06:29:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.156+0000 7f520da78740 1 Processor -- start 2024-02-26T06:29:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.156+0000 7f520da78740 1 -- start start 2024-02-26T06:29:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.156+0000 7f520da78740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b8b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.156+0000 7f520da78740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b8ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.156+0000 7f520da78740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b96f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.156+0000 7f520da78740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927b3e5f0 con 0x55a927b8b590 2024-02-26T06:29:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.156+0000 7f520da78740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927af0890 con 0x55a927b8c300 2024-02-26T06:29:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.156+0000 7f520da78740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927aeddc0 con 0x55a927b94b10 2024-02-26T06:29:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.157+0000 7f5203e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b8ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.157+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b96f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.157+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b8b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.157+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b8b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54854/0 (socket says 172.21.15.49:54854) 2024-02-26T06:29:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.157+0000 7f5203e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b8ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45482/0 (socket says 172.21.15.49:45482) 2024-02-26T06:29:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.157+0000 7f51fbe00640 1 -- 172.21.15.49:0/4055762359 learned_addr learned my addr 172.21.15.49:0/4055762359 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.158+0000 7f5203e00640 1 -- 172.21.15.49:0/4055762359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 msgr2=0x55a927b96f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.158+0000 7f5203e00640 1 --2- 172.21.15.49:0/4055762359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b96f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.158+0000 7f5203e00640 1 -- 172.21.15.49:0/4055762359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 msgr2=0x55a927b8b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.158+0000 7f5203e00640 1 --2- 172.21.15.49:0/4055762359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b8b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.158+0000 7f5203e00640 1 -- 172.21.15.49:0/4055762359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a927a6aad0 con 0x55a927b8b590 2024-02-26T06:29:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.158+0000 7f5203e00640 1 --2- 172.21.15.49:0/4055762359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b8ba10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f51f400b760 tx=0x7f51f400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7d7f6388796d06 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.159+0000 7f5201600640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51e80a23d0 0x7f51e80a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.159+0000 7f5201600640 1 -- 172.21.15.49:0/3934446849 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f51fc0cdee0 con 0x7f520410cb60 2024-02-26T06:29:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.159+0000 7f51f9200640 1 -- 172.21.15.49:0/4055762359 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f400be70 con 0x55a927b8b590 2024-02-26T06:29:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.159+0000 7f51fbe00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51e80a23d0 0x7f51e80a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.160+0000 7f51f9200640 1 -- 172.21.15.49:0/4055762359 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51f400de30 con 0x55a927b8b590 2024-02-26T06:29:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.160+0000 7f51f9200640 1 -- 172.21.15.49:0/4055762359 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40123e0 con 0x55a927b8b590 2024-02-26T06:29:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.160+0000 7f520da78740 1 -- 172.21.15.49:0/4055762359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 msgr2=0x55a927b8ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.160+0000 7f520da78740 1 --2- 172.21.15.49:0/4055762359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b8ba10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f51f400b760 tx=0x7f51f400bc20 comp rx=0 tx=0).stop 2024-02-26T06:29:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.161+0000 7f520da78740 1 -- 172.21.15.49:0/4055762359 shutdown_connections 2024-02-26T06:29:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.161+0000 7f520da78740 1 --2- 172.21.15.49:0/4055762359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b96f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.161+0000 7f520da78740 1 --2- 172.21.15.49:0/4055762359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b8ba10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.161+0000 7f520da78740 1 --2- 172.21.15.49:0/4055762359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b8b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.161+0000 7f520da78740 1 -- 172.21.15.49:0/4055762359 >> 172.21.15.49:0/4055762359 conn(0x55a927b4a480 msgr2=0x55a927b86280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.162+0000 7f51fbe00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51e80a23d0 0x7f51e80a4890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55a927b87410 tx=0x7f51ec000bc0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.162+0000 7f520da78740 1 -- 172.21.15.49:0/4055762359 shutdown_connections 2024-02-26T06:29:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.162+0000 7f520da78740 1 -- 172.21.15.49:0/4055762359 wait complete. 2024-02-26T06:29:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.162+0000 7f520da78740 1 -- start start 2024-02-26T06:29:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.163+0000 7f520da78740 1 Processor -- start 2024-02-26T06:29:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f520da78740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b91b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f520da78740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8c300 0x55a927b92080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f520da78740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b94b10 0x55a927b925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f520da78740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927ae95a0 con 0x55a927b8c300 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f520da78740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927a8eb20 con 0x55a927b94b10 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f520da78740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927a8e7c0 con 0x55a927b8b590 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f520da78740 10 client.0 _ll_register_callbacks cb 0x55a927aa2520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f5203e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b91b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f5203e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b91b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41818/0 (socket says 172.21.15.49:41818) 2024-02-26T06:29:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f5203e00640 1 -- 172.21.15.49:0/4286939430 learned_addr learned my addr 172.21.15.49:0/4286939430 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f51fbe00640 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8c300 0x55a927b92080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f5203e00640 1 -- 172.21.15.49:0/4286939430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b94b10 msgr2=0x55a927b925c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f5203e00640 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b94b10 0x55a927b925c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f5203e00640 1 -- 172.21.15.49:0/4286939430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8c300 msgr2=0x55a927b92080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f5203e00640 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8c300 0x55a927b92080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.164+0000 7f5203e00640 1 -- 172.21.15.49:0/4286939430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f400e040 con 0x55a927b8b590 2024-02-26T06:29:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.165+0000 7f5203e00640 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b91b40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f51f4014b00 tx=0x7f51f4007fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1aa9c7b0967e62b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.165+0000 7f51f1600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.165+0000 7f51f1600640 1 -- 172.21.15.49:0/4286939430 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4014920 con 0x55a927b8b590 2024-02-26T06:29:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.165+0000 7f51f1600640 1 -- 172.21.15.49:0/4286939430 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f51f400db60 con 0x55a927b8b590 2024-02-26T06:29:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.165+0000 7f520da78740 1 -- 172.21.15.49:0/4286939430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a927abbca0 con 0x55a927b8b590 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f51f1600640 1 -- 172.21.15.49:0/4286939430 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f400be70 con 0x55a927b8b590 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f520da78740 10 client.26866 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f520da78740 1 -- 172.21.15.49:0/4286939430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a927ae7c20 con 0x55a927b8b590 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f51f1600640 1 -- 172.21.15.49:0/4286939430 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a927abbca0 con 0x55a927b8b590 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f51f1600640 1 -- 172.21.15.49:0/4286939430 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51f401d440 con 0x55a927b8b590 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f520da78740 10 client.26866 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f520da78740 10 client.26866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f520da78740 20 client.26866 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f520da78740 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a927ba4a30 0x55a927ba6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f520da78740 4 client.26866 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.166+0000 7f520da78740 1 -- 172.21.15.49:0/4286939430 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a927adfcb0 con 0x55a927ba4a30 2024-02-26T06:29:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.167+0000 7f51fbe00640 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a927ba4a30 0x55a927ba6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.168+0000 7f51fbe00640 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a927ba4a30 0x55a927ba6e50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c31a3b20d1ecc9e server_cookie=6f13a7b82a835d5e in_seq=0 out_seq=0 2024-02-26T06:29:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.168+0000 7f51f1600640 10 client.26866 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.169+0000 7f51f1600640 1 -- 172.21.15.49:0/4286939430 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a927adfcb0 con 0x55a927ba4a30 2024-02-26T06:29:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.169+0000 7f51f1600640 10 client.26866 handle_command_reply: tid=0 2024-02-26T06:29:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.169+0000 7f520da78740 1 client.26866 shutdown 2024-02-26T06:29:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.171+0000 7f520da78740 1 -- 172.21.15.49:0/4286939430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 msgr2=0x55a927b91b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.171+0000 7f520da78740 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b91b40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f51f4014b00 tx=0x7f51f4007fd0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.171+0000 7f520da78740 1 -- 172.21.15.49:0/4286939430 shutdown_connections 2024-02-26T06:29:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.171+0000 7f520da78740 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a927ba4a30 0x55a927ba6e50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.171+0000 7f520da78740 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b94b10 0x55a927b925c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.171+0000 7f51f1600640 0 client.26866 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.171+0000 7f520da78740 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8c300 0x55a927b92080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.171+0000 7f520da78740 1 --2- 172.21.15.49:0/4286939430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b91b40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.172+0000 7f520da78740 1 -- 172.21.15.49:0/4286939430 >> 172.21.15.49:0/4286939430 conn(0x55a927b4a480 msgr2=0x55a927b87740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.172+0000 7f520da78740 1 -- 172.21.15.49:0/4286939430 shutdown_connections 2024-02-26T06:29:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.172+0000 7f520da78740 1 -- 172.21.15.49:0/4286939430 wait complete. 2024-02-26T06:29:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.173+0000 7f520da78740 20 client.26866 trim_cache size 0 max 16384 2024-02-26T06:29:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.204+0000 7f520da78740 1 Processor -- start 2024-02-26T06:29:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.204+0000 7f520da78740 1 -- start start 2024-02-26T06:29:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f520da78740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b94270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f520da78740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b9b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f520da78740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b94b10 0x55a927b9be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f520da78740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927831e60 con 0x55a927b94b10 2024-02-26T06:29:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f520da78740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927b6f670 con 0x55a927b8c300 2024-02-26T06:29:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f520da78740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927b79340 con 0x55a927b8b590 2024-02-26T06:29:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b9b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b9b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54864/0 (socket says 172.21.15.49:54864) 2024-02-26T06:29:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.205+0000 7f51fbe00640 1 -- 172.21.15.49:0/2557437242 learned_addr learned my addr 172.21.15.49:0/2557437242 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.206+0000 7f51fbe00640 1 -- 172.21.15.49:0/2557437242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 msgr2=0x55a927b94270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.206+0000 7f51fbe00640 1 --2- 172.21.15.49:0/2557437242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b94270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.206+0000 7f51fbe00640 1 -- 172.21.15.49:0/2557437242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b94b10 msgr2=0x55a927b9be80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.206+0000 7f51fbe00640 1 --2- 172.21.15.49:0/2557437242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b94b10 0x55a927b9be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.206+0000 7f51fbe00640 1 -- 172.21.15.49:0/2557437242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a927a84500 con 0x55a927b8c300 2024-02-26T06:29:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.207+0000 7f51fbe00640 1 --2- 172.21.15.49:0/2557437242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b9b940 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55a927b8c030 tx=0x7f51ec00ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a696f85268b8b76b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.207+0000 7f51f3e00640 1 -- 172.21.15.49:0/2557437242 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec004450 con 0x55a927b8c300 2024-02-26T06:29:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.208+0000 7f51f3e00640 1 -- 172.21.15.49:0/2557437242 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51ec03a070 con 0x55a927b8c300 2024-02-26T06:29:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.208+0000 7f51f3e00640 1 -- 172.21.15.49:0/2557437242 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0532d0 con 0x55a927b8c300 2024-02-26T06:29:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 -- 172.21.15.49:0/2557437242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 msgr2=0x55a927b9b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 --2- 172.21.15.49:0/2557437242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b9b940 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55a927b8c030 tx=0x7f51ec00ac00 comp rx=0 tx=0).stop 2024-02-26T06:29:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 -- 172.21.15.49:0/2557437242 shutdown_connections 2024-02-26T06:29:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 --2- 172.21.15.49:0/2557437242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b9b940 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 --2- 172.21.15.49:0/2557437242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b94b10 0x55a927b9be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 --2- 172.21.15.49:0/2557437242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b8b590 0x55a927b94270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 -- 172.21.15.49:0/2557437242 >> 172.21.15.49:0/2557437242 conn(0x55a927b4a480 msgr2=0x55a927b87440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 -- 172.21.15.49:0/2557437242 shutdown_connections 2024-02-26T06:29:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 -- 172.21.15.49:0/2557437242 wait complete. 2024-02-26T06:29:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.209+0000 7f520da78740 1 -- start start 2024-02-26T06:29:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.210+0000 7f520da78740 1 Processor -- start 2024-02-26T06:29:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.211+0000 7f520da78740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b9bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f520da78740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b99f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f520da78740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b9a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f520da78740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927b9aac0 con 0x55a927b8b590 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f520da78740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927b9ac30 con 0x55a927b8c300 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f520da78740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a927b9adf0 con 0x55a927b94b10 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f520da78740 10 client.0 _ll_register_callbacks cb 0x7f51ec0075f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b99f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f5208c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b9a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b99f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54874/0 (socket says 172.21.15.49:54874) 2024-02-26T06:29:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.212+0000 7f51fbe00640 1 -- 172.21.15.49:0/3605483189 learned_addr learned my addr 172.21.15.49:0/3605483189 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.213+0000 7f5203e00640 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b9bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.213+0000 7f5208c00640 1 -- 172.21.15.49:0/3605483189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 msgr2=0x55a927b99f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.213+0000 7f5208c00640 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b99f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.213+0000 7f5208c00640 1 -- 172.21.15.49:0/3605483189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 msgr2=0x55a927b9bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.213+0000 7f5208c00640 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b9bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.213+0000 7f5208c00640 1 -- 172.21.15.49:0/3605483189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec03f040 con 0x55a927b94b10 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.213+0000 7f51fbe00640 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b99f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.213+0000 7f5203e00640 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b9bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.214+0000 7f5208c00640 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b9a4b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0cb5e0 tx=0x7f51fc09b0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f4749fea57952dc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.214+0000 7f51f1600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.214+0000 7f51f1600640 1 -- 172.21.15.49:0/3605483189 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0cb860 con 0x55a927b94b10 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.214+0000 7f51f1600640 1 -- 172.21.15.49:0/3605483189 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f51fc0cb9c0 con 0x55a927b94b10 2024-02-26T06:29:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.214+0000 7f520da78740 1 -- 172.21.15.49:0/3605483189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a927b0ed00 con 0x55a927b94b10 2024-02-26T06:29:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.214+0000 7f51f1600640 1 -- 172.21.15.49:0/3605483189 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51fc0a2b90 con 0x55a927b94b10 2024-02-26T06:29:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.214+0000 7f520da78740 10 client.26869 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.214+0000 7f520da78740 1 -- 172.21.15.49:0/3605483189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a927aaa290 con 0x55a927b94b10 2024-02-26T06:29:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f51f1600640 1 -- 172.21.15.49:0/3605483189 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a927b0ed00 con 0x55a927b94b10 2024-02-26T06:29:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f51f1600640 1 -- 172.21.15.49:0/3605483189 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51fc09bbb0 con 0x55a927b94b10 2024-02-26T06:29:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f520da78740 10 client.26869 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f520da78740 10 client.26869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f520da78740 20 client.26869 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f520da78740 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a927ba4a30 0x55a927bfe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f520da78740 4 client.26869 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f520da78740 1 -- 172.21.15.49:0/3605483189 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9277e9600 con 0x55a927ba4a30 2024-02-26T06:29:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.215+0000 7f5203e00640 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a927ba4a30 0x55a927bfe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.216+0000 7f51f1600640 10 client.26869 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.217+0000 7f5203e00640 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a927ba4a30 0x55a927bfe680 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652446f2cccc4996 server_cookie=c69a1cf545611a06 in_seq=0 out_seq=0 2024-02-26T06:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.232+0000 7f51f1600640 1 -- 172.21.15.49:0/3605483189 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9277e9600 con 0x55a927ba4a30 2024-02-26T06:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.232+0000 7f51f1600640 10 client.26869 handle_command_reply: tid=0 2024-02-26T06:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.232+0000 7f520da78740 1 client.26869 shutdown 2024-02-26T06:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.232+0000 7f520da78740 1 -- 172.21.15.49:0/3605483189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 msgr2=0x55a927b9a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.232+0000 7f520da78740 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b9a4b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0cb5e0 tx=0x7f51fc09b0c0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f520da78740 1 -- 172.21.15.49:0/3605483189 shutdown_connections 2024-02-26T06:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f520da78740 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a927ba4a30 0x55a927bfe680 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f51f1600640 0 client.26869 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f520da78740 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a927b8c300 0x55a927b99f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f520da78740 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a927b94b10 0x55a927b9a4b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f520da78740 1 --2- 172.21.15.49:0/3605483189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a927b8b590 0x55a927b9bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f520da78740 1 -- 172.21.15.49:0/3605483189 >> 172.21.15.49:0/3605483189 conn(0x55a927b4a480 msgr2=0x55a927b85db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f520da78740 1 -- 172.21.15.49:0/3605483189 shutdown_connections 2024-02-26T06:29:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.233+0000 7f520da78740 1 -- 172.21.15.49:0/3605483189 wait complete. 2024-02-26T06:29:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.234+0000 7f520da78740 20 client.26869 trim_cache size 0 max 16384 2024-02-26T06:29:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:29:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:29:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:29:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 -- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51e80a23d0 msgr2=0x7f51e80a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51e80a23d0 0x7f51e80a4890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55a927b87410 tx=0x7f51ec000bc0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 -- 172.21.15.49:0/3934446849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f520410cb60 msgr2=0x7f52041a2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f520410cb60 0x7f52041a2700 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00db20 tx=0x7f51fc00b260 comp rx=0 tx=0).stop 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 -- 172.21.15.49:0/3934446849 shutdown_connections 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f51e80a23d0 0x7f51e80a4890 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f520410cb60 0x7f52041a2700 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5204108b20 0x7f520419e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 --2- 172.21.15.49:0/3934446849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5204108180 0x7f520419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.236+0000 7f520ac00640 1 -- 172.21.15.49:0/3934446849 >> 172.21.15.49:0/3934446849 conn(0x7f520406c220 msgr2=0x7f5204070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.237+0000 7f520ac00640 1 -- 172.21.15.49:0/3934446849 shutdown_connections 2024-02-26T06:29:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:12.237+0000 7f520ac00640 1 -- 172.21.15.49:0/3934446849 wait complete. 2024-02-26T06:29:13.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:12 smithi049 ceph-mon[31696]: pgmap v177: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-02-26T06:29:13.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:12 smithi080 ceph-mon[36206]: pgmap v177: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-02-26T06:29:13.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:12 smithi115 ceph-mon[36426]: pgmap v177: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-02-26T06:29:15.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:14 smithi049 ceph-mon[31696]: pgmap v178: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-02-26T06:29:15.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:14 smithi080 ceph-mon[36206]: pgmap v178: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-02-26T06:29:15.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:14 smithi115 ceph-mon[36426]: pgmap v178: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-02-26T06:29:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 325.0 s, 2347.3 tps, lat 42.889 ms stddev 51.146 2024-02-26T06:29:17.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:16 smithi049 ceph-mon[31696]: pgmap v179: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-02-26T06:29:17.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:16 smithi080 ceph-mon[36206]: pgmap v179: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-02-26T06:29:17.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:16 smithi115 ceph-mon[36426]: pgmap v179: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-02-26T06:29:19.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:18 smithi049 ceph-mon[31696]: pgmap v180: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 567 op/s 2024-02-26T06:29:19.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:18 smithi080 ceph-mon[36206]: pgmap v180: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 567 op/s 2024-02-26T06:29:19.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:18 smithi115 ceph-mon[36426]: pgmap v180: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 567 op/s 2024-02-26T06:29:21.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:20 smithi049 ceph-mon[31696]: pgmap v181: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.4 MiB/s wr, 715 op/s 2024-02-26T06:29:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 330.0 s, 2153.8 tps, lat 46.366 ms stddev 52.642 2024-02-26T06:29:21.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:20 smithi080 ceph-mon[36206]: pgmap v181: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.4 MiB/s wr, 715 op/s 2024-02-26T06:29:21.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:20 smithi115 ceph-mon[36426]: pgmap v181: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.4 MiB/s wr, 715 op/s 2024-02-26T06:29:23.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:22 smithi049 ceph-mon[31696]: pgmap v182: 97 pgs: 97 active+clean; 3.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.4 MiB/s wr, 652 op/s 2024-02-26T06:29:23.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:22 smithi080 ceph-mon[36206]: pgmap v182: 97 pgs: 97 active+clean; 3.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.4 MiB/s wr, 652 op/s 2024-02-26T06:29:23.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:22 smithi115 ceph-mon[36426]: pgmap v182: 97 pgs: 97 active+clean; 3.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.4 MiB/s wr, 652 op/s 2024-02-26T06:29:24.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:29:24.508 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 -- 172.21.15.49:0/538232979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead0073520 msgr2=0x7fead0073900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 --2- 172.21.15.49:0/538232979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead0073520 0x7fead0073900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feac4009900 tx=0x7feac402bcc0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 -- 172.21.15.49:0/538232979 shutdown_connections 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 --2- 172.21.15.49:0/538232979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fead0073e40 0x7fead0111bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 --2- 172.21.15.49:0/538232979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fead0112100 0x7fead01144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 --2- 172.21.15.49:0/538232979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead0073520 0x7fead0073900 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 -- 172.21.15.49:0/538232979 >> 172.21.15.49:0/538232979 conn(0x7fead00fcbe0 msgr2=0x7fead00ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 -- 172.21.15.49:0/538232979 shutdown_connections 2024-02-26T06:29:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.816+0000 7fead7800640 1 -- 172.21.15.49:0/538232979 wait complete. 2024-02-26T06:29:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.817+0000 7fead7800640 1 Processor -- start 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.817+0000 7fead7800640 1 -- start start 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.817+0000 7fead7800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fead0073520 0x7fead019de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead7800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fead0073e40 0x7fead019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead7800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead0112100 0x7fead01a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead7800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead01a2ce0 con 0x7fead0073e40 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead7800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead01a2e50 con 0x7fead0073520 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead7800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead01a3010 con 0x7fead0112100 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead0112100 0x7fead01a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7feacfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fead0073e40 0x7fead019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead4e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fead0073520 0x7fead019de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead4e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fead0073520 0x7fead019de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51224/0 (socket says 172.21.15.49:51224) 2024-02-26T06:29:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.818+0000 7fead4e00640 1 -- 172.21.15.49:0/2055645079 learned_addr learned my addr 172.21.15.49:0/2055645079 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:24.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.819+0000 7fead4e00640 1 -- 172.21.15.49:0/2055645079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead0112100 msgr2=0x7fead01a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.819+0000 7fead4e00640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead0112100 0x7fead01a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.819+0000 7fead4e00640 1 -- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fead0073e40 msgr2=0x7fead019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.819+0000 7fead4e00640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fead0073e40 0x7fead019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.819+0000 7fead4e00640 1 -- 172.21.15.49:0/2055645079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4009560 con 0x7fead0073520 2024-02-26T06:29:24.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.819+0000 7fead4e00640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fead0073520 0x7fead019de70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feac4000c00 tx=0x7feac402d880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:24.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.820+0000 7feacd600640 1 -- 172.21.15.49:0/2055645079 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac402dd40 con 0x7fead0073520 2024-02-26T06:29:24.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.820+0000 7feacd600640 1 -- 172.21.15.49:0/2055645079 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feac40052d0 con 0x7fead0073520 2024-02-26T06:29:24.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.820+0000 7feacd600640 1 -- 172.21.15.49:0/2055645079 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4003c90 con 0x7fead0073520 2024-02-26T06:29:24.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.820+0000 7fead7800640 1 -- 172.21.15.49:0/2055645079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead01a3290 con 0x7fead0073520 2024-02-26T06:29:24.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.821+0000 7fead7800640 1 -- 172.21.15.49:0/2055645079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead01aab70 con 0x7fead0073520 2024-02-26T06:29:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.824+0000 7feacd600640 1 -- 172.21.15.49:0/2055645079 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feac403e450 con 0x7fead0073520 2024-02-26T06:29:24.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.827+0000 7feacd600640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feab80a2230 0x7feab80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.827+0000 7feada7dc740 1 Processor -- start 2024-02-26T06:29:24.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.828+0000 7feacfe00640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feab80a2230 0x7feab80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.828+0000 7feacd600640 1 -- 172.21.15.49:0/2055645079 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7feac40f07e0 con 0x7fead0073520 2024-02-26T06:29:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.828+0000 7feada7dc740 1 -- start start 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feada7dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c24cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feada7dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c25710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feada7dc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feada7dc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034b94120 con 0x563034c2e600 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feada7dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034b91650 con 0x563034c25290 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feada7dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034b8ce30 con 0x563034c25f70 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feacfe00640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feab80a2230 0x7feab80a46f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fd0 tx=0x7feabc005850 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feacfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c25710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feacfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c25710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51226/0 (socket says 172.21.15.49:51226) 2024-02-26T06:29:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.829+0000 7feacfe00640 1 -- 172.21.15.49:0/1155281161 learned_addr learned my addr 172.21.15.49:0/1155281161 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.830+0000 7feacfe00640 1 -- 172.21.15.49:0/1155281161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 msgr2=0x563034c24cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.830+0000 7fead4e00640 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c24cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.830+0000 7feacfe00640 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c24cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.830+0000 7feacfe00640 1 -- 172.21.15.49:0/1155281161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 msgr2=0x563034c309f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.830+0000 7fead5800640 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.830+0000 7feacfe00640 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c309f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.830+0000 7feacfe00640 1 -- 172.21.15.49:0/1155281161 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563034b46030 con 0x563034c25290 2024-02-26T06:29:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.830+0000 7fead4e00640 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c24cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.831+0000 7fead5800640 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.831+0000 7feacfe00640 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c25710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feabc00fec0 tx=0x7feabc012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95801905c3d7b2a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.831+0000 7feac2000640 1 -- 172.21.15.49:0/1155281161 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc01e650 con 0x563034c25290 2024-02-26T06:29:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.831+0000 7feac2000640 1 -- 172.21.15.49:0/1155281161 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feabc011e50 con 0x563034c25290 2024-02-26T06:29:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.831+0000 7feac2000640 1 -- 172.21.15.49:0/1155281161 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc01dba0 con 0x563034c25290 2024-02-26T06:29:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.833+0000 7feada7dc740 1 -- 172.21.15.49:0/1155281161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 msgr2=0x563034c25710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.833+0000 7feada7dc740 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c25710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feabc00fec0 tx=0x7feabc012000 comp rx=0 tx=0).stop 2024-02-26T06:29:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.833+0000 7feada7dc740 1 -- 172.21.15.49:0/1155281161 shutdown_connections 2024-02-26T06:29:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.833+0000 7feada7dc740 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c25710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.833+0000 7feada7dc740 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.833+0000 7feada7dc740 1 --2- 172.21.15.49:0/1155281161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c24cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.833+0000 7feada7dc740 1 -- 172.21.15.49:0/1155281161 >> 172.21.15.49:0/1155281161 conn(0x563034bec570 msgr2=0x563034c1fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.833+0000 7feada7dc740 1 -- 172.21.15.49:0/1155281161 shutdown_connections 2024-02-26T06:29:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.834+0000 7feada7dc740 1 -- 172.21.15.49:0/1155281161 wait complete. 2024-02-26T06:29:24.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.834+0000 7feada7dc740 1 -- start start 2024-02-26T06:29:24.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.835+0000 7feada7dc740 1 Processor -- start 2024-02-26T06:29:24.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.835+0000 7feada7dc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25290 0x563034c2b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.835+0000 7feada7dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25f70 0x563034c2ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.835+0000 7feada7dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.835+0000 7feada7dc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034b31e90 con 0x563034c25290 2024-02-26T06:29:24.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.835+0000 7feada7dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034b31b30 con 0x563034c25f70 2024-02-26T06:29:24.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.835+0000 7feada7dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630348fb1c0 con 0x563034c2e600 2024-02-26T06:29:24.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feada7dc740 10 client.0 _ll_register_callbacks cb 0x563034b45930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:24.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feacfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25f70 0x563034c2ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feacfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25f70 0x563034c2ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51232/0 (socket says 172.21.15.49:51232) 2024-02-26T06:29:24.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feacfe00640 1 -- 172.21.15.49:0/4115560731 learned_addr learned my addr 172.21.15.49:0/4115560731 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7fead4e00640 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25290 0x563034c2b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7fead5800640 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feacfe00640 1 -- 172.21.15.49:0/4115560731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 msgr2=0x563034c2bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feacfe00640 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feacfe00640 1 -- 172.21.15.49:0/4115560731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25290 msgr2=0x563034c2b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feacfe00640 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25290 0x563034c2b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.836+0000 7feacfe00640 1 -- 172.21.15.49:0/4115560731 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc00fb70 con 0x563034c25f70 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.837+0000 7fead5800640 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.837+0000 7feacfe00640 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25f70 0x563034c2ba80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7feabc00fe90 tx=0x7feabc01db50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a25aad6a54f6125d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:24.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.837+0000 7feab6a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:29:24.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.838+0000 7feab6a00640 1 -- 172.21.15.49:0/4115560731 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc02d070 con 0x563034c25f70 2024-02-26T06:29:24.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.839+0000 7feab6a00640 1 -- 172.21.15.49:0/4115560731 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7feabc032470 con 0x563034c25f70 2024-02-26T06:29:24.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.839+0000 7feada7dc740 1 -- 172.21.15.49:0/4115560731 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56303488c7e0 con 0x563034c25f70 2024-02-26T06:29:24.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.840+0000 7feab6a00640 1 -- 172.21.15.49:0/4115560731 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7feabc01df00 con 0x563034c25f70 2024-02-26T06:29:24.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.840+0000 7feada7dc740 10 client.26891 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:24.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.840+0000 7feada7dc740 1 -- 172.21.15.49:0/4115560731 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563034b8b4b0 con 0x563034c25f70 2024-02-26T06:29:24.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.841+0000 7feab6a00640 1 -- 172.21.15.49:0/4115560731 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56303488c7e0 con 0x563034c25f70 2024-02-26T06:29:24.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.841+0000 7feab6a00640 1 -- 172.21.15.49:0/4115560731 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feabc03a070 con 0x563034c25f70 2024-02-26T06:29:24.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.841+0000 7feada7dc740 10 client.26891 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:24.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.841+0000 7feada7dc740 10 client.26891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:24.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.841+0000 7feada7dc740 20 client.26891 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:24.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.841+0000 7feada7dc740 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563034c3e460 0x563034c408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.841+0000 7feada7dc740 4 client.26891 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:24.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.841+0000 7feada7dc740 1 -- 172.21.15.49:0/4115560731 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630348c9720 con 0x563034c3e460 2024-02-26T06:29:24.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.842+0000 7fead4e00640 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563034c3e460 0x563034c408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.842+0000 7fead4e00640 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563034c3e460 0x563034c408c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ae120a43900028 server_cookie=70e11a285176053d in_seq=0 out_seq=0 2024-02-26T06:29:24.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.842+0000 7feab6a00640 10 client.26891 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:24.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.843+0000 7feab6a00640 1 -- 172.21.15.49:0/4115560731 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5630348c9720 con 0x563034c3e460 2024-02-26T06:29:24.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.843+0000 7feab6a00640 10 client.26891 handle_command_reply: tid=0 2024-02-26T06:29:24.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.843+0000 7feada7dc740 1 client.26891 shutdown 2024-02-26T06:29:24.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 -- 172.21.15.49:0/4115560731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25f70 msgr2=0x563034c2ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25f70 0x563034c2ba80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7feabc00fe90 tx=0x7feabc01db50 comp rx=0 tx=0).stop 2024-02-26T06:29:24.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 -- 172.21.15.49:0/4115560731 shutdown_connections 2024-02-26T06:29:24.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563034c3e460 0x563034c408c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25f70 0x563034c2ba80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 --2- 172.21.15.49:0/4115560731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25290 0x563034c2b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feab6a00640 0 client.26891 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:24.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 -- 172.21.15.49:0/4115560731 >> 172.21.15.49:0/4115560731 conn(0x563034bec570 msgr2=0x563034c21a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:24.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 -- 172.21.15.49:0/4115560731 shutdown_connections 2024-02-26T06:29:24.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.844+0000 7feada7dc740 1 -- 172.21.15.49:0/4115560731 wait complete. 2024-02-26T06:29:24.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.845+0000 7feada7dc740 20 client.26891 trim_cache size 0 max 16384 2024-02-26T06:29:24.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.891+0000 7feada7dc740 1 Processor -- start 2024-02-26T06:29:24.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.891+0000 7feada7dc740 1 -- start start 2024-02-26T06:29:24.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.891+0000 7feada7dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c2dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.891+0000 7feada7dc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25f70 0x563034c45260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.891+0000 7feada7dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.891+0000 7feada7dc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034c3ab70 con 0x563034c25f70 2024-02-26T06:29:24.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.891+0000 7feada7dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034c1f580 con 0x563034c25290 2024-02-26T06:29:24.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.891+0000 7feada7dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034c457d0 con 0x563034c2e600 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.892+0000 7fead4e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c2dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.892+0000 7feacfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25f70 0x563034c45260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.892+0000 7fead5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.892+0000 7fead5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48876/0 (socket says 172.21.15.49:48876) 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.892+0000 7feacfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25f70 0x563034c45260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53062/0 (socket says 172.21.15.49:53062) 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.892+0000 7fead5800640 1 -- 172.21.15.49:0/4018466361 learned_addr learned my addr 172.21.15.49:0/4018466361 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.893+0000 7feacfe00640 1 -- 172.21.15.49:0/4018466361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 msgr2=0x563034c2bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.893+0000 7feacfe00640 1 --2- 172.21.15.49:0/4018466361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.893+0000 7feacfe00640 1 -- 172.21.15.49:0/4018466361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 msgr2=0x563034c2dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:24.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.893+0000 7feacfe00640 1 --2- 172.21.15.49:0/4018466361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c2dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.893+0000 7feacfe00640 1 -- 172.21.15.49:0/4018466361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563034b27920 con 0x563034c25f70 2024-02-26T06:29:24.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.893+0000 7feacfe00640 1 --2- 172.21.15.49:0/4018466361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25f70 0x563034c45260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feabc00fec0 tx=0x7feabc011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7b19fe45e84d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:24.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.899+0000 7feac1600640 1 -- 172.21.15.49:0/4018466361 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc02d070 con 0x563034c25f70 2024-02-26T06:29:24.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.899+0000 7feac1600640 1 -- 172.21.15.49:0/4018466361 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feabc01df00 con 0x563034c25f70 2024-02-26T06:29:24.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.899+0000 7feac1600640 1 -- 172.21.15.49:0/4018466361 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc01e450 con 0x563034c25f70 2024-02-26T06:29:24.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 -- 172.21.15.49:0/4018466361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25f70 msgr2=0x563034c45260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 --2- 172.21.15.49:0/4018466361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25f70 0x563034c45260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feabc00fec0 tx=0x7feabc011ca0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 -- 172.21.15.49:0/4018466361 shutdown_connections 2024-02-26T06:29:24.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 --2- 172.21.15.49:0/4018466361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c2e600 0x563034c2bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 --2- 172.21.15.49:0/4018466361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c25f70 0x563034c45260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 --2- 172.21.15.49:0/4018466361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c2dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 -- 172.21.15.49:0/4018466361 >> 172.21.15.49:0/4018466361 conn(0x563034bec570 msgr2=0x563034c29e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:24.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 -- 172.21.15.49:0/4018466361 shutdown_connections 2024-02-26T06:29:24.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.901+0000 7feada7dc740 1 -- 172.21.15.49:0/4018466361 wait complete. 2024-02-26T06:29:24.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.902+0000 7feada7dc740 1 -- start start 2024-02-26T06:29:24.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.902+0000 7feada7dc740 1 Processor -- start 2024-02-26T06:29:24.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.903+0000 7feada7dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c35190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.903+0000 7feada7dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.904+0000 7feada7dc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.904+0000 7feada7dc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034c33ce0 con 0x563034c2e600 2024-02-26T06:29:24.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.904+0000 7feada7dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034c33e50 con 0x563034c25290 2024-02-26T06:29:24.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.904+0000 7feada7dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563034c34010 con 0x563034c25f70 2024-02-26T06:29:24.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.904+0000 7feada7dc740 10 client.0 _ll_register_callbacks cb 0x7feabc00fd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:24.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.904+0000 7feacfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.904+0000 7feacfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48890/0 (socket says 172.21.15.49:48890) 2024-02-26T06:29:24.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.904+0000 7feacfe00640 1 -- 172.21.15.49:0/882371869 learned_addr learned my addr 172.21.15.49:0/882371869 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:24.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.905+0000 7fead5800640 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.905+0000 7fead4e00640 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c35190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.905+0000 7feacfe00640 1 -- 172.21.15.49:0/882371869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 msgr2=0x563034c35190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:24.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.905+0000 7feacfe00640 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c35190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.905+0000 7feacfe00640 1 -- 172.21.15.49:0/882371869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 msgr2=0x563034c336d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:24.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.905+0000 7feacfe00640 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c336d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.906+0000 7feacfe00640 1 -- 172.21.15.49:0/882371869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563034b8b4b0 con 0x563034c25f70 2024-02-26T06:29:24.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.907+0000 7feacfe00640 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c356d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feabc012fd0 tx=0x7feabc01ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5db34b11d8a528 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:24.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.907+0000 7feab6a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:24.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.908+0000 7feab6a00640 1 -- 172.21.15.49:0/882371869 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc02d030 con 0x563034c25f70 2024-02-26T06:29:24.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.909+0000 7feab6a00640 1 -- 172.21.15.49:0/882371869 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7feabc02e070 con 0x563034c25f70 2024-02-26T06:29:24.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.909+0000 7feada7dc740 1 -- 172.21.15.49:0/882371869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feabc011e50 con 0x563034c25f70 2024-02-26T06:29:24.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.910+0000 7feab6a00640 1 -- 172.21.15.49:0/882371869 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7feabc01d790 con 0x563034c25f70 2024-02-26T06:29:24.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.910+0000 7feada7dc740 10 client.26881 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:24.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.910+0000 7feada7dc740 1 -- 172.21.15.49:0/882371869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563034b4d400 con 0x563034c25f70 2024-02-26T06:29:24.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.913+0000 7feab6a00640 1 -- 172.21.15.49:0/882371869 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feabc011e50 con 0x563034c25f70 2024-02-26T06:29:24.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.913+0000 7feab6a00640 1 -- 172.21.15.49:0/882371869 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feabc03a050 con 0x563034c25f70 2024-02-26T06:29:24.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.914+0000 7feada7dc740 10 client.26881 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:24.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.914+0000 7feada7dc740 10 client.26881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:24.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.914+0000 7feada7dc740 20 client.26881 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:24.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.914+0000 7feada7dc740 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563034c95650 0x563034c97a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:24.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.914+0000 7feada7dc740 4 client.26881 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:29:24.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.914+0000 7feada7dc740 1 -- 172.21.15.49:0/882371869 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563034918890 con 0x563034c95650 2024-02-26T06:29:24.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.915+0000 7fead4e00640 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563034c95650 0x563034c97a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:24.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.917+0000 7fead4e00640 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563034c95650 0x563034c97a70 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb687612a0cd46c server_cookie=966be0502efa6e9c in_seq=0 out_seq=0 2024-02-26T06:29:24.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.917+0000 7feab6a00640 10 client.26881 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:24.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.917+0000 7feab6a00640 1 -- 172.21.15.49:0/882371869 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563034918890 con 0x563034c95650 2024-02-26T06:29:24.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.917+0000 7feab6a00640 10 client.26881 handle_command_reply: tid=0 2024-02-26T06:29:24.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.917+0000 7feada7dc740 1 client.26881 shutdown 2024-02-26T06:29:24.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feada7dc740 1 -- 172.21.15.49:0/882371869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 msgr2=0x563034c356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feada7dc740 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c356d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feabc012fd0 tx=0x7feabc01ef70 comp rx=0 tx=0).stop 2024-02-26T06:29:24.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feada7dc740 1 -- 172.21.15.49:0/882371869 shutdown_connections 2024-02-26T06:29:24.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feada7dc740 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563034c95650 0x563034c97a70 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feab6a00640 0 client.26881 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:24.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feada7dc740 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563034c2e600 0x563034c336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feada7dc740 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563034c25f70 0x563034c356d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feada7dc740 1 --2- 172.21.15.49:0/882371869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563034c25290 0x563034c35190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.918+0000 7feada7dc740 1 -- 172.21.15.49:0/882371869 >> 172.21.15.49:0/882371869 conn(0x563034bec570 msgr2=0x563034c1ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:24.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.919+0000 7feada7dc740 1 -- 172.21.15.49:0/882371869 shutdown_connections 2024-02-26T06:29:24.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.919+0000 7feada7dc740 1 -- 172.21.15.49:0/882371869 wait complete. 2024-02-26T06:29:24.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.919+0000 7feada7dc740 20 client.26881 trim_cache size 0 max 16384 2024-02-26T06:29:24.919 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:29:24.919 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:29:24.919 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:29:24.920 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:29:24.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 -- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feab80a2230 msgr2=0x7feab80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feab80a2230 0x7feab80a46f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fd0 tx=0x7feabc005850 comp rx=0 tx=0).stop 2024-02-26T06:29:24.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 -- 172.21.15.49:0/2055645079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fead0073520 msgr2=0x7fead019de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:24.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fead0073520 0x7fead019de70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feac4000c00 tx=0x7feac402d880 comp rx=0 tx=0).stop 2024-02-26T06:29:24.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 -- 172.21.15.49:0/2055645079 shutdown_connections 2024-02-26T06:29:24.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feab80a2230 0x7feab80a46f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead0112100 0x7fead01a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fead0073e40 0x7fead019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 --2- 172.21.15.49:0/2055645079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fead0073520 0x7fead019de70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:24.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 -- 172.21.15.49:0/2055645079 >> 172.21.15.49:0/2055645079 conn(0x7fead00fcbe0 msgr2=0x7fead00fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:24.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.921+0000 7fead7800640 1 -- 172.21.15.49:0/2055645079 shutdown_connections 2024-02-26T06:29:24.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:24.922+0000 7fead7800640 1 -- 172.21.15.49:0/2055645079 wait complete. 2024-02-26T06:29:24.959 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:29:24.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:29:25.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:24 smithi049 ceph-mon[31696]: pgmap v183: 97 pgs: 97 active+clean; 3.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-02-26T06:29:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.267+0000 7fe46cc00640 1 -- 172.21.15.49:0/1938508521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468075fd0 msgr2=0x7fe468111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.267+0000 7fe46cc00640 1 --2- 172.21.15.49:0/1938508521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468075fd0 0x7fe468111a70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe460009c10 tx=0x7fe46002bef0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.268+0000 7fe46cc00640 1 -- 172.21.15.49:0/1938508521 shutdown_connections 2024-02-26T06:29:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.268+0000 7fe46cc00640 1 --2- 172.21.15.49:0/1938508521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468075fd0 0x7fe468111a70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.268+0000 7fe46cc00640 1 --2- 172.21.15.49:0/1938508521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe468111fb0 0x7fe4681143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.268+0000 7fe46cc00640 1 --2- 172.21.15.49:0/1938508521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4680756b0 0x7fe468075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.268+0000 7fe46cc00640 1 -- 172.21.15.49:0/1938508521 >> 172.21.15.49:0/1938508521 conn(0x7fe4680fede0 msgr2=0x7fe468101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:25.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.268+0000 7fe46cc00640 1 -- 172.21.15.49:0/1938508521 shutdown_connections 2024-02-26T06:29:25.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.268+0000 7fe46cc00640 1 -- 172.21.15.49:0/1938508521 wait complete. 2024-02-26T06:29:25.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.270+0000 7fe46cc00640 1 Processor -- start 2024-02-26T06:29:25.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.271+0000 7fe46cc00640 1 -- start start 2024-02-26T06:29:25.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.271+0000 7fe46cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4680756b0 0x7fe46819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.271+0000 7fe46cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468075fd0 0x7fe46819e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.272+0000 7fe46cc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468111fb0 0x7fe4681a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.272+0000 7fe46cc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4681a2ce0 con 0x7fe468075fd0 2024-02-26T06:29:25.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.272+0000 7fe46cc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4681a2e50 con 0x7fe468111fb0 2024-02-26T06:29:25.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.272+0000 7fe46cc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4681a3010 con 0x7fe4680756b0 2024-02-26T06:29:25.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.272+0000 7fe466800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468111fb0 0x7fe4681a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.273+0000 7fe465e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4680756b0 0x7fe46819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.273+0000 7fe465400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468075fd0 0x7fe46819e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.273+0000 7fe466800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468111fb0 0x7fe4681a26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51266/0 (socket says 172.21.15.49:51266) 2024-02-26T06:29:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.273+0000 7fe466800640 1 -- 172.21.15.49:0/641636078 learned_addr learned my addr 172.21.15.49:0/641636078 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.273+0000 7fe465e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4680756b0 0x7fe46819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48904/0 (socket says 172.21.15.49:48904) 2024-02-26T06:29:25.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.273+0000 7fe465e00640 1 -- 172.21.15.49:0/641636078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468111fb0 msgr2=0x7fe4681a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.273+0000 7fe465e00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468111fb0 0x7fe4681a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.273+0000 7fe465e00640 1 -- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468075fd0 msgr2=0x7fe46819e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.274+0000 7fe465e00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468075fd0 0x7fe46819e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.274+0000 7fe465e00640 1 -- 172.21.15.49:0/641636078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe454009560 con 0x7fe4680756b0 2024-02-26T06:29:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.274+0000 7fe465400640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468075fd0 0x7fe46819e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.276+0000 7fe465e00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4680756b0 0x7fe46819de20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe458005fd0 tx=0x7fe45800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.277+0000 7fe45ea00640 1 -- 172.21.15.49:0/641636078 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458009c30 con 0x7fe4680756b0 2024-02-26T06:29:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.277+0000 7fe45ea00640 1 -- 172.21.15.49:0/641636078 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe458004510 con 0x7fe4680756b0 2024-02-26T06:29:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.277+0000 7fe45ea00640 1 -- 172.21.15.49:0/641636078 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458003e70 con 0x7fe4680756b0 2024-02-26T06:29:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.278+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe460009840 con 0x7fe4680756b0 2024-02-26T06:29:25.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.278+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4681aa990 con 0x7fe4680756b0 2024-02-26T06:29:25.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.281+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4680683c0 con 0x7fe4680756b0 2024-02-26T06:29:25.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.282+0000 7fe45ea00640 1 -- 172.21.15.49:0/641636078 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe458004000 con 0x7fe4680756b0 2024-02-26T06:29:25.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.285+0000 7fe45ea00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe44c0a23c0 0x7fe44c0a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.285+0000 7fe45ea00640 1 -- 172.21.15.49:0/641636078 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe4580cddb0 con 0x7fe4680756b0 2024-02-26T06:29:25.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.286+0000 7fe465400640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe44c0a23c0 0x7fe44c0a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.288+0000 7fe465400640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe44c0a23c0 0x7fe44c0a4880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe460000f80 tx=0x7fe460005fd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:25.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.289+0000 7fe45ea00640 1 -- 172.21.15.49:0/641636078 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4580d2070 con 0x7fe4680756b0 2024-02-26T06:29:25.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:24 smithi080 ceph-mon[36206]: pgmap v183: 97 pgs: 97 active+clean; 3.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-02-26T06:29:25.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:24 smithi115 ceph-mon[36426]: pgmap v183: 97 pgs: 97 active+clean; 3.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-02-26T06:29:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.474+0000 7efeaac00640 1 -- 172.21.15.49:0/3832264697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efea4100b00 msgr2=0x7efea4104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.474+0000 7efeaac00640 1 --2- 172.21.15.49:0/3832264697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efea4100b00 0x7efea4104fc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009900 tx=0x7efe8c02bd70 comp rx=0 tx=0).stop 2024-02-26T06:29:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.475+0000 7efeaac00640 1 -- 172.21.15.49:0/3832264697 shutdown_connections 2024-02-26T06:29:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.475+0000 7efeaac00640 1 --2- 172.21.15.49:0/3832264697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efea4100b00 0x7efea4104fc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.475+0000 7efeaac00640 1 --2- 172.21.15.49:0/3832264697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efea4105500 0x7efea4105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.475+0000 7efeaac00640 1 --2- 172.21.15.49:0/3832264697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea41001e0 0x7efea41005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.475+0000 7efeaac00640 1 -- 172.21.15.49:0/3832264697 >> 172.21.15.49:0/3832264697 conn(0x7efea40771a0 msgr2=0x7efea40775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.476+0000 7efeaac00640 1 -- 172.21.15.49:0/3832264697 shutdown_connections 2024-02-26T06:29:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.476+0000 7efeaac00640 1 -- 172.21.15.49:0/3832264697 wait complete. 2024-02-26T06:29:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.478+0000 7efeaac00640 1 Processor -- start 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.480+0000 7efeaac00640 1 -- start start 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.480+0000 7efeaac00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efea41001e0 0x7efea41a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.480+0000 7efeaac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efea4100b00 0x7efea41a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.480+0000 7efeaac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea4105500 0x7efea419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.480+0000 7efeaac00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea419e2f0 con 0x7efea41001e0 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.480+0000 7efeaac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea419e430 con 0x7efea4100b00 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.480+0000 7efeaac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea419e5f0 con 0x7efea4105500 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.481+0000 7efea8c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea4105500 0x7efea419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.481+0000 7efea8c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea4105500 0x7efea419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48924/0 (socket says 172.21.15.49:48924) 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.481+0000 7efea8c00640 1 -- 172.21.15.49:0/2342722332 learned_addr learned my addr 172.21.15.49:0/2342722332 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.481+0000 7efea3e00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efea41001e0 0x7efea41a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.481+0000 7efe9be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efea4100b00 0x7efea41a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.482+0000 7efea8c00640 1 -- 172.21.15.49:0/2342722332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efea4100b00 msgr2=0x7efea41a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.482+0000 7efea8c00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efea4100b00 0x7efea41a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.482+0000 7efea8c00640 1 -- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efea41001e0 msgr2=0x7efea41a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.482+0000 7efea8c00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efea41001e0 0x7efea41a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.482+0000 7efea8c00640 1 -- 172.21.15.49:0/2342722332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c009560 con 0x7efea4105500 2024-02-26T06:29:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.482+0000 7efea8c00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea4105500 0x7efea419db60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00db20 tx=0x7efe9c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.483+0000 7efea1600640 1 -- 172.21.15.49:0/2342722332 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00bea0 con 0x7efea4105500 2024-02-26T06:29:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.483+0000 7efea1600640 1 -- 172.21.15.49:0/2342722332 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe9c009940 con 0x7efea4105500 2024-02-26T06:29:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.483+0000 7efea1600640 1 -- 172.21.15.49:0/2342722332 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00bea0 con 0x7efea4105500 2024-02-26T06:29:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.483+0000 7efeaac00640 1 -- 172.21.15.49:0/2342722332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea419e8d0 con 0x7efea4105500 2024-02-26T06:29:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.483+0000 7efeaac00640 1 -- 172.21.15.49:0/2342722332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea406ad80 con 0x7efea4105500 2024-02-26T06:29:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.489+0000 7efea1600640 1 -- 172.21.15.49:0/2342722332 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7efe9c003970 con 0x7efea4105500 2024-02-26T06:29:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.492+0000 7efea1600640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7efe880a2430 0x7efe880a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.492+0000 7efea1600640 1 -- 172.21.15.49:0/2342722332 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7efe9c0cd8b0 con 0x7efea4105500 2024-02-26T06:29:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.494+0000 7efea3e00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7efe880a2430 0x7efe880a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.494+0000 7efeadb92740 1 Processor -- start 2024-02-26T06:29:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.495+0000 7efeadb92740 1 -- start start 2024-02-26T06:29:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.495+0000 7efeadb92740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb8af0 0x560275bb7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.495+0000 7efeadb92740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb7d80 0x560275bb8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.495+0000 7efeadb92740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275bc36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.496+0000 7efeadb92740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275b735f0 con 0x560275bb7d80 2024-02-26T06:29:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.496+0000 7efeadb92740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275b25890 con 0x560275bc1300 2024-02-26T06:29:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.496+0000 7efeadb92740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275b22dc0 con 0x560275bb8af0 2024-02-26T06:29:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.496+0000 7efea8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275bc36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.497+0000 7efea8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275bc36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51290/0 (socket says 172.21.15.49:51290) 2024-02-26T06:29:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.497+0000 7efea8c00640 1 -- 172.21.15.49:0/2983787838 learned_addr learned my addr 172.21.15.49:0/2983787838 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.497+0000 7efe9be00640 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb7d80 0x560275bb8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.497+0000 7efea3e00640 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb8af0 0x560275bb7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.498+0000 7efea3e00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7efe880a2430 0x7efe880a48f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efe9400bbe0 tx=0x7efe9400b390 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.498+0000 7efe9be00640 1 -- 172.21.15.49:0/2983787838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb8af0 msgr2=0x560275bb7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.498+0000 7efe9be00640 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb8af0 0x560275bb7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.498+0000 7efe9be00640 1 -- 172.21.15.49:0/2983787838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 msgr2=0x560275bc36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.498+0000 7efe9be00640 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275bc36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.498+0000 7efe9be00640 1 -- 172.21.15.49:0/2983787838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560275a9fad0 con 0x560275bb7d80 2024-02-26T06:29:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.499+0000 7efe9be00640 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb7d80 0x560275bb8200 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efe8c02c2b0 tx=0x7efe8c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fb99fa46866a88 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.500+0000 7efe99200640 1 -- 172.21.15.49:0/2983787838 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c004930 con 0x560275bb7d80 2024-02-26T06:29:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.500+0000 7efe99200640 1 -- 172.21.15.49:0/2983787838 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe8c004a90 con 0x560275bb7d80 2024-02-26T06:29:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.500+0000 7efe99200640 1 -- 172.21.15.49:0/2983787838 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c005030 con 0x560275bb7d80 2024-02-26T06:29:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.502+0000 7efeadb92740 1 -- 172.21.15.49:0/2983787838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb7d80 msgr2=0x560275bb8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.502+0000 7efeadb92740 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb7d80 0x560275bb8200 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efe8c02c2b0 tx=0x7efe8c004740 comp rx=0 tx=0).stop 2024-02-26T06:29:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.503+0000 7efeadb92740 1 -- 172.21.15.49:0/2983787838 shutdown_connections 2024-02-26T06:29:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.503+0000 7efeadb92740 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275bc36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.503+0000 7efeadb92740 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb7d80 0x560275bb8200 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.503+0000 7efeadb92740 1 --2- 172.21.15.49:0/2983787838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb8af0 0x560275bb7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.503+0000 7efeadb92740 1 -- 172.21.15.49:0/2983787838 >> 172.21.15.49:0/2983787838 conn(0x560275b7f480 msgr2=0x560275bb2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.503+0000 7efeadb92740 1 -- 172.21.15.49:0/2983787838 shutdown_connections 2024-02-26T06:29:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.504+0000 7efeadb92740 1 -- 172.21.15.49:0/2983787838 wait complete. 2024-02-26T06:29:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.504+0000 7efeadb92740 1 -- start start 2024-02-26T06:29:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.505+0000 7efeadb92740 1 Processor -- start 2024-02-26T06:29:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.505+0000 7efeadb92740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb7d80 0x560275bbe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.505+0000 7efeadb92740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.505+0000 7efeadb92740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bc1300 0x560275bbee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.506+0000 7efea3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb7d80 0x560275bbe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.506+0000 7efea3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb7d80 0x560275bbe3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51292/0 (socket says 172.21.15.49:51292) 2024-02-26T06:29:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.507+0000 7efe9be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.507+0000 7efe9be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbe930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53138/0 (socket says 172.21.15.49:53138) 2024-02-26T06:29:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.508+0000 7efea3e00640 1 -- 172.21.15.49:0/2886690823 learned_addr learned my addr 172.21.15.49:0/2886690823 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.508+0000 7efea8c00640 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bc1300 0x560275bbee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.508+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275b1e5a0 con 0x560275bb8af0 2024-02-26T06:29:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.509+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275ac3b20 con 0x560275bb7d80 2024-02-26T06:29:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.509+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275ac37c0 con 0x560275bc1300 2024-02-26T06:29:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.509+0000 7efeadb92740 10 client.0 _ll_register_callbacks cb 0x560275ad7520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.509+0000 7efe9be00640 1 -- 172.21.15.49:0/2886690823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bc1300 msgr2=0x560275bbee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.509+0000 7efe9be00640 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bc1300 0x560275bbee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.509+0000 7efe9be00640 1 -- 172.21.15.49:0/2886690823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb7d80 msgr2=0x560275bbe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.509+0000 7efe9be00640 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb7d80 0x560275bbe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.509+0000 7efe9be00640 1 -- 172.21.15.49:0/2886690823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea4114e50 con 0x560275bb8af0 2024-02-26T06:29:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.510+0000 7efe9be00640 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbe930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efe8c02c2b0 tx=0x7efe8c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f749cb8e40cd4f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.511+0000 7efe91600640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:29:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.516+0000 7efe91600640 1 -- 172.21.15.49:0/2886690823 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0052c0 con 0x560275bb8af0 2024-02-26T06:29:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.517+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560275af0ca0 con 0x560275bb8af0 2024-02-26T06:29:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.519+0000 7efe91600640 1 -- 172.21.15.49:0/2886690823 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7efe8c005420 con 0x560275bb8af0 2024-02-26T06:29:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.522+0000 7efe91600640 1 -- 172.21.15.49:0/2886690823 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe8c0055e0 con 0x560275bb8af0 2024-02-26T06:29:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.523+0000 7efeadb92740 10 client.16404 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.523+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560275b1cc20 con 0x560275bb8af0 2024-02-26T06:29:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.524+0000 7efe91600640 1 -- 172.21.15.49:0/2886690823 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560275af0ca0 con 0x560275bb8af0 2024-02-26T06:29:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.524+0000 7efe91600640 1 -- 172.21.15.49:0/2886690823 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efe8c03c050 con 0x560275bb8af0 2024-02-26T06:29:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.527+0000 7efeadb92740 10 client.16404 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.527+0000 7efeadb92740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.527+0000 7efeadb92740 20 client.16404 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.527+0000 7efeadb92740 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560275bd0fd0 0x560275bd3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.527+0000 7efeadb92740 4 client.16404 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.527+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560275b14cb0 con 0x560275bd0fd0 2024-02-26T06:29:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.528+0000 7efea3e00640 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560275bd0fd0 0x560275bd3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.529+0000 7efea3e00640 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560275bd0fd0 0x560275bd3400 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24bc0a68d7af24af server_cookie=89de58ca21b7615a in_seq=0 out_seq=0 2024-02-26T06:29:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.529+0000 7efe91600640 10 client.16404 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.530+0000 7efe91600640 1 -- 172.21.15.49:0/2886690823 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560275b14cb0 con 0x560275bd0fd0 2024-02-26T06:29:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.530+0000 7efe91600640 10 client.16404 handle_command_reply: tid=0 2024-02-26T06:29:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.530+0000 7efeadb92740 1 client.16404 shutdown 2024-02-26T06:29:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.530+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 msgr2=0x560275bbe930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.530+0000 7efeadb92740 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbe930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efe8c02c2b0 tx=0x7efe8c005fd0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.531+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 shutdown_connections 2024-02-26T06:29:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.531+0000 7efeadb92740 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560275bd0fd0 0x560275bd3400 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.531+0000 7efeadb92740 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bc1300 0x560275bbee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.531+0000 7efeadb92740 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbe930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.531+0000 7efeadb92740 1 --2- 172.21.15.49:0/2886690823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb7d80 0x560275bbe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.531+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 >> 172.21.15.49:0/2886690823 conn(0x560275b7f480 msgr2=0x560275bb47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.531+0000 7efe91600640 0 client.16404 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.535+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 shutdown_connections 2024-02-26T06:29:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.535+0000 7efeadb92740 1 -- 172.21.15.49:0/2886690823 wait complete. 2024-02-26T06:29:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.535+0000 7efeadb92740 20 client.16404 trim_cache size 0 max 16384 2024-02-26T06:29:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.577+0000 7efeadb92740 1 Processor -- start 2024-02-26T06:29:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.577+0000 7efeadb92740 1 -- start start 2024-02-26T06:29:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.577+0000 7efeadb92740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bc0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.577+0000 7efeadb92740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb8af0 0x560275bbec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.577+0000 7efeadb92740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bc1300 0x560275bc80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.578+0000 7efea3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bc0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.578+0000 7efea3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bc0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48952/0 (socket says 172.21.15.49:48952) 2024-02-26T06:29:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.578+0000 7efe9be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb8af0 0x560275bbec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.578+0000 7efe9be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb8af0 0x560275bbec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51300/0 (socket says 172.21.15.49:51300) 2024-02-26T06:29:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.579+0000 7efeadb92740 1 -- 172.21.15.49:0/933016039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275866e60 con 0x560275bc1300 2024-02-26T06:29:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.579+0000 7efeadb92740 1 -- 172.21.15.49:0/933016039 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275af0ca0 con 0x560275bb8af0 2024-02-26T06:29:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.579+0000 7efeadb92740 1 -- 172.21.15.49:0/933016039 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275bb9230 con 0x560275bb7d80 2024-02-26T06:29:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.579+0000 7efea3e00640 1 -- 172.21.15.49:0/933016039 learned_addr learned my addr 172.21.15.49:0/933016039 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.579+0000 7efea3e00640 1 -- 172.21.15.49:0/933016039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb8af0 msgr2=0x560275bbec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.580+0000 7efea8c00640 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bc1300 0x560275bc80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.580+0000 7efea3e00640 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb8af0 0x560275bbec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.580+0000 7efea3e00640 1 -- 172.21.15.49:0/933016039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bc1300 msgr2=0x560275bc80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.580+0000 7efea3e00640 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bc1300 0x560275bc80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.580+0000 7efea3e00640 1 -- 172.21.15.49:0/933016039 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560275ab9500 con 0x560275bb7d80 2024-02-26T06:29:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.580+0000 7efea8c00640 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bc1300 0x560275bc80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.581+0000 7efea3e00640 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bc0b20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efe94010860 tx=0x7efe94013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e44a1346654a2089 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.582+0000 7efe93e00640 1 -- 172.21.15.49:0/933016039 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9402a070 con 0x560275bb7d80 2024-02-26T06:29:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.582+0000 7efe93e00640 1 -- 172.21.15.49:0/933016039 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe94015070 con 0x560275bb7d80 2024-02-26T06:29:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.583+0000 7efeadb92740 1 -- 172.21.15.49:0/933016039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 msgr2=0x560275bc0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.583+0000 7efe93e00640 1 -- 172.21.15.49:0/933016039 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9402a070 con 0x560275bb7d80 2024-02-26T06:29:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.583+0000 7efeadb92740 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bc0b20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efe94010860 tx=0x7efe94013000 comp rx=0 tx=0).stop 2024-02-26T06:29:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.584+0000 7efeadb92740 1 -- 172.21.15.49:0/933016039 shutdown_connections 2024-02-26T06:29:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.584+0000 7efeadb92740 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bb8af0 0x560275bbec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.584+0000 7efeadb92740 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bc1300 0x560275bc80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.584+0000 7efeadb92740 1 --2- 172.21.15.49:0/933016039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bc0b20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.584+0000 7efeadb92740 1 -- 172.21.15.49:0/933016039 >> 172.21.15.49:0/933016039 conn(0x560275b7f480 msgr2=0x560275bbccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.587+0000 7efeadb92740 1 -- 172.21.15.49:0/933016039 shutdown_connections 2024-02-26T06:29:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.587+0000 7efeadb92740 1 -- 172.21.15.49:0/933016039 wait complete. 2024-02-26T06:29:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.588+0000 7efeadb92740 1 -- start start 2024-02-26T06:29:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.588+0000 7efeadb92740 1 Processor -- start 2024-02-26T06:29:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.589+0000 7efeadb92740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.590+0000 7efea3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.590+0000 7efea3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bbb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48968/0 (socket says 172.21.15.49:48968) 2024-02-26T06:29:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.590+0000 7efea3e00640 1 -- 172.21.15.49:0/441684704 learned_addr learned my addr 172.21.15.49:0/441684704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.590+0000 7efeadb92740 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.590+0000 7efeadb92740 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275c274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.591+0000 7efea8c00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275c274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.591+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275bbc2b0 con 0x560275bb8af0 2024-02-26T06:29:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.591+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275c27a00 con 0x560275bc1300 2024-02-26T06:29:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.591+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560275c27b40 con 0x560275bb7d80 2024-02-26T06:29:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.591+0000 7efe9be00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.591+0000 7efea8c00640 1 -- 172.21.15.49:0/441684704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 msgr2=0x560275bbb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.591+0000 7efea8c00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bbb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.591+0000 7efea8c00640 1 -- 172.21.15.49:0/441684704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 msgr2=0x560275bbb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.592+0000 7efeadb92740 10 client.0 _ll_register_callbacks cb 0x7efe9c0965a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.595+0000 7efea8c00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.595+0000 7efea8c00640 1 -- 172.21.15.49:0/441684704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe94010d50 con 0x560275bc1300 2024-02-26T06:29:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.595+0000 7efea3e00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.597+0000 7efea8c00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275c274c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe9c096760 tx=0x7efe9c09e460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8f4476457278966 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.597+0000 7efe91600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:29:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.597+0000 7efe91600640 1 -- 172.21.15.49:0/441684704 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0cb230 con 0x560275bc1300 2024-02-26T06:29:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.598+0000 7efe91600640 1 -- 172.21.15.49:0/441684704 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7efe9c0cb390 con 0x560275bc1300 2024-02-26T06:29:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.598+0000 7efe9be00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.598+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560275b43d00 con 0x560275bc1300 2024-02-26T06:29:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.599+0000 7efe91600640 1 -- 172.21.15.49:0/441684704 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9c004b40 con 0x560275bc1300 2024-02-26T06:29:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.599+0000 7efeadb92740 10 client.26921 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.599+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x560275adf290 con 0x560275bc1300 2024-02-26T06:29:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.600+0000 7efe91600640 1 -- 172.21.15.49:0/441684704 <== mon.2 v2:172.21.15.80:3300/0 4 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efe9c004ca0 con 0x560275bc1300 2024-02-26T06:29:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.600+0000 7efeadb92740 10 client.26921 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.600+0000 7efeadb92740 10 client.26921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.600+0000 7efeadb92740 20 client.26921 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.600+0000 7efeadb92740 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560275c28fa0 0x560275c2b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.600+0000 7efeadb92740 4 client.26921 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.601+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560275b43d00 con 0x560275c28fa0 2024-02-26T06:29:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.601+0000 7efea3e00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560275c28fa0 0x560275c2b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.602+0000 7efe91600640 10 client.26921 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.602+0000 7efea3e00640 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560275c28fa0 0x560275c2b3c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960ecbdbf29ae70f server_cookie=743670d8737d2788 in_seq=0 out_seq=0 2024-02-26T06:29:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.674+0000 7efe91600640 1 -- 172.21.15.49:0/441684704 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560275b43d00 con 0x560275c28fa0 2024-02-26T06:29:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.674+0000 7efe91600640 10 client.26921 handle_command_reply: tid=0 2024-02-26T06:29:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.674+0000 7efeadb92740 1 client.26921 shutdown 2024-02-26T06:29:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 msgr2=0x560275c274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275c274c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe9c096760 tx=0x7efe9c09e460 comp rx=0 tx=0).stop 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 shutdown_connections 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560275c28fa0 0x560275c2b3c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560275bc1300 0x560275c274c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560275bb8af0 0x560275bbb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 --2- 172.21.15.49:0/441684704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560275bb7d80 0x560275bbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 >> 172.21.15.49:0/441684704 conn(0x560275b7f480 msgr2=0x560275bb1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efe91600640 0 client.26921 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 shutdown_connections 2024-02-26T06:29:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 1 -- 172.21.15.49:0/441684704 wait complete. 2024-02-26T06:29:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.675+0000 7efeadb92740 20 client.26921 trim_cache size 0 max 16384 2024-02-26T06:29:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:29:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:29:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:29:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:29:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.677+0000 7efeaac00640 1 -- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7efe880a2430 msgr2=0x7efe880a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.677+0000 7efeaac00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7efe880a2430 0x7efe880a48f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efe9400bbe0 tx=0x7efe9400b390 comp rx=0 tx=0).stop 2024-02-26T06:29:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.677+0000 7efeaac00640 1 -- 172.21.15.49:0/2342722332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea4105500 msgr2=0x7efea419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.677+0000 7efeaac00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea4105500 0x7efea419db60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00db20 tx=0x7efe9c00b260 comp rx=0 tx=0).stop 2024-02-26T06:29:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.678+0000 7efeaac00640 1 -- 172.21.15.49:0/2342722332 shutdown_connections 2024-02-26T06:29:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.678+0000 7efeaac00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7efe880a2430 0x7efe880a48f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.678+0000 7efeaac00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efea4100b00 0x7efea41a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.678+0000 7efeaac00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea4105500 0x7efea419db60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.678+0000 7efeaac00640 1 --2- 172.21.15.49:0/2342722332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efea41001e0 0x7efea41a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.678+0000 7efeaac00640 1 -- 172.21.15.49:0/2342722332 >> 172.21.15.49:0/2342722332 conn(0x7efea40771a0 msgr2=0x7efea4102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.678+0000 7efeaac00640 1 -- 172.21.15.49:0/2342722332 shutdown_connections 2024-02-26T06:29:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:25.678+0000 7efeaac00640 1 -- 172.21.15.49:0/2342722332 wait complete. 2024-02-26T06:29:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.820+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe46810d140 con 0x7fe4680756b0 2024-02-26T06:29:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.821+0000 7fe45ea00640 1 -- 172.21.15.49:0/641636078 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11334 (secure 0 0 0) 0x7fe458020090 con 0x7fe4680756b0 2024-02-26T06:29:25.821 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:29:25.821 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:27:29.193881+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:29:25.821 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 35 2024-02-26T06:29:25.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe44c0a23c0 msgr2=0x7fe44c0a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe44c0a23c0 0x7fe44c0a4880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe460000f80 tx=0x7fe460005fd0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4680756b0 msgr2=0x7fe46819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:25.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4680756b0 0x7fe46819de20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe458005fd0 tx=0x7fe45800dde0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 shutdown_connections 2024-02-26T06:29:25.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe44c0a23c0 0x7fe44c0a4880 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe468111fb0 0x7fe4681a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe468075fd0 0x7fe46819e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 --2- 172.21.15.49:0/641636078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4680756b0 0x7fe46819de20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:25.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.826+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 >> 172.21.15.49:0/641636078 conn(0x7fe4680fede0 msgr2=0x7fe468113b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:25.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.827+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 shutdown_connections 2024-02-26T06:29:25.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:25.827+0000 7fe46cc00640 1 -- 172.21.15.49:0/641636078 wait complete. 2024-02-26T06:29:25.851 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:25.851 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:29:26.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.167+0000 7f80e2200640 1 -- 172.21.15.49:0/1853759956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc101360 msgr2=0x7f80dc101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.167+0000 7f80e2200640 1 --2- 172.21.15.49:0/1853759956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc101360 0x7f80dc101740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009890 tx=0x7f80cc02bb70 comp rx=0 tx=0).stop 2024-02-26T06:29:26.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.167+0000 7f80e2200640 1 -- 172.21.15.49:0/1853759956 shutdown_connections 2024-02-26T06:29:26.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.167+0000 7f80e2200640 1 --2- 172.21.15.49:0/1853759956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80dc101d10 0x7f80dc105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.167+0000 7f80e2200640 1 --2- 172.21.15.49:0/1853759956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc106430 0x7f80dc10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.167+0000 7f80e2200640 1 --2- 172.21.15.49:0/1853759956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc101360 0x7f80dc101740 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.167+0000 7f80e2200640 1 -- 172.21.15.49:0/1853759956 >> 172.21.15.49:0/1853759956 conn(0x7f80dc0fcbe0 msgr2=0x7f80dc0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.169+0000 7f80e2200640 1 -- 172.21.15.49:0/1853759956 shutdown_connections 2024-02-26T06:29:26.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.169+0000 7f80e2200640 1 -- 172.21.15.49:0/1853759956 wait complete. 2024-02-26T06:29:26.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:25 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/641636078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:26.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.171+0000 7f80e2200640 1 Processor -- start 2024-02-26T06:29:26.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.171+0000 7f80e2200640 1 -- start start 2024-02-26T06:29:26.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.171+0000 7f80e2200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc106430 0x7f80dc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.171+0000 7f80e2200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80dc19ec80 0x7f80dc10d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.171+0000 7f80e2200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc10d850 0x7f80dc10dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.172+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc10d850 0x7f80dc10dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.172+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc10d850 0x7f80dc10dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48998/0 (socket says 172.21.15.49:48998) 2024-02-26T06:29:26.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.174+0000 7f80e2200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc19a360 con 0x7f80dc106430 2024-02-26T06:29:26.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.174+0000 7f80e2200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc19a4d0 con 0x7f80dc19ec80 2024-02-26T06:29:26.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.174+0000 7f80e2200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc19a690 con 0x7f80dc10d850 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.174+0000 7f80db400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc106430 0x7f80dc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.174+0000 7f80db400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc106430 0x7f80dc19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53176/0 (socket says 172.21.15.49:53176) 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.174+0000 7f80db400640 1 -- 172.21.15.49:0/2726396708 learned_addr learned my addr 172.21.15.49:0/2726396708 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.175+0000 7f80db400640 1 -- 172.21.15.49:0/2726396708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc10d850 msgr2=0x7f80dc10dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.175+0000 7f80db400640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc10d850 0x7f80dc10dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.175+0000 7f80db400640 1 -- 172.21.15.49:0/2726396708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80dc19ec80 msgr2=0x7f80dc10d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.175+0000 7f80db400640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80dc19ec80 0x7f80dc10d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.175+0000 7f80db400640 1 -- 172.21.15.49:0/2726396708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc009490 con 0x7f80dc106430 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.175+0000 7f80dbe00640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc10d850 0x7f80dc10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.175+0000 7f80db400640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc106430 0x7f80dc19e740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000c00 tx=0x7f80cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.176+0000 7f80d3e00640 1 -- 172.21.15.49:0/2726396708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc034850 con 0x7f80dc106430 2024-02-26T06:29:26.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.176+0000 7f80d3e00640 1 -- 172.21.15.49:0/2726396708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80cc0044e0 con 0x7f80dc106430 2024-02-26T06:29:26.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.176+0000 7f80d3e00640 1 -- 172.21.15.49:0/2726396708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc03d650 con 0x7f80dc106430 2024-02-26T06:29:26.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.176+0000 7f80e2200640 1 -- 172.21.15.49:0/2726396708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80dc19a910 con 0x7f80dc106430 2024-02-26T06:29:26.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.178+0000 7f80e2200640 1 -- 172.21.15.49:0/2726396708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80dc19ad70 con 0x7f80dc106430 2024-02-26T06:29:26.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.182+0000 7f80e5020740 1 Processor -- start 2024-02-26T06:29:26.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.183+0000 7f80e5020740 1 -- start start 2024-02-26T06:29:26.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.184+0000 7f80e5020740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea235ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.184+0000 7f80e5020740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea23ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.184+0000 7f80e5020740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea241500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.184+0000 7f80e5020740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea1a41e0 con 0x5576ea23f110 2024-02-26T06:29:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.184+0000 7f80e5020740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea1a1710 con 0x5576ea236530 2024-02-26T06:29:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.184+0000 7f80e5020740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea19cef0 con 0x5576ea235c10 2024-02-26T06:29:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.185+0000 7f80daa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea241500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.185+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea235ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.185+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea235ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49012/0 (socket says 172.21.15.49:49012) 2024-02-26T06:29:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.185+0000 7f80dbe00640 1 -- 172.21.15.49:0/311077122 learned_addr learned my addr 172.21.15.49:0/311077122 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.186+0000 7f80dbe00640 1 -- 172.21.15.49:0/311077122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 msgr2=0x5576ea23ebd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.186+0000 7f80dbe00640 1 --2- 172.21.15.49:0/311077122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea23ebd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.186+0000 7f80dbe00640 1 -- 172.21.15.49:0/311077122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 msgr2=0x5576ea241500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.186+0000 7f80dbe00640 1 --2- 172.21.15.49:0/311077122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea241500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.186+0000 7f80dbe00640 1 -- 172.21.15.49:0/311077122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576ea15edc0 con 0x5576ea235c10 2024-02-26T06:29:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.186+0000 7f80dbe00640 1 --2- 172.21.15.49:0/311077122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea235ff0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c450 tx=0x7f80d400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1649ea1c4468aa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.187+0000 7f80c7e00640 1 -- 172.21.15.49:0/311077122 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4011070 con 0x5576ea235c10 2024-02-26T06:29:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.187+0000 7f80c7e00640 1 -- 172.21.15.49:0/311077122 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80d400a8e0 con 0x5576ea235c10 2024-02-26T06:29:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.187+0000 7f80c7e00640 1 -- 172.21.15.49:0/311077122 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4015920 con 0x5576ea235c10 2024-02-26T06:29:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.187+0000 7f80d3e00640 1 -- 172.21.15.49:0/2726396708 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f80cc03b040 con 0x7f80dc106430 2024-02-26T06:29:26.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.188+0000 7f80e5020740 1 -- 172.21.15.49:0/311077122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 msgr2=0x5576ea235ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.188+0000 7f80e5020740 1 --2- 172.21.15.49:0/311077122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea235ff0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c450 tx=0x7f80d400c850 comp rx=0 tx=0).stop 2024-02-26T06:29:26.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.189+0000 7f80e5020740 1 -- 172.21.15.49:0/311077122 shutdown_connections 2024-02-26T06:29:26.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.189+0000 7f80e5020740 1 --2- 172.21.15.49:0/311077122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea23ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.189+0000 7f80e5020740 1 --2- 172.21.15.49:0/311077122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea241500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.189+0000 7f80e5020740 1 --2- 172.21.15.49:0/311077122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea235ff0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.189+0000 7f80e5020740 1 -- 172.21.15.49:0/311077122 >> 172.21.15.49:0/311077122 conn(0x5576ea230340 msgr2=0x5576ea230720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.192+0000 7f80e5020740 1 -- 172.21.15.49:0/311077122 shutdown_connections 2024-02-26T06:29:26.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.192+0000 7f80e5020740 1 -- 172.21.15.49:0/311077122 wait complete. 2024-02-26T06:29:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.193+0000 7f80d3e00640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f80c00a2240 0x7f80c00a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.193+0000 7f80e5020740 1 -- start start 2024-02-26T06:29:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.193+0000 7f80d3e00640 1 -- 172.21.15.49:0/2726396708 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f80cc0ef100 con 0x7f80dc106430 2024-02-26T06:29:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.194+0000 7f80e5020740 1 Processor -- start 2024-02-26T06:29:26.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.195+0000 7f80dbe00640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f80c00a2240 0x7f80c00a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.195+0000 7f80e5020740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.195+0000 7f80e5020740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea23c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.195+0000 7f80e5020740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea23cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.195+0000 7f80e5020740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea142120 con 0x5576ea23f110 2024-02-26T06:29:26.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.195+0000 7f80e5020740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea141dc0 con 0x5576ea236530 2024-02-26T06:29:26.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.195+0000 7f80e5020740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e9f0b1c0 con 0x5576ea235c10 2024-02-26T06:29:26.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.195+0000 7f80e5020740 10 client.0 _ll_register_callbacks cb 0x5576ea191280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:26.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.196+0000 7f80db400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.196+0000 7f80db400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49024/0 (socket says 172.21.15.49:49024) 2024-02-26T06:29:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.196+0000 7f80db400640 1 -- 172.21.15.49:0/3044786399 learned_addr learned my addr 172.21.15.49:0/3044786399 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.196+0000 7f80daa00640 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea23c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.197+0000 7f80dbe00640 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea23cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.198+0000 7f80db400640 1 -- 172.21.15.49:0/3044786399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 msgr2=0x5576ea23c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.198+0000 7f80db400640 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea23c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.198+0000 7f80db400640 1 -- 172.21.15.49:0/3044786399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 msgr2=0x5576ea23cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.198+0000 7f80db400640 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea23cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.198+0000 7f80db400640 1 -- 172.21.15.49:0/3044786399 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d4009ca0 con 0x5576ea235c10 2024-02-26T06:29:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.199+0000 7f80dbe00640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f80c00a2240 0x7f80c00a4700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c420 tx=0x7f80d4015460 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.199+0000 7f80db400640 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23c160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0bb8b0 tx=0x7f80cc0bb510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e523ea731d5e5b6b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.199+0000 7f80c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.200+0000 7f80c4c00640 1 -- 172.21.15.49:0/3044786399 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0c40c0 con 0x5576ea235c10 2024-02-26T06:29:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.201+0000 7f80c4c00640 1 -- 172.21.15.49:0/3044786399 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f80cc0bbbb0 con 0x5576ea235c10 2024-02-26T06:29:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.201+0000 7f80e5020740 1 -- 172.21.15.49:0/3044786399 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576e9e9c7e0 con 0x5576ea235c10 2024-02-26T06:29:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.201+0000 7f80c4c00640 1 -- 172.21.15.49:0/3044786399 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80cc0ecc90 con 0x5576ea235c10 2024-02-26T06:29:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.201+0000 7f80e5020740 10 client.26914 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.201+0000 7f80e5020740 1 -- 172.21.15.49:0/3044786399 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5576ea19b570 con 0x5576ea235c10 2024-02-26T06:29:26.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.202+0000 7f80c4c00640 1 -- 172.21.15.49:0/3044786399 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5576e9e9c7e0 con 0x5576ea235c10 2024-02-26T06:29:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.203+0000 7f80c4c00640 1 -- 172.21.15.49:0/3044786399 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f80cc0baa70 con 0x5576ea235c10 2024-02-26T06:29:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.203+0000 7f80e5020740 10 client.26914 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.203+0000 7f80e5020740 10 client.26914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:29:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.203+0000 7f80e5020740 20 client.26914 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.203+0000 7f80e5020740 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5576ea24ed60 0x5576ea251150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.203+0000 7f80e5020740 4 client.26914 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:26.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.203+0000 7f80e5020740 1 -- 172.21.15.49:0/3044786399 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576e9ed9720 con 0x5576ea24ed60 2024-02-26T06:29:26.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.204+0000 7f80daa00640 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5576ea24ed60 0x5576ea251150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.206+0000 7f80daa00640 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5576ea24ed60 0x5576ea251150 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b65b0f0b4f138e4 server_cookie=9ad205a5399bbcb1 in_seq=0 out_seq=0 2024-02-26T06:29:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.206+0000 7f80c4c00640 10 client.26914 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:29:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.207+0000 7f80c4c00640 1 -- 172.21.15.49:0/3044786399 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576e9ed9720 con 0x5576ea24ed60 2024-02-26T06:29:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.207+0000 7f80c4c00640 10 client.26914 handle_command_reply: tid=0 2024-02-26T06:29:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.207+0000 7f80e5020740 1 client.26914 shutdown 2024-02-26T06:29:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.208+0000 7f80e5020740 1 -- 172.21.15.49:0/3044786399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 msgr2=0x5576ea23c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.208+0000 7f80e5020740 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23c160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0bb8b0 tx=0x7f80cc0bb510 comp rx=0 tx=0).stop 2024-02-26T06:29:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.208+0000 7f80e5020740 1 -- 172.21.15.49:0/3044786399 shutdown_connections 2024-02-26T06:29:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.209+0000 7f80e5020740 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5576ea24ed60 0x5576ea251150 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.209+0000 7f80e5020740 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea23c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.209+0000 7f80c4c00640 0 client.26914 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:29:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.209+0000 7f80e5020740 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea23cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.209+0000 7f80e5020740 1 --2- 172.21.15.49:0/3044786399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23c160 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.209+0000 7f80e5020740 1 -- 172.21.15.49:0/3044786399 >> 172.21.15.49:0/3044786399 conn(0x5576ea230340 msgr2=0x5576ea232090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.210+0000 7f80e5020740 1 -- 172.21.15.49:0/3044786399 shutdown_connections 2024-02-26T06:29:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.210+0000 7f80e5020740 1 -- 172.21.15.49:0/3044786399 wait complete. 2024-02-26T06:29:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.211+0000 7f80e5020740 20 client.26914 trim_cache size 0 max 16384 2024-02-26T06:29:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.238+0000 7f80e5020740 1 Processor -- start 2024-02-26T06:29:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.238+0000 7f80e5020740 1 -- start start 2024-02-26T06:29:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80e5020740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80e5020740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea231e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80e5020740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea246020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80e5020740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e9e9c7e0 con 0x5576ea23f110 2024-02-26T06:29:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80e5020740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea22fbc0 con 0x5576ea236530 2024-02-26T06:29:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80e5020740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea234f00 con 0x5576ea235c10 2024-02-26T06:29:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea231e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea231e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51332/0 (socket says 172.21.15.49:51332) 2024-02-26T06:29:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.239+0000 7f80daa00640 1 -- 172.21.15.49:0/3120241109 learned_addr learned my addr 172.21.15.49:0/3120241109 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.240+0000 7f80db400640 1 --2- 172.21.15.49:0/3120241109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.240+0000 7f80daa00640 1 -- 172.21.15.49:0/3120241109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 msgr2=0x5576ea23ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.240+0000 7f80daa00640 1 --2- 172.21.15.49:0/3120241109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.240+0000 7f80daa00640 1 -- 172.21.15.49:0/3120241109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 msgr2=0x5576ea246020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.240+0000 7f80daa00640 1 --2- 172.21.15.49:0/3120241109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea246020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.240+0000 7f80daa00640 1 -- 172.21.15.49:0/3120241109 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576ea137a80 con 0x5576ea236530 2024-02-26T06:29:26.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.240+0000 7f80daa00640 1 --2- 172.21.15.49:0/3120241109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea231e80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x5576ea2314d0 tx=0x7f80c8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aacdbe0a27bac948 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.241+0000 7f80c7400640 1 -- 172.21.15.49:0/3120241109 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8017040 con 0x5576ea236530 2024-02-26T06:29:26.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.241+0000 7f80c7400640 1 -- 172.21.15.49:0/3120241109 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80c8020070 con 0x5576ea236530 2024-02-26T06:29:26.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.241+0000 7f80c7400640 1 -- 172.21.15.49:0/3120241109 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c801b380 con 0x5576ea236530 2024-02-26T06:29:26.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.242+0000 7f80e5020740 1 -- 172.21.15.49:0/3120241109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 msgr2=0x5576ea231e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.242+0000 7f80e5020740 1 --2- 172.21.15.49:0/3120241109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea231e80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x5576ea2314d0 tx=0x7f80c8010800 comp rx=0 tx=0).stop 2024-02-26T06:29:26.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.243+0000 7f80e5020740 1 -- 172.21.15.49:0/3120241109 shutdown_connections 2024-02-26T06:29:26.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.243+0000 7f80e5020740 1 --2- 172.21.15.49:0/3120241109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea231e80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.243+0000 7f80e5020740 1 --2- 172.21.15.49:0/3120241109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea23f110 0x5576ea246020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.243+0000 7f80e5020740 1 --2- 172.21.15.49:0/3120241109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea235c10 0x5576ea23ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.243+0000 7f80e5020740 1 -- 172.21.15.49:0/3120241109 >> 172.21.15.49:0/3120241109 conn(0x5576ea230340 msgr2=0x5576ea231770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.243+0000 7f80e5020740 1 -- 172.21.15.49:0/3120241109 shutdown_connections 2024-02-26T06:29:26.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.243+0000 7f80e5020740 1 -- 172.21.15.49:0/3120241109 wait complete. 2024-02-26T06:29:26.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.244+0000 7f80e5020740 1 -- start start 2024-02-26T06:29:26.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.244+0000 7f80e5020740 1 Processor -- start 2024-02-26T06:29:26.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.246+0000 7f80e5020740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea235c10 0x5576ea244d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.246+0000 7f80e5020740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea2452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.246+0000 7f80e5020740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea23f110 0x5576ea2a5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.247+0000 7f80e5020740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea245a70 con 0x5576ea235c10 2024-02-26T06:29:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.247+0000 7f80e5020740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea245be0 con 0x5576ea236530 2024-02-26T06:29:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.247+0000 7f80e5020740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ea2a5740 con 0x5576ea23f110 2024-02-26T06:29:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.247+0000 7f80e5020740 10 client.0 _ll_register_callbacks cb 0x7f80d4009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:26.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.247+0000 7f80daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea2452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.247+0000 7f80daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea2452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51348/0 (socket says 172.21.15.49:51348) 2024-02-26T06:29:26.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.247+0000 7f80daa00640 1 -- 172.21.15.49:0/864249718 learned_addr learned my addr 172.21.15.49:0/864249718 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.247+0000 7f80db400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea235c10 0x5576ea244d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.248+0000 7f80dbe00640 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea23f110 0x5576ea2a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.249+0000 7f80daa00640 1 -- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea23f110 msgr2=0x5576ea2a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.249+0000 7f80daa00640 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea23f110 0x5576ea2a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.249+0000 7f80daa00640 1 -- 172.21.15.49:0/864249718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea235c10 msgr2=0x5576ea244d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.249+0000 7f80daa00640 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea235c10 0x5576ea244d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.249+0000 7f80daa00640 1 -- 172.21.15.49:0/864249718 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c8009de0 con 0x5576ea236530 2024-02-26T06:29:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.250+0000 7f80db400640 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea235c10 0x5576ea244d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.250+0000 7f80dbe00640 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea23f110 0x5576ea2a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:26.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.251+0000 7f80daa00640 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea2452b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f80c8000c00 tx=0x7f80c800ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4aecfe98f06608 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.251+0000 7f80c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:29:26.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.252+0000 7f80c4c00640 1 -- 172.21.15.49:0/864249718 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8017070 con 0x5576ea236530 2024-02-26T06:29:26.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.252+0000 7f80c4c00640 1 -- 172.21.15.49:0/864249718 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f80c801c5a0 con 0x5576ea236530 2024-02-26T06:29:26.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.252+0000 7f80e5020740 1 -- 172.21.15.49:0/864249718 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80d400a8e0 con 0x5576ea236530 2024-02-26T06:29:26.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.253+0000 7f80c4c00640 1 -- 172.21.15.49:0/864249718 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c801c760 con 0x5576ea236530 2024-02-26T06:29:26.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.253+0000 7f80e5020740 10 client.26933 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:26.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.253+0000 7f80e5020740 1 -- 172.21.15.49:0/864249718 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5576ea15d620 con 0x5576ea236530 2024-02-26T06:29:26.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.253+0000 7f80c4c00640 1 -- 172.21.15.49:0/864249718 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80c801c8c0 con 0x5576ea236530 2024-02-26T06:29:26.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.254+0000 7f80c4c00640 1 -- 172.21.15.49:0/864249718 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f80c800f3c0 con 0x5576ea236530 2024-02-26T06:29:26.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.254+0000 7f80e5020740 10 client.26933 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:26.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.254+0000 7f80e5020740 10 client.26933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:29:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.254+0000 7f80e5020740 20 client.26933 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.254+0000 7f80e5020740 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5576ea24ed60 0x5576ea2a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.254+0000 7f80e5020740 4 client.26933 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.254+0000 7f80e5020740 1 -- 172.21.15.49:0/864249718 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f80d400a8e0 con 0x5576ea24ed60 2024-02-26T06:29:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.254+0000 7f80db400640 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5576ea24ed60 0x5576ea2a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.255+0000 7f80db400640 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5576ea24ed60 0x5576ea2a8f10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8b8ebbf8619ee86 server_cookie=4c87642d676792c0 in_seq=0 out_seq=0 2024-02-26T06:29:26.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.255+0000 7f80c4c00640 10 client.26933 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:29:26.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.255+0000 7f80c4c00640 1 -- 172.21.15.49:0/864249718 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f80d400a8e0 con 0x5576ea24ed60 2024-02-26T06:29:26.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.255+0000 7f80c4c00640 10 client.26933 handle_command_reply: tid=0 2024-02-26T06:29:26.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.255+0000 7f80e5020740 1 client.26933 shutdown 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.255+0000 7f80e5020740 1 -- 172.21.15.49:0/864249718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 msgr2=0x5576ea2452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.255+0000 7f80e5020740 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea2452b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f80c8000c00 tx=0x7f80c800ff90 comp rx=0 tx=0).stop 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.256+0000 7f80e5020740 1 -- 172.21.15.49:0/864249718 shutdown_connections 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.256+0000 7f80e5020740 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5576ea24ed60 0x5576ea2a8f10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.256+0000 7f80e5020740 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5576ea236530 0x5576ea2452b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.256+0000 7f80e5020740 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576ea23f110 0x5576ea2a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.256+0000 7f80e5020740 1 --2- 172.21.15.49:0/864249718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576ea235c10 0x5576ea244d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.256+0000 7f80e5020740 1 -- 172.21.15.49:0/864249718 >> 172.21.15.49:0/864249718 conn(0x5576ea230340 msgr2=0x5576ea22f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.256+0000 7f80c4c00640 0 client.26933 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:29:26.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.256+0000 7f80e5020740 1 -- 172.21.15.49:0/864249718 shutdown_connections 2024-02-26T06:29:26.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.257+0000 7f80e5020740 1 -- 172.21.15.49:0/864249718 wait complete. 2024-02-26T06:29:26.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.257+0000 7f80e5020740 20 client.26933 trim_cache size 0 max 16384 2024-02-26T06:29:26.257 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:29:26.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.259+0000 7f80e2200640 1 -- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f80c00a2240 msgr2=0x7f80c00a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.259+0000 7f80e2200640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f80c00a2240 0x7f80c00a4700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c420 tx=0x7f80d4015460 comp rx=0 tx=0).stop 2024-02-26T06:29:26.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.259+0000 7f80e2200640 1 -- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc106430 msgr2=0x7f80dc19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.259+0000 7f80e2200640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc106430 0x7f80dc19e740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000c00 tx=0x7f80cc0042f0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.259+0000 7f80e2200640 1 -- 172.21.15.49:0/2726396708 shutdown_connections 2024-02-26T06:29:26.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.259+0000 7f80e2200640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f80c00a2240 0x7f80c00a4700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.259+0000 7f80e2200640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80dc19ec80 0x7f80dc10d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.259+0000 7f80e2200640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc10d850 0x7f80dc10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.260+0000 7f80e2200640 1 --2- 172.21.15.49:0/2726396708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80dc106430 0x7f80dc19e740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.260+0000 7f80e2200640 1 -- 172.21.15.49:0/2726396708 >> 172.21.15.49:0/2726396708 conn(0x7f80dc0fcbe0 msgr2=0x7f80dc0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.260+0000 7f80e2200640 1 -- 172.21.15.49:0/2726396708 shutdown_connections 2024-02-26T06:29:26.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.260+0000 7f80e2200640 1 -- 172.21.15.49:0/2726396708 wait complete. 2024-02-26T06:29:26.283 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:26.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:29:26.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 335.0 s, 2519.6 tps, lat 39.799 ms stddev 43.191 2024-02-26T06:29:26.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:25 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/641636078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:26.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:25 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/641636078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 -- 172.21.15.49:0/4069811168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d0107720 msgr2=0x7f86d0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 --2- 172.21.15.49:0/4069811168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d0107720 0x7f86d0107ba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009900 tx=0x7f86bc02bc50 comp rx=0 tx=0).stop 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 -- 172.21.15.49:0/4069811168 shutdown_connections 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 --2- 172.21.15.49:0/4069811168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d0107720 0x7f86d0107ba0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 --2- 172.21.15.49:0/4069811168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d01080e0 0x7f86d01145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 --2- 172.21.15.49:0/4069811168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d010d730 0x7f86d010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 -- 172.21.15.49:0/4069811168 >> 172.21.15.49:0/4069811168 conn(0x7f86d00756c0 msgr2=0x7f86d0077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 -- 172.21.15.49:0/4069811168 shutdown_connections 2024-02-26T06:29:26.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.585+0000 7f86d7200640 1 -- 172.21.15.49:0/4069811168 wait complete. 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.586+0000 7f86d7200640 1 Processor -- start 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.586+0000 7f86d7200640 1 -- start start 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.586+0000 7f86d7200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d0107720 0x7f86d019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.586+0000 7f86d7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d01080e0 0x7f86d019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86d7200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d010d730 0x7f86d01a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86d7200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01a2c90 con 0x7f86d010d730 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86d7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01a2e00 con 0x7f86d01080e0 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86d7200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01a2fc0 con 0x7f86d0107720 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d0107720 0x7f86d019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d01080e0 0x7f86d019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d0107720 0x7f86d019ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49066/0 (socket says 172.21.15.49:49066) 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d01080e0 0x7f86d019e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51358/0 (socket says 172.21.15.49:51358) 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cfe00640 1 -- 172.21.15.49:0/1819269158 learned_addr learned my addr 172.21.15.49:0/1819269158 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86d5200640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d010d730 0x7f86d01a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cf400640 1 -- 172.21.15.49:0/1819269158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d0107720 msgr2=0x7f86d019ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cf400640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d0107720 0x7f86d019ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cf400640 1 -- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d010d730 msgr2=0x7f86d01a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cf400640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d010d730 0x7f86d01a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86cf400640 1 -- 172.21.15.49:0/1819269158 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009560 con 0x7f86d01080e0 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.587+0000 7f86d5200640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d010d730 0x7f86d01a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.588+0000 7f86cf400640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d01080e0 0x7f86d019e310 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86bc02c190 tx=0x7f86bc02c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.588+0000 7f86ccc00640 1 -- 172.21.15.49:0/1819269158 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0042d0 con 0x7f86d01080e0 2024-02-26T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.588+0000 7f86ccc00640 1 -- 172.21.15.49:0/1819269158 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86bc004430 con 0x7f86d01080e0 2024-02-26T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.588+0000 7f86ccc00640 1 -- 172.21.15.49:0/1819269158 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc003bb0 con 0x7f86d01080e0 2024-02-26T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.588+0000 7f86d7200640 1 -- 172.21.15.49:0/1819269158 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d01a3240 con 0x7f86d01080e0 2024-02-26T06:29:26.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.588+0000 7f86d7200640 1 -- 172.21.15.49:0/1819269158 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d01aab20 con 0x7f86d01080e0 2024-02-26T06:29:26.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.598+0000 7f86da200740 1 Processor -- start 2024-02-26T06:29:26.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.599+0000 7f86ccc00640 1 -- 172.21.15.49:0/1819269158 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f86bc003d10 con 0x7f86d01080e0 2024-02-26T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.603+0000 7f86da200740 1 -- start start 2024-02-26T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.603+0000 7f86da200740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x565546078ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86da200740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x565546081bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86da200740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x565546084500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86da200740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x565545fe71e0 con 0x565546078c10 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86da200740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x565545fe4710 con 0x565546082110 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86da200740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x565545fdfef0 con 0x565546079530 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86cf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x565546084500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86cf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x565546084500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51366/0 (socket says 172.21.15.49:51366) 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86cf400640 1 -- 172.21.15.49:0/89598381 learned_addr learned my addr 172.21.15.49:0/89598381 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.604+0000 7f86d5200640 1 --2- 172.21.15.49:0/89598381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x565546078ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.605+0000 7f86d5200640 1 -- 172.21.15.49:0/89598381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 msgr2=0x565546081bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.605+0000 7f86d5200640 1 --2- 172.21.15.49:0/89598381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x565546081bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.605+0000 7f86d5200640 1 -- 172.21.15.49:0/89598381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 msgr2=0x565546084500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.605+0000 7f86d5200640 1 --2- 172.21.15.49:0/89598381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x565546084500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.605+0000 7f86d5200640 1 -- 172.21.15.49:0/89598381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565545fa1dc0 con 0x565546078c10 2024-02-26T06:29:26.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.606+0000 7f86d5200640 1 --2- 172.21.15.49:0/89598381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x565546078ff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009ec0 tx=0x7f86c800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df64823b01b54bb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.608+0000 7f86c5600640 1 -- 172.21.15.49:0/89598381 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8003b70 con 0x565546078c10 2024-02-26T06:29:26.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.609+0000 7f86ccc00640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86b40a2330 0x7f86b40a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.609+0000 7f86c5600640 1 -- 172.21.15.49:0/89598381 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86c8003cd0 con 0x565546078c10 2024-02-26T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.609+0000 7f86c5600640 1 -- 172.21.15.49:0/89598381 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8011940 con 0x565546078c10 2024-02-26T06:29:26.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.609+0000 7f86cfe00640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86b40a2330 0x7f86b40a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.611+0000 7f86ccc00640 1 -- 172.21.15.49:0/1819269158 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f86bc0f0220 con 0x7f86d01080e0 2024-02-26T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.612+0000 7f86da200740 1 -- 172.21.15.49:0/89598381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 msgr2=0x565546078ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.612+0000 7f86cfe00640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86b40a2330 0x7f86b40a47f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5655460786e0 tx=0x7f86c00098c0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.612+0000 7f86da200740 1 --2- 172.21.15.49:0/89598381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x565546078ff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009ec0 tx=0x7f86c800f730 comp rx=0 tx=0).stop 2024-02-26T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.615+0000 7f86da200740 1 -- 172.21.15.49:0/89598381 shutdown_connections 2024-02-26T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.615+0000 7f86da200740 1 --2- 172.21.15.49:0/89598381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x565546084500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.615+0000 7f86da200740 1 --2- 172.21.15.49:0/89598381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x565546081bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.615+0000 7f86da200740 1 --2- 172.21.15.49:0/89598381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x565546078ff0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.615+0000 7f86da200740 1 -- 172.21.15.49:0/89598381 >> 172.21.15.49:0/89598381 conn(0x565546073340 msgr2=0x565546073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.616+0000 7f86da200740 1 -- 172.21.15.49:0/89598381 shutdown_connections 2024-02-26T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.616+0000 7f86da200740 1 -- 172.21.15.49:0/89598381 wait complete. 2024-02-26T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.617+0000 7f86da200740 1 -- start start 2024-02-26T06:29:26.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.617+0000 7f86da200740 1 Processor -- start 2024-02-26T06:29:26.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.619+0000 7f86da200740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546078c10 0x56554607f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.620+0000 7f86da200740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.621+0000 7f86da200740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546082110 0x56554607fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.621+0000 7f86cfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546078c10 0x56554607f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.621+0000 7f86cfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546078c10 0x56554607f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51370/0 (socket says 172.21.15.49:51370) 2024-02-26T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.621+0000 7f86cf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.621+0000 7f86cf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49084/0 (socket says 172.21.15.49:49084) 2024-02-26T06:29:26.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.622+0000 7f86da200740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x565545f85120 con 0x565546082110 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.622+0000 7f86da200740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x565545f84dc0 con 0x565546078c10 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.622+0000 7f86da200740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x565545d4e1c0 con 0x565546079530 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.623+0000 7f86da200740 10 client.0 _ll_register_callbacks cb 0x565545fd4280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.623+0000 7f86cfe00640 1 -- 172.21.15.49:0/3791665283 learned_addr learned my addr 172.21.15.49:0/3791665283 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.624+0000 7f86cf400640 1 -- 172.21.15.49:0/3791665283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546078c10 msgr2=0x56554607f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.624+0000 7f86cf400640 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546078c10 0x56554607f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.624+0000 7f86cf400640 1 -- 172.21.15.49:0/3791665283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546082110 msgr2=0x56554607fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.624+0000 7f86cf400640 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546082110 0x56554607fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.624+0000 7f86cf400640 1 -- 172.21.15.49:0/3791665283 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c8009ba0 con 0x565546079530 2024-02-26T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.625+0000 7f86cf400640 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607f710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0bc9d0 tx=0x7f86bc0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a117f467d5190a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.626+0000 7f86ba000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.626+0000 7f86ba000640 1 -- 172.21.15.49:0/3791665283 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0f0030 con 0x565546079530 2024-02-26T06:29:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.626+0000 7f86ba000640 1 -- 172.21.15.49:0/3791665283 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f86bc0bd3d0 con 0x565546079530 2024-02-26T06:29:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.627+0000 7f86da200740 1 -- 172.21.15.49:0/3791665283 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565545cdf7e0 con 0x565546079530 2024-02-26T06:29:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.627+0000 7f86ba000640 1 -- 172.21.15.49:0/3791665283 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86bc0bc810 con 0x565546079530 2024-02-26T06:29:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86da200740 10 client.26929 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86da200740 1 -- 172.21.15.49:0/3791665283 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x565545fde570 con 0x565546079530 2024-02-26T06:29:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86ba000640 1 -- 172.21.15.49:0/3791665283 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565545cdf7e0 con 0x565546079530 2024-02-26T06:29:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86ba000640 1 -- 172.21.15.49:0/3791665283 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f86bc0d0d20 con 0x565546079530 2024-02-26T06:29:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86da200740 10 client.26929 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86da200740 10 client.26929 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:29:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86da200740 20 client.26929 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86da200740 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5655460a2e70 0x5655460a5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86da200740 4 client.26929 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:26.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.628+0000 7f86da200740 1 -- 172.21.15.49:0/3791665283 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565545d1c720 con 0x5655460a2e70 2024-02-26T06:29:26.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.629+0000 7f86cfe00640 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5655460a2e70 0x5655460a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.630+0000 7f86ba000640 10 client.26929 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:29:26.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.630+0000 7f86cfe00640 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5655460a2e70 0x5655460a5290 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=205bf0ffe6e70cd8 server_cookie=7129dfc9a3a08911 in_seq=0 out_seq=0 2024-02-26T06:29:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.631+0000 7f86ba000640 1 -- 172.21.15.49:0/3791665283 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565545d1c720 con 0x5655460a2e70 2024-02-26T06:29:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.631+0000 7f86ba000640 10 client.26929 handle_command_reply: tid=0 2024-02-26T06:29:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.631+0000 7f86da200740 1 client.26929 shutdown 2024-02-26T06:29:26.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.633+0000 7f86da200740 1 -- 172.21.15.49:0/3791665283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 msgr2=0x56554607f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.633+0000 7f86da200740 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607f710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0bc9d0 tx=0x7f86bc0035f0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86da200740 1 -- 172.21.15.49:0/3791665283 shutdown_connections 2024-02-26T06:29:26.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86da200740 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5655460a2e70 0x5655460a5290 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86da200740 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546082110 0x56554607fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86da200740 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607f710 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86da200740 1 --2- 172.21.15.49:0/3791665283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546078c10 0x56554607f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86da200740 1 -- 172.21.15.49:0/3791665283 >> 172.21.15.49:0/3791665283 conn(0x565546073340 msgr2=0x565546075090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86ba000640 0 client.26929 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:29:26.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86da200740 1 -- 172.21.15.49:0/3791665283 shutdown_connections 2024-02-26T06:29:26.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.634+0000 7f86da200740 1 -- 172.21.15.49:0/3791665283 wait complete. 2024-02-26T06:29:26.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.635+0000 7f86da200740 20 client.26929 trim_cache size 0 max 16384 2024-02-26T06:29:26.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.659+0000 7f86da200740 1 Processor -- start 2024-02-26T06:29:26.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.660+0000 7f86da200740 1 -- start start 2024-02-26T06:29:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.660+0000 7f86da200740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x56554607f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.660+0000 7f86da200740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.660+0000 7f86da200740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x56554607c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.660+0000 7f86da200740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x565545d1c720 con 0x565546078c10 2024-02-26T06:29:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.660+0000 7f86da200740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x565546077d00 con 0x565546082110 2024-02-26T06:29:26.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.660+0000 7f86da200740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x565546072bc0 con 0x565546079530 2024-02-26T06:29:26.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49096/0 (socket says 172.21.15.49:49096) 2024-02-26T06:29:26.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cf400640 1 -- 172.21.15.49:0/3232560533 learned_addr learned my addr 172.21.15.49:0/3232560533 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cf400640 1 -- 172.21.15.49:0/3232560533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 msgr2=0x56554607c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86d5200640 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x56554607c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cf400640 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x56554607c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cf400640 1 -- 172.21.15.49:0/3232560533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 msgr2=0x56554607f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cfe00640 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x56554607f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cf400640 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x56554607f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.661+0000 7f86cf400640 1 -- 172.21.15.49:0/3232560533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565545f7aa80 con 0x565546079530 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.662+0000 7f86cfe00640 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x56554607f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.662+0000 7f86cf400640 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607fc60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0bcc60 tx=0x7f86bc0bb1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7272aab4c281b4f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.662+0000 7f86c4c00640 1 -- 172.21.15.49:0/3232560533 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0d5070 con 0x565546079530 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.662+0000 7f86c4c00640 1 -- 172.21.15.49:0/3232560533 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86bc0f0040 con 0x565546079530 2024-02-26T06:29:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.662+0000 7f86c4c00640 1 -- 172.21.15.49:0/3232560533 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0f38a0 con 0x565546079530 2024-02-26T06:29:26.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 -- 172.21.15.49:0/3232560533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 msgr2=0x56554607fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607fc60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0bcc60 tx=0x7f86bc0bb1e0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 -- 172.21.15.49:0/3232560533 shutdown_connections 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x56554607c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546079530 0x56554607fc60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 --2- 172.21.15.49:0/3232560533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546078c10 0x56554607f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 -- 172.21.15.49:0/3232560533 >> 172.21.15.49:0/3232560533 conn(0x565546073340 msgr2=0x5655460a9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 -- 172.21.15.49:0/3232560533 shutdown_connections 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 -- 172.21.15.49:0/3232560533 wait complete. 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.663+0000 7f86da200740 1 -- start start 2024-02-26T06:29:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.664+0000 7f86da200740 1 Processor -- start 2024-02-26T06:29:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.665+0000 7f86da200740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546078c10 0x56554609a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.665+0000 7f86da200740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546079530 0x5655460984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.665+0000 7f86da200740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x5655460989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.665+0000 7f86da200740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x565546098f20 con 0x565546079530 2024-02-26T06:29:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.665+0000 7f86da200740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x565546099090 con 0x565546082110 2024-02-26T06:29:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.665+0000 7f86da200740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x565546099250 con 0x565546078c10 2024-02-26T06:29:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.665+0000 7f86da200740 10 client.0 _ll_register_callbacks cb 0x7f86c8009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:26.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.666+0000 7f86cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546078c10 0x56554609a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.666+0000 7f86cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546078c10 0x56554609a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49102/0 (socket says 172.21.15.49:49102) 2024-02-26T06:29:26.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.666+0000 7f86cfe00640 1 -- 172.21.15.49:0/1687057519 learned_addr learned my addr 172.21.15.49:0/1687057519 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:26.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.667+0000 7f86d5200640 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x5655460989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.667+0000 7f86cf400640 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546079530 0x5655460984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.667+0000 7f86cfe00640 1 -- 172.21.15.49:0/1687057519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 msgr2=0x5655460989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.667+0000 7f86cfe00640 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x5655460989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.667+0000 7f86cfe00640 1 -- 172.21.15.49:0/1687057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546079530 msgr2=0x5655460984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:26.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.667+0000 7f86cfe00640 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546079530 0x5655460984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.667+0000 7f86cfe00640 1 -- 172.21.15.49:0/1687057519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565545fde570 con 0x565546078c10 2024-02-26T06:29:26.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.667+0000 7f86cf400640 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546079530 0x5655460984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:26.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.668+0000 7f86cfe00640 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546078c10 0x56554609a100 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f86c0011ca0 tx=0x7f86c0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d1c4fee0716965e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:26.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.668+0000 7f86ba000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:26.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.668+0000 7f86ba000640 1 -- 172.21.15.49:0/1687057519 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0022040 con 0x565546078c10 2024-02-26T06:29:26.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.668+0000 7f86ba000640 1 -- 172.21.15.49:0/1687057519 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f86c0028620 con 0x565546078c10 2024-02-26T06:29:26.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.668+0000 7f86da200740 1 -- 172.21.15.49:0/1687057519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565545d6b890 con 0x565546078c10 2024-02-26T06:29:26.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.668+0000 7f86ba000640 1 -- 172.21.15.49:0/1687057519 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c00287e0 con 0x565546078c10 2024-02-26T06:29:26.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.669+0000 7f86da200740 10 client.26935 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:26.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.669+0000 7f86da200740 1 -- 172.21.15.49:0/1687057519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x565545fa0620 con 0x565546078c10 2024-02-26T06:29:26.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86ba000640 1 -- 172.21.15.49:0/1687057519 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565545d6b890 con 0x565546078c10 2024-02-26T06:29:26.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86ba000640 1 -- 172.21.15.49:0/1687057519 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f86c001d860 con 0x565546078c10 2024-02-26T06:29:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86da200740 10 client.26935 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86da200740 10 client.26935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:29:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86da200740 20 client.26935 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86da200740 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5655460fa7f0 0x5655460fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86da200740 4 client.26935 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86da200740 1 -- 172.21.15.49:0/1687057519 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565545cce4e0 con 0x5655460fa7f0 2024-02-26T06:29:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.670+0000 7f86cf400640 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5655460fa7f0 0x5655460fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:26.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86cf400640 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5655460fa7f0 0x5655460fcc10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be5cbc54fb30c006 server_cookie=df318e6ce03867f0 in_seq=0 out_seq=0 2024-02-26T06:29:26.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86ba000640 10 client.26935 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:29:26.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86ba000640 1 -- 172.21.15.49:0/1687057519 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565545cce4e0 con 0x5655460fa7f0 2024-02-26T06:29:26.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86ba000640 10 client.26935 handle_command_reply: tid=0 2024-02-26T06:29:26.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 client.26935 shutdown 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 -- 172.21.15.49:0/1687057519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546078c10 msgr2=0x56554609a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546078c10 0x56554609a100 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f86c0011ca0 tx=0x7f86c0005f90 comp rx=0 tx=0).stop 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 -- 172.21.15.49:0/1687057519 shutdown_connections 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x5655460fa7f0 0x5655460fcc10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565546082110 0x5655460989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565546079530 0x5655460984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 --2- 172.21.15.49:0/1687057519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565546078c10 0x56554609a100 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86da200740 1 -- 172.21.15.49:0/1687057519 >> 172.21.15.49:0/1687057519 conn(0x565546073340 msgr2=0x565546071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.671+0000 7f86ba000640 0 client.26935 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86da200740 1 -- 172.21.15.49:0/1687057519 shutdown_connections 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86da200740 1 -- 172.21.15.49:0/1687057519 wait complete. 2024-02-26T06:29:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86da200740 20 client.26935 trim_cache size 0 max 16384 2024-02-26T06:29:26.673 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:29:26.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86d7200640 1 -- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86b40a2330 msgr2=0x7f86b40a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86d7200640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86b40a2330 0x7f86b40a47f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5655460786e0 tx=0x7f86c00098c0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86d7200640 1 -- 172.21.15.49:0/1819269158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d01080e0 msgr2=0x7f86d019e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:26.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86d7200640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d01080e0 0x7f86d019e310 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86bc02c190 tx=0x7f86bc02c690 comp rx=0 tx=0).stop 2024-02-26T06:29:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86d7200640 1 -- 172.21.15.49:0/1819269158 shutdown_connections 2024-02-26T06:29:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.672+0000 7f86d7200640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f86b40a2330 0x7f86b40a47f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.673+0000 7f86d7200640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d01080e0 0x7f86d019e310 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.673+0000 7f86d7200640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d010d730 0x7f86d01a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.673+0000 7f86d7200640 1 --2- 172.21.15.49:0/1819269158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d0107720 0x7f86d019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.673+0000 7f86d7200640 1 -- 172.21.15.49:0/1819269158 >> 172.21.15.49:0/1819269158 conn(0x7f86d00756c0 msgr2=0x7f86d0075dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.673+0000 7f86d7200640 1 -- 172.21.15.49:0/1819269158 shutdown_connections 2024-02-26T06:29:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:26.673+0000 7f86d7200640 1 -- 172.21.15.49:0/1819269158 wait complete. 2024-02-26T06:29:26.696 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:26.696 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:29:27.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.061+0000 7fee8b200640 1 -- 172.21.15.49:0/523524385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840055d0 msgr2=0x7fee840a24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.061+0000 7fee8b200640 1 --2- 172.21.15.49:0/523524385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840055d0 0x7fee840a24c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009900 tx=0x7fee7c02bc40 comp rx=0 tx=0).stop 2024-02-26T06:29:27.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.062+0000 7fee8b200640 1 -- 172.21.15.49:0/523524385 shutdown_connections 2024-02-26T06:29:27.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.062+0000 7fee8b200640 1 --2- 172.21.15.49:0/523524385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee840a2a00 0x7fee840a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.062+0000 7fee8b200640 1 --2- 172.21.15.49:0/523524385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840055d0 0x7fee840a24c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.062+0000 7fee8b200640 1 --2- 172.21.15.49:0/523524385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee84004cb0 0x7fee84005090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.062+0000 7fee8b200640 1 -- 172.21.15.49:0/523524385 >> 172.21.15.49:0/523524385 conn(0x7fee8400a740 msgr2=0x7fee8400ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:27.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.062+0000 7fee8b200640 1 -- 172.21.15.49:0/523524385 shutdown_connections 2024-02-26T06:29:27.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.062+0000 7fee8b200640 1 -- 172.21.15.49:0/523524385 wait complete. 2024-02-26T06:29:27.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.063+0000 7fee8b200640 1 Processor -- start 2024-02-26T06:29:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.066+0000 7fee8b200640 1 -- start start 2024-02-26T06:29:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.067+0000 7fee8b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84004cb0 0x7fee8412e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.067+0000 7fee8b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee840055d0 0x7fee8412e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.068+0000 7fee89e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84004cb0 0x7fee8412e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.068+0000 7fee89e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84004cb0 0x7fee8412e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53282/0 (socket says 172.21.15.49:53282) 2024-02-26T06:29:27.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.068+0000 7fee89400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee840055d0 0x7fee8412e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.068+0000 7fee89400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee840055d0 0x7fee8412e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51426/0 (socket says 172.21.15.49:51426) 2024-02-26T06:29:27.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee8b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840a2a00 0x7fee84132cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee8b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee841332c0 con 0x7fee84004cb0 2024-02-26T06:29:27.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee8b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84133430 con 0x7fee840055d0 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee8b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee841335f0 con 0x7fee840a2a00 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee89e00640 1 -- 172.21.15.49:0/56708841 learned_addr learned my addr 172.21.15.49:0/56708841 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee8a800640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840a2a00 0x7fee84132cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee89400640 1 -- 172.21.15.49:0/56708841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840a2a00 msgr2=0x7fee84132cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee89400640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840a2a00 0x7fee84132cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee89400640 1 -- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84004cb0 msgr2=0x7fee8412e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.069+0000 7fee89400640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84004cb0 0x7fee8412e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.070+0000 7fee89400640 1 -- 172.21.15.49:0/56708841 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c009560 con 0x7fee840055d0 2024-02-26T06:29:27.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.070+0000 7fee8a800640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840a2a00 0x7fee84132cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:27.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.070+0000 7fee89400640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee840055d0 0x7fee8412e990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fee7c02c180 tx=0x7fee7c0040e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:27.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.071+0000 7fee82a00640 1 -- 172.21.15.49:0/56708841 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0044e0 con 0x7fee840055d0 2024-02-26T06:29:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.071+0000 7fee82a00640 1 -- 172.21.15.49:0/56708841 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee7c0052d0 con 0x7fee840055d0 2024-02-26T06:29:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.071+0000 7fee82a00640 1 -- 172.21.15.49:0/56708841 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003e20 con 0x7fee840055d0 2024-02-26T06:29:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.071+0000 7fee8b200640 1 -- 172.21.15.49:0/56708841 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee84133870 con 0x7fee840055d0 2024-02-26T06:29:27.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.072+0000 7fee8b200640 1 -- 172.21.15.49:0/56708841 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84097060 con 0x7fee840055d0 2024-02-26T06:29:27.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.079+0000 7fee82a00640 1 -- 172.21.15.49:0/56708841 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fee7c03e450 con 0x7fee840055d0 2024-02-26T06:29:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.087+0000 7fee82a00640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee700a2400 0x7fee700a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.087+0000 7fee82a00640 1 -- 172.21.15.49:0/56708841 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fee7c0f1800 con 0x7fee840055d0 2024-02-26T06:29:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.087+0000 7fee89e00640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee700a2400 0x7fee700a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.090+0000 7fee89e00640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee700a2400 0x7fee700a48c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fee78012fd0 tx=0x7fee78014040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.093+0000 7fee93fd1740 1 Processor -- start 2024-02-26T06:29:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.093+0000 7fee93fd1740 1 -- start start 2024-02-26T06:29:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e322151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e3221dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e322206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee93fd1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e321831e0 con 0x555e3221e2c0 2024-02-26T06:29:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee93fd1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e32180710 con 0x555e322156e0 2024-02-26T06:29:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee93fd1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3217bef0 con 0x555e32214dc0 2024-02-26T06:29:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee89e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e322151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee89400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e3221dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee89e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e322151a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49114/0 (socket says 172.21.15.49:49114) 2024-02-26T06:29:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.094+0000 7fee89e00640 1 -- 172.21.15.49:0/3651104781 learned_addr learned my addr 172.21.15.49:0/3651104781 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.095+0000 7fee89400640 1 -- 172.21.15.49:0/3651104781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 msgr2=0x555e322151a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:27.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.095+0000 7fee89400640 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e322151a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.095+0000 7fee89400640 1 -- 172.21.15.49:0/3651104781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 msgr2=0x555e322206b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:27.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.096+0000 7fee8a800640 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e322206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.096+0000 7fee89400640 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e322206b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.096+0000 7fee89400640 1 -- 172.21.15.49:0/3651104781 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e3213ddc0 con 0x555e322156e0 2024-02-26T06:29:27.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.096+0000 7fee8a800640 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e322206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:27.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.096+0000 7fee89400640 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e3221dd80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0ef010 tx=0x7fee7c0bdeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee52657d8dc4555f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:27.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.097+0000 7fee77400640 1 -- 172.21.15.49:0/3651104781 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0c2400 con 0x555e322156e0 2024-02-26T06:29:27.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.097+0000 7fee77400640 1 -- 172.21.15.49:0/3651104781 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee7c0c6a50 con 0x555e322156e0 2024-02-26T06:29:27.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.097+0000 7fee77400640 1 -- 172.21.15.49:0/3651104781 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0c2020 con 0x555e322156e0 2024-02-26T06:29:27.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.098+0000 7fee93fd1740 1 -- 172.21.15.49:0/3651104781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 msgr2=0x555e3221dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.099+0000 7fee93fd1740 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e3221dd80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0ef010 tx=0x7fee7c0bdeb0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.099+0000 7fee93fd1740 1 -- 172.21.15.49:0/3651104781 shutdown_connections 2024-02-26T06:29:27.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.099+0000 7fee93fd1740 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e3221dd80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.099+0000 7fee93fd1740 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e322206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.099+0000 7fee93fd1740 1 --2- 172.21.15.49:0/3651104781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e322151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.099+0000 7fee93fd1740 1 -- 172.21.15.49:0/3651104781 >> 172.21.15.49:0/3651104781 conn(0x555e3220f4f0 msgr2=0x555e3220f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:27.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.100+0000 7fee93fd1740 1 -- 172.21.15.49:0/3651104781 shutdown_connections 2024-02-26T06:29:27.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.100+0000 7fee93fd1740 1 -- 172.21.15.49:0/3651104781 wait complete. 2024-02-26T06:29:27.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.101+0000 7fee93fd1740 1 -- start start 2024-02-26T06:29:27.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.102+0000 7fee93fd1740 1 Processor -- start 2024-02-26T06:29:27.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.103+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e3221b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e3221b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e3221bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee93fd1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e32121120 con 0x555e322156e0 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee93fd1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e32120dc0 con 0x555e3221e2c0 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee93fd1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e31eea1c0 con 0x555e32214dc0 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee93fd1740 10 client.0 _ll_register_callbacks cb 0x555e32170280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee89e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e3221b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee89e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e3221b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49130/0 (socket says 172.21.15.49:49130) 2024-02-26T06:29:27.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.104+0000 7fee89e00640 1 -- 172.21.15.49:0/174486061 learned_addr learned my addr 172.21.15.49:0/174486061 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:27.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.105+0000 7fee89e00640 1 -- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 msgr2=0x555e3221bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:27.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.105+0000 7fee8a800640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e3221bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.105+0000 7fee89400640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e3221b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.105+0000 7fee89e00640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e3221bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.105+0000 7fee89e00640 1 -- 172.21.15.49:0/174486061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 msgr2=0x555e3221b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:27.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.105+0000 7fee89e00640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e3221b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.105+0000 7fee89e00640 1 -- 172.21.15.49:0/174486061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84097060 con 0x555e32214dc0 2024-02-26T06:29:27.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.106+0000 7fee8a800640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e3221bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:27.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.106+0000 7fee89400640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e3221b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:27.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.107+0000 7fee89e00640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e3221b2e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fee7801ef00 tx=0x7fee780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed21cc28d7591a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:27.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.107+0000 7fee6be00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:27.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.108+0000 7fee6be00640 1 -- 172.21.15.49:0/174486061 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee780104e0 con 0x555e32214dc0 2024-02-26T06:29:27.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.108+0000 7fee6be00640 1 -- 172.21.15.49:0/174486061 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fee78010640 con 0x555e32214dc0 2024-02-26T06:29:27.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.108+0000 7fee93fd1740 1 -- 172.21.15.49:0/174486061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e31e7b7e0 con 0x555e32214dc0 2024-02-26T06:29:27.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.109+0000 7fee6be00640 1 -- 172.21.15.49:0/174486061 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee7801ce90 con 0x555e32214dc0 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.109+0000 7fee93fd1740 10 client.26947 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.109+0000 7fee93fd1740 1 -- 172.21.15.49:0/174486061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x555e3217a570 con 0x555e32214dc0 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.110+0000 7fee6be00640 1 -- 172.21.15.49:0/174486061 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555e31e7b7e0 con 0x555e32214dc0 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.110+0000 7fee6be00640 1 -- 172.21.15.49:0/174486061 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fee7800b050 con 0x555e32214dc0 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.110+0000 7fee93fd1740 10 client.26947 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.110+0000 7fee93fd1740 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.110+0000 7fee93fd1740 20 client.26947 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.110+0000 7fee93fd1740 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555e3223f0a0 0x555e322414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.110+0000 7fee93fd1740 4 client.26947 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.110+0000 7fee93fd1740 1 -- 172.21.15.49:0/174486061 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e31eb8720 con 0x555e3223f0a0 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.111+0000 7fee89400640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555e3223f0a0 0x555e322414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.112+0000 7fee89400640 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555e3223f0a0 0x555e322414c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c4d7c81b551223 server_cookie=7a5565080f64120a in_seq=0 out_seq=0 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.112+0000 7fee6be00640 10 client.26947 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.113+0000 7fee6be00640 1 -- 172.21.15.49:0/174486061 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e31eb8720 con 0x555e3223f0a0 2024-02-26T06:29:27.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.113+0000 7fee6be00640 10 client.26947 handle_command_reply: tid=0 2024-02-26T06:29:27.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.113+0000 7fee93fd1740 1 client.26947 shutdown 2024-02-26T06:29:27.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee93fd1740 1 -- 172.21.15.49:0/174486061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 msgr2=0x555e3221b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee93fd1740 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e3221b2e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fee7801ef00 tx=0x7fee780046c0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee93fd1740 1 -- 172.21.15.49:0/174486061 shutdown_connections 2024-02-26T06:29:27.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee93fd1740 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555e3223f0a0 0x555e322414c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee93fd1740 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e3221bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee93fd1740 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e3221b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee93fd1740 1 --2- 172.21.15.49:0/174486061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e3221b2e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee93fd1740 1 -- 172.21.15.49:0/174486061 >> 172.21.15.49:0/174486061 conn(0x555e3220f4f0 msgr2=0x555e32211240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:27.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.114+0000 7fee6be00640 0 client.26947 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:27.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.115+0000 7fee93fd1740 1 -- 172.21.15.49:0/174486061 shutdown_connections 2024-02-26T06:29:27.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.115+0000 7fee93fd1740 1 -- 172.21.15.49:0/174486061 wait complete. 2024-02-26T06:29:27.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.115+0000 7fee93fd1740 20 client.26947 trim_cache size 0 max 16384 2024-02-26T06:29:27.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.137+0000 7fee93fd1740 1 Processor -- start 2024-02-26T06:29:27.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.138+0000 7fee93fd1740 1 -- start start 2024-02-26T06:29:27.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.138+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32216600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.138+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e32216b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.138+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e3221b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.138+0000 7fee93fd1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e322103c0 con 0x555e3221e2c0 2024-02-26T06:29:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.138+0000 7fee93fd1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3222aad0 con 0x555e322156e0 2024-02-26T06:29:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.138+0000 7fee93fd1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e322100c0 con 0x555e32214dc0 2024-02-26T06:29:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee8a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e3221b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e32216b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32216600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e32216b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51446/0 (socket says 172.21.15.49:51446) 2024-02-26T06:29:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89400640 1 -- 172.21.15.49:0/1636323946 learned_addr learned my addr 172.21.15.49:0/1636323946 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89400640 1 -- 172.21.15.49:0/1636323946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 msgr2=0x555e32216600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89400640 1 --2- 172.21.15.49:0/1636323946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32216600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89400640 1 -- 172.21.15.49:0/1636323946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 msgr2=0x555e3221b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89400640 1 --2- 172.21.15.49:0/1636323946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e3221b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.139+0000 7fee89400640 1 -- 172.21.15.49:0/1636323946 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e32116a80 con 0x555e322156e0 2024-02-26T06:29:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.140+0000 7fee89400640 1 --2- 172.21.15.49:0/1636323946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e32216b40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0bdfb0 tx=0x7fee7c003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f70c2e60a561519 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.141+0000 7fee76a00640 1 -- 172.21.15.49:0/1636323946 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c005440 con 0x555e322156e0 2024-02-26T06:29:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.141+0000 7fee76a00640 1 -- 172.21.15.49:0/1636323946 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555e31eb8720 con 0x555e322156e0 2024-02-26T06:29:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.141+0000 7fee76a00640 1 -- 172.21.15.49:0/1636323946 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0ef180 con 0x555e322156e0 2024-02-26T06:29:27.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.142+0000 7fee93fd1740 1 -- 172.21.15.49:0/1636323946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 msgr2=0x555e32216b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.142+0000 7fee93fd1740 1 --2- 172.21.15.49:0/1636323946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e32216b40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0bdfb0 tx=0x7fee7c003920 comp rx=0 tx=0).stop 2024-02-26T06:29:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.142+0000 7fee93fd1740 1 -- 172.21.15.49:0/1636323946 shutdown_connections 2024-02-26T06:29:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.142+0000 7fee93fd1740 1 --2- 172.21.15.49:0/1636323946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e322156e0 0x555e32216b40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.142+0000 7fee93fd1740 1 --2- 172.21.15.49:0/1636323946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e3221e2c0 0x555e3221b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.142+0000 7fee93fd1740 1 --2- 172.21.15.49:0/1636323946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32216600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.142+0000 7fee93fd1740 1 -- 172.21.15.49:0/1636323946 >> 172.21.15.49:0/1636323946 conn(0x555e3220f4f0 msgr2=0x555e3220a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:27.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.143+0000 7fee93fd1740 1 -- 172.21.15.49:0/1636323946 shutdown_connections 2024-02-26T06:29:27.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.143+0000 7fee93fd1740 1 -- 172.21.15.49:0/1636323946 wait complete. 2024-02-26T06:29:27.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 1 -- start start 2024-02-26T06:29:27.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 1 Processor -- start 2024-02-26T06:29:27.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32225040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e322233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e32223920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e32223e60 con 0x555e322156e0 2024-02-26T06:29:27.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e32223fd0 con 0x555e3221e2c0 2024-02-26T06:29:27.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e32224190 con 0x555e32214dc0 2024-02-26T06:29:27.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.144+0000 7fee93fd1740 10 client.0 _ll_register_callbacks cb 0x7fee7c005720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:27.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.145+0000 7fee89e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32225040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.146+0000 7fee8a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e32223920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.146+0000 7fee8a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e32223920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51456/0 (socket says 172.21.15.49:51456) 2024-02-26T06:29:27.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.146+0000 7fee8a800640 1 -- 172.21.15.49:0/4067607176 learned_addr learned my addr 172.21.15.49:0/4067607176 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:27.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.146+0000 7fee89400640 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e322233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.147+0000 7fee8a800640 1 -- 172.21.15.49:0/4067607176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 msgr2=0x555e32225040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.147+0000 7fee8a800640 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32225040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.147+0000 7fee8a800640 1 -- 172.21.15.49:0/4067607176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 msgr2=0x555e322233e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.147+0000 7fee8a800640 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e322233e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.147+0000 7fee8a800640 1 -- 172.21.15.49:0/4067607176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e3213ddc0 con 0x555e3221e2c0 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.147+0000 7fee89e00640 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32225040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.147+0000 7fee89400640 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e322233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.147+0000 7fee8a800640 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e32223920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0668b0 tx=0x7fee8c074bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ac003691d65959 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:27.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.148+0000 7fee6be00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:29:27.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.148+0000 7fee6be00640 1 -- 172.21.15.49:0/4067607176 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c071780 con 0x555e3221e2c0 2024-02-26T06:29:27.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.148+0000 7fee6be00640 1 -- 172.21.15.49:0/4067607176 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fee8c0718e0 con 0x555e3221e2c0 2024-02-26T06:29:27.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.148+0000 7fee93fd1740 1 -- 172.21.15.49:0/4067607176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee7c0c6a50 con 0x555e3221e2c0 2024-02-26T06:29:27.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.149+0000 7fee6be00640 1 -- 172.21.15.49:0/4067607176 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8c071ac0 con 0x555e3221e2c0 2024-02-26T06:29:27.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.149+0000 7fee93fd1740 10 client.26960 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:27.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.149+0000 7fee93fd1740 1 -- 172.21.15.49:0/4067607176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x555e3213c620 con 0x555e3221e2c0 2024-02-26T06:29:27.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.149+0000 7fee6be00640 1 -- 172.21.15.49:0/4067607176 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee8c06f050 con 0x555e3221e2c0 2024-02-26T06:29:27.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.150+0000 7fee6be00640 1 -- 172.21.15.49:0/4067607176 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fee8c086070 con 0x555e3221e2c0 2024-02-26T06:29:27.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.150+0000 7fee93fd1740 10 client.26960 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:27.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.150+0000 7fee93fd1740 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.150+0000 7fee93fd1740 20 client.26960 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.150+0000 7fee93fd1740 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555e32290ba0 0x555e32290fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.150+0000 7fee93fd1740 4 client.26960 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.150+0000 7fee93fd1740 1 -- 172.21.15.49:0/4067607176 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fee7c0c6a50 con 0x555e32290ba0 2024-02-26T06:29:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.150+0000 7fee89e00640 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555e32290ba0 0x555e32290fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.151+0000 7fee89e00640 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555e32290ba0 0x555e32290fb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11882176ba79e509 server_cookie=d936caf0d3b36bf6 in_seq=0 out_seq=0 2024-02-26T06:29:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.151+0000 7fee6be00640 10 client.26960 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.152+0000 7fee6be00640 1 -- 172.21.15.49:0/4067607176 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fee7c0c6a50 con 0x555e32290ba0 2024-02-26T06:29:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.152+0000 7fee6be00640 10 client.26960 handle_command_reply: tid=0 2024-02-26T06:29:27.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.152+0000 7fee93fd1740 1 client.26960 shutdown 2024-02-26T06:29:27.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.152+0000 7fee93fd1740 1 -- 172.21.15.49:0/4067607176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 msgr2=0x555e32223920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.152+0000 7fee93fd1740 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e32223920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0668b0 tx=0x7fee8c074bf0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.152+0000 7fee93fd1740 1 -- 172.21.15.49:0/4067607176 shutdown_connections 2024-02-26T06:29:27.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.152+0000 7fee93fd1740 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555e32290ba0 0x555e32290fb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.153+0000 7fee93fd1740 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555e3221e2c0 0x555e32223920 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.153+0000 7fee93fd1740 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e322156e0 0x555e322233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.153+0000 7fee93fd1740 1 --2- 172.21.15.49:0/4067607176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e32214dc0 0x555e32225040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.153+0000 7fee93fd1740 1 -- 172.21.15.49:0/4067607176 >> 172.21.15.49:0/4067607176 conn(0x555e3220f4f0 msgr2=0x555e32219780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:27.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.153+0000 7fee6be00640 0 client.26960 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:27.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.153+0000 7fee93fd1740 1 -- 172.21.15.49:0/4067607176 shutdown_connections 2024-02-26T06:29:27.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.153+0000 7fee93fd1740 1 -- 172.21.15.49:0/4067607176 wait complete. 2024-02-26T06:29:27.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.153+0000 7fee93fd1740 20 client.26960 trim_cache size 0 max 16384 2024-02-26T06:29:27.154 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 -- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee700a2400 msgr2=0x7fee700a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee700a2400 0x7fee700a48c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fee78012fd0 tx=0x7fee78014040 comp rx=0 tx=0).stop 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 -- 172.21.15.49:0/56708841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee840055d0 msgr2=0x7fee8412e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee840055d0 0x7fee8412e990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fee7c02c180 tx=0x7fee7c0040e0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 -- 172.21.15.49:0/56708841 shutdown_connections 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fee700a2400 0x7fee700a48c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee840055d0 0x7fee8412e990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee840a2a00 0x7fee84132cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 --2- 172.21.15.49:0/56708841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84004cb0 0x7fee8412e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.155+0000 7fee8b200640 1 -- 172.21.15.49:0/56708841 >> 172.21.15.49:0/56708841 conn(0x7fee8400a740 msgr2=0x7fee840a3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:27.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.156+0000 7fee8b200640 1 -- 172.21.15.49:0/56708841 shutdown_connections 2024-02-26T06:29:27.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:27.156+0000 7fee8b200640 1 -- 172.21.15.49:0/56708841 wait complete. 2024-02-26T06:29:27.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:26 smithi049 ceph-mon[31696]: pgmap v184: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 892 op/s 2024-02-26T06:29:27.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:26 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:27.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:26 smithi080 ceph-mon[36206]: pgmap v184: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 892 op/s 2024-02-26T06:29:27.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:26 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:26 smithi115 ceph-mon[36426]: pgmap v184: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 892 op/s 2024-02-26T06:29:27.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:26 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:28.184 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:29:28.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:29:28.184 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:29:28.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.484+0000 7f2a93e00640 1 -- 172.21.15.49:0/1522422954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a94108210 msgr2=0x7f2a94108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.484+0000 7f2a93e00640 1 --2- 172.21.15.49:0/1522422954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a94108210 0x7f2a94108690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009930 tx=0x7f2a7c02bc20 comp rx=0 tx=0).stop 2024-02-26T06:29:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.485+0000 7f2a93e00640 1 -- 172.21.15.49:0/1522422954 shutdown_connections 2024-02-26T06:29:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.485+0000 7f2a93e00640 1 --2- 172.21.15.49:0/1522422954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9411a960 0x7f2a9411cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.485+0000 7f2a93e00640 1 --2- 172.21.15.49:0/1522422954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a94108210 0x7f2a94108690 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.485+0000 7f2a93e00640 1 --2- 172.21.15.49:0/1522422954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a9410f7a0 0x7f2a9410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.485+0000 7f2a93e00640 1 -- 172.21.15.49:0/1522422954 >> 172.21.15.49:0/1522422954 conn(0x7f2a9406c0d0 msgr2=0x7f2a9406c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.487+0000 7f2a93e00640 1 -- 172.21.15.49:0/1522422954 shutdown_connections 2024-02-26T06:29:28.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.491+0000 7f2a93e00640 1 -- 172.21.15.49:0/1522422954 wait complete. 2024-02-26T06:29:28.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.492+0000 7f2a93e00640 1 Processor -- start 2024-02-26T06:29:28.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.493+0000 7f2a93e00640 1 -- start start 2024-02-26T06:29:28.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a94108210 0x7f2a94119d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a9410f7a0 0x7f2a9411a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9411a960 0x7f2a94112e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a93e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94113610 con 0x7f2a9410f7a0 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a93e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94113780 con 0x7f2a94108210 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a93e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94113940 con 0x7f2a9411a960 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a92a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a94108210 0x7f2a94119d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a92a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a94108210 0x7f2a94119d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51480/0 (socket says 172.21.15.49:51480) 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.494+0000 7f2a92a00640 1 -- 172.21.15.49:0/730583433 learned_addr learned my addr 172.21.15.49:0/730583433 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.495+0000 7f2a92a00640 1 -- 172.21.15.49:0/730583433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9411a960 msgr2=0x7f2a94112e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.495+0000 7f2a92a00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9411a960 0x7f2a94112e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.495+0000 7f2a92a00640 1 -- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a9410f7a0 msgr2=0x7f2a9411a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.495+0000 7f2a92000640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a9410f7a0 0x7f2a9411a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.495+0000 7f2a92a00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a9410f7a0 0x7f2a9411a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.495+0000 7f2a92a00640 1 -- 172.21.15.49:0/730583433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c009560 con 0x7f2a94108210 2024-02-26T06:29:28.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.496+0000 7f2a92000640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a9410f7a0 0x7f2a9411a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:28.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.497+0000 7f2a92a00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a94108210 0x7f2a94119d80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c920 tx=0x7f2a8c00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.497+0000 7f2a8b400640 1 -- 172.21.15.49:0/730583433 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c007c70 con 0x7f2a94108210 2024-02-26T06:29:28.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.497+0000 7f2a8b400640 1 -- 172.21.15.49:0/730583433 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c007dd0 con 0x7f2a94108210 2024-02-26T06:29:28.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.497+0000 7f2a8b400640 1 -- 172.21.15.49:0/730583433 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c018860 con 0x7f2a94108210 2024-02-26T06:29:28.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.497+0000 7f2a93e00640 1 -- 172.21.15.49:0/730583433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a94113c20 con 0x7f2a94108210 2024-02-26T06:29:28.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.498+0000 7f2a93e00640 1 -- 172.21.15.49:0/730583433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a94117b70 con 0x7f2a94108210 2024-02-26T06:29:28.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.505+0000 7f2a9c9c3740 1 Processor -- start 2024-02-26T06:29:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.506+0000 7f2a9c9c3740 1 -- start start 2024-02-26T06:29:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.506+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.506+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.506+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x5635739d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.506+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635739872d0 con 0x5635739cc0e0 2024-02-26T06:29:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.506+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563573939e20 con 0x5635739d55e0 2024-02-26T06:29:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.506+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563573937350 con 0x5635739cca00 2024-02-26T06:29:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.507+0000 7f2a92000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x5635739d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.507+0000 7f2a92000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x5635739d79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51490/0 (socket says 172.21.15.49:51490) 2024-02-26T06:29:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.507+0000 7f2a92000640 1 -- 172.21.15.49:0/2478782379 learned_addr learned my addr 172.21.15.49:0/2478782379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.507+0000 7f2a93400640 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.508+0000 7f2a8b400640 1 -- 172.21.15.49:0/730583433 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2a8c00fac0 con 0x7f2a94108210 2024-02-26T06:29:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.508+0000 7f2a93400640 1 -- 172.21.15.49:0/2478782379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 msgr2=0x5635739d50a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.508+0000 7f2a92a00640 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.508+0000 7f2a93400640 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739d50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.508+0000 7f2a93400640 1 -- 172.21.15.49:0/2478782379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 msgr2=0x5635739d79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.508+0000 7f2a93400640 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x5635739d79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.508+0000 7f2a93400640 1 -- 172.21.15.49:0/2478782379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635738fbc50 con 0x5635739cc0e0 2024-02-26T06:29:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.508+0000 7f2a92000640 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x5635739d79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.509+0000 7f2a92a00640 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.510+0000 7f2a93400640 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739cc4c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009c90 tx=0x7f2a8400d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a91f491fe591dc3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.510+0000 7f2a83e00640 1 -- 172.21.15.49:0/2478782379 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84003970 con 0x5635739cc0e0 2024-02-26T06:29:28.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.511+0000 7f2a8b400640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2a780a2230 0x7f2a780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.511+0000 7f2a83e00640 1 -- 172.21.15.49:0/2478782379 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a84009770 con 0x5635739cc0e0 2024-02-26T06:29:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.511+0000 7f2a83e00640 1 -- 172.21.15.49:0/2478782379 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84010790 con 0x5635739cc0e0 2024-02-26T06:29:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.512+0000 7f2a8b400640 1 -- 172.21.15.49:0/730583433 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2a8c0ce290 con 0x7f2a94108210 2024-02-26T06:29:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.512+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/2478782379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 msgr2=0x5635739cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.512+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739cc4c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009c90 tx=0x7f2a8400d6b0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.512+0000 7f2a92a00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2a780a2230 0x7f2a780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.513+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/2478782379 shutdown_connections 2024-02-26T06:29:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.513+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x5635739d79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.513+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.513+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/2478782379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739cc4c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.513+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/2478782379 >> 172.21.15.49:0/2478782379 conn(0x5635739c6810 msgr2=0x5635739c6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.513+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/2478782379 shutdown_connections 2024-02-26T06:29:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.514+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/2478782379 wait complete. 2024-02-26T06:29:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.514+0000 7f2a92a00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2a780a2230 0x7f2a780a46f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0cbaa0 tx=0x7f2a8c000bc0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.515+0000 7f2a9c9c3740 1 -- start start 2024-02-26T06:29:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.515+0000 7f2a9c9c3740 1 Processor -- start 2024-02-26T06:29:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.516+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.516+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cca00 0x5635739d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.516+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.516+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563573932b30 con 0x5635739cc0e0 2024-02-26T06:29:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.516+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635738d7fa0 con 0x5635739cca00 2024-02-26T06:29:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.516+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635738d7c40 con 0x5635739d55e0 2024-02-26T06:29:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.516+0000 7f2a9c9c3740 10 client.0 _ll_register_callbacks cb 0x563573643c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.517+0000 7f2a93400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.517+0000 7f2a93400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739d3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49186/0 (socket says 172.21.15.49:49186) 2024-02-26T06:29:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.517+0000 7f2a93400640 1 -- 172.21.15.49:0/3488491251 learned_addr learned my addr 172.21.15.49:0/3488491251 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.517+0000 7f2a92000640 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cca00 0x5635739d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.519+0000 7f2a93400640 1 -- 172.21.15.49:0/3488491251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cca00 msgr2=0x5635739d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.519+0000 7f2a93400640 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cca00 0x5635739d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.519+0000 7f2a93400640 1 -- 172.21.15.49:0/3488491251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 msgr2=0x5635739d26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.519+0000 7f2a93400640 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.519+0000 7f2a93400640 1 -- 172.21.15.49:0/3488491251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a840098f0 con 0x5635739d55e0 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.519+0000 7f2a93400640 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739d3130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a84004770 tx=0x7f2a84009440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd1e2b7c5afcbaca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.519+0000 7f2a80c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.520+0000 7f2a80c00640 1 -- 172.21.15.49:0/3488491251 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84004d20 con 0x5635739d55e0 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.520+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3488491251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563573683410 con 0x5635739d55e0 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.520+0000 7f2a80c00640 1 -- 172.21.15.49:0/3488491251 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2a84005020 con 0x5635739d55e0 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.520+0000 7f2a80c00640 1 -- 172.21.15.49:0/3488491251 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a84004e80 con 0x5635739d55e0 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.520+0000 7f2a9c9c3740 10 client.26959 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.520+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3488491251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5635739311b0 con 0x5635739d55e0 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.520+0000 7f2a80c00640 1 -- 172.21.15.49:0/3488491251 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a84031080 con 0x5635739d55e0 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.521+0000 7f2a80c00640 1 -- 172.21.15.49:0/3488491251 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2a84019450 con 0x5635739d55e0 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.521+0000 7f2a9c9c3740 10 client.26959 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.521+0000 7f2a9c9c3740 10 client.26959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.521+0000 7f2a9c9c3740 20 client.26959 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.521+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5635739e5980 0x5635739e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.521+0000 7f2a9c9c3740 4 client.26959 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.521+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3488491251 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56357366f7b0 con 0x5635739e5980 2024-02-26T06:29:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.521+0000 7f2a92000640 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5635739e5980 0x5635739e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.523+0000 7f2a92000640 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5635739e5980 0x5635739e7da0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03e08ef3084aaa6 server_cookie=29ca1c6680f5574b in_seq=0 out_seq=0 2024-02-26T06:29:28.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.524+0000 7f2a80c00640 10 client.26959 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:28.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a80c00640 1 -- 172.21.15.49:0/3488491251 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56357366f7b0 con 0x5635739e5980 2024-02-26T06:29:28.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a80c00640 10 client.26959 handle_command_reply: tid=0 2024-02-26T06:29:28.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 client.26959 shutdown 2024-02-26T06:29:28.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3488491251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 msgr2=0x5635739d3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739d3130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a84004770 tx=0x7f2a84009440 comp rx=0 tx=0).stop 2024-02-26T06:29:28.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3488491251 shutdown_connections 2024-02-26T06:29:28.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5635739e5980 0x5635739e7da0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cca00 0x5635739d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739d3130 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3488491251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739d26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3488491251 >> 172.21.15.49:0/3488491251 conn(0x5635739c6810 msgr2=0x5635739c8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.525+0000 7f2a80c00640 0 client.26959 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:28.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.526+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3488491251 shutdown_connections 2024-02-26T06:29:28.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.526+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3488491251 wait complete. 2024-02-26T06:29:28.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.526+0000 7f2a9c9c3740 20 client.26959 trim_cache size 0 max 16384 2024-02-26T06:29:28.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.565+0000 7f2a9c9c3740 1 Processor -- start 2024-02-26T06:29:28.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.565+0000 7f2a9c9c3740 1 -- start start 2024-02-26T06:29:28.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cc0e0 0x5635739d4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cca00 0x5635739d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357367b0c0 con 0x5635739cca00 2024-02-26T06:29:28.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635739cbb20 con 0x5635739cc0e0 2024-02-26T06:29:28.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635739c60d0 con 0x5635739d55e0 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a93400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cc0e0 0x5635739d4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a93400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cc0e0 0x5635739d4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51504/0 (socket says 172.21.15.49:51504) 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a93400640 1 -- 172.21.15.49:0/756622274 learned_addr learned my addr 172.21.15.49:0/756622274 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a93400640 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.566+0000 7f2a92000640 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cca00 0x5635739d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a93400640 1 -- 172.21.15.49:0/756622274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 msgr2=0x5635739cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a93400640 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a93400640 1 -- 172.21.15.49:0/756622274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cca00 msgr2=0x5635739d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a93400640 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cca00 0x5635739d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a93400640 1 -- 172.21.15.49:0/756622274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635738cda60 con 0x5635739cc0e0 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a93400640 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cc0e0 0x5635739d4de0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2a84019ad0 tx=0x7f2a8401f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f2c4e518ee4b676 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a83400640 1 -- 172.21.15.49:0/756622274 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84019bd0 con 0x5635739cc0e0 2024-02-26T06:29:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a83400640 1 -- 172.21.15.49:0/756622274 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8401e070 con 0x5635739cc0e0 2024-02-26T06:29:28.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.567+0000 7f2a83400640 1 -- 172.21.15.49:0/756622274 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84005020 con 0x5635739cc0e0 2024-02-26T06:29:28.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.569+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/756622274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cc0e0 msgr2=0x5635739d4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.569+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cc0e0 0x5635739d4de0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2a84019ad0 tx=0x7f2a8401f040 comp rx=0 tx=0).stop 2024-02-26T06:29:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.570+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/756622274 shutdown_connections 2024-02-26T06:29:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.570+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739d55e0 0x5635739cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.570+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cca00 0x5635739d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.570+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/756622274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739cc0e0 0x5635739d4de0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.570+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/756622274 >> 172.21.15.49:0/756622274 conn(0x5635739c6810 msgr2=0x5635739d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.570+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/756622274 shutdown_connections 2024-02-26T06:29:28.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.570+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/756622274 wait complete. 2024-02-26T06:29:28.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.571+0000 7f2a9c9c3740 1 -- start start 2024-02-26T06:29:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.571+0000 7f2a9c9c3740 1 Processor -- start 2024-02-26T06:29:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.572+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739db240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.572+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739db780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.572+0000 7f2a9c9c3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x563573a3be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.572+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635739dc050 con 0x5635739cc0e0 2024-02-26T06:29:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.572+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563573a3c3d0 con 0x5635739d55e0 2024-02-26T06:29:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.572+0000 7f2a9c9c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563573a3c510 con 0x5635739cca00 2024-02-26T06:29:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.572+0000 7f2a9c9c3740 10 client.0 _ll_register_callbacks cb 0x7f2a840092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a93400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x563573a3be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a92000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739db780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a92000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739db780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49206/0 (socket says 172.21.15.49:49206) 2024-02-26T06:29:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a92000640 1 -- 172.21.15.49:0/3322660156 learned_addr learned my addr 172.21.15.49:0/3322660156 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a93400640 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a92000640 1 -- 172.21.15.49:0/3322660156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 msgr2=0x563573a3be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a92000640 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x563573a3be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a92000640 1 -- 172.21.15.49:0/3322660156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 msgr2=0x5635739db240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a92000640 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739db240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.573+0000 7f2a92000640 1 -- 172.21.15.49:0/3322660156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635739311b0 con 0x5635739cca00 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.574+0000 7f2a93400640 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.574+0000 7f2a92000640 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739db780 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c02c770 tx=0x7f2a7c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a28d7fa5654aeee server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.574+0000 7f2a80c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.574+0000 7f2a80c00640 1 -- 172.21.15.49:0/3322660156 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c004040 con 0x5635739cca00 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.574+0000 7f2a80c00640 1 -- 172.21.15.49:0/3322660156 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2a7c047070 con 0x5635739cca00 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.574+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3322660156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563573632860 con 0x5635739cca00 2024-02-26T06:29:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.575+0000 7f2a80c00640 1 -- 172.21.15.49:0/3322660156 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a7c0041a0 con 0x5635739cca00 2024-02-26T06:29:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.575+0000 7f2a9c9c3740 10 client.26965 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.575+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3322660156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5635738f3670 con 0x5635739cca00 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.575+0000 7f2a80c00640 1 -- 172.21.15.49:0/3322660156 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563573632860 con 0x5635739cca00 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.576+0000 7f2a80c00640 1 -- 172.21.15.49:0/3322660156 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2a7c052070 con 0x5635739cca00 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.576+0000 7f2a9c9c3740 10 client.26965 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.576+0000 7f2a9c9c3740 10 client.26965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.576+0000 7f2a9c9c3740 20 client.26965 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.576+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563573a3dc80 0x563573a400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.577+0000 7f2a9c9c3740 4 client.26965 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.577+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3322660156 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5635735de990 con 0x563573a3dc80 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.577+0000 7f2a93400640 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563573a3dc80 0x563573a400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.578+0000 7f2a93400640 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563573a3dc80 0x563573a400a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf1aa62520f1c1f server_cookie=b4f7aa670bccbd97 in_seq=0 out_seq=0 2024-02-26T06:29:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.578+0000 7f2a80c00640 10 client.26965 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.579+0000 7f2a80c00640 1 -- 172.21.15.49:0/3322660156 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5635735de990 con 0x563573a3dc80 2024-02-26T06:29:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.579+0000 7f2a80c00640 10 client.26965 handle_command_reply: tid=0 2024-02-26T06:29:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.579+0000 7f2a9c9c3740 1 client.26965 shutdown 2024-02-26T06:29:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3322660156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 msgr2=0x5635739db780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739db780 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c02c770 tx=0x7f2a7c0046c0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3322660156 shutdown_connections 2024-02-26T06:29:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x563573a3dc80 0x563573a400a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5635739d55e0 0x563573a3be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635739cca00 0x5635739db780 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 --2- 172.21.15.49:0/3322660156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635739cc0e0 0x5635739db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3322660156 >> 172.21.15.49:0/3322660156 conn(0x5635739c6810 msgr2=0x5635739d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a80c00640 0 client.26965 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.580+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3322660156 shutdown_connections 2024-02-26T06:29:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.581+0000 7f2a9c9c3740 1 -- 172.21.15.49:0/3322660156 wait complete. 2024-02-26T06:29:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.581+0000 7f2a9c9c3740 20 client.26965 trim_cache size 0 max 16384 2024-02-26T06:29:28.581 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:29:28.581 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2024-02-26T06:29:28.581 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "a439ddc6-10fc-4cb4-b884-f55c883fa0a9", 2024-02-26T06:29:28.581 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2024-02-26T06:29:28.582 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:29:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.583+0000 7f2a93e00640 1 -- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2a780a2230 msgr2=0x7f2a780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.583+0000 7f2a93e00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2a780a2230 0x7f2a780a46f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0cbaa0 tx=0x7f2a8c000bc0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.583+0000 7f2a93e00640 1 -- 172.21.15.49:0/730583433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a94108210 msgr2=0x7f2a94119d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.583+0000 7f2a93e00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a94108210 0x7f2a94119d80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c920 tx=0x7f2a8c00cde0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.584+0000 7f2a93e00640 1 -- 172.21.15.49:0/730583433 shutdown_connections 2024-02-26T06:29:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.584+0000 7f2a93e00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2a780a2230 0x7f2a780a46f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.584+0000 7f2a93e00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9411a960 0x7f2a94112e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.584+0000 7f2a93e00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a9410f7a0 0x7f2a9411a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.584+0000 7f2a93e00640 1 --2- 172.21.15.49:0/730583433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a94108210 0x7f2a94119d80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.584+0000 7f2a93e00640 1 -- 172.21.15.49:0/730583433 >> 172.21.15.49:0/730583433 conn(0x7f2a9406c0d0 msgr2=0x7f2a9411ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.584+0000 7f2a93e00640 1 -- 172.21.15.49:0/730583433 shutdown_connections 2024-02-26T06:29:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.585+0000 7f2a93e00640 1 -- 172.21.15.49:0/730583433 wait complete. 2024-02-26T06:29:28.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:29:28.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.587+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x30000000230 [7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000034 auth{1=2} v1395 ap=1 s=16777216 n(v0 rc2024-02-26T06:28:25.376763+0000 b16777216 1=1+0) (iversion lock) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=0 remoteparent=0 freezing=0 frozen=0 exportingcaps=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 scrubqueue=0 0x560db746a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x30000000230:[<0x10000000007/000000010000000000000034 v1395>,<0x10000000006/pg_wal v1611>,<0x10000000005/data v503>,<0x100000001fb/postgres v257>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v271>,<0x10000000001/sv_1 v293>,<0x10000000000/qa v273>,<0x1/volumes v259>]//[]","memoryvalue":"(3)0x30000000230:[<0x10000000007/000000010000000000000034 v1395>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.587+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000212 [7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000030 auth{1=1} v1432 ap=1 s=16777216 n(v0 rc2024-02-26T06:28:20.338901+0000 b16777216 1=1+0) (iversion lock) | replicated=1 authpin=1 scrubqueue=0 0x560db7153180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000212:[<0x10000000007/000000010000000000000030 v1314>,<0x10000000006/pg_wal v1398>,<0x10000000005/data v445>,<0x100000001fb/postgres v227>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v247>,<0x10000000001/sv_1 v268>,<0x10000000000/qa v251>,<0x1/volumes v237>]//[]","memoryvalue":"(3)0x20000000212:[<0x10000000007/000000010000000000000030 v1432>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.587+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000020c [7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002B auth{1=1} v1398 ap=1 s=16777216 n(v0 rc2024-02-26T06:28:13.851079+0000 b16777216 1=1+0) (iversion lock) caps={25193=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x560db6ffbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000020c:[<0x10000000007/00000001000000000000002B v1281>,<0x10000000006/pg_wal v1398>,<0x10000000005/data v445>,<0x100000001fb/postgres v227>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v247>,<0x10000000001/sv_1 v268>,<0x10000000000/qa v251>,<0x1/volumes v237>]//[]","memoryvalue":"(3)0x2000000020c:[<0x10000000007/00000001000000000000002B v1398>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.587+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000020f [7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002D auth{1=1} v1434 ap=1 s=16777216 n(v0 rc2024-02-26T06:28:16.195014+0000 b16777216 1=1+0) (iversion lock) caps={25193=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x560db7353180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000020f:[<0x10000000007/00000001000000000000002D v1289>,<0x10000000006/pg_wal v1398>,<0x10000000005/data v445>,<0x100000001fb/postgres v227>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v247>,<0x10000000001/sv_1 v268>,<0x10000000000/qa v251>,<0x1/volumes v237>]//[]","memoryvalue":"(3)0x2000000020f:[<0x10000000007/00000001000000000000002D v1434>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.588+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000211 [7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002F auth{1=1} v1433 ap=1 s=16777216 n(v0 rc2024-02-26T06:28:18.857942+0000 b16777216 1=1+0) (iversion lock) | replicated=1 authpin=1 scrubqueue=0 0x560db6f65600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000211:[<0x10000000007/00000001000000000000002F v1310>,<0x10000000006/pg_wal v1398>,<0x10000000005/data v445>,<0x100000001fb/postgres v227>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v247>,<0x10000000001/sv_1 v268>,<0x10000000000/qa v251>,<0x1/volumes v237>]//[]","memoryvalue":"(3)0x20000000211:[<0x10000000007/00000001000000000000002F v1433>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.588+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e5 [...7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000053 auth v1453 ap=1 snaprealm=0x560db6f97d40 s=16777216 n(v0 rc2024-02-26T06:28:48.617126+0000 b16777216 1=1+0) (iversion lock) caps={25193=pAsLsXsFs/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x560db7441600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005e5:[<0x10000000007/000000010000000000000053 v1399>,<0x10000000006/pg_wal v1611>,<0x10000000005/data v503>,<0x100000001fb/postgres v257>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v271>,<0x10000000001/sv_1 v293>,<0x10000000000/qa v273>,<0x1/volumes v259>]//[]","memoryvalue":"(3)0x100000005e5:[<0x10000000007/000000010000000000000053 v1453>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.588+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000204 [...7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000061 auth v1471 ap=1 snaprealm=0x560db6f96d80 s=16777216 n(v0 rc2024-02-26T06:28:48.867120+0000 b16777216 1=1+0) (iversion lock) caps={25193=pAsLsXsFs/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x560db7441080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000204:[<0x10000000007/000000010000000000000061 v1426>,<0x10000000006/pg_wal v1611>,<0x10000000005/data v503>,<0x100000001fb/postgres v257>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v271>,<0x10000000001/sv_1 v293>,<0x10000000000/qa v273>,<0x1/volumes v259>]//[]","memoryvalue":"(3)0x20000000204:[<0x10000000007/000000010000000000000061 v1471>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.588+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000611 [7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000003B auth v1473 ap=1 s=16777216 n(v0 rc2024-02-26T06:28:36.086470+0000 b16777216 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25193=0-37748736@7} caps={25193=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25193 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x560db7440b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000611:[<0x10000000007/00000001000000000000003B v1378>,<0x10000000006/pg_wal v1477>,<0x10000000005/data v467>,<0x100000001fb/postgres v243>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v259>,<0x10000000001/sv_1 v280>,<0x10000000000/qa v263>,<0x1/volumes v249>]//[]","memoryvalue":"(3)0x10000000611:[<0x10000000007/00000001000000000000003B v1473>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.589+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000218 [7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000037 auth{1=1} v1435 ap=1 s=16777216 n(v0 rc2024-02-26T06:28:29.356654+0000 b16777216 1=1+0) (iversion lock) | replicated=1 authpin=1 scrubqueue=0 0x560db732d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000218:[<0x10000000007/000000010000000000000037 v1355>,<0x10000000006/pg_wal v1398>,<0x10000000005/data v445>,<0x100000001fb/postgres v227>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v247>,<0x10000000001/sv_1 v268>,<0x10000000000/qa v251>,<0x1/volumes v237>]//[]","memoryvalue":"(3)0x20000000218:[<0x10000000007/000000010000000000000037 v1435>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.841 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:29:28.590+0000 7f0da91b8700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000214 [7,head] /volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000032 auth{1=1} v1475 ap=1 s=16777216 n(v0 rc2024-02-26T06:28:22.720836+0000 b16777216 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25193=0-37748736@7} caps={25193=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25193 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x560db7351600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000214:[<0x10000000007/000000010000000000000032 v1337>,<0x10000000006/pg_wal v1398>,<0x10000000005/data v445>,<0x100000001fb/postgres v227>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v247>,<0x10000000001/sv_1 v268>,<0x10000000000/qa v251>,<0x1/volumes v237>]//[]","memoryvalue":"(3)0x20000000214:[<0x10000000007/000000010000000000000032 v1475>,<0x10000000006/pg_wal v1718>,<0x10000000005/data v543>,<0x100000001fb/postgres v288>,<0x100000001fa/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6 v293>,<0x10000000001/sv_1 v232>,<0x10000000000/qa v295>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:28.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.897+0000 7f9c28000640 1 -- 172.21.15.49:0/372130480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c20108c80 msgr2=0x7f9c20113d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.897+0000 7f9c28000640 1 --2- 172.21.15.49:0/372130480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c20108c80 0x7f9c20113d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009900 tx=0x7f9c0c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.899+0000 7f9c28000640 1 -- 172.21.15.49:0/372130480 shutdown_connections 2024-02-26T06:29:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.899+0000 7f9c28000640 1 --2- 172.21.15.49:0/372130480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c201142d0 0x7f9c201166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.899+0000 7f9c28000640 1 --2- 172.21.15.49:0/372130480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c20108c80 0x7f9c20113d90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.899+0000 7f9c28000640 1 --2- 172.21.15.49:0/372130480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c20108360 0x7f9c20108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.899+0000 7f9c28000640 1 -- 172.21.15.49:0/372130480 >> 172.21.15.49:0/372130480 conn(0x7f9c2006c260 msgr2=0x7f9c2006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.901+0000 7f9c28000640 1 -- 172.21.15.49:0/372130480 shutdown_connections 2024-02-26T06:29:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.902+0000 7f9c28000640 1 -- 172.21.15.49:0/372130480 wait complete. 2024-02-26T06:29:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.903+0000 7f9c28000640 1 Processor -- start 2024-02-26T06:29:28.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.903+0000 7f9c28000640 1 -- start start 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.904+0000 7f9c28000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c20108360 0x7f9c201a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.904+0000 7f9c28000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c20108c80 0x7f9c201a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.904+0000 7f9c28000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c201142d0 0x7f9c201a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.904+0000 7f9c28000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201a7140 con 0x7f9c20108360 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.904+0000 7f9c28000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201a72b0 con 0x7f9c20108c80 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.904+0000 7f9c28000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201a7470 con 0x7f9c201142d0 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c24c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c20108c80 0x7f9c201a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c24c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c20108c80 0x7f9c201a27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51526/0 (socket says 172.21.15.49:51526) 2024-02-26T06:29:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c24c00640 1 -- 172.21.15.49:0/4138917885 learned_addr learned my addr 172.21.15.49:0/4138917885 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c25600640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c20108360 0x7f9c201a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c24c00640 1 -- 172.21.15.49:0/4138917885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c201142d0 msgr2=0x7f9c201a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c24c00640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c201142d0 0x7f9c201a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c24c00640 1 -- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c20108360 msgr2=0x7f9c201a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c24c00640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c20108360 0x7f9c201a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.905+0000 7f9c24c00640 1 -- 172.21.15.49:0/4138917885 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c009560 con 0x7f9c20108c80 2024-02-26T06:29:28.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.906+0000 7f9c24c00640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c20108c80 0x7f9c201a27c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c000c00 tx=0x7f9c0c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.906+0000 7f9c1e000640 1 -- 172.21.15.49:0/4138917885 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c02cc90 con 0x7f9c20108c80 2024-02-26T06:29:28.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.906+0000 7f9c1e000640 1 -- 172.21.15.49:0/4138917885 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c0c0052d0 con 0x7f9c20108c80 2024-02-26T06:29:28.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.906+0000 7f9c1e000640 1 -- 172.21.15.49:0/4138917885 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c003d30 con 0x7f9c20108c80 2024-02-26T06:29:28.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.907+0000 7f9c28000640 1 -- 172.21.15.49:0/4138917885 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c201a76f0 con 0x7f9c20108c80 2024-02-26T06:29:28.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.908+0000 7f9c28000640 1 -- 172.21.15.49:0/4138917885 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c201a79a0 con 0x7f9c20108c80 2024-02-26T06:29:28.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.914+0000 7f9c2ae68740 1 Processor -- start 2024-02-26T06:29:28.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.916+0000 7f9c2ae68740 1 -- start start 2024-02-26T06:29:28.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.916+0000 7f9c1e000640 1 -- 172.21.15.49:0/4138917885 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9c0c005440 con 0x7f9c20108c80 2024-02-26T06:29:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.916+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fb7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.917+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fb8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.917+0000 7f9c26000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fb7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.917+0000 7f9c26000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fb7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49232/0 (socket says 172.21.15.49:49232) 2024-02-26T06:29:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.917+0000 7f9c26000640 1 -- 172.21.15.49:0/2711829456 learned_addr learned my addr 172.21.15.49:0/2711829456 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.918+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.918+0000 7f9c2ae68740 1 -- 172.21.15.49:0/2711829456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5f27120 con 0x55e3a5fb8290 2024-02-26T06:29:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.918+0000 7f9c2ae68740 1 -- 172.21.15.49:0/2711829456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5f24650 con 0x55e3a5fc1600 2024-02-26T06:29:28.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.918+0000 7f9c2ae68740 1 -- 172.21.15.49:0/2711829456 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5f1fe30 con 0x55e3a5fb8f70 2024-02-26T06:29:28.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.918+0000 7f9c25600640 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fb8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.918+0000 7f9c1e000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c080a2440 0x7f9c080a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.918+0000 7f9c1e000640 1 -- 172.21.15.49:0/4138917885 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9c0c0f0580 con 0x7f9c20108c80 2024-02-26T06:29:28.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.919+0000 7f9c26000640 1 -- 172.21.15.49:0/2711829456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fc1600 msgr2=0x55e3a5fc39f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:29:28.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.919+0000 7f9c26000640 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.919+0000 7f9c26000640 1 -- 172.21.15.49:0/2711829456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 msgr2=0x55e3a5fb8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.919+0000 7f9c26000640 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fb8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.919+0000 7f9c26000640 1 -- 172.21.15.49:0/2711829456 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3a5ed9030 con 0x55e3a5fb8f70 2024-02-26T06:29:28.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.920+0000 7f9c26000640 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fb7cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009cd0 tx=0x7f9c1800b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=885c4bd2a57cf9c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.920+0000 7f9c26000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c080a2440 0x7f9c080a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.922+0000 7f9c12600640 1 -- 172.21.15.49:0/2711829456 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c18018070 con 0x55e3a5fb8f70 2024-02-26T06:29:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.922+0000 7f9c12600640 1 -- 172.21.15.49:0/2711829456 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c18012910 con 0x55e3a5fb8f70 2024-02-26T06:29:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.922+0000 7f9c12600640 1 -- 172.21.15.49:0/2711829456 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800e070 con 0x55e3a5fb8f70 2024-02-26T06:29:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.923+0000 7f9c2ae68740 1 -- 172.21.15.49:0/2711829456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8f70 msgr2=0x55e3a5fb7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.923+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fb7cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009cd0 tx=0x7f9c1800b420 comp rx=0 tx=0).stop 2024-02-26T06:29:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.924+0000 7f9c2ae68740 1 -- 172.21.15.49:0/2711829456 shutdown_connections 2024-02-26T06:29:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.924+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.924+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fb8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.924+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/2711829456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fb7cc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.924+0000 7f9c2ae68740 1 -- 172.21.15.49:0/2711829456 >> 172.21.15.49:0/2711829456 conn(0x55e3a5f7f570 msgr2=0x55e3a5fb2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.925+0000 7f9c26000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c080a2440 0x7f9c080a4900 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c1801b000 tx=0x7f9c18011af0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.925+0000 7f9c2ae68740 1 -- 172.21.15.49:0/2711829456 shutdown_connections 2024-02-26T06:29:28.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.925+0000 7f9c2ae68740 1 -- 172.21.15.49:0/2711829456 wait complete. 2024-02-26T06:29:28.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.928+0000 7f9c2ae68740 1 -- start start 2024-02-26T06:29:28.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.928+0000 7f9c2ae68740 1 Processor -- start 2024-02-26T06:29:28.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.929+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8290 0x55e3a5fbe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.930+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fbea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.930+0000 7f9c25600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8290 0x55e3a5fbe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.930+0000 7f9c25600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8290 0x55e3a5fbe550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49248/0 (socket says 172.21.15.49:49248) 2024-02-26T06:29:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.930+0000 7f9c25600640 1 -- 172.21.15.49:0/1132160127 learned_addr learned my addr 172.21.15.49:0/1132160127 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.930+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fc1600 0x55e3a5fbefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.930+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5ec4e90 con 0x55e3a5fc1600 2024-02-26T06:29:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.930+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5ec4b30 con 0x55e3a5fb8f70 2024-02-26T06:29:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.930+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5c8e1c0 con 0x55e3a5fb8290 2024-02-26T06:29:28.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.931+0000 7f9c2ae68740 10 client.0 _ll_register_callbacks cb 0x55e3a5ed8930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:28.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.932+0000 7f9c25600640 1 -- 172.21.15.49:0/1132160127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 msgr2=0x55e3a5fbea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.932+0000 7f9c25600640 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fbea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.932+0000 7f9c25600640 1 -- 172.21.15.49:0/1132160127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fc1600 msgr2=0x55e3a5fbefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.932+0000 7f9c25600640 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fc1600 0x55e3a5fbefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.932+0000 7f9c25600640 1 -- 172.21.15.49:0/1132160127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c18009980 con 0x55e3a5fb8290 2024-02-26T06:29:28.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.933+0000 7f9c25600640 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8290 0x55e3a5fbe550 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400df50 tx=0x7f9c14011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be0a25c3a04506c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.933+0000 7f9c06a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.934+0000 7f9c06a00640 1 -- 172.21.15.49:0/1132160127 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14004230 con 0x55e3a5fb8290 2024-02-26T06:29:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.935+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3a5c1f7e0 con 0x55e3a5fb8290 2024-02-26T06:29:28.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.936+0000 7f9c06a00640 1 -- 172.21.15.49:0/1132160127 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9c14004390 con 0x55e3a5fb8290 2024-02-26T06:29:28.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.937+0000 7f9c06a00640 1 -- 172.21.15.49:0/1132160127 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c14004230 con 0x55e3a5fb8290 2024-02-26T06:29:28.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.940+0000 7f9c2ae68740 10 client.26974 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:28.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.940+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e3a5f1e4b0 con 0x55e3a5fb8290 2024-02-26T06:29:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.940+0000 7f9c06a00640 1 -- 172.21.15.49:0/1132160127 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e3a5c1f7e0 con 0x55e3a5fb8290 2024-02-26T06:29:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.941+0000 7f9c06a00640 1 -- 172.21.15.49:0/1132160127 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9c14022070 con 0x55e3a5fb8290 2024-02-26T06:29:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.941+0000 7f9c2ae68740 10 client.26974 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.941+0000 7f9c2ae68740 10 client.26974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.941+0000 7f9c2ae68740 20 client.26974 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.941+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e3a5fd1820 0x55e3a5fd3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.941+0000 7f9c2ae68740 4 client.26974 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.941+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3a5c5c720 con 0x55e3a5fd1820 2024-02-26T06:29:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.942+0000 7f9c24c00640 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e3a5fd1820 0x55e3a5fd3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.944+0000 7f9c06a00640 10 client.26974 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:28.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.944+0000 7f9c24c00640 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e3a5fd1820 0x55e3a5fd3c60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49cdc5fda1cd4614 server_cookie=9b7be41bd78caf20 in_seq=0 out_seq=0 2024-02-26T06:29:28.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.945+0000 7f9c06a00640 1 -- 172.21.15.49:0/1132160127 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3a5c5c720 con 0x55e3a5fd1820 2024-02-26T06:29:28.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.945+0000 7f9c06a00640 10 client.26974 handle_command_reply: tid=0 2024-02-26T06:29:28.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.945+0000 7f9c2ae68740 1 client.26974 shutdown 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8290 msgr2=0x55e3a5fbe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8290 0x55e3a5fbe550 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400df50 tx=0x7f9c14011870 comp rx=0 tx=0).stop 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 shutdown_connections 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e3a5fd1820 0x55e3a5fd3c60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fbea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fc1600 0x55e3a5fbefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1132160127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fb8290 0x55e3a5fbe550 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c06a00640 0 client.26974 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:28.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.946+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 >> 172.21.15.49:0/1132160127 conn(0x55e3a5f7f570 msgr2=0x55e3a5fb4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.948+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 shutdown_connections 2024-02-26T06:29:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.952+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1132160127 wait complete. 2024-02-26T06:29:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.952+0000 7f9c2ae68740 20 client.26974 trim_cache size 0 max 16384 2024-02-26T06:29:28.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c2ae68740 1 Processor -- start 2024-02-26T06:29:28.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c2ae68740 1 -- start start 2024-02-26T06:29:28.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c2ae68740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5fbf6c0 con 0x55e3a5fb8f70 2024-02-26T06:29:28.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c2ae68740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5fd8c70 con 0x55e3a5fb8290 2024-02-26T06:29:28.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c2ae68740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a5c1f7e0 con 0x55e3a5fc1600 2024-02-26T06:29:28.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c26000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c26000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49262/0 (socket says 172.21.15.49:49262) 2024-02-26T06:29:28.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c26000640 1 -- 172.21.15.49:0/1584320747 learned_addr learned my addr 172.21.15.49:0/1584320747 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.985+0000 7f9c24c00640 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.986+0000 7f9c25600640 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.986+0000 7f9c25600640 1 -- 172.21.15.49:0/1584320747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 msgr2=0x55e3a5fc29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.986+0000 7f9c25600640 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.986+0000 7f9c25600640 1 -- 172.21.15.49:0/1584320747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8f70 msgr2=0x55e3a5fc8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.986+0000 7f9c25600640 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.986+0000 7f9c25600640 1 -- 172.21.15.49:0/1584320747 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3a5eba920 con 0x55e3a5fb8290 2024-02-26T06:29:28.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.987+0000 7f9c26000640 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:28.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.987+0000 7f9c24c00640 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:28.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.987+0000 7f9c25600640 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc8560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9c140128a0 tx=0x7f9c1400d340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86639cfbd222ebf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.987+0000 7f9c11c00640 1 -- 172.21.15.49:0/1584320747 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14012590 con 0x55e3a5fb8290 2024-02-26T06:29:28.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.988+0000 7f9c11c00640 1 -- 172.21.15.49:0/1584320747 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c14016070 con 0x55e3a5fb8290 2024-02-26T06:29:28.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.988+0000 7f9c11c00640 1 -- 172.21.15.49:0/1584320747 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14003ad0 con 0x55e3a5fb8290 2024-02-26T06:29:28.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.988+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1584320747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8290 msgr2=0x55e3a5fc8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.988+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc8560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9c140128a0 tx=0x7f9c1400d340 comp rx=0 tx=0).stop 2024-02-26T06:29:28.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.989+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1584320747 shutdown_connections 2024-02-26T06:29:28.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.989+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.989+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.989+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/1584320747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc8560 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.989+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1584320747 >> 172.21.15.49:0/1584320747 conn(0x55e3a5f7f570 msgr2=0x55e3a5fbce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.989+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1584320747 shutdown_connections 2024-02-26T06:29:28.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.989+0000 7f9c2ae68740 1 -- 172.21.15.49:0/1584320747 wait complete. 2024-02-26T06:29:28.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 1 -- start start 2024-02-26T06:29:28.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 1 Processor -- start 2024-02-26T06:29:28.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a6030750 con 0x55e3a5fb8290 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a60308c0 con 0x55e3a5fb8f70 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a6030a80 con 0x55e3a5fc1600 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.990+0000 7f9c2ae68740 10 client.0 _ll_register_callbacks cb 0x7f9c18011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c24c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c24c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:51544/0 (socket says 172.21.15.49:51544) 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c24c00640 1 -- 172.21.15.49:0/494768985 learned_addr learned my addr 172.21.15.49:0/494768985 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c26000640 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c25600640 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c24c00640 1 -- 172.21.15.49:0/494768985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 msgr2=0x55e3a5fc5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c24c00640 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c24c00640 1 -- 172.21.15.49:0/494768985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 msgr2=0x55e3a5fc5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c24c00640 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c24c00640 1 -- 172.21.15.49:0/494768985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c201a79a0 con 0x55e3a5fb8f70 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.991+0000 7f9c25600640 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.992+0000 7f9c24c00640 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc5770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0b9490 tx=0x7f9c0c02c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78010d9b3a474b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.992+0000 7f9c06a00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.992+0000 7f9c06a00640 1 -- 172.21.15.49:0/494768985 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0c1180 con 0x55e3a5fb8f70 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.992+0000 7f9c06a00640 1 -- 172.21.15.49:0/494768985 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f9c0c0c57d0 con 0x55e3a5fb8f70 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.992+0000 7f9c2ae68740 1 -- 172.21.15.49:0/494768985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c18012910 con 0x55e3a5fb8f70 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.992+0000 7f9c06a00640 1 -- 172.21.15.49:0/494768985 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c0c0bb330 con 0x55e3a5fb8f70 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.992+0000 7f9c2ae68740 10 client.26987 fetch_fsmap learned FSMap version 36 2024-02-26T06:29:28.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.992+0000 7f9c2ae68740 1 -- 172.21.15.49:0/494768985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e3a5ed9030 con 0x55e3a5fb8f70 2024-02-26T06:29:28.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.993+0000 7f9c06a00640 1 -- 172.21.15.49:0/494768985 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c18012910 con 0x55e3a5fb8f70 2024-02-26T06:29:28.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.993+0000 7f9c06a00640 1 -- 172.21.15.49:0/494768985 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 36) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9c0c03c070 con 0x55e3a5fb8f70 2024-02-26T06:29:28.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.994+0000 7f9c2ae68740 10 client.26987 fetch_fsmap finished waiting for FSMap version 36 2024-02-26T06:29:28.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.994+0000 7f9c2ae68740 10 client.26987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:28.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.994+0000 7f9c2ae68740 20 client.26987 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:28.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.994+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e3a6032080 0x55e3a60344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.994+0000 7f9c2ae68740 4 client.26987 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:29:28.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.994+0000 7f9c2ae68740 1 -- 172.21.15.49:0/494768985 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e3a5cab890 con 0x55e3a6032080 2024-02-26T06:29:28.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.995+0000 7f9c25600640 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e3a6032080 0x55e3a60344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.996+0000 7f9c25600640 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e3a6032080 0x55e3a60344a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7877a2540acf6ba server_cookie=e14a10db060ca988 in_seq=0 out_seq=0 2024-02-26T06:29:28.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.996+0000 7f9c06a00640 10 client.26987 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:28.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.996+0000 7f9c06a00640 1 -- 172.21.15.49:0/494768985 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e3a5cab890 con 0x55e3a6032080 2024-02-26T06:29:28.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.996+0000 7f9c06a00640 10 client.26987 handle_command_reply: tid=0 2024-02-26T06:29:28.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 client.26987 shutdown 2024-02-26T06:29:28.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 -- 172.21.15.49:0/494768985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 msgr2=0x55e3a5fc5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc5770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0b9490 tx=0x7f9c0c02c720 comp rx=0 tx=0).stop 2024-02-26T06:29:28.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 -- 172.21.15.49:0/494768985 shutdown_connections 2024-02-26T06:29:28.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e3a6032080 0x55e3a60344a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e3a5fb8f70 0x55e3a5fc5770 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3a5fc1600 0x55e3a5fc5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 --2- 172.21.15.49:0/494768985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3a5fb8290 0x55e3a5fc5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c2ae68740 1 -- 172.21.15.49:0/494768985 >> 172.21.15.49:0/494768985 conn(0x55e3a5f7f570 msgr2=0x55e3a5fbd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.997+0000 7f9c06a00640 0 client.26987 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:28.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.998+0000 7f9c2ae68740 1 -- 172.21.15.49:0/494768985 shutdown_connections 2024-02-26T06:29:28.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.998+0000 7f9c2ae68740 1 -- 172.21.15.49:0/494768985 wait complete. 2024-02-26T06:29:28.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.998+0000 7f9c2ae68740 20 client.26987 trim_cache size 0 max 16384 2024-02-26T06:29:28.998 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout: "a439ddc6-10fc-4cb4-b884-f55c883fa0a9": { 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "a439ddc6-10fc-4cb4-b884-f55c883fa0a9", 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:29:28.999 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 -- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c080a2440 msgr2=0x7f9c080a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c080a2440 0x7f9c080a4900 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c1801b000 tx=0x7f9c18011af0 comp rx=0 tx=0).stop 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 -- 172.21.15.49:0/4138917885 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c20108c80 msgr2=0x7f9c201a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c20108c80 0x7f9c201a27c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c000c00 tx=0x7f9c0c004270 comp rx=0 tx=0).stop 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 -- 172.21.15.49:0/4138917885 shutdown_connections 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9c080a2440 0x7f9c080a4900 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c20108c80 0x7f9c201a27c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c201142d0 0x7f9c201a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:29.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 --2- 172.21.15.49:0/4138917885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c20108360 0x7f9c201a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:29.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 -- 172.21.15.49:0/4138917885 >> 172.21.15.49:0/4138917885 conn(0x7f9c2006c260 msgr2=0x7f9c20071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:29.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 -- 172.21.15.49:0/4138917885 shutdown_connections 2024-02-26T06:29:29.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:28.999+0000 7f9c28000640 1 -- 172.21.15.49:0/4138917885 wait complete. 2024-02-26T06:29:29.025 INFO:tasks.cephfs.filesystem:scrub status for tag:a439ddc6-10fc-4cb4-b884-f55c883fa0a9 - {'path': '/', 'tag': 'a439ddc6-10fc-4cb4-b884-f55c883fa0a9', 'options': 'recursive,force'} 2024-02-26T06:29:29.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:28 smithi080 ceph-mon[36206]: pgmap v185: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-02-26T06:29:29.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:28 smithi115 ceph-mon[36426]: pgmap v185: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-02-26T06:29:29.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:28 smithi049 ceph-mon[31696]: pgmap v185: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-02-26T06:29:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub queued for path: / 2024-02-26T06:29:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:29:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:29:30.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: active paths [/] 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x30000000230 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000034) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x20000000212 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000030) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x2000000020c (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002B) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x2000000020f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002D) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x20000000211 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002F) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x100000005e5 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000053) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x20000000204 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000061) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x10000000611 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000003B) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x20000000218 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000037) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: Scrub error on inode 0x20000000214 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000032) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:29 smithi080 ceph-mon[36206]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub queued for path: / 2024-02-26T06:29:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:29:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:29:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:29:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: active paths [/] 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x30000000230 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000034) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x20000000212 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000030) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x2000000020c (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002B) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x2000000020f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002D) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x20000000211 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002F) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x100000005e5 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000053) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x20000000204 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000061) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x10000000611 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000003B) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x20000000218 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000037) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: Scrub error on inode 0x20000000214 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000032) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:29 smithi115 ceph-mon[36426]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub queued for path: / 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: active paths [/] 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x30000000230 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000034) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x20000000212 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000030) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x2000000020c (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002B) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x2000000020f (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002D) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x20000000211 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002F) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x100000005e5 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000053) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x20000000204 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000061) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.422 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x10000000611 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000003B) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.423 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x20000000218 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000037) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.423 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: Scrub error on inode 0x20000000214 (/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000032) see mds.c log and `damage ls` output for details 2024-02-26T06:29:30.423 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.423 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:30.423 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:29 smithi049 ceph-mon[31696]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 340.0 s, 2176.8 tps, lat 45.824 ms stddev 50.521 2024-02-26T06:29:31.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:30 smithi080 ceph-mon[36206]: pgmap v186: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.8 MiB/s wr, 812 op/s 2024-02-26T06:29:31.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:30 smithi115 ceph-mon[36426]: pgmap v186: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.8 MiB/s wr, 812 op/s 2024-02-26T06:29:31.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:30 smithi049 ceph-mon[31696]: pgmap v186: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.8 MiB/s wr, 812 op/s 2024-02-26T06:29:32.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:31 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:32.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:31 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:32.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:31 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:33.230 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:32 smithi080 ceph-mon[36206]: pgmap v187: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 8.7 MiB/s wr, 844 op/s 2024-02-26T06:29:33.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:32 smithi115 ceph-mon[36426]: pgmap v187: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 8.7 MiB/s wr, 844 op/s 2024-02-26T06:29:33.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:32 smithi049 ceph-mon[31696]: pgmap v187: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 8.7 MiB/s wr, 844 op/s 2024-02-26T06:29:35.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:34 smithi080 ceph-mon[36206]: pgmap v188: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 7.1 MiB/s wr, 719 op/s 2024-02-26T06:29:35.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:34 smithi115 ceph-mon[36426]: pgmap v188: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 7.1 MiB/s wr, 719 op/s 2024-02-26T06:29:35.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:34 smithi049 ceph-mon[31696]: pgmap v188: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 7.1 MiB/s wr, 719 op/s 2024-02-26T06:29:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 345.0 s, 2301.2 tps, lat 43.489 ms stddev 48.292 2024-02-26T06:29:36.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:36 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:29:36.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:36 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:29:36.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:35 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:29:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:37 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:29:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:37 smithi080 ceph-mon[36206]: scrub summary: idle 2024-02-26T06:29:37.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:37 smithi080 ceph-mon[36206]: pgmap v189: 97 pgs: 97 active+clean; 3.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-02-26T06:29:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:37 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:29:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:37 smithi115 ceph-mon[36426]: scrub summary: idle 2024-02-26T06:29:37.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:37 smithi115 ceph-mon[36426]: pgmap v189: 97 pgs: 97 active+clean; 3.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-02-26T06:29:37.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:37 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:29:37.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:37 smithi049 ceph-mon[31696]: scrub summary: idle 2024-02-26T06:29:37.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:37 smithi049 ceph-mon[31696]: pgmap v189: 97 pgs: 97 active+clean; 3.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-02-26T06:29:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:38 smithi080 ceph-mon[36206]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T06:29:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:38 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:active 2024-02-26T06:29:38.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:38 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:38 smithi115 ceph-mon[36426]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T06:29:38.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:38 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:active 2024-02-26T06:29:38.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:38 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31692]: 2024-02-26T06:29:38.004+0000 7f751c200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T06:29:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:38 smithi049 ceph-mon[31696]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T06:29:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:38 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] up:active 2024-02-26T06:29:38.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:38 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:29:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.893+0000 7fd64a400640 1 -- 172.21.15.49:0/1728174254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6441013a0 msgr2=0x7fd644101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.893+0000 7fd64a400640 1 --2- 172.21.15.49:0/1728174254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6441013a0 0x7fd644101780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd634009930 tx=0x7fd63402bc20 comp rx=0 tx=0).stop 2024-02-26T06:29:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.894+0000 7fd64a400640 1 -- 172.21.15.49:0/1728174254 shutdown_connections 2024-02-26T06:29:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.894+0000 7fd64a400640 1 --2- 172.21.15.49:0/1728174254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd644106470 0x7fd64410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.894+0000 7fd64a400640 1 --2- 172.21.15.49:0/1728174254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd644101d50 0x7fd644105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.894+0000 7fd64a400640 1 --2- 172.21.15.49:0/1728174254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6441013a0 0x7fd644101780 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.894+0000 7fd64a400640 1 -- 172.21.15.49:0/1728174254 >> 172.21.15.49:0/1728174254 conn(0x7fd6440fcc40 msgr2=0x7fd6440ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.894+0000 7fd64a400640 1 -- 172.21.15.49:0/1728174254 shutdown_connections 2024-02-26T06:29:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.894+0000 7fd64a400640 1 -- 172.21.15.49:0/1728174254 wait complete. 2024-02-26T06:29:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.897+0000 7fd64a400640 1 Processor -- start 2024-02-26T06:29:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.898+0000 7fd64a400640 1 -- start start 2024-02-26T06:29:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd64a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6441013a0 0x7fd64419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd64a400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd644106470 0x7fd64419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd64a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd644199800 0x7fd644199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd64a400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64419a2b0 con 0x7fd644106470 2024-02-26T06:29:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd64a400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64419a420 con 0x7fd644199800 2024-02-26T06:29:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd64a400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64419a590 con 0x7fd6441013a0 2024-02-26T06:29:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd643400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6441013a0 0x7fd64419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd643400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6441013a0 0x7fd64419f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:37908/0 (socket says 172.21.15.49:37908) 2024-02-26T06:29:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd643400640 1 -- 172.21.15.49:0/959448383 learned_addr learned my addr 172.21.15.49:0/959448383 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.899+0000 7fd643e00640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd644199800 0x7fd644199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.900+0000 7fd643400640 1 -- 172.21.15.49:0/959448383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd644199800 msgr2=0x7fd644199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.900+0000 7fd643400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd644199800 0x7fd644199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.900+0000 7fd643400640 1 -- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd644106470 msgr2=0x7fd64419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.900+0000 7fd643400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd644106470 0x7fd64419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.900+0000 7fd643400640 1 -- 172.21.15.49:0/959448383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009490 con 0x7fd6441013a0 2024-02-26T06:29:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.901+0000 7fd643400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6441013a0 0x7fd64419f690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd634000c00 tx=0x7fd63402cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.901+0000 7fd63be00640 1 -- 172.21.15.49:0/959448383 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63402dea0 con 0x7fd6441013a0 2024-02-26T06:29:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.902+0000 7fd63be00640 1 -- 172.21.15.49:0/959448383 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd634004350 con 0x7fd6441013a0 2024-02-26T06:29:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.902+0000 7fd63be00640 1 -- 172.21.15.49:0/959448383 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634003b50 con 0x7fd6441013a0 2024-02-26T06:29:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.902+0000 7fd64a400640 1 -- 172.21.15.49:0/959448383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd634009560 con 0x7fd6441013a0 2024-02-26T06:29:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.902+0000 7fd64a400640 1 -- 172.21.15.49:0/959448383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6441a67e0 con 0x7fd6441013a0 2024-02-26T06:29:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.909+0000 7fd63be00640 1 -- 172.21.15.49:0/959448383 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd634003cb0 con 0x7fd6441013a0 2024-02-26T06:29:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.913+0000 7fd63be00640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd6280a2350 0x7fd6280a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.914+0000 7fd63be00640 1 -- 172.21.15.49:0/959448383 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd6340f0510 con 0x7fd6441013a0 2024-02-26T06:29:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.915+0000 7fd642a00640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd6280a2350 0x7fd6280a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.916+0000 7fd642a00640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd6280a2350 0x7fd6280a4810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd630005d00 tx=0x7fd630005c50 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.918+0000 7fd64d376740 1 Processor -- start 2024-02-26T06:29:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.919+0000 7fd64d376740 1 -- start start 2024-02-26T06:29:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.919+0000 7fd64d376740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4552af0 0x561fe4551840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.919+0000 7fd64d376740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4552200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.919+0000 7fd64d376740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe455b300 0x561fe455d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.919+0000 7fd64d376740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe450d5f0 con 0x561fe4552af0 2024-02-26T06:29:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.920+0000 7fd64d376740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe44bf890 con 0x561fe4551d80 2024-02-26T06:29:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.920+0000 7fd64d376740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe44bcdc0 con 0x561fe455b300 2024-02-26T06:29:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.920+0000 7fd643400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4552af0 0x561fe4551840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.920+0000 7fd642a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4552200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.920+0000 7fd642a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4552200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:41096/0 (socket says 172.21.15.49:41096) 2024-02-26T06:29:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.920+0000 7fd642a00640 1 -- 172.21.15.49:0/1259676445 learned_addr learned my addr 172.21.15.49:0/1259676445 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.921+0000 7fd643400640 1 -- 172.21.15.49:0/1259676445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe455b300 msgr2=0x561fe455d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:29:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.921+0000 7fd643400640 1 --2- 172.21.15.49:0/1259676445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe455b300 0x561fe455d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.921+0000 7fd643400640 1 -- 172.21.15.49:0/1259676445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 msgr2=0x561fe4552200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.921+0000 7fd643400640 1 --2- 172.21.15.49:0/1259676445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4552200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.921+0000 7fd643400640 1 -- 172.21.15.49:0/1259676445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fe4439ad0 con 0x561fe4552af0 2024-02-26T06:29:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.922+0000 7fd643400640 1 --2- 172.21.15.49:0/1259676445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4552af0 0x561fe4551840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd6340edd80 tx=0x7fd6340bcfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91412bac566697 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.922+0000 7fd638c00640 1 -- 172.21.15.49:0/1259676445 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340c1200 con 0x561fe4552af0 2024-02-26T06:29:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.922+0000 7fd638c00640 1 -- 172.21.15.49:0/1259676445 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6340c57c0 con 0x561fe4552af0 2024-02-26T06:29:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.922+0000 7fd638c00640 1 -- 172.21.15.49:0/1259676445 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340c5330 con 0x561fe4552af0 2024-02-26T06:29:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.923+0000 7fd64d376740 1 -- 172.21.15.49:0/1259676445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4552af0 msgr2=0x561fe4551840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.923+0000 7fd64d376740 1 --2- 172.21.15.49:0/1259676445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4552af0 0x561fe4551840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd6340edd80 tx=0x7fd6340bcfb0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.924+0000 7fd64d376740 1 -- 172.21.15.49:0/1259676445 shutdown_connections 2024-02-26T06:29:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.924+0000 7fd64d376740 1 --2- 172.21.15.49:0/1259676445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4552200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.924+0000 7fd64d376740 1 --2- 172.21.15.49:0/1259676445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe455b300 0x561fe455d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.924+0000 7fd64d376740 1 --2- 172.21.15.49:0/1259676445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4552af0 0x561fe4551840 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd6340edd80 tx=0x7fd6340bcfb0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.924+0000 7fd64d376740 1 -- 172.21.15.49:0/1259676445 >> 172.21.15.49:0/1259676445 conn(0x561fe4519480 msgr2=0x561fe454ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.924+0000 7fd64d376740 1 -- 172.21.15.49:0/1259676445 shutdown_connections 2024-02-26T06:29:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.925+0000 7fd64d376740 1 -- 172.21.15.49:0/1259676445 wait complete. 2024-02-26T06:29:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.925+0000 7fd64d376740 1 -- start start 2024-02-26T06:29:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.926+0000 7fd64d376740 1 Processor -- start 2024-02-26T06:29:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.928+0000 7fd64d376740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4558410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.928+0000 7fd64d376740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe455b300 0x561fe4558950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.928+0000 7fd64d376740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe456f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.928+0000 7fd64d376740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe44b85a0 con 0x561fe455b300 2024-02-26T06:29:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.928+0000 7fd64d376740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe445db20 con 0x561fe4551d80 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.928+0000 7fd64d376740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe445d7c0 con 0x561fe4558e90 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.928+0000 7fd64d376740 10 client.0 _ll_register_callbacks cb 0x561fe4471520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.929+0000 7fd643400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4558410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.929+0000 7fd643400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4558410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:41108/0 (socket says 172.21.15.49:41108) 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.929+0000 7fd643400640 1 -- 172.21.15.49:0/1185598070 learned_addr learned my addr 172.21.15.49:0/1185598070 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.929+0000 7fd643400640 1 -- 172.21.15.49:0/1185598070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 msgr2=0x561fe456f7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.929+0000 7fd643400640 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe456f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.929+0000 7fd643400640 1 -- 172.21.15.49:0/1185598070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe455b300 msgr2=0x561fe4558950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.929+0000 7fd643400640 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe455b300 0x561fe4558950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.929+0000 7fd643400640 1 -- 172.21.15.49:0/1185598070 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6441a67e0 con 0x561fe4551d80 2024-02-26T06:29:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.930+0000 7fd643400640 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4558410 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd6340ede30 tx=0x7fd634005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948ac731740e4cb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.930+0000 7fd62d600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:29:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.930+0000 7fd62d600640 1 -- 172.21.15.49:0/1185598070 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340b9900 con 0x561fe4551d80 2024-02-26T06:29:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.930+0000 7fd62d600640 1 -- 172.21.15.49:0/1185598070 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd6340c1200 con 0x561fe4551d80 2024-02-26T06:29:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.932+0000 7fd64d376740 1 -- 172.21.15.49:0/1185598070 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fe448aca0 con 0x561fe4551d80 2024-02-26T06:29:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.932+0000 7fd62d600640 1 -- 172.21.15.49:0/1185598070 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6340c57c0 con 0x561fe4551d80 2024-02-26T06:29:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.933+0000 7fd64d376740 10 client.26999 fetch_fsmap learned FSMap version 38 2024-02-26T06:29:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.933+0000 7fd64d376740 1 -- 172.21.15.49:0/1185598070 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561fe44b6c20 con 0x561fe4551d80 2024-02-26T06:29:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.934+0000 7fd62d600640 1 -- 172.21.15.49:0/1185598070 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6340c57c0 con 0x561fe4551d80 2024-02-26T06:29:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.934+0000 7fd62d600640 1 -- 172.21.15.49:0/1185598070 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd63403c070 con 0x561fe4551d80 2024-02-26T06:29:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.934+0000 7fd64d376740 10 client.26999 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:29:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.934+0000 7fd64d376740 10 client.26999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.934+0000 7fd64d376740 20 client.26999 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.935+0000 7fd64d376740 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561fe4573970 0x561fe4575d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.935+0000 7fd64d376740 4 client.26999 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.935+0000 7fd64d376740 1 -- 172.21.15.49:0/1185598070 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fe44aecb0 con 0x561fe4573970 2024-02-26T06:29:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.935+0000 7fd642a00640 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561fe4573970 0x561fe4575d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.937+0000 7fd642a00640 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561fe4573970 0x561fe4575d90 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c638c01056b31070 server_cookie=a5d5790eacaa20fc in_seq=0 out_seq=0 2024-02-26T06:29:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.937+0000 7fd62d600640 10 client.26999 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.940+0000 7fd62d600640 1 -- 172.21.15.49:0/1185598070 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fe44aecb0 con 0x561fe4573970 2024-02-26T06:29:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.940+0000 7fd62d600640 10 client.26999 handle_command_reply: tid=0 2024-02-26T06:29:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.940+0000 7fd64d376740 1 client.26999 shutdown 2024-02-26T06:29:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.942+0000 7fd64d376740 1 -- 172.21.15.49:0/1185598070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 msgr2=0x561fe4558410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.942+0000 7fd64d376740 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4558410 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd6340ede30 tx=0x7fd634005670 comp rx=0 tx=0).stop 2024-02-26T06:29:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.942+0000 7fd64d376740 1 -- 172.21.15.49:0/1185598070 shutdown_connections 2024-02-26T06:29:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.942+0000 7fd64d376740 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561fe4573970 0x561fe4575d90 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.943+0000 7fd64d376740 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe456f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.943+0000 7fd64d376740 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe455b300 0x561fe4558950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.943+0000 7fd64d376740 1 --2- 172.21.15.49:0/1185598070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe4551d80 0x561fe4558410 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.943+0000 7fd62d600640 0 client.26999 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.943+0000 7fd64d376740 1 -- 172.21.15.49:0/1185598070 >> 172.21.15.49:0/1185598070 conn(0x561fe4519480 msgr2=0x561fe454e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.943+0000 7fd64d376740 1 -- 172.21.15.49:0/1185598070 shutdown_connections 2024-02-26T06:29:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.944+0000 7fd64d376740 1 -- 172.21.15.49:0/1185598070 wait complete. 2024-02-26T06:29:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.944+0000 7fd64d376740 20 client.26999 trim_cache size 0 max 16384 2024-02-26T06:29:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.982+0000 7fd64d376740 1 Processor -- start 2024-02-26T06:29:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.982+0000 7fd64d376740 1 -- start start 2024-02-26T06:29:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.982+0000 7fd64d376740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4551d80 0x561fe456a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.982+0000 7fd64d376740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4558e90 0x561fe456acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.982+0000 7fd64d376740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 0x561fe455c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.982+0000 7fd64d376740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe4200e60 con 0x561fe4558e90 2024-02-26T06:29:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.982+0000 7fd64d376740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe45530e0 con 0x561fe455b300 2024-02-26T06:29:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.982+0000 7fd64d376740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe454da70 con 0x561fe4551d80 2024-02-26T06:29:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.983+0000 7fd643400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4551d80 0x561fe456a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.983+0000 7fd643400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4551d80 0x561fe456a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:37932/0 (socket says 172.21.15.49:37932) 2024-02-26T06:29:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.983+0000 7fd643400640 1 -- 172.21.15.49:0/2378181921 learned_addr learned my addr 172.21.15.49:0/2378181921 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.983+0000 7fd643e00640 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 0x561fe455c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.983+0000 7fd642a00640 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4558e90 0x561fe456acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.984+0000 7fd643400640 1 -- 172.21.15.49:0/2378181921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 msgr2=0x561fe455c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.984+0000 7fd643400640 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 0x561fe455c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.984+0000 7fd643400640 1 -- 172.21.15.49:0/2378181921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4558e90 msgr2=0x561fe456acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.984+0000 7fd643400640 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4558e90 0x561fe456acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.984+0000 7fd643400640 1 -- 172.21.15.49:0/2378181921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fe4453500 con 0x561fe4551d80 2024-02-26T06:29:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.984+0000 7fd642a00640 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4558e90 0x561fe456acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.985+0000 7fd643400640 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4551d80 0x561fe456a770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd6340b9510 tx=0x7fd6340bcfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d3135b29ee9105 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.986+0000 7fd62fe00640 1 -- 172.21.15.49:0/2378181921 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340c57c0 con 0x561fe4551d80 2024-02-26T06:29:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.986+0000 7fd62fe00640 1 -- 172.21.15.49:0/2378181921 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561fe448aca0 con 0x561fe4551d80 2024-02-26T06:29:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.986+0000 7fd62fe00640 1 -- 172.21.15.49:0/2378181921 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340bb420 con 0x561fe4551d80 2024-02-26T06:29:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.988+0000 7fd64d376740 1 -- 172.21.15.49:0/2378181921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4551d80 msgr2=0x561fe456a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.988+0000 7fd64d376740 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4551d80 0x561fe456a770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd6340b9510 tx=0x7fd6340bcfb0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.988+0000 7fd64d376740 1 -- 172.21.15.49:0/2378181921 shutdown_connections 2024-02-26T06:29:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.988+0000 7fd64d376740 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 0x561fe455c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.988+0000 7fd64d376740 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4558e90 0x561fe456acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.988+0000 7fd64d376740 1 --2- 172.21.15.49:0/2378181921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4551d80 0x561fe456a770 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.988+0000 7fd64d376740 1 -- 172.21.15.49:0/2378181921 >> 172.21.15.49:0/2378181921 conn(0x561fe4519480 msgr2=0x561fe4556d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.988+0000 7fd64d376740 1 -- 172.21.15.49:0/2378181921 shutdown_connections 2024-02-26T06:29:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.989+0000 7fd64d376740 1 -- 172.21.15.49:0/2378181921 wait complete. 2024-02-26T06:29:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.989+0000 7fd64d376740 1 -- start start 2024-02-26T06:29:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.990+0000 7fd64d376740 1 Processor -- start 2024-02-26T06:29:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.990+0000 7fd64d376740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4551d80 0x561fe456a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.990+0000 7fd64d376740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe4568760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd64d376740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 0x561fe4568ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd64d376740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe456af10 con 0x561fe4551d80 2024-02-26T06:29:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd64d376740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe456b080 con 0x561fe455b300 2024-02-26T06:29:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd64d376740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe4569230 con 0x561fe4558e90 2024-02-26T06:29:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd64d376740 10 client.0 _ll_register_callbacks cb 0x7fd630014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd642a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe4568760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd642a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe4568760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:37940/0 (socket says 172.21.15.49:37940) 2024-02-26T06:29:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd642a00640 1 -- 172.21.15.49:0/2837960256 learned_addr learned my addr 172.21.15.49:0/2837960256 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd643400640 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4551d80 0x561fe456a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.991+0000 7fd643e00640 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 0x561fe4568ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.992+0000 7fd642a00640 1 -- 172.21.15.49:0/2837960256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 msgr2=0x561fe4568ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.992+0000 7fd642a00640 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 0x561fe4568ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.992+0000 7fd642a00640 1 -- 172.21.15.49:0/2837960256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4551d80 msgr2=0x561fe456a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.992+0000 7fd642a00640 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4551d80 0x561fe456a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.992+0000 7fd642a00640 1 -- 172.21.15.49:0/2837960256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fe44b6c20 con 0x561fe4558e90 2024-02-26T06:29:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.992+0000 7fd642a00640 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe4568760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd630013040 tx=0x7fd6300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126923e59fc32560 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.992+0000 7fd62d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.993+0000 7fd62d600640 1 -- 172.21.15.49:0/2837960256 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630010e50 con 0x561fe4558e90 2024-02-26T06:29:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.993+0000 7fd62d600640 1 -- 172.21.15.49:0/2837960256 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd630022050 con 0x561fe4558e90 2024-02-26T06:29:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.993+0000 7fd64d376740 1 -- 172.21.15.49:0/2837960256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fe44ddd00 con 0x561fe4558e90 2024-02-26T06:29:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.993+0000 7fd62d600640 1 -- 172.21.15.49:0/2837960256 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd630004020 con 0x561fe4558e90 2024-02-26T06:29:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd64d376740 10 client.26992 fetch_fsmap learned FSMap version 38 2024-02-26T06:29:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd64d376740 1 -- 172.21.15.49:0/2837960256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561fe4479290 con 0x561fe4558e90 2024-02-26T06:29:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd62d600640 1 -- 172.21.15.49:0/2837960256 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd630024760 con 0x561fe4558e90 2024-02-26T06:29:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd62d600640 1 -- 172.21.15.49:0/2837960256 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd630018040 con 0x561fe4558e90 2024-02-26T06:29:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd64d376740 10 client.26992 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:29:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd64d376740 10 client.26992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd64d376740 20 client.26992 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd64d376740 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561fe45dd270 0x561fe45df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd64d376740 4 client.26992 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.994+0000 7fd64d376740 1 -- 172.21.15.49:0/2837960256 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561fe44ddd00 con 0x561fe45dd270 2024-02-26T06:29:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.995+0000 7fd643400640 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561fe45dd270 0x561fe45df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.995+0000 7fd643400640 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561fe45dd270 0x561fe45df690 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d984bc2dccc058f server_cookie=f8d3a098b85e8899 in_seq=0 out_seq=0 2024-02-26T06:29:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:38.995+0000 7fd62d600640 10 client.26992 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.013+0000 7fd62d600640 1 -- 172.21.15.49:0/2837960256 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561fe44ddd00 con 0x561fe45dd270 2024-02-26T06:29:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.013+0000 7fd62d600640 10 client.26992 handle_command_reply: tid=0 2024-02-26T06:29:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 client.26992 shutdown 2024-02-26T06:29:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 -- 172.21.15.49:0/2837960256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 msgr2=0x561fe4568760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe4568760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd630013040 tx=0x7fd6300046c0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 -- 172.21.15.49:0/2837960256 shutdown_connections 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561fe45dd270 0x561fe45df690 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561fe455b300 0x561fe4568ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd62d600640 0 client.26992 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fe4558e90 0x561fe4568760 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 --2- 172.21.15.49:0/2837960256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561fe4551d80 0x561fe456a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.014+0000 7fd64d376740 1 -- 172.21.15.49:0/2837960256 >> 172.21.15.49:0/2837960256 conn(0x561fe4519480 msgr2=0x561fe454b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.015+0000 7fd64d376740 1 -- 172.21.15.49:0/2837960256 shutdown_connections 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.015+0000 7fd64d376740 1 -- 172.21.15.49:0/2837960256 wait complete. 2024-02-26T06:29:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.015+0000 7fd64d376740 20 client.26992 trim_cache size 0 max 16384 2024-02-26T06:29:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:29:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:29:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:29:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:29:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.016+0000 7fd64a400640 1 -- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd6280a2350 msgr2=0x7fd6280a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.016+0000 7fd64a400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd6280a2350 0x7fd6280a4810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd630005d00 tx=0x7fd630005c50 comp rx=0 tx=0).stop 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.016+0000 7fd64a400640 1 -- 172.21.15.49:0/959448383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6441013a0 msgr2=0x7fd64419f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.016+0000 7fd64a400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6441013a0 0x7fd64419f690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd634000c00 tx=0x7fd63402cd70 comp rx=0 tx=0).stop 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.017+0000 7fd64a400640 1 -- 172.21.15.49:0/959448383 shutdown_connections 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.017+0000 7fd64a400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd6280a2350 0x7fd6280a4810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.017+0000 7fd64a400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd644199800 0x7fd644199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.017+0000 7fd64a400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd644106470 0x7fd64419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.017+0000 7fd64a400640 1 --2- 172.21.15.49:0/959448383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6441013a0 0x7fd64419f690 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.017+0000 7fd64a400640 1 -- 172.21.15.49:0/959448383 >> 172.21.15.49:0/959448383 conn(0x7fd6440fcc40 msgr2=0x7fd6440fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.017+0000 7fd64a400640 1 -- 172.21.15.49:0/959448383 shutdown_connections 2024-02-26T06:29:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:39.017+0000 7fd64a400640 1 -- 172.21.15.49:0/959448383 wait complete. 2024-02-26T06:29:39.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:39 smithi080 ceph-mon[36206]: pgmap v190: 97 pgs: 97 active+clean; 3.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-02-26T06:29:39.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:39 smithi115 ceph-mon[36426]: pgmap v190: 97 pgs: 97 active+clean; 3.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-02-26T06:29:39.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:39 smithi049 ceph-mon[31696]: pgmap v190: 97 pgs: 97 active+clean; 3.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-02-26T06:29:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 350.0 s, 2858.2 tps, lat 34.966 ms stddev 39.000 2024-02-26T06:29:41.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:41 smithi080 ceph-mon[36206]: pgmap v191: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 9.0 MiB/s wr, 878 op/s 2024-02-26T06:29:41.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:41 smithi115 ceph-mon[36426]: pgmap v191: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 9.0 MiB/s wr, 878 op/s 2024-02-26T06:29:41.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:41 smithi049 ceph-mon[31696]: pgmap v191: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 9.0 MiB/s wr, 878 op/s 2024-02-26T06:29:43.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:43 smithi080 ceph-mon[36206]: pgmap v192: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.0 MiB/s wr, 788 op/s 2024-02-26T06:29:43.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:43 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:29:43.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:43 smithi115 ceph-mon[36426]: pgmap v192: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.0 MiB/s wr, 788 op/s 2024-02-26T06:29:43.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:43 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:29:43.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:43 smithi049 ceph-mon[31696]: pgmap v192: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.0 MiB/s wr, 788 op/s 2024-02-26T06:29:43.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:43 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:29:45.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:45 smithi080 ceph-mon[36206]: pgmap v193: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-02-26T06:29:45.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:45 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:29:45.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:45 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:29:45.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:45 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:29:45.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:45 smithi115 ceph-mon[36426]: pgmap v193: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-02-26T06:29:45.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:45 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:29:45.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:45 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:29:45.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:45 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:29:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:45 smithi049 ceph-mon[31696]: pgmap v193: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-02-26T06:29:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:45 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:29:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:45 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:29:45.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:45 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:29:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 355.0 s, 2060.0 tps, lat 48.279 ms stddev 59.101 2024-02-26T06:29:47.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:47 smithi080 ceph-mon[36206]: pgmap v194: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 908 op/s 2024-02-26T06:29:47.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:47 smithi115 ceph-mon[36426]: pgmap v194: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 908 op/s 2024-02-26T06:29:47.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:47 smithi049 ceph-mon[31696]: pgmap v194: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 908 op/s 2024-02-26T06:29:49.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:49 smithi115 ceph-mon[36426]: pgmap v195: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-02-26T06:29:49.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:49 smithi049 ceph-mon[31696]: pgmap v195: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-02-26T06:29:49.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:49 smithi080 ceph-mon[36206]: pgmap v195: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-02-26T06:29:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 360.0 s, 1675.4 tps, lat 58.971 ms stddev 62.012 2024-02-26T06:29:51.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:51 smithi115 ceph-mon[36426]: pgmap v196: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 684 op/s 2024-02-26T06:29:51.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:51 smithi049 ceph-mon[31696]: pgmap v196: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 684 op/s 2024-02-26T06:29:51.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:51 smithi080 ceph-mon[36206]: pgmap v196: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 684 op/s 2024-02-26T06:29:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.205+0000 7f5e49600640 1 -- 172.21.15.49:0/3661003201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e4410d9d0 msgr2=0x7f5e4410fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.205+0000 7f5e49600640 1 --2- 172.21.15.49:0/3661003201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e4410d9d0 0x7f5e4410fde0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009be0 tx=0x7f5e3c02bf80 comp rx=0 tx=0).stop 2024-02-26T06:29:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.205+0000 7f5e49600640 1 -- 172.21.15.49:0/3661003201 shutdown_connections 2024-02-26T06:29:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.205+0000 7f5e49600640 1 --2- 172.21.15.49:0/3661003201 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e44068d40 0x7f5e44069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.205+0000 7f5e49600640 1 --2- 172.21.15.49:0/3661003201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e4410d9d0 0x7f5e4410fde0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.205+0000 7f5e49600640 1 --2- 172.21.15.49:0/3661003201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e44102060 0x7f5e44068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.205+0000 7f5e49600640 1 -- 172.21.15.49:0/3661003201 >> 172.21.15.49:0/3661003201 conn(0x7f5e44076ed0 msgr2=0x7f5e440772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.205+0000 7f5e49600640 1 -- 172.21.15.49:0/3661003201 shutdown_connections 2024-02-26T06:29:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.206+0000 7f5e49600640 1 -- 172.21.15.49:0/3661003201 wait complete. 2024-02-26T06:29:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.206+0000 7f5e49600640 1 Processor -- start 2024-02-26T06:29:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e49600640 1 -- start start 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e49600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e44068d40 0x7f5e4419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e49600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e44102060 0x7f5e4419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e49600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e4410d9d0 0x7f5e44199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e49600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e44199e90 con 0x7f5e44102060 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e49600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4419a000 con 0x7f5e44068d40 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e49600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4419a1c0 con 0x7f5e4410d9d0 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e42000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e44102060 0x7f5e4419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e44068d40 0x7f5e4419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e42000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e44102060 0x7f5e4419fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39412/0 (socket says 172.21.15.49:39412) 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e42000640 1 -- 172.21.15.49:0/4255962604 learned_addr learned my addr 172.21.15.49:0/4255962604 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.207+0000 7f5e43400640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e4410d9d0 0x7f5e44199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.208+0000 7f5e42000640 1 -- 172.21.15.49:0/4255962604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e4410d9d0 msgr2=0x7f5e44199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.208+0000 7f5e42000640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e4410d9d0 0x7f5e44199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.208+0000 7f5e42000640 1 -- 172.21.15.49:0/4255962604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e44068d40 msgr2=0x7f5e4419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.208+0000 7f5e42000640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e44068d40 0x7f5e4419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.208+0000 7f5e42000640 1 -- 172.21.15.49:0/4255962604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34009560 con 0x7f5e44102060 2024-02-26T06:29:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.208+0000 7f5e42000640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e44102060 0x7f5e4419fb80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000e9f0 tx=0x7f5e3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.209+0000 7f5e3b400640 1 -- 172.21.15.49:0/4255962604 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30009720 con 0x7f5e44102060 2024-02-26T06:29:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.209+0000 7f5e3b400640 1 -- 172.21.15.49:0/4255962604 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e30005500 con 0x7f5e44102060 2024-02-26T06:29:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.209+0000 7f5e3b400640 1 -- 172.21.15.49:0/4255962604 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30010490 con 0x7f5e44102060 2024-02-26T06:29:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.209+0000 7f5e49600640 1 -- 172.21.15.49:0/4255962604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c009840 con 0x7f5e44102060 2024-02-26T06:29:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.209+0000 7f5e49600640 1 -- 172.21.15.49:0/4255962604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e441026c0 con 0x7f5e44102060 2024-02-26T06:29:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.214+0000 7f5e4c5bd740 1 Processor -- start 2024-02-26T06:29:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.216+0000 7f5e4c5bd740 1 -- start start 2024-02-26T06:29:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.217+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d358840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.217+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d359200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.217+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d3646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.217+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d3145f0 con 0x55a19d358d80 2024-02-26T06:29:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.217+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d2c6890 con 0x55a19d359af0 2024-02-26T06:29:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.217+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d2c3dc0 con 0x55a19d362300 2024-02-26T06:29:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.218+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d359200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.218+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d359200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39426/0 (socket says 172.21.15.49:39426) 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.218+0000 7f5e42a00640 1 -- 172.21.15.49:0/347002753 learned_addr learned my addr 172.21.15.49:0/347002753 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.218+0000 7f5e43400640 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d358840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.218+0000 7f5e3b400640 1 -- 172.21.15.49:0/4255962604 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5e30005020 con 0x7f5e44102060 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e42000640 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d3646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e42a00640 1 -- 172.21.15.49:0/347002753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 msgr2=0x55a19d3646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e42a00640 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d3646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e42a00640 1 -- 172.21.15.49:0/347002753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 msgr2=0x55a19d358840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e42a00640 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d358840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e42a00640 1 -- 172.21.15.49:0/347002753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a19d240ad0 con 0x55a19d358d80 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e42000640 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d3646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e43400640 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d358840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e42a00640 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d359200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009de0 tx=0x7f5e34011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1911a85b485450 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.219+0000 7f5e2f400640 1 -- 172.21.15.49:0/347002753 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34005220 con 0x55a19d358d80 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.220+0000 7f5e2f400640 1 -- 172.21.15.49:0/347002753 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e34004c80 con 0x55a19d358d80 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.220+0000 7f5e2f400640 1 -- 172.21.15.49:0/347002753 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400e970 con 0x55a19d358d80 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.220+0000 7f5e3b400640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5e280a2500 0x7f5e280a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.220+0000 7f5e3b400640 1 -- 172.21.15.49:0/4255962604 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5e30014070 con 0x7f5e44102060 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.220+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/347002753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 msgr2=0x55a19d359200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.220+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d359200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009de0 tx=0x7f5e34011810 comp rx=0 tx=0).stop 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.220+0000 7f5e43400640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5e280a2500 0x7f5e280a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.221+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/347002753 shutdown_connections 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.221+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d3646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.221+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d359200 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.221+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/347002753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d358840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.221+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/347002753 >> 172.21.15.49:0/347002753 conn(0x55a19d320480 msgr2=0x55a19d353a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.221+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/347002753 shutdown_connections 2024-02-26T06:29:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.221+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/347002753 wait complete. 2024-02-26T06:29:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.222+0000 7f5e43400640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5e280a2500 0x7f5e280a49c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c004af0 tx=0x7f5e3c005fd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.222+0000 7f5e4c5bd740 1 -- start start 2024-02-26T06:29:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.223+0000 7f5e4c5bd740 1 Processor -- start 2024-02-26T06:29:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.224+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d35f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.224+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d35f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.224+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d35fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.224+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d2bf5a0 con 0x55a19d358d80 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.224+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d264b20 con 0x55a19d359af0 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.224+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d2647c0 con 0x55a19d362300 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.224+0000 7f5e4c5bd740 10 client.0 _ll_register_callbacks cb 0x55a19d278520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.224+0000 7f5e42000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d35f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.225+0000 7f5e43400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d35fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.225+0000 7f5e43400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d35fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:45850/0 (socket says 172.21.15.49:45850) 2024-02-26T06:29:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.225+0000 7f5e43400640 1 -- 172.21.15.49:0/1013435955 learned_addr learned my addr 172.21.15.49:0/1013435955 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.225+0000 7f5e43400640 1 -- 172.21.15.49:0/1013435955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 msgr2=0x55a19d35f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.225+0000 7f5e43400640 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d35f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.225+0000 7f5e43400640 1 -- 172.21.15.49:0/1013435955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 msgr2=0x55a19d35f2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.226+0000 7f5e42a00640 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d35f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.226+0000 7f5e43400640 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d35f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.226+0000 7f5e43400640 1 -- 172.21.15.49:0/1013435955 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34009a10 con 0x55a19d362300 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.227+0000 7f5e42000640 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d35f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.227+0000 7f5e42a00640 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d35f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.227+0000 7f5e43400640 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d35fd20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c035000 tx=0x7f5e3c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d332c040050fbba0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.227+0000 7f5e23e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.227+0000 7f5e23e00640 1 -- 172.21.15.49:0/1013435955 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c004690 con 0x55a19d362300 2024-02-26T06:29:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.227+0000 7f5e23e00640 1 -- 172.21.15.49:0/1013435955 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5e3c0047f0 con 0x55a19d362300 2024-02-26T06:29:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/1013435955 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a19d291ca0 con 0x55a19d362300 2024-02-26T06:29:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e23e00640 1 -- 172.21.15.49:0/1013435955 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e3c0349b0 con 0x55a19d362300 2024-02-26T06:29:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 10 client.27001 fetch_fsmap learned FSMap version 38 2024-02-26T06:29:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/1013435955 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a19d2bdc20 con 0x55a19d362300 2024-02-26T06:29:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e23e00640 1 -- 172.21.15.49:0/1013435955 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a19d291ca0 con 0x55a19d362300 2024-02-26T06:29:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e23e00640 1 -- 172.21.15.49:0/1013435955 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5e3c048040 con 0x55a19d362300 2024-02-26T06:29:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 10 client.27001 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:29:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 10 client.27001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 20 client.27001 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a19d372260 0x55a19d374680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 4 client.27001 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.228+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/1013435955 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a19d2b5cb0 con 0x55a19d372260 2024-02-26T06:29:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.229+0000 7f5e42a00640 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a19d372260 0x55a19d374680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.230+0000 7f5e23e00640 10 client.27001 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.230+0000 7f5e42a00640 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a19d372260 0x55a19d374680 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ca39f0841af1a0d server_cookie=c030710ad7dc6229 in_seq=0 out_seq=0 2024-02-26T06:29:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.231+0000 7f5e23e00640 1 -- 172.21.15.49:0/1013435955 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a19d2b5cb0 con 0x55a19d372260 2024-02-26T06:29:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.231+0000 7f5e23e00640 10 client.27001 handle_command_reply: tid=0 2024-02-26T06:29:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.231+0000 7f5e4c5bd740 1 client.27001 shutdown 2024-02-26T06:29:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.232+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/1013435955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 msgr2=0x55a19d35fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.232+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d35fd20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c035000 tx=0x7f5e3c009500 comp rx=0 tx=0).stop 2024-02-26T06:29:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.232+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/1013435955 shutdown_connections 2024-02-26T06:29:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.232+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a19d372260 0x55a19d374680 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.232+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d359af0 0x55a19d35f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.232+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d362300 0x55a19d35fd20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.232+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/1013435955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d358d80 0x55a19d35f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.233+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/1013435955 >> 172.21.15.49:0/1013435955 conn(0x55a19d320480 msgr2=0x55a19d3557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.233+0000 7f5e23e00640 0 client.27001 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.233+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/1013435955 shutdown_connections 2024-02-26T06:29:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.233+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/1013435955 wait complete. 2024-02-26T06:29:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.233+0000 7f5e4c5bd740 20 client.27001 trim_cache size 0 max 16384 2024-02-26T06:29:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.256+0000 7f5e4c5bd740 1 Processor -- start 2024-02-26T06:29:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.256+0000 7f5e4c5bd740 1 -- start start 2024-02-26T06:29:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.256+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d358d80 0x55a19d3619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.256+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d359af0 0x55a19d35c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.256+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d362300 0x55a19d35caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.256+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d007e60 con 0x55a19d362300 2024-02-26T06:29:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.256+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d35a0e0 con 0x55a19d358d80 2024-02-26T06:29:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.257+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d354870 con 0x55a19d359af0 2024-02-26T06:29:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.257+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d358d80 0x55a19d3619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.257+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d358d80 0x55a19d3619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50122/0 (socket says 172.21.15.49:50122) 2024-02-26T06:29:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.257+0000 7f5e42a00640 1 -- 172.21.15.49:0/26991593 learned_addr learned my addr 172.21.15.49:0/26991593 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.258+0000 7f5e42a00640 1 -- 172.21.15.49:0/26991593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d359af0 msgr2=0x55a19d35c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.258+0000 7f5e42a00640 1 --2- 172.21.15.49:0/26991593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d359af0 0x55a19d35c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.258+0000 7f5e42a00640 1 -- 172.21.15.49:0/26991593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d362300 msgr2=0x55a19d35caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.258+0000 7f5e42a00640 1 --2- 172.21.15.49:0/26991593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d362300 0x55a19d35caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.258+0000 7f5e42a00640 1 -- 172.21.15.49:0/26991593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a19d25a500 con 0x55a19d358d80 2024-02-26T06:29:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.258+0000 7f5e42a00640 1 --2- 172.21.15.49:0/26991593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d358d80 0x55a19d3619d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5e3401b040 tx=0x7f5e34005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed01611a374681f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.259+0000 7f5e2ea00640 1 -- 172.21.15.49:0/26991593 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400eaf0 con 0x55a19d358d80 2024-02-26T06:29:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.259+0000 7f5e2ea00640 1 -- 172.21.15.49:0/26991593 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e3400ec50 con 0x55a19d358d80 2024-02-26T06:29:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.259+0000 7f5e2ea00640 1 -- 172.21.15.49:0/26991593 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400eaf0 con 0x55a19d358d80 2024-02-26T06:29:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.260+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/26991593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d358d80 msgr2=0x55a19d3619d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.260+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/26991593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d358d80 0x55a19d3619d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5e3401b040 tx=0x7f5e34005fd0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.260+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/26991593 shutdown_connections 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.260+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/26991593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d362300 0x55a19d35caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.260+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/26991593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d359af0 0x55a19d35c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.260+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/26991593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d358d80 0x55a19d3619d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.260+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/26991593 >> 172.21.15.49:0/26991593 conn(0x55a19d320480 msgr2=0x55a19d35dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.261+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/26991593 shutdown_connections 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.261+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/26991593 wait complete. 2024-02-26T06:29:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.261+0000 7f5e4c5bd740 1 -- start start 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e4c5bd740 1 Processor -- start 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d358d80 0x55a19d368290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d359af0 0x55a19d3687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e4c5bd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d362300 0x55a19d3c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d3c2cc0 con 0x55a19d359af0 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d3c2e30 con 0x55a19d362300 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e4c5bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19d3c2ff0 con 0x55a19d358d80 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e4c5bd740 10 client.0 _ll_register_callbacks cb 0x7f5e3c003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d358d80 0x55a19d368290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.262+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d358d80 0x55a19d368290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:45860/0 (socket says 172.21.15.49:45860) 2024-02-26T06:29:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42a00640 1 -- 172.21.15.49:0/4087083168 learned_addr learned my addr 172.21.15.49:0/4087083168 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42a00640 1 -- 172.21.15.49:0/4087083168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d362300 msgr2=0x55a19d3c2780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42a00640 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d362300 0x55a19d3c2780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42a00640 1 -- 172.21.15.49:0/4087083168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d359af0 msgr2=0x55a19d3687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42000640 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d359af0 0x55a19d3687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42a00640 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d359af0 0x55a19d3687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42a00640 1 -- 172.21.15.49:0/4087083168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a19d240ad0 con 0x55a19d358d80 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42000640 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d359af0 0x55a19d3687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e42a00640 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d358d80 0x55a19d368290 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009430 tx=0x7f5e34005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca73edd1d83894dc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.263+0000 7f5e23e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.264+0000 7f5e23e00640 1 -- 172.21.15.49:0/4087083168 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400edc0 con 0x55a19d358d80 2024-02-26T06:29:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.264+0000 7f5e23e00640 1 -- 172.21.15.49:0/4087083168 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f5e3400e8f0 con 0x55a19d358d80 2024-02-26T06:29:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.264+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/4087083168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a19d2e4d00 con 0x55a19d358d80 2024-02-26T06:29:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.265+0000 7f5e23e00640 1 -- 172.21.15.49:0/4087083168 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e3400eab0 con 0x55a19d358d80 2024-02-26T06:29:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.265+0000 7f5e4c5bd740 10 client.27004 fetch_fsmap learned FSMap version 38 2024-02-26T06:29:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.265+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/4087083168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a19d280290 con 0x55a19d358d80 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.266+0000 7f5e23e00640 1 -- 172.21.15.49:0/4087083168 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a19d2e4d00 con 0x55a19d358d80 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.266+0000 7f5e23e00640 1 -- 172.21.15.49:0/4087083168 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5e34030680 con 0x55a19d358d80 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.266+0000 7f5e4c5bd740 10 client.27004 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.266+0000 7f5e4c5bd740 10 client.27004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.266+0000 7f5e4c5bd740 20 client.27004 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.266+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a19d3db7b0 0x55a19d3ddba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.266+0000 7f5e4c5bd740 4 client.27004 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.266+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/4087083168 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a19cfbf600 con 0x55a19d3db7b0 2024-02-26T06:29:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.267+0000 7f5e42000640 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a19d3db7b0 0x55a19d3ddba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.267+0000 7f5e42000640 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a19d3db7b0 0x55a19d3ddba0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb190899c4b7b90 server_cookie=61c76080ca05532f in_seq=0 out_seq=0 2024-02-26T06:29:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.267+0000 7f5e23e00640 10 client.27004 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.329+0000 7f5e23e00640 1 -- 172.21.15.49:0/4087083168 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a19cfbf600 con 0x55a19d3db7b0 2024-02-26T06:29:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.329+0000 7f5e23e00640 10 client.27004 handle_command_reply: tid=0 2024-02-26T06:29:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.329+0000 7f5e4c5bd740 1 client.27004 shutdown 2024-02-26T06:29:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/4087083168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d358d80 msgr2=0x55a19d368290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d358d80 0x55a19d368290 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009430 tx=0x7f5e34005e80 comp rx=0 tx=0).stop 2024-02-26T06:29:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/4087083168 shutdown_connections 2024-02-26T06:29:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a19d3db7b0 0x55a19d3ddba0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a19d362300 0x55a19d3c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e23e00640 0 client.27004 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a19d359af0 0x55a19d3687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 --2- 172.21.15.49:0/4087083168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a19d358d80 0x55a19d368290 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/4087083168 >> 172.21.15.49:0/4087083168 conn(0x55a19d320480 msgr2=0x55a19d352480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/4087083168 shutdown_connections 2024-02-26T06:29:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.330+0000 7f5e4c5bd740 1 -- 172.21.15.49:0/4087083168 wait complete. 2024-02-26T06:29:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.331+0000 7f5e4c5bd740 20 client.27004 trim_cache size 0 max 16384 2024-02-26T06:29:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:29:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:29:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:29:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:29:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 -- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5e280a2500 msgr2=0x7f5e280a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5e280a2500 0x7f5e280a49c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c004af0 tx=0x7f5e3c005fd0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 -- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e44102060 msgr2=0x7f5e4419fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e44102060 0x7f5e4419fb80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000e9f0 tx=0x7f5e3000edf0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 -- 172.21.15.49:0/4255962604 shutdown_connections 2024-02-26T06:29:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5e280a2500 0x7f5e280a49c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e4410d9d0 0x7f5e44199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e44102060 0x7f5e4419fb80 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000e9f0 tx=0x7f5e3000edf0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 --2- 172.21.15.49:0/4255962604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e44068d40 0x7f5e4419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 -- 172.21.15.49:0/4255962604 >> 172.21.15.49:0/4255962604 conn(0x7f5e44076ed0 msgr2=0x7f5e440ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.333+0000 7f5e49600640 1 -- 172.21.15.49:0/4255962604 shutdown_connections 2024-02-26T06:29:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:29:52.334+0000 7f5e49600640 1 -- 172.21.15.49:0/4255962604 wait complete. 2024-02-26T06:29:53.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:53 smithi049 ceph-mon[31696]: pgmap v197: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 673 op/s 2024-02-26T06:29:53.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:53 smithi080 ceph-mon[36206]: pgmap v197: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 673 op/s 2024-02-26T06:29:53.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:53 smithi115 ceph-mon[36426]: pgmap v197: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 673 op/s 2024-02-26T06:29:55.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:55 smithi049 ceph-mon[31696]: pgmap v198: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-02-26T06:29:55.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:55 smithi080 ceph-mon[36206]: pgmap v198: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-02-26T06:29:55.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:55 smithi115 ceph-mon[36426]: pgmap v198: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-02-26T06:29:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 365.0 s, 2317.6 tps, lat 43.824 ms stddev 51.946 2024-02-26T06:29:57.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:57 smithi049 ceph-mon[31696]: pgmap v199: 97 pgs: 97 active+clean; 4.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.7 MiB/s wr, 840 op/s 2024-02-26T06:29:57.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:57 smithi080 ceph-mon[36206]: pgmap v199: 97 pgs: 97 active+clean; 4.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.7 MiB/s wr, 840 op/s 2024-02-26T06:29:57.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:57 smithi115 ceph-mon[36426]: pgmap v199: 97 pgs: 97 active+clean; 4.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.7 MiB/s wr, 840 op/s 2024-02-26T06:29:59.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:29:59.026 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:29:59.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.325+0000 7f2fd5000640 1 -- 172.21.15.49:0/3051174373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fd0103590 msgr2=0x7f2fd0103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.325+0000 7f2fd5000640 1 --2- 172.21.15.49:0/3051174373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fd0103590 0x7f2fd0103970 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009890 tx=0x7f2fc002bb70 comp rx=0 tx=0).stop 2024-02-26T06:29:59.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.325+0000 7f2fd5000640 1 -- 172.21.15.49:0/3051174373 shutdown_connections 2024-02-26T06:29:59.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.325+0000 7f2fd5000640 1 --2- 172.21.15.49:0/3051174373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0108660 0x7f2fd0110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.325+0000 7f2fd5000640 1 --2- 172.21.15.49:0/3051174373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fd0103f40 0x7f2fd0107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.325+0000 7f2fd5000640 1 --2- 172.21.15.49:0/3051174373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fd0103590 0x7f2fd0103970 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.325+0000 7f2fd5000640 1 -- 172.21.15.49:0/3051174373 >> 172.21.15.49:0/3051174373 conn(0x7f2fd00fee30 msgr2=0x7f2fd0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:59.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.329+0000 7f2fd5000640 1 -- 172.21.15.49:0/3051174373 shutdown_connections 2024-02-26T06:29:59.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.329+0000 7f2fd5000640 1 -- 172.21.15.49:0/3051174373 wait complete. 2024-02-26T06:29:59.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.330+0000 7f2fd5000640 1 Processor -- start 2024-02-26T06:29:59.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.330+0000 7f2fd5000640 1 -- start start 2024-02-26T06:29:59.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.331+0000 7f2fd5000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fd0103590 0x7f2fd019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.331+0000 7f2fd5000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0108660 0x7f2fd019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.331+0000 7f2fd5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fd0199860 0x7f2fd0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.331+0000 7f2fd5000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd019a250 con 0x7f2fd0103590 2024-02-26T06:29:59.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.331+0000 7f2fd5000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd019a3c0 con 0x7f2fd0199860 2024-02-26T06:29:59.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.331+0000 7f2fd5000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd019a580 con 0x7f2fd0108660 2024-02-26T06:29:59.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.332+0000 7f2fcde00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fd0103590 0x7f2fd019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.332+0000 7f2fcde00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fd0103590 0x7f2fd019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39462/0 (socket says 172.21.15.49:39462) 2024-02-26T06:29:59.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.332+0000 7f2fcde00640 1 -- 172.21.15.49:0/1444575419 learned_addr learned my addr 172.21.15.49:0/1444575419 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:59.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.333+0000 7f2fcde00640 1 -- 172.21.15.49:0/1444575419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0108660 msgr2=0x7f2fd019fbf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:29:59.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.334+0000 7f2fcde00640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0108660 0x7f2fd019fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.334+0000 7f2fcde00640 1 -- 172.21.15.49:0/1444575419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fd0199860 msgr2=0x7f2fd0199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.334+0000 7f2fce800640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fd0199860 0x7f2fd0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.334+0000 7f2fcde00640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fd0199860 0x7f2fd0199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.334+0000 7f2fcde00640 1 -- 172.21.15.49:0/1444575419 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc0009490 con 0x7f2fd0103590 2024-02-26T06:29:59.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.335+0000 7f2fcde00640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fd0103590 0x7f2fd019f6b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0000c00 tx=0x7f2fc00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.335+0000 7f2fc6a00640 1 -- 172.21.15.49:0/1444575419 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00349d0 con 0x7f2fd0103590 2024-02-26T06:29:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.335+0000 7f2fd5000640 1 -- 172.21.15.49:0/1444575419 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd01a6570 con 0x7f2fd0103590 2024-02-26T06:29:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.335+0000 7f2fd5000640 1 -- 172.21.15.49:0/1444575419 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd01a68e0 con 0x7f2fd0103590 2024-02-26T06:29:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.335+0000 7f2fc6a00640 1 -- 172.21.15.49:0/1444575419 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fc002c9f0 con 0x7f2fd0103590 2024-02-26T06:29:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.336+0000 7f2fc6a00640 1 -- 172.21.15.49:0/1444575419 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0032600 con 0x7f2fd0103590 2024-02-26T06:29:59.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.340+0000 7f2fc6a00640 1 -- 172.21.15.49:0/1444575419 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2fc003b040 con 0x7f2fd0103590 2024-02-26T06:29:59.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.343+0000 7f2fc6a00640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fb40a22b0 0x7f2fb40a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.343+0000 7f2fc6a00640 1 -- 172.21.15.49:0/1444575419 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2fc00efc60 con 0x7f2fd0103590 2024-02-26T06:29:59.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.343+0000 7f2fd7f86740 1 Processor -- start 2024-02-26T06:29:59.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.343+0000 7f2fcd400640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fb40a22b0 0x7f2fb40a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.343+0000 7f2fd7f86740 1 -- start start 2024-02-26T06:29:59.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cb7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cb8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cc39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786c27120 con 0x55a786cb8290 2024-02-26T06:29:59.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786c24650 con 0x55a786cb8f70 2024-02-26T06:29:59.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786c1fe30 con 0x55a786cc1600 2024-02-26T06:29:59.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cb7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cb7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50156/0 (socket says 172.21.15.49:50156) 2024-02-26T06:29:59.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcde00640 1 -- 172.21.15.49:0/3464534286 learned_addr learned my addr 172.21.15.49:0/3464534286 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:59.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcd400640 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cb8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fce800640 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cc39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcde00640 1 -- 172.21.15.49:0/3464534286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 msgr2=0x55a786cc39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcde00640 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cc39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcde00640 1 -- 172.21.15.49:0/3464534286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 msgr2=0x55a786cb8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcde00640 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cb8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcde00640 1 -- 172.21.15.49:0/3464534286 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a786bd9030 con 0x55a786cb8f70 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcd400640 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cb8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.344+0000 7f2fcd400640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fb40a22b0 0x7f2fb40a4770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0109c80 tx=0x7f2fbc00b370 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.345+0000 7f2fcde00640 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cb7cc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00b90e0 tx=0x7f2fc00035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b54e730f8da9d81 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.345+0000 7f2fbb400640 1 -- 172.21.15.49:0/3464534286 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00a70e0 con 0x55a786cb8f70 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.345+0000 7f2fbb400640 1 -- 172.21.15.49:0/3464534286 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fc00bc710 con 0x55a786cb8f70 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.345+0000 7f2fbb400640 1 -- 172.21.15.49:0/3464534286 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0003680 con 0x55a786cb8f70 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 -- 172.21.15.49:0/3464534286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 msgr2=0x55a786cb7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cb7cc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00b90e0 tx=0x7f2fc00035f0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 -- 172.21.15.49:0/3464534286 shutdown_connections 2024-02-26T06:29:59.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cc39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cb8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/3464534286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cb7cc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 -- 172.21.15.49:0/3464534286 >> 172.21.15.49:0/3464534286 conn(0x55a786c7f570 msgr2=0x55a786cb2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:59.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 -- 172.21.15.49:0/3464534286 shutdown_connections 2024-02-26T06:29:59.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.346+0000 7f2fd7f86740 1 -- 172.21.15.49:0/3464534286 wait complete. 2024-02-26T06:29:59.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.347+0000 7f2fd7f86740 1 -- start start 2024-02-26T06:29:59.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.347+0000 7f2fd7f86740 1 Processor -- start 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.348+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cbe4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.348+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cbe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.348+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cbef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.348+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786bc4e90 con 0x55a786cb8290 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.348+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786bc4b30 con 0x55a786cb8f70 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.348+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78698e1c0 con 0x55a786cc1600 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.348+0000 7f2fd7f86740 10 client.0 _ll_register_callbacks cb 0x55a786bd8930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.349+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cbe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.349+0000 7f2fce800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cbef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.349+0000 7f2fce800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cbef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:45894/0 (socket says 172.21.15.49:45894) 2024-02-26T06:29:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.349+0000 7f2fce800640 1 -- 172.21.15.49:0/347792348 learned_addr learned my addr 172.21.15.49:0/347792348 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.349+0000 7f2fcde00640 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cbe4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.350+0000 7f2fce800640 1 -- 172.21.15.49:0/347792348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 msgr2=0x55a786cbe9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.350+0000 7f2fce800640 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cbe9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.350+0000 7f2fce800640 1 -- 172.21.15.49:0/347792348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 msgr2=0x55a786cbe4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.350+0000 7f2fce800640 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cbe4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.350+0000 7f2fce800640 1 -- 172.21.15.49:0/347792348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd01a68e0 con 0x55a786cc1600 2024-02-26T06:29:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.350+0000 7f2fcd400640 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cbe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.350+0000 7f2fce800640 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cbef30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8010be0 tx=0x7f2fc800e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d0ffed736096743 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.350+0000 7f2fafe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.351+0000 7f2fafe00640 1 -- 172.21.15.49:0/347792348 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8015070 con 0x55a786cc1600 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.351+0000 7f2fafe00640 1 -- 172.21.15.49:0/347792348 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2fc8003970 con 0x55a786cc1600 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.351+0000 7f2fd7f86740 1 -- 172.21.15.49:0/347792348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a78691f7e0 con 0x55a786cc1600 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.351+0000 7f2fafe00640 1 -- 172.21.15.49:0/347792348 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc8009af0 con 0x55a786cc1600 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.351+0000 7f2fd7f86740 10 client.27013 fetch_fsmap learned FSMap version 38 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.351+0000 7f2fd7f86740 1 -- 172.21.15.49:0/347792348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a786c1e4b0 con 0x55a786cc1600 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.351+0000 7f2fafe00640 1 -- 172.21.15.49:0/347792348 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a78691f7e0 con 0x55a786cc1600 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.352+0000 7f2fafe00640 1 -- 172.21.15.49:0/347792348 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2fc8021870 con 0x55a786cc1600 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.352+0000 7f2fd7f86740 10 client.27013 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.352+0000 7f2fd7f86740 10 client.27013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.352+0000 7f2fd7f86740 20 client.27013 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.352+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a786cc8390 0x55a786cca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.352+0000 7f2fd7f86740 4 client.27013 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:59.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.352+0000 7f2fd7f86740 1 -- 172.21.15.49:0/347792348 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a78695c720 con 0x55a786cc8390 2024-02-26T06:29:59.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.353+0000 7f2fcde00640 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a786cc8390 0x55a786cca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.354+0000 7f2fcde00640 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a786cc8390 0x55a786cca7b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d38c65e1fed2973 server_cookie=be5faa7dfbe892e3 in_seq=0 out_seq=0 2024-02-26T06:29:59.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.354+0000 7f2fafe00640 10 client.27013 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.356+0000 7f2fafe00640 1 -- 172.21.15.49:0/347792348 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a78695c720 con 0x55a786cc8390 2024-02-26T06:29:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.356+0000 7f2fafe00640 10 client.27013 handle_command_reply: tid=0 2024-02-26T06:29:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.356+0000 7f2fd7f86740 1 client.27013 shutdown 2024-02-26T06:29:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fd7f86740 1 -- 172.21.15.49:0/347792348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 msgr2=0x55a786cbef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cbef30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8010be0 tx=0x7f2fc800e7c0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fd7f86740 1 -- 172.21.15.49:0/347792348 shutdown_connections 2024-02-26T06:29:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a786cc8390 0x55a786cca7b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786cbe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786cbef30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/347792348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cbe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fd7f86740 1 -- 172.21.15.49:0/347792348 >> 172.21.15.49:0/347792348 conn(0x55a786c7f570 msgr2=0x55a786cb4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.357+0000 7f2fafe00640 0 client.27013 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.358+0000 7f2fd7f86740 1 -- 172.21.15.49:0/347792348 shutdown_connections 2024-02-26T06:29:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.358+0000 7f2fd7f86740 1 -- 172.21.15.49:0/347792348 wait complete. 2024-02-26T06:29:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.358+0000 7f2fd7f86740 20 client.27013 trim_cache size 0 max 16384 2024-02-26T06:29:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.396+0000 7f2fd7f86740 1 Processor -- start 2024-02-26T06:29:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.396+0000 7f2fd7f86740 1 -- start start 2024-02-26T06:29:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.397+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cc0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.397+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.397+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.397+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.397+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50170/0 (socket says 172.21.15.49:50170) 2024-02-26T06:29:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.399+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786cadc90 con 0x55a786cb8290 2024-02-26T06:29:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.399+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786cbf480 con 0x55a786cb8f70 2024-02-26T06:29:59.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.399+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786cc4d50 con 0x55a786cc1600 2024-02-26T06:29:59.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.399+0000 7f2fcd400640 1 -- 172.21.15.49:0/2382374132 learned_addr learned my addr 172.21.15.49:0/2382374132 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.399+0000 7f2fce800640 1 --2- 172.21.15.49:0/2382374132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.400+0000 7f2fcd400640 1 -- 172.21.15.49:0/2382374132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 msgr2=0x55a786ccf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.400+0000 7f2fcd400640 1 --2- 172.21.15.49:0/2382374132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.400+0000 7f2fcd400640 1 -- 172.21.15.49:0/2382374132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 msgr2=0x55a786cc0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.400+0000 7f2fcd400640 1 --2- 172.21.15.49:0/2382374132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cc0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.400+0000 7f2fcd400640 1 -- 172.21.15.49:0/2382374132 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a786bba920 con 0x55a786cb8f70 2024-02-26T06:29:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.400+0000 7f2fcd400640 1 --2- 172.21.15.49:0/2382374132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccf110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2fd019b050 tx=0x7f2fbc010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b19cbe890cd1f1fa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.401+0000 7f2fbaa00640 1 -- 172.21.15.49:0/2382374132 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc004020 con 0x55a786cb8f70 2024-02-26T06:29:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.401+0000 7f2fbaa00640 1 -- 172.21.15.49:0/2382374132 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc004180 con 0x55a786cb8f70 2024-02-26T06:29:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.401+0000 7f2fbaa00640 1 -- 172.21.15.49:0/2382374132 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc01f660 con 0x55a786cb8f70 2024-02-26T06:29:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.402+0000 7f2fd7f86740 1 -- 172.21.15.49:0/2382374132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 msgr2=0x55a786ccf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.402+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/2382374132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccf110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2fd019b050 tx=0x7f2fbc010000 comp rx=0 tx=0).stop 2024-02-26T06:29:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.404+0000 7f2fd7f86740 1 -- 172.21.15.49:0/2382374132 shutdown_connections 2024-02-26T06:29:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.404+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/2382374132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccf110 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.404+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/2382374132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.404+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/2382374132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786cc0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.404+0000 7f2fd7f86740 1 -- 172.21.15.49:0/2382374132 >> 172.21.15.49:0/2382374132 conn(0x55a786c7f570 msgr2=0x55a786cbcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.405+0000 7f2fd7f86740 1 -- 172.21.15.49:0/2382374132 shutdown_connections 2024-02-26T06:29:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.405+0000 7f2fd7f86740 1 -- 172.21.15.49:0/2382374132 wait complete. 2024-02-26T06:29:59.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.405+0000 7f2fd7f86740 1 -- start start 2024-02-26T06:29:59.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.406+0000 7f2fd7f86740 1 Processor -- start 2024-02-26T06:29:59.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.406+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786ccf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.408+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.408+0000 7f2fd7f86740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.408+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786ccdb80 con 0x55a786cb8290 2024-02-26T06:29:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.408+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786ccdcf0 con 0x55a786cb8f70 2024-02-26T06:29:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.408+0000 7f2fd7f86740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a786ccdeb0 con 0x55a786cc1600 2024-02-26T06:29:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.409+0000 7f2fd7f86740 10 client.0 _ll_register_callbacks cb 0x7f2fc8009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.409+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.409+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50176/0 (socket says 172.21.15.49:50176) 2024-02-26T06:29:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.409+0000 7f2fcd400640 1 -- 172.21.15.49:0/140318056 learned_addr learned my addr 172.21.15.49:0/140318056 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.409+0000 7f2fce800640 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.410+0000 7f2fce800640 1 -- 172.21.15.49:0/140318056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 msgr2=0x55a786ccd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:59.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.410+0000 7f2fce800640 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.410+0000 7f2fce800640 1 -- 172.21.15.49:0/140318056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 msgr2=0x55a786ccf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:59.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.410+0000 7f2fce800640 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786ccf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.410+0000 7f2fce800640 1 -- 172.21.15.49:0/140318056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc00dae0 con 0x55a786cc1600 2024-02-26T06:29:59.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.411+0000 7f2fce800640 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccd640 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8021c20 tx=0x7f2fc8004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ba761ab0586d6c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:59.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.411+0000 7f2fafe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:29:59.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.415+0000 7f2fafe00640 1 -- 172.21.15.49:0/140318056 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8015070 con 0x55a786cc1600 2024-02-26T06:29:59.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.415+0000 7f2fafe00640 1 -- 172.21.15.49:0/140318056 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f2fc8021800 con 0x55a786cc1600 2024-02-26T06:29:59.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.415+0000 7f2fd7f86740 1 -- 172.21.15.49:0/140318056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fc00bc710 con 0x55a786cc1600 2024-02-26T06:29:59.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.416+0000 7f2fafe00640 1 -- 172.21.15.49:0/140318056 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc80219e0 con 0x55a786cc1600 2024-02-26T06:29:59.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.416+0000 7f2fd7f86740 10 client.27019 fetch_fsmap learned FSMap version 38 2024-02-26T06:29:59.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.416+0000 7f2fd7f86740 1 -- 172.21.15.49:0/140318056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a786be0400 con 0x55a786cc1600 2024-02-26T06:29:59.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.416+0000 7f2fafe00640 1 -- 172.21.15.49:0/140318056 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fc00bc710 con 0x55a786cc1600 2024-02-26T06:29:59.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.417+0000 7f2fafe00640 1 -- 172.21.15.49:0/140318056 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2fc802a5e0 con 0x55a786cc1600 2024-02-26T06:29:59.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.417+0000 7f2fd7f86740 10 client.27019 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:29:59.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.417+0000 7f2fd7f86740 10 client.27019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:59.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.417+0000 7f2fd7f86740 20 client.27019 populate_metadata read hostname 'smithi049' 2024-02-26T06:29:59.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.417+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a786d3a6f0 0x55a786d3cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.417+0000 7f2fd7f86740 4 client.27019 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:29:59.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.417+0000 7f2fd7f86740 1 -- 172.21.15.49:0/140318056 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7869ab890 con 0x55a786d3a6f0 2024-02-26T06:29:59.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.418+0000 7f2fcde00640 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a786d3a6f0 0x55a786d3cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:29:59 smithi049 ceph-mon[31696]: pgmap v200: 97 pgs: 97 active+clean; 4.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-02-26T06:29:59.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.420+0000 7f2fcde00640 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a786d3a6f0 0x55a786d3cb10 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bebcd3a1c305bd5 server_cookie=d86dcde6c01e2844 in_seq=0 out_seq=0 2024-02-26T06:29:59.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.420+0000 7f2fafe00640 10 client.27019 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.422+0000 7f2fafe00640 1 -- 172.21.15.49:0/140318056 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7869ab890 con 0x55a786d3a6f0 2024-02-26T06:29:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.422+0000 7f2fafe00640 10 client.27019 handle_command_reply: tid=0 2024-02-26T06:29:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.422+0000 7f2fd7f86740 1 client.27019 shutdown 2024-02-26T06:29:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.422+0000 7f2fd7f86740 1 -- 172.21.15.49:0/140318056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 msgr2=0x55a786ccd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.422+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccd640 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8021c20 tx=0x7f2fc8004f90 comp rx=0 tx=0).stop 2024-02-26T06:29:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.422+0000 7f2fd7f86740 1 -- 172.21.15.49:0/140318056 shutdown_connections 2024-02-26T06:29:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.423+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a786d3a6f0 0x55a786d3cb10 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.423+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a786cb8f70 0x55a786ccd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.423+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a786cc1600 0x55a786ccd640 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.423+0000 7f2fd7f86740 1 --2- 172.21.15.49:0/140318056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a786cb8290 0x55a786ccf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.423+0000 7f2fafe00640 0 client.27019 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:29:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.423+0000 7f2fd7f86740 1 -- 172.21.15.49:0/140318056 >> 172.21.15.49:0/140318056 conn(0x55a786c7f570 msgr2=0x55a786cbe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.423+0000 7f2fd7f86740 1 -- 172.21.15.49:0/140318056 shutdown_connections 2024-02-26T06:29:59.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.424+0000 7f2fd7f86740 1 -- 172.21.15.49:0/140318056 wait complete. 2024-02-26T06:29:59.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.424+0000 7f2fd7f86740 20 client.27019 trim_cache size 0 max 16384 2024-02-26T06:29:59.424 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:29:59.424 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2024-02-26T06:29:59.424 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2024-02-26T06:29:59.424 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:29:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.426+0000 7f2fd5000640 1 -- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fb40a22b0 msgr2=0x7f2fb40a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.426+0000 7f2fd5000640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fb40a22b0 0x7f2fb40a4770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0109c80 tx=0x7f2fbc00b370 comp rx=0 tx=0).stop 2024-02-26T06:29:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.426+0000 7f2fd5000640 1 -- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fd0103590 msgr2=0x7f2fd019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.426+0000 7f2fd5000640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fd0103590 0x7f2fd019f6b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0000c00 tx=0x7f2fc00045e0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.427+0000 7f2fd5000640 1 -- 172.21.15.49:0/1444575419 shutdown_connections 2024-02-26T06:29:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.427+0000 7f2fd5000640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2fb40a22b0 0x7f2fb40a4770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.427+0000 7f2fd5000640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fd0199860 0x7f2fd0199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.427+0000 7f2fd5000640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0108660 0x7f2fd019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.427+0000 7f2fd5000640 1 --2- 172.21.15.49:0/1444575419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fd0103590 0x7f2fd019f6b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.427+0000 7f2fd5000640 1 -- 172.21.15.49:0/1444575419 >> 172.21.15.49:0/1444575419 conn(0x7f2fd00fee30 msgr2=0x7f2fd0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.428+0000 7f2fd5000640 1 -- 172.21.15.49:0/1444575419 shutdown_connections 2024-02-26T06:29:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.428+0000 7f2fd5000640 1 -- 172.21.15.49:0/1444575419 wait complete. 2024-02-26T06:29:59.457 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:29:59.457 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:29:59.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:29:59 smithi080 ceph-mon[36206]: pgmap v200: 97 pgs: 97 active+clean; 4.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-02-26T06:29:59.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:29:59 smithi115 ceph-mon[36426]: pgmap v200: 97 pgs: 97 active+clean; 4.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-02-26T06:29:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.728+0000 7f53dda00640 1 -- 172.21.15.49:0/1400023954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8111fb0 msgr2=0x7f53d81143a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.728+0000 7f53dda00640 1 --2- 172.21.15.49:0/1400023954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8111fb0 0x7f53d81143a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009900 tx=0x7f53c802bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.729+0000 7f53dda00640 1 -- 172.21.15.49:0/1400023954 shutdown_connections 2024-02-26T06:29:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.729+0000 7f53dda00640 1 --2- 172.21.15.49:0/1400023954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8111fb0 0x7f53d81143a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.729+0000 7f53dda00640 1 --2- 172.21.15.49:0/1400023954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d8075fd0 0x7f53d8111a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.729+0000 7f53dda00640 1 --2- 172.21.15.49:0/1400023954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53d80756b0 0x7f53d8075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.729+0000 7f53dda00640 1 -- 172.21.15.49:0/1400023954 >> 172.21.15.49:0/1400023954 conn(0x7f53d80fede0 msgr2=0x7f53d8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:59.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.729+0000 7f53dda00640 1 -- 172.21.15.49:0/1400023954 shutdown_connections 2024-02-26T06:29:59.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.729+0000 7f53dda00640 1 -- 172.21.15.49:0/1400023954 wait complete. 2024-02-26T06:29:59.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.731+0000 7f53dda00640 1 Processor -- start 2024-02-26T06:29:59.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.731+0000 7f53dda00640 1 -- start start 2024-02-26T06:29:59.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.731+0000 7f53dda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53d80756b0 0x7f53d819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.731+0000 7f53dda00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8075fd0 0x7f53d819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.731+0000 7f53dda00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d8111fb0 0x7f53d81a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.731+0000 7f53dda00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81a2cd0 con 0x7f53d8075fd0 2024-02-26T06:29:59.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.731+0000 7f53dda00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81a2e40 con 0x7f53d80756b0 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.731+0000 7f53dda00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81a3000 con 0x7f53d8111fb0 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53d80756b0 0x7f53d819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53d80756b0 0x7f53d819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50196/0 (socket says 172.21.15.49:50196) 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d6a00640 1 -- 172.21.15.49:0/1189055521 learned_addr learned my addr 172.21.15.49:0/1189055521 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d7400640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d8111fb0 0x7f53d81a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d7400640 1 -- 172.21.15.49:0/1189055521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53d80756b0 msgr2=0x7f53d819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d7400640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53d80756b0 0x7f53d819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d7400640 1 -- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8075fd0 msgr2=0x7f53d819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d6000640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8075fd0 0x7f53d819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d7400640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8075fd0 0x7f53d819e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.732+0000 7f53d7400640 1 -- 172.21.15.49:0/1189055521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8009560 con 0x7f53d8111fb0 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.733+0000 7f53d6000640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8075fd0 0x7f53d819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:59.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.733+0000 7f53d7400640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d8111fb0 0x7f53d81a26d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009230 tx=0x7f53c802c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:59.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.733+0000 7f53cf400640 1 -- 172.21.15.49:0/1189055521 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c802dea0 con 0x7f53d8111fb0 2024-02-26T06:29:59.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.734+0000 7f53cf400640 1 -- 172.21.15.49:0/1189055521 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53c8004410 con 0x7f53d8111fb0 2024-02-26T06:29:59.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.734+0000 7f53cf400640 1 -- 172.21.15.49:0/1189055521 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8003b80 con 0x7f53d8111fb0 2024-02-26T06:29:59.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.735+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d81a3280 con 0x7f53d8111fb0 2024-02-26T06:29:59.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.735+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d81a3530 con 0x7f53d8111fb0 2024-02-26T06:29:59.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.738+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d8076df0 con 0x7f53d8111fb0 2024-02-26T06:29:59.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.740+0000 7f53cf400640 1 -- 172.21.15.49:0/1189055521 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f53c8003ce0 con 0x7f53d8111fb0 2024-02-26T06:29:59.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.742+0000 7f53cf400640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f53bc0a22b0 0x7f53bc0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:59.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.743+0000 7f53cf400640 1 -- 172.21.15.49:0/1189055521 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f53c80f0490 con 0x7f53d8111fb0 2024-02-26T06:29:59.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.743+0000 7f53d6a00640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f53bc0a22b0 0x7f53bc0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:59.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.744+0000 7f53d6a00640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f53bc0a22b0 0x7f53bc0a4770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009fd0 tx=0x7f53d0009720 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:29:59.746+0000 7f53cf400640 1 -- 172.21.15.49:0/1189055521 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53c80b9240 con 0x7f53d8111fb0 2024-02-26T06:30:00.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.148+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53d80683c0 con 0x7f53d8111fb0 2024-02-26T06:30:00.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.148+0000 7f53cf400640 1 -- 172.21.15.49:0/1189055521 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+11339 (secure 0 0 0) 0x7f53c80bc900 con 0x7f53d8111fb0 2024-02-26T06:30:00.148 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:30:00.148 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:29:37.125296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":144,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:30:00.149 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 38 2024-02-26T06:30:00.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.152+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f53bc0a22b0 msgr2=0x7f53bc0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.152+0000 7f53dda00640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f53bc0a22b0 0x7f53bc0a4770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009fd0 tx=0x7f53d0009720 comp rx=0 tx=0).stop 2024-02-26T06:30:00.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.152+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d8111fb0 msgr2=0x7f53d81a26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.152+0000 7f53dda00640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d8111fb0 0x7f53d81a26d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009230 tx=0x7f53c802c6a0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.153+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 shutdown_connections 2024-02-26T06:30:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.153+0000 7f53dda00640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f53bc0a22b0 0x7f53bc0a4770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.153+0000 7f53dda00640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d8111fb0 0x7f53d81a26d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.153+0000 7f53dda00640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d8075fd0 0x7f53d819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.153+0000 7f53dda00640 1 --2- 172.21.15.49:0/1189055521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53d80756b0 0x7f53d819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.153+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 >> 172.21.15.49:0/1189055521 conn(0x7f53d80fede0 msgr2=0x7f53d8113600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.153+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 shutdown_connections 2024-02-26T06:30:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.153+0000 7f53dda00640 1 -- 172.21.15.49:0/1189055521 wait complete. 2024-02-26T06:30:00.179 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:30:00.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:30:00.296 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31692]: 2024-02-26T06:29:59.999+0000 7f751c200700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:30:00.296 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31692]: 2024-02-26T06:29:59.999+0000 7f751c200700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-26T06:30:00.296 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31692]: 2024-02-26T06:29:59.999+0000 7f751c200700 -1 log_channel(cluster) log [ERR] : mds.c(mds.2): Metadata damage detected 2024-02-26T06:30:00.296 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:00 smithi049 ceph-mon[31696]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:30:00.296 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:00 smithi049 ceph-mon[31696]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-26T06:30:00.296 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:00 smithi049 ceph-mon[31696]: mds.c(mds.2): Metadata damage detected 2024-02-26T06:30:00.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.498+0000 7f388b200640 1 -- 172.21.15.49:0/371738398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 msgr2=0x7f3884103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.498+0000 7f388b200640 1 --2- 172.21.15.49:0/371738398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 0x7f3884103970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3874009890 tx=0x7f387402bb90 comp rx=0 tx=0).stop 2024-02-26T06:30:00.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.500+0000 7f388b200640 1 -- 172.21.15.49:0/371738398 shutdown_connections 2024-02-26T06:30:00.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.500+0000 7f388b200640 1 --2- 172.21.15.49:0/371738398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3884103f40 0x7f3884107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.500+0000 7f388b200640 1 --2- 172.21.15.49:0/371738398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3884108660 0x7f3884110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.500+0000 7f388b200640 1 --2- 172.21.15.49:0/371738398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 0x7f3884103970 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.500+0000 7f388b200640 1 -- 172.21.15.49:0/371738398 >> 172.21.15.49:0/371738398 conn(0x7f38840fee30 msgr2=0x7f3884101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.500+0000 7f388b200640 1 -- 172.21.15.49:0/371738398 shutdown_connections 2024-02-26T06:30:00.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.500+0000 7f388b200640 1 -- 172.21.15.49:0/371738398 wait complete. 2024-02-26T06:30:00.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.502+0000 7f388b200640 1 Processor -- start 2024-02-26T06:30:00.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.502+0000 7f388b200640 1 -- start start 2024-02-26T06:30:00.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.504+0000 7f388b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 0x7f3884072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.504+0000 7f388b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3884108660 0x7f3884072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.504+0000 7f388b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388406b9f0 0x7f388406be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.504+0000 7f388b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388406c600 con 0x7f3884103590 2024-02-26T06:30:00.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.504+0000 7f388b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388406c770 con 0x7f3884108660 2024-02-26T06:30:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.504+0000 7f388b200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38840703a0 con 0x7f388406b9f0 2024-02-26T06:30:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3884108660 0x7f3884072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 0x7f3884072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3884108660 0x7f3884072dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54686/0 (socket says 172.21.15.49:54686) 2024-02-26T06:30:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883400640 1 -- 172.21.15.49:0/2561105320 learned_addr learned my addr 172.21.15.49:0/2561105320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3889200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388406b9f0 0x7f388406be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883e00640 1 -- 172.21.15.49:0/2561105320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388406b9f0 msgr2=0x7f388406be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883e00640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388406b9f0 0x7f388406be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883e00640 1 -- 172.21.15.49:0/2561105320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3884108660 msgr2=0x7f3884072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883e00640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3884108660 0x7f3884072dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883e00640 1 -- 172.21.15.49:0/2561105320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874009490 con 0x7f3884103590 2024-02-26T06:30:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.505+0000 7f3883400640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3884108660 0x7f3884072dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.506+0000 7f3883e00640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 0x7f3884072880 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3874037040 tx=0x7f3874034930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.506+0000 7f3880c00640 1 -- 172.21.15.49:0/2561105320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874005460 con 0x7f3884103590 2024-02-26T06:30:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.507+0000 7f388b200640 1 -- 172.21.15.49:0/2561105320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38840705f0 con 0x7f3884103590 2024-02-26T06:30:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.507+0000 7f3880c00640 1 -- 172.21.15.49:0/2561105320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38740055c0 con 0x7f3884103590 2024-02-26T06:30:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.507+0000 7f3880c00640 1 -- 172.21.15.49:0/2561105320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874005460 con 0x7f3884103590 2024-02-26T06:30:00.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.507+0000 7f388b200640 1 -- 172.21.15.49:0/2561105320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3884070ac0 con 0x7f3884103590 2024-02-26T06:30:00.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.512+0000 7f3880c00640 1 -- 172.21.15.49:0/2561105320 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f387403b040 con 0x7f3884103590 2024-02-26T06:30:00.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.513+0000 7f388e1e2740 1 Processor -- start 2024-02-26T06:30:00.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.513+0000 7f388e1e2740 1 -- start start 2024-02-26T06:30:00.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.513+0000 7f3880c00640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f38680a2390 0x7f38680a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.513+0000 7f3880c00640 1 -- 172.21.15.49:0/2561105320 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f38740ef630 con 0x7f3884103590 2024-02-26T06:30:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f3889200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f38680a2390 0x7f38680a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab634800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab63fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f388e1e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab59a1e0 con 0x5605ab634d40 2024-02-26T06:30:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f388e1e2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab597710 con 0x5605ab634420 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f388e1e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab592ef0 con 0x5605ab63d920 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f3883400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f3883e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab634800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f3883e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab634800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54700/0 (socket says 172.21.15.49:54700) 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f3883e00640 1 -- 172.21.15.49:0/2535265130 learned_addr learned my addr 172.21.15.49:0/2535265130 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.514+0000 7f3889200640 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab63fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.515+0000 7f3883400640 1 -- 172.21.15.49:0/2535265130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 msgr2=0x5605ab63fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.515+0000 7f3883400640 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab63fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.515+0000 7f3883400640 1 -- 172.21.15.49:0/2535265130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 msgr2=0x5605ab634800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.515+0000 7f3883400640 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab634800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.515+0000 7f3889200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f38680a2390 0x7f38680a4850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f387c005fd0 tx=0x7f387c0095c0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.515+0000 7f3883400640 1 -- 172.21.15.49:0/2535265130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ab554dc0 con 0x5605ab634d40 2024-02-26T06:30:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.515+0000 7f3883e00640 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab634800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.515+0000 7f3883400640 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63d3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f388406d150 tx=0x7f38700118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85fef714345ca28 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.516+0000 7f3879600640 1 -- 172.21.15.49:0/2535265130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870004030 con 0x5605ab634d40 2024-02-26T06:30:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.516+0000 7f3879600640 1 -- 172.21.15.49:0/2535265130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38700035f0 con 0x5605ab634d40 2024-02-26T06:30:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.516+0000 7f3879600640 1 -- 172.21.15.49:0/2535265130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387000f7f0 con 0x5605ab634d40 2024-02-26T06:30:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.517+0000 7f388e1e2740 1 -- 172.21.15.49:0/2535265130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 msgr2=0x5605ab63d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.517+0000 7f388e1e2740 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63d3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f388406d150 tx=0x7f38700118d0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.517+0000 7f388e1e2740 1 -- 172.21.15.49:0/2535265130 shutdown_connections 2024-02-26T06:30:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.517+0000 7f388e1e2740 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab63fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.517+0000 7f388e1e2740 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63d3e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.517+0000 7f388e1e2740 1 --2- 172.21.15.49:0/2535265130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab634800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.517+0000 7f388e1e2740 1 -- 172.21.15.49:0/2535265130 >> 172.21.15.49:0/2535265130 conn(0x5605ab62eb50 msgr2=0x5605ab62ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.518+0000 7f388e1e2740 1 -- 172.21.15.49:0/2535265130 shutdown_connections 2024-02-26T06:30:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.518+0000 7f388e1e2740 1 -- 172.21.15.49:0/2535265130 wait complete. 2024-02-26T06:30:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.518+0000 7f388e1e2740 1 -- start start 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.519+0000 7f388e1e2740 1 Processor -- start 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.519+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.519+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab63ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.519+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab63d920 0x5605ab63b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.519+0000 7f388e1e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab538120 con 0x5605ab63d920 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.519+0000 7f388e1e2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab537dc0 con 0x5605ab634420 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.519+0000 7f388e1e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab3011c0 con 0x5605ab634d40 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.519+0000 7f388e1e2740 10 client.0 _ll_register_callbacks cb 0x5605ab587280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54704/0 (socket says 172.21.15.49:54704) 2024-02-26T06:30:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 -- 172.21.15.49:0/1916139870 learned_addr learned my addr 172.21.15.49:0/1916139870 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3889200640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab63d920 0x5605ab63b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883400640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab63ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 -- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 msgr2=0x5605ab63ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab63ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 -- 172.21.15.49:0/1916139870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab63d920 msgr2=0x5605ab63b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab63d920 0x5605ab63b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 -- 172.21.15.49:0/1916139870 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3870009c10 con 0x5605ab634420 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883400640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab63ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:00.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3889200640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab63d920 0x5605ab63b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f3883e00640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63a930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f38740b8ab0 tx=0x7f38740bba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ddcd014807cda6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.520+0000 7f386e000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.521+0000 7f386e000640 1 -- 172.21.15.49:0/1916139870 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740d1070 con 0x5605ab634420 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.521+0000 7f386e000640 1 -- 172.21.15.49:0/1916139870 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f38740c0230 con 0x5605ab634420 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.521+0000 7f388e1e2740 1 -- 172.21.15.49:0/1916139870 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605ab2927e0 con 0x5605ab634420 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.522+0000 7f386e000640 1 -- 172.21.15.49:0/1916139870 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38740c4880 con 0x5605ab634420 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.522+0000 7f388e1e2740 10 client.27053 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.522+0000 7f388e1e2740 1 -- 172.21.15.49:0/1916139870 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5605ab591570 con 0x5605ab634420 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.522+0000 7f386e000640 1 -- 172.21.15.49:0/1916139870 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5605ab2927e0 con 0x5605ab634420 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.523+0000 7f386e000640 1 -- 172.21.15.49:0/1916139870 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3874044070 con 0x5605ab634420 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.523+0000 7f388e1e2740 10 client.27053 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.523+0000 7f388e1e2740 10 client.27053 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.523+0000 7f388e1e2740 20 client.27053 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:00.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.523+0000 7f388e1e2740 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5605ab64d540 0x5605ab64f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.523+0000 7f388e1e2740 4 client.27053 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:00.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.523+0000 7f388e1e2740 1 -- 172.21.15.49:0/1916139870 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605ab2cf720 con 0x5605ab64d540 2024-02-26T06:30:00.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.524+0000 7f3883400640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5605ab64d540 0x5605ab64f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.525+0000 7f3883400640 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5605ab64d540 0x5605ab64f930 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb807ff874a4fb72 server_cookie=993bfb5f7cfaf0d4 in_seq=0 out_seq=0 2024-02-26T06:30:00.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.525+0000 7f386e000640 10 client.27053 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:30:00.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.525+0000 7f386e000640 1 -- 172.21.15.49:0/1916139870 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605ab2cf720 con 0x5605ab64d540 2024-02-26T06:30:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.525+0000 7f386e000640 10 client.27053 handle_command_reply: tid=0 2024-02-26T06:30:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.525+0000 7f388e1e2740 1 client.27053 shutdown 2024-02-26T06:30:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 -- 172.21.15.49:0/1916139870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 msgr2=0x5605ab63a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63a930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f38740b8ab0 tx=0x7f38740bba40 comp rx=0 tx=0).stop 2024-02-26T06:30:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 -- 172.21.15.49:0/1916139870 shutdown_connections 2024-02-26T06:30:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5605ab64d540 0x5605ab64f930 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab63d920 0x5605ab63b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab63ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 --2- 172.21.15.49:0/1916139870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63a930 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 -- 172.21.15.49:0/1916139870 >> 172.21.15.49:0/1916139870 conn(0x5605ab62eb50 msgr2=0x5605ab6308a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f386e000640 0 client.27053 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:30:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 -- 172.21.15.49:0/1916139870 shutdown_connections 2024-02-26T06:30:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 1 -- 172.21.15.49:0/1916139870 wait complete. 2024-02-26T06:30:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.526+0000 7f388e1e2740 20 client.27053 trim_cache size 0 max 16384 2024-02-26T06:30:00.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.551+0000 7f388e1e2740 1 Processor -- start 2024-02-26T06:30:00.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.551+0000 7f388e1e2740 1 -- start start 2024-02-26T06:30:00.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.551+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.551+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab637bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f388e1e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab6496e0 con 0x5605ab634d40 2024-02-26T06:30:00.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f388e1e2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab62e680 con 0x5605ab634420 2024-02-26T06:30:00.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f388e1e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab621c10 con 0x5605ab63d920 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54712/0 (socket says 172.21.15.49:54712) 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883e00640 1 -- 172.21.15.49:0/3334745391 learned_addr learned my addr 172.21.15.49:0/3334745391 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883400640 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3889200640 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab637bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883e00640 1 -- 172.21.15.49:0/3334745391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 msgr2=0x5605ab637bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883e00640 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab637bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883e00640 1 -- 172.21.15.49:0/3334745391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 msgr2=0x5605ab63b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883e00640 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3883e00640 1 -- 172.21.15.49:0/3334745391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ab52da80 con 0x5605ab634420 2024-02-26T06:30:00.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.552+0000 7f3889200640 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab637bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:00.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.553+0000 7f3883e00640 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63d060 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f38740b8a60 tx=0x7f387402ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fac5a195a08a8c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.553+0000 7f3878c00640 1 -- 172.21.15.49:0/3334745391 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740d1070 con 0x5605ab634420 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.553+0000 7f3878c00640 1 -- 172.21.15.49:0/3334745391 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38740c0230 con 0x5605ab634420 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.553+0000 7f3878c00640 1 -- 172.21.15.49:0/3334745391 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740ed100 con 0x5605ab634420 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.556+0000 7f388e1e2740 1 -- 172.21.15.49:0/3334745391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 msgr2=0x5605ab63d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.556+0000 7f388e1e2740 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63d060 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f38740b8a60 tx=0x7f387402ca40 comp rx=0 tx=0).stop 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.556+0000 7f388e1e2740 1 -- 172.21.15.49:0/3334745391 shutdown_connections 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.556+0000 7f388e1e2740 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab63d920 0x5605ab637bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.556+0000 7f388e1e2740 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634d40 0x5605ab63b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.556+0000 7f388e1e2740 1 --2- 172.21.15.49:0/3334745391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab634420 0x5605ab63d060 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.556+0000 7f388e1e2740 1 -- 172.21.15.49:0/3334745391 >> 172.21.15.49:0/3334745391 conn(0x5605ab62eb50 msgr2=0x5605ab639240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.557+0000 7f388e1e2740 1 -- 172.21.15.49:0/3334745391 shutdown_connections 2024-02-26T06:30:00.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.557+0000 7f388e1e2740 1 -- 172.21.15.49:0/3334745391 wait complete. 2024-02-26T06:30:00.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.558+0000 7f388e1e2740 1 -- start start 2024-02-26T06:30:00.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.558+0000 7f388e1e2740 1 Processor -- start 2024-02-26T06:30:00.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.558+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634420 0x5605ab643210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.558+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab643750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.558+0000 7f388e1e2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab63d920 0x5605ab643c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.559+0000 7f388e1e2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab6a39f0 con 0x5605ab634420 2024-02-26T06:30:00.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.559+0000 7f388e1e2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab6a3b30 con 0x5605ab63d920 2024-02-26T06:30:00.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.559+0000 7f388e1e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ab6a3c70 con 0x5605ab634d40 2024-02-26T06:30:00.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.559+0000 7f388e1e2740 10 client.0 _ll_register_callbacks cb 0x7f3884073300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:00.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.559+0000 7f3889200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab63d920 0x5605ab643c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.559+0000 7f3889200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab63d920 0x5605ab643c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54726/0 (socket says 172.21.15.49:54726) 2024-02-26T06:30:00.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.559+0000 7f3889200640 1 -- 172.21.15.49:0/629853771 learned_addr learned my addr 172.21.15.49:0/629853771 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.559+0000 7f3883400640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab643750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.560+0000 7f3883e00640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634420 0x5605ab643210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.560+0000 7f3883400640 1 -- 172.21.15.49:0/629853771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab63d920 msgr2=0x5605ab643c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.561+0000 7f3883400640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab63d920 0x5605ab643c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.561+0000 7f3883400640 1 -- 172.21.15.49:0/629853771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634420 msgr2=0x5605ab643210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.561+0000 7f3883400640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634420 0x5605ab643210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.561+0000 7f3883400640 1 -- 172.21.15.49:0/629853771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605ab591570 con 0x5605ab634d40 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.561+0000 7f3883e00640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634420 0x5605ab643210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:00.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.561+0000 7f3889200640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab63d920 0x5605ab643c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:00.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.562+0000 7f3883400640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab643750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f387001b040 tx=0x7f3870003d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27134a48aa973dd6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.562+0000 7f386e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:30:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.562+0000 7f386e000640 1 -- 172.21.15.49:0/629853771 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387002a070 con 0x5605ab634d40 2024-02-26T06:30:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.562+0000 7f388e1e2740 1 -- 172.21.15.49:0/629853771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38700035f0 con 0x5605ab634d40 2024-02-26T06:30:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.562+0000 7f386e000640 1 -- 172.21.15.49:0/629853771 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f3870003880 con 0x5605ab634d40 2024-02-26T06:30:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.563+0000 7f386e000640 1 -- 172.21.15.49:0/629853771 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f387002a070 con 0x5605ab634d40 2024-02-26T06:30:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.563+0000 7f388e1e2740 10 client.27043 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.563+0000 7f388e1e2740 1 -- 172.21.15.49:0/629853771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5605ab553620 con 0x5605ab634d40 2024-02-26T06:30:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.563+0000 7f386e000640 1 -- 172.21.15.49:0/629853771 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f387003c020 con 0x5605ab634d40 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.563+0000 7f386e000640 1 -- 172.21.15.49:0/629853771 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3870021040 con 0x5605ab634d40 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.563+0000 7f388e1e2740 10 client.27043 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.563+0000 7f388e1e2740 10 client.27043 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.563+0000 7f388e1e2740 20 client.27043 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.564+0000 7f388e1e2740 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5605ab6a5420 0x5605ab6a7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.564+0000 7f388e1e2740 4 client.27043 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.564+0000 7f388e1e2740 1 -- 172.21.15.49:0/629853771 --> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5605ab31e890 con 0x5605ab6a5420 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.564+0000 7f3883e00640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5605ab6a5420 0x5605ab6a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.565+0000 7f3883e00640 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5605ab6a5420 0x5605ab6a7840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84c57396eb3bc0bc server_cookie=384c7a22689bb285 in_seq=0 out_seq=0 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.565+0000 7f386e000640 10 client.27043 ms_handle_connect on v2:172.21.15.115:6836/3720601525 2024-02-26T06:30:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.565+0000 7f386e000640 1 -- 172.21.15.49:0/629853771 <== mds.2 v2:172.21.15.115:6836/3720601525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+2231 (crc 0 0 0) 0x5605ab31e890 con 0x5605ab6a5420 2024-02-26T06:30:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.565+0000 7f386e000640 10 client.27043 handle_command_reply: tid=0 2024-02-26T06:30:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.565+0000 7f388e1e2740 1 client.27043 shutdown 2024-02-26T06:30:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.565+0000 7f388e1e2740 1 -- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 msgr2=0x5605ab643750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.565+0000 7f388e1e2740 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab643750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f387001b040 tx=0x7f3870003d70 comp rx=0 tx=0).stop 2024-02-26T06:30:00.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f388e1e2740 1 -- 172.21.15.49:0/629853771 shutdown_connections 2024-02-26T06:30:00.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f388e1e2740 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:6836/3720601525,v1:172.21.15.115:6837/3720601525] conn(0x5605ab6a5420 0x5605ab6a7840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f388e1e2740 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605ab63d920 0x5605ab643c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f388e1e2740 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605ab634d40 0x5605ab643750 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f388e1e2740 1 --2- 172.21.15.49:0/629853771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605ab634420 0x5605ab643210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f388e1e2740 1 -- 172.21.15.49:0/629853771 >> 172.21.15.49:0/629853771 conn(0x5605ab62eb50 msgr2=0x5605ab624bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f386e000640 0 client.27043 ms_handle_reset on v2:172.21.15.115:6836/3720601525 2024-02-26T06:30:00.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f388e1e2740 1 -- 172.21.15.49:0/629853771 shutdown_connections 2024-02-26T06:30:00.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.566+0000 7f388e1e2740 1 -- 172.21.15.49:0/629853771 wait complete. 2024-02-26T06:30:00.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.567+0000 7f388e1e2740 20 client.27043 trim_cache size 0 max 16384 2024-02-26T06:30:00.568 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 313119143, 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 2199023256084, 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000032" 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 949296035, 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 1099511629329, 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000003B" 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 1288646021, 2024-02-26T06:30:00.569 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 2199023256081, 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002F" 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 1562620774, 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 2199023256076, 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002B" 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.570 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 1909543629, 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 3298534883888, 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000034" 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 3078138514, 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 1099511629285, 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000053" 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 3113980660, 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 2199023256079, 2024-02-26T06:30:00.571 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002D" 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 3184263012, 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 2199023256082, 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000030" 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4012606509, 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 2199023256088, 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000037" 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:30:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:30:00.573 INFO:teuthology.orchestra.run.smithi049.stdout: "damage_type": "backtrace", 2024-02-26T06:30:00.573 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4279780572, 2024-02-26T06:30:00.573 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 2199023256068, 2024-02-26T06:30:00.573 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000061" 2024-02-26T06:30:00.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:30:00.573 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 -- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f38680a2390 msgr2=0x7f38680a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f38680a2390 0x7f38680a4850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f387c005fd0 tx=0x7f387c0095c0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 -- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 msgr2=0x7f3884072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 0x7f3884072880 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3874037040 tx=0x7f3874034930 comp rx=0 tx=0).stop 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 -- 172.21.15.49:0/2561105320 shutdown_connections 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f38680a2390 0x7f38680a4850 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3884108660 0x7f3884072dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388406b9f0 0x7f388406be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 --2- 172.21.15.49:0/2561105320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3884103590 0x7f3884072880 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 -- 172.21.15.49:0/2561105320 >> 172.21.15.49:0/2561105320 conn(0x7f38840fee30 msgr2=0x7f38841001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 -- 172.21.15.49:0/2561105320 shutdown_connections 2024-02-26T06:30:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.568+0000 7f388b200640 1 -- 172.21.15.49:0/2561105320 wait complete. 2024-02-26T06:30:00.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:00 smithi080 ceph-mon[36206]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:30:00.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:00 smithi080 ceph-mon[36206]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-26T06:30:00.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:00 smithi080 ceph-mon[36206]: mds.c(mds.2): Metadata damage detected 2024-02-26T06:30:00.591 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:30:00.591 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:30:00.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:00 smithi115 ceph-mon[36426]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:30:00.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:00 smithi115 ceph-mon[36426]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-26T06:30:00.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:00 smithi115 ceph-mon[36426]: mds.c(mds.2): Metadata damage detected 2024-02-26T06:30:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.889+0000 7f6681400640 1 -- 172.21.15.49:0/817603003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 msgr2=0x7f667c1166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.890+0000 7f6681400640 1 --2- 172.21.15.49:0/817603003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 0x7f667c1166c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6674009ca0 tx=0x7f667402c0f0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.890+0000 7f6681400640 1 -- 172.21.15.49:0/817603003 shutdown_connections 2024-02-26T06:30:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.891+0000 7f6681400640 1 --2- 172.21.15.49:0/817603003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f667c108c80 0x7f667c113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.891+0000 7f6681400640 1 --2- 172.21.15.49:0/817603003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 0x7f667c1166c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.891+0000 7f6681400640 1 --2- 172.21.15.49:0/817603003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f667c108360 0x7f667c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.891+0000 7f6681400640 1 -- 172.21.15.49:0/817603003 >> 172.21.15.49:0/817603003 conn(0x7f667c06c260 msgr2=0x7f667c06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.891+0000 7f6681400640 1 -- 172.21.15.49:0/817603003 shutdown_connections 2024-02-26T06:30:00.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.891+0000 7f6681400640 1 -- 172.21.15.49:0/817603003 wait complete. 2024-02-26T06:30:00.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.892+0000 7f6681400640 1 Processor -- start 2024-02-26T06:30:00.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.894+0000 7f6681400640 1 -- start start 2024-02-26T06:30:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.894+0000 7f6681400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f667c108360 0x7f667c10f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.894+0000 7f6681400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f667c108c80 0x7f667c10f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.894+0000 7f6681400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 0x7f667c10a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.894+0000 7f6681400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c10ad50 con 0x7f667c108c80 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.894+0000 7f6681400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c10aec0 con 0x7f667c108360 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.894+0000 7f6681400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c10b080 con 0x7f667c1142d0 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f667b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 0x7f667c10a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f667b200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 0x7f667c10a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33894/0 (socket says 172.21.15.49:33894) 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f667b200640 1 -- 172.21.15.49:0/4194869856 learned_addr learned my addr 172.21.15.49:0/4194869856 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f6679e00640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f667c108c80 0x7f667c10f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f667b200640 1 -- 172.21.15.49:0/4194869856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f667c108360 msgr2=0x7f667c10f4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f667b200640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f667c108360 0x7f667c10f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f667b200640 1 -- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f667c108c80 msgr2=0x7f667c10f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f667b200640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f667c108c80 0x7f667c10f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f667b200640 1 -- 172.21.15.49:0/4194869856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6674009900 con 0x7f667c1142d0 2024-02-26T06:30:00.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.895+0000 7f6679e00640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f667c108c80 0x7f667c10f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:30:00.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.896+0000 7f667b200640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 0x7f667c10a5c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6674005e30 tx=0x7f6674034cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.896+0000 7f6673400640 1 -- 172.21.15.49:0/4194869856 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674032ac0 con 0x7f667c1142d0 2024-02-26T06:30:00.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.896+0000 7f6673400640 1 -- 172.21.15.49:0/4194869856 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6674032eb0 con 0x7f667c1142d0 2024-02-26T06:30:00.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.896+0000 7f6673400640 1 -- 172.21.15.49:0/4194869856 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674040450 con 0x7f667c1142d0 2024-02-26T06:30:00.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.897+0000 7f6681400640 1 -- 172.21.15.49:0/4194869856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c10b300 con 0x7f667c1142d0 2024-02-26T06:30:00.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.897+0000 7f6681400640 1 -- 172.21.15.49:0/4194869856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c113390 con 0x7f667c1142d0 2024-02-26T06:30:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.903+0000 7f668425d740 1 Processor -- start 2024-02-26T06:30:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.904+0000 7f6673400640 1 -- 172.21.15.49:0/4194869856 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f667400a040 con 0x7f667c1142d0 2024-02-26T06:30:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.905+0000 7f668425d740 1 -- start start 2024-02-26T06:30:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.905+0000 7f668425d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.905+0000 7f668425d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 0x55f1ba0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.905+0000 7f668425d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 0x55f1ba102500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.905+0000 7f668425d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba0651e0 con 0x55f1ba0f7530 2024-02-26T06:30:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.905+0000 7f668425d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba062710 con 0x55f1ba0f6c10 2024-02-26T06:30:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.905+0000 7f668425d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba05def0 con 0x55f1ba100110 2024-02-26T06:30:00.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.906+0000 7f6679e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.906+0000 7f6679e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54760/0 (socket says 172.21.15.49:54760) 2024-02-26T06:30:00.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.906+0000 7f6679e00640 1 -- 172.21.15.49:0/765780287 learned_addr learned my addr 172.21.15.49:0/765780287 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.906+0000 7f6679e00640 1 --2- 172.21.15.49:0/765780287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 0x55f1ba102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.908+0000 7f6673400640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66600a22b0 0x7f66600a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.908+0000 7f6679e00640 1 -- 172.21.15.49:0/765780287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 msgr2=0x55f1ba102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.908+0000 7f6679e00640 1 --2- 172.21.15.49:0/765780287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 0x55f1ba102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.908+0000 7f6679e00640 1 -- 172.21.15.49:0/765780287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 msgr2=0x55f1ba0ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.908+0000 7f6679e00640 1 --2- 172.21.15.49:0/765780287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 0x55f1ba0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.908+0000 7f6679e00640 1 -- 172.21.15.49:0/765780287 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1ba01fdc0 con 0x55f1ba0f6c10 2024-02-26T06:30:00.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.908+0000 7f6673400640 1 -- 172.21.15.49:0/4194869856 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f66740f03d0 con 0x7f667c1142d0 2024-02-26T06:30:00.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.908+0000 7f6679e00640 1 --2- 172.21.15.49:0/765780287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0f6ff0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6668013860 tx=0x7f666800b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd979ea084035f45 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.909+0000 7f6667400640 1 -- 172.21.15.49:0/765780287 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668013ea0 con 0x55f1ba0f6c10 2024-02-26T06:30:00.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.909+0000 7f6667400640 1 -- 172.21.15.49:0/765780287 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6668019070 con 0x55f1ba0f6c10 2024-02-26T06:30:00.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.909+0000 7f6667400640 1 -- 172.21.15.49:0/765780287 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666800e950 con 0x55f1ba0f6c10 2024-02-26T06:30:00.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.910+0000 7f667b200640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66600a22b0 0x7f66600a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.911+0000 7f668425d740 1 -- 172.21.15.49:0/765780287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 msgr2=0x55f1ba0f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.911+0000 7f668425d740 1 --2- 172.21.15.49:0/765780287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0f6ff0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6668013860 tx=0x7f666800b400 comp rx=0 tx=0).stop 2024-02-26T06:30:00.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.912+0000 7f668425d740 1 -- 172.21.15.49:0/765780287 shutdown_connections 2024-02-26T06:30:00.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.912+0000 7f668425d740 1 --2- 172.21.15.49:0/765780287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 0x55f1ba102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.912+0000 7f668425d740 1 --2- 172.21.15.49:0/765780287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 0x55f1ba0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.912+0000 7f668425d740 1 --2- 172.21.15.49:0/765780287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0f6ff0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.912+0000 7f668425d740 1 -- 172.21.15.49:0/765780287 >> 172.21.15.49:0/765780287 conn(0x55f1ba0f1340 msgr2=0x55f1ba0f1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.912+0000 7f667b200640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66600a22b0 0x7f66600a4770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66740b98b0 tx=0x7f6674004370 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.915+0000 7f668425d740 1 -- 172.21.15.49:0/765780287 shutdown_connections 2024-02-26T06:30:00.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.915+0000 7f668425d740 1 -- 172.21.15.49:0/765780287 wait complete. 2024-02-26T06:30:00.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.916+0000 7f668425d740 1 -- start start 2024-02-26T06:30:00.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.916+0000 7f668425d740 1 Processor -- start 2024-02-26T06:30:00.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.919+0000 7f668425d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0fd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.919+0000 7f668425d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.919+0000 7f668425d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba100110 0x55f1ba0fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.919+0000 7f668425d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba003120 con 0x55f1ba0f6c10 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.919+0000 7f668425d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba002dc0 con 0x55f1ba100110 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.919+0000 7f668425d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1b9dcc1c0 con 0x55f1ba0f7530 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.919+0000 7f668425d740 10 client.0 _ll_register_callbacks cb 0x55f1ba052280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.921+0000 7f667a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.921+0000 7f667a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33906/0 (socket says 172.21.15.49:33906) 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.921+0000 7f667a800640 1 -- 172.21.15.49:0/2004003388 learned_addr learned my addr 172.21.15.49:0/2004003388 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.921+0000 7f6679e00640 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.924+0000 7f6679e00640 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba100110 0x55f1ba0fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.924+0000 7f667a800640 1 -- 172.21.15.49:0/2004003388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba100110 msgr2=0x55f1ba0fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.924+0000 7f667a800640 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba100110 0x55f1ba0fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.924+0000 7f667a800640 1 -- 172.21.15.49:0/2004003388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f6c10 msgr2=0x55f1ba0fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.924+0000 7f667a800640 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.924+0000 7f667a800640 1 -- 172.21.15.49:0/2004003388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6668013540 con 0x55f1ba0f7530 2024-02-26T06:30:00.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.925+0000 7f667a800640 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fd750 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f666c009e00 tx=0x7f666c00b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76db9769b9f6c4ce server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.925+0000 7f665be00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:30:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.925+0000 7f665be00640 1 -- 172.21.15.49:0/2004003388 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c011620 con 0x55f1ba0f7530 2024-02-26T06:30:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.925+0000 7f665be00640 1 -- 172.21.15.49:0/2004003388 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f666c011780 con 0x55f1ba0f7530 2024-02-26T06:30:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.926+0000 7f668425d740 1 -- 172.21.15.49:0/2004003388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1b9d5d7e0 con 0x55f1ba0f7530 2024-02-26T06:30:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.926+0000 7f665be00640 1 -- 172.21.15.49:0/2004003388 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f666c00be90 con 0x55f1ba0f7530 2024-02-26T06:30:00.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.926+0000 7f668425d740 10 client.27055 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:00.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.926+0000 7f668425d740 1 -- 172.21.15.49:0/2004003388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55f1ba05c570 con 0x55f1ba0f7530 2024-02-26T06:30:00.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.927+0000 7f665be00640 1 -- 172.21.15.49:0/2004003388 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f1b9d5d7e0 con 0x55f1ba0f7530 2024-02-26T06:30:00.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.927+0000 7f665be00640 1 -- 172.21.15.49:0/2004003388 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f666c01aba0 con 0x55f1ba0f7530 2024-02-26T06:30:00.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.927+0000 7f668425d740 10 client.27055 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:00.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.927+0000 7f668425d740 10 client.27055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:30:00.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.927+0000 7f668425d740 20 client.27055 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:00.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.927+0000 7f668425d740 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55f1ba110290 0x55f1ba1126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.927+0000 7f668425d740 4 client.27055 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:00.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.927+0000 7f668425d740 1 -- 172.21.15.49:0/2004003388 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1b9d9a720 con 0x55f1ba110290 2024-02-26T06:30:00.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.928+0000 7f6679e00640 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55f1ba110290 0x55f1ba1126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.929+0000 7f6679e00640 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55f1ba110290 0x55f1ba1126b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fa07998c29dd9ea server_cookie=f8d9c7afe0269a45 in_seq=0 out_seq=0 2024-02-26T06:30:00.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.929+0000 7f665be00640 10 client.27055 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:30:00.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.933+0000 7f665be00640 1 -- 172.21.15.49:0/2004003388 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1b9d9a720 con 0x55f1ba110290 2024-02-26T06:30:00.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.933+0000 7f665be00640 10 client.27055 handle_command_reply: tid=0 2024-02-26T06:30:00.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.933+0000 7f668425d740 1 client.27055 shutdown 2024-02-26T06:30:00.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.935+0000 7f668425d740 1 -- 172.21.15.49:0/2004003388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f7530 msgr2=0x55f1ba0fd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.935+0000 7f668425d740 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fd750 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f666c009e00 tx=0x7f666c00b4b0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 1 -- 172.21.15.49:0/2004003388 shutdown_connections 2024-02-26T06:30:00.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55f1ba110290 0x55f1ba1126b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba100110 0x55f1ba0fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fd750 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 1 --2- 172.21.15.49:0/2004003388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f665be00640 0 client.27055 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:30:00.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 1 -- 172.21.15.49:0/2004003388 >> 172.21.15.49:0/2004003388 conn(0x55f1ba0f1340 msgr2=0x55f1ba0f3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 1 -- 172.21.15.49:0/2004003388 shutdown_connections 2024-02-26T06:30:00.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 1 -- 172.21.15.49:0/2004003388 wait complete. 2024-02-26T06:30:00.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.937+0000 7f668425d740 20 client.27055 trim_cache size 0 max 16384 2024-02-26T06:30:00.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.965+0000 7f668425d740 1 Processor -- start 2024-02-26T06:30:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.965+0000 7f668425d740 1 -- start start 2024-02-26T06:30:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.966+0000 7f668425d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0ff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.966+0000 7f668425d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.966+0000 7f668425d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 0x55f1ba0f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.966+0000 7f668425d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba10eab0 con 0x55f1ba0f7530 2024-02-26T06:30:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.966+0000 7f668425d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba0f11a0 con 0x55f1ba0f6c10 2024-02-26T06:30:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.966+0000 7f668425d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba0fe560 con 0x55f1ba100110 2024-02-26T06:30:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.967+0000 7f6679e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.967+0000 7f6679e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 0x55f1ba0f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.967+0000 7f6679e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0ff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54782/0 (socket says 172.21.15.49:54782) 2024-02-26T06:30:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.967+0000 7f6679e00640 1 -- 172.21.15.49:0/1994900343 learned_addr learned my addr 172.21.15.49:0/1994900343 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.967+0000 7f667a800640 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.968+0000 7f6679e00640 1 -- 172.21.15.49:0/1994900343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 msgr2=0x55f1ba0f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.968+0000 7f6679e00640 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 0x55f1ba0f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.968+0000 7f6679e00640 1 -- 172.21.15.49:0/1994900343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 msgr2=0x55f1ba0fdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.968+0000 7f6679e00640 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.968+0000 7f6679e00640 1 -- 172.21.15.49:0/1994900343 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1b9ff8a80 con 0x55f1ba0f6c10 2024-02-26T06:30:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.968+0000 7f667a800640 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.969+0000 7f6679e00640 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0ff940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6668004e00 tx=0x7f6668015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be69c835ac0a521e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.969+0000 7f6666a00640 1 -- 172.21.15.49:0/1994900343 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668013ea0 con 0x55f1ba0f6c10 2024-02-26T06:30:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.969+0000 7f6666a00640 1 -- 172.21.15.49:0/1994900343 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6668034070 con 0x55f1ba0f6c10 2024-02-26T06:30:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.969+0000 7f6666a00640 1 -- 172.21.15.49:0/1994900343 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666802d8f0 con 0x55f1ba0f6c10 2024-02-26T06:30:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.970+0000 7f668425d740 1 -- 172.21.15.49:0/1994900343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 msgr2=0x55f1ba0ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.970+0000 7f668425d740 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0ff940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6668004e00 tx=0x7f6668015040 comp rx=0 tx=0).stop 2024-02-26T06:30:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.971+0000 7f668425d740 1 -- 172.21.15.49:0/1994900343 shutdown_connections 2024-02-26T06:30:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.971+0000 7f668425d740 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba100110 0x55f1ba0f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.971+0000 7f668425d740 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba0f7530 0x55f1ba0fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.971+0000 7f668425d740 1 --2- 172.21.15.49:0/1994900343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f6c10 0x55f1ba0ff940 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.971+0000 7f668425d740 1 -- 172.21.15.49:0/1994900343 >> 172.21.15.49:0/1994900343 conn(0x55f1ba0f1340 msgr2=0x55f1ba0f1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.972+0000 7f668425d740 1 -- 172.21.15.49:0/1994900343 shutdown_connections 2024-02-26T06:30:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.972+0000 7f668425d740 1 -- 172.21.15.49:0/1994900343 wait complete. 2024-02-26T06:30:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.972+0000 7f668425d740 1 -- start start 2024-02-26T06:30:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.973+0000 7f668425d740 1 Processor -- start 2024-02-26T06:30:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.973+0000 7f668425d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f6c10 0x55f1ba105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.973+0000 7f668425d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f7530 0x55f1ba105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.973+0000 7f668425d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba100110 0x55f1ba1063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.974+0000 7f668425d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba107910 con 0x55f1ba100110 2024-02-26T06:30:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.974+0000 7f668425d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba1667f0 con 0x55f1ba0f7530 2024-02-26T06:30:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.974+0000 7f668425d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ba1669b0 con 0x55f1ba0f6c10 2024-02-26T06:30:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.974+0000 7f668425d740 10 client.0 _ll_register_callbacks cb 0x7f666800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.974+0000 7f667a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f7530 0x55f1ba105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.974+0000 7f667a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f7530 0x55f1ba105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54784/0 (socket says 172.21.15.49:54784) 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.974+0000 7f667a800640 1 -- 172.21.15.49:0/279731321 learned_addr learned my addr 172.21.15.49:0/279731321 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.974+0000 7f6679e00640 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f6c10 0x55f1ba105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f6679e00640 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba100110 0x55f1ba1063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f6679e00640 1 -- 172.21.15.49:0/279731321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f7530 msgr2=0x55f1ba105eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f6679e00640 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f7530 0x55f1ba105eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f6679e00640 1 -- 172.21.15.49:0/279731321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba100110 msgr2=0x55f1ba1063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f6679e00640 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba100110 0x55f1ba1063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f6679e00640 1 -- 172.21.15.49:0/279731321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1ba05c570 con 0x55f1ba0f6c10 2024-02-26T06:30:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f6679e00640 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f6c10 0x55f1ba105970 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6668004e00 tx=0x7f6668005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca332781a0040fd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f665be00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:30:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f665be00640 1 -- 172.21.15.49:0/279731321 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666802d830 con 0x55f1ba0f6c10 2024-02-26T06:30:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.975+0000 7f665be00640 1 -- 172.21.15.49:0/279731321 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7f6668004810 con 0x55f1ba0f6c10 2024-02-26T06:30:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.976+0000 7f668425d740 1 -- 172.21.15.49:0/279731321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6668019070 con 0x55f1ba0f6c10 2024-02-26T06:30:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.976+0000 7f665be00640 1 -- 172.21.15.49:0/279731321 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66680049d0 con 0x55f1ba0f6c10 2024-02-26T06:30:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.976+0000 7f668425d740 10 client.27061 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.976+0000 7f668425d740 1 -- 172.21.15.49:0/279731321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55f1ba01fdc0 con 0x55f1ba0f6c10 2024-02-26T06:30:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.976+0000 7f665be00640 1 -- 172.21.15.49:0/279731321 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6668019070 con 0x55f1ba0f6c10 2024-02-26T06:30:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.977+0000 7f665be00640 1 -- 172.21.15.49:0/279731321 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f666800e880 con 0x55f1ba0f6c10 2024-02-26T06:30:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.977+0000 7f668425d740 10 client.27061 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.977+0000 7f668425d740 10 client.27061 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T06:30:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.977+0000 7f668425d740 20 client.27061 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.977+0000 7f668425d740 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55f1ba167eb0 0x55f1ba16a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:00.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.977+0000 7f668425d740 4 client.27061 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:30:00.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.977+0000 7f668425d740 1 -- 172.21.15.49:0/279731321 --> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f1b9de9890 con 0x55f1ba167eb0 2024-02-26T06:30:00.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.978+0000 7f667a800640 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55f1ba167eb0 0x55f1ba16a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:00.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.980+0000 7f667a800640 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55f1ba167eb0 0x55f1ba16a2d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac006761ed07c5f3 server_cookie=118f280854de3f9f in_seq=0 out_seq=0 2024-02-26T06:30:00.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.980+0000 7f665be00640 10 client.27061 ms_handle_connect on v2:172.21.15.115:6838/318392314 2024-02-26T06:30:00.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.980+0000 7f665be00640 1 -- 172.21.15.49:0/279731321 <== mds.1 v2:172.21.15.115:6838/318392314 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f1b9de9890 con 0x55f1ba167eb0 2024-02-26T06:30:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.980+0000 7f665be00640 10 client.27061 handle_command_reply: tid=0 2024-02-26T06:30:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.980+0000 7f668425d740 1 client.27061 shutdown 2024-02-26T06:30:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f668425d740 1 -- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f6c10 msgr2=0x55f1ba105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f668425d740 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f6c10 0x55f1ba105970 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6668004e00 tx=0x7f6668005850 comp rx=0 tx=0).stop 2024-02-26T06:30:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f668425d740 1 -- 172.21.15.49:0/279731321 shutdown_connections 2024-02-26T06:30:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f668425d740 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:6838/318392314,v1:172.21.15.115:6839/318392314] conn(0x55f1ba167eb0 0x55f1ba16a2d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f668425d740 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1ba0f7530 0x55f1ba105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f668425d740 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1ba100110 0x55f1ba1063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f668425d740 1 --2- 172.21.15.49:0/279731321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1ba0f6c10 0x55f1ba105970 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f668425d740 1 -- 172.21.15.49:0/279731321 >> 172.21.15.49:0/279731321 conn(0x55f1ba0f1340 msgr2=0x55f1ba0fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.981+0000 7f665be00640 0 client.27061 ms_handle_reset on v2:172.21.15.115:6838/318392314 2024-02-26T06:30:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.982+0000 7f668425d740 1 -- 172.21.15.49:0/279731321 shutdown_connections 2024-02-26T06:30:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.982+0000 7f668425d740 1 -- 172.21.15.49:0/279731321 wait complete. 2024-02-26T06:30:00.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.982+0000 7f668425d740 20 client.27061 trim_cache size 0 max 16384 2024-02-26T06:30:00.982 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:30:00.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.983+0000 7f6681400640 1 -- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66600a22b0 msgr2=0x7f66600a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.983+0000 7f6681400640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66600a22b0 0x7f66600a4770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66740b98b0 tx=0x7f6674004370 comp rx=0 tx=0).stop 2024-02-26T06:30:00.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.983+0000 7f6681400640 1 -- 172.21.15.49:0/4194869856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 msgr2=0x7f667c10a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:00.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.983+0000 7f6681400640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 0x7f667c10a5c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6674005e30 tx=0x7f6674034cc0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.984+0000 7f6681400640 1 -- 172.21.15.49:0/4194869856 shutdown_connections 2024-02-26T06:30:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.984+0000 7f6681400640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f66600a22b0 0x7f66600a4770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.984+0000 7f6681400640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f667c1142d0 0x7f667c10a5c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.984+0000 7f6681400640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f667c108c80 0x7f667c10f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.984+0000 7f6681400640 1 --2- 172.21.15.49:0/4194869856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f667c108360 0x7f667c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.984+0000 7f6681400640 1 -- 172.21.15.49:0/4194869856 >> 172.21.15.49:0/4194869856 conn(0x7f667c06c260 msgr2=0x7f667c06fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.984+0000 7f6681400640 1 -- 172.21.15.49:0/4194869856 shutdown_connections 2024-02-26T06:30:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:00.984+0000 7f6681400640 1 -- 172.21.15.49:0/4194869856 wait complete. 2024-02-26T06:30:01.012 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:30:01.012 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 -- 172.21.15.49:0/2189866095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588101380 msgr2=0x7fe588101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2189866095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588101380 0x7fe588101760 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe578009860 tx=0x7fe57802bb70 comp rx=0 tx=0).stop 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 -- 172.21.15.49:0/2189866095 shutdown_connections 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2189866095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe588101d30 0x7fe588105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2189866095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe588106450 0x7fe58810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2189866095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588101380 0x7fe588101760 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 -- 172.21.15.49:0/2189866095 >> 172.21.15.49:0/2189866095 conn(0x7fe588078580 msgr2=0x7fe588078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 -- 172.21.15.49:0/2189866095 shutdown_connections 2024-02-26T06:30:01.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.249+0000 7fe58ee00640 1 -- 172.21.15.49:0/2189866095 wait complete. 2024-02-26T06:30:01.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.251+0000 7fe58ee00640 1 Processor -- start 2024-02-26T06:30:01.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.251+0000 7fe58ee00640 1 -- start start 2024-02-26T06:30:01.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.251+0000 7fe58ee00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe588101380 0x7fe58819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.251+0000 7fe58ee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe588106450 0x7fe58819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.251+0000 7fe58ee00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5881975b0 0x7fe588197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.251+0000 7fe58ee00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588197fa0 con 0x7fe588101380 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.251+0000 7fe58ee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588198110 con 0x7fe588106450 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.251+0000 7fe58ee00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5881982d0 con 0x7fe5881975b0 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe588101380 0x7fe58819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe588106450 0x7fe58819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe588106450 0x7fe58819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54808/0 (socket says 172.21.15.49:54808) 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 -- 172.21.15.49:0/2120032337 learned_addr learned my addr 172.21.15.49:0/2120032337 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe58ce00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5881975b0 0x7fe588197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 -- 172.21.15.49:0/2120032337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5881975b0 msgr2=0x7fe588197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5881975b0 0x7fe588197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 -- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe588101380 msgr2=0x7fe58819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe588101380 0x7fe58819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 -- 172.21.15.49:0/2120032337 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580009900 con 0x7fe588106450 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.252+0000 7fe587400640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe588106450 0x7fe58819d940 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe57400e9c0 tx=0x7fe57400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.253+0000 7fe584c00640 1 -- 172.21.15.49:0/2120032337 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574009700 con 0x7fe588106450 2024-02-26T06:30:01.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.253+0000 7fe584c00640 1 -- 172.21.15.49:0/2120032337 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe57400cd80 con 0x7fe588106450 2024-02-26T06:30:01.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.253+0000 7fe584c00640 1 -- 172.21.15.49:0/2120032337 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574010430 con 0x7fe588106450 2024-02-26T06:30:01.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.253+0000 7fe58ee00640 1 -- 172.21.15.49:0/2120032337 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe578009490 con 0x7fe588106450 2024-02-26T06:30:01.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.253+0000 7fe58ee00640 1 -- 172.21.15.49:0/2120032337 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5881a4690 con 0x7fe588106450 2024-02-26T06:30:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe591d8b740 1 Processor -- start 2024-02-26T06:30:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe584c00640 1 -- 172.21.15.49:0/2120032337 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe574005000 con 0x7fe588106450 2024-02-26T06:30:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe591d8b740 1 -- start start 2024-02-26T06:30:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 0x561ee64d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d8220 0x561ee64e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64e5520 0x561ee64e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe591d8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee64461e0 con 0x561ee64d7900 2024-02-26T06:30:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe591d8b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee6443710 con 0x561ee64e5520 2024-02-26T06:30:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.257+0000 7fe591d8b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee643eef0 con 0x561ee64d8220 2024-02-26T06:30:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d8220 0x561ee64e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d8220 0x561ee64e4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33946/0 (socket says 172.21.15.49:33946) 2024-02-26T06:30:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 -- 172.21.15.49:0/3870831544 learned_addr learned my addr 172.21.15.49:0/3870831544 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe58ce00640 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 0x561ee64d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587400640 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64e5520 0x561ee64e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 -- 172.21.15.49:0/3870831544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64e5520 msgr2=0x561ee64e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64e5520 0x561ee64e7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 -- 172.21.15.49:0/3870831544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 msgr2=0x561ee64d7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 0x561ee64d7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 -- 172.21.15.49:0/3870831544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ee6400dc0 con 0x561ee64d8220 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587400640 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64e5520 0x561ee64e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe587e00640 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d8220 0x561ee64e4fe0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe578005e00 tx=0x7fe5780323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fa8e7b515674a48 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.258+0000 7fe584c00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe56c0a22b0 0x7fe56c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.259+0000 7fe584c00640 1 -- 172.21.15.49:0/2120032337 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe574014070 con 0x7fe588106450 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.259+0000 7fe57d600640 1 -- 172.21.15.49:0/3870831544 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578004180 con 0x561ee64d8220 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.259+0000 7fe58ce00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe56c0a22b0 0x7fe56c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.259+0000 7fe57d600640 1 -- 172.21.15.49:0/3870831544 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe578034920 con 0x561ee64d8220 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.259+0000 7fe57d600640 1 -- 172.21.15.49:0/3870831544 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578032600 con 0x561ee64d8220 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 -- 172.21.15.49:0/3870831544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d8220 msgr2=0x561ee64e4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe58ce00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe56c0a22b0 0x7fe56c0a4770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe580009fd0 tx=0x7fe580010040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d8220 0x561ee64e4fe0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe578005e00 tx=0x7fe5780323c0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 -- 172.21.15.49:0/3870831544 shutdown_connections 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64e5520 0x561ee64e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d8220 0x561ee64e4fe0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 --2- 172.21.15.49:0/3870831544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 0x561ee64d7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 -- 172.21.15.49:0/3870831544 >> 172.21.15.49:0/3870831544 conn(0x561ee64d2340 msgr2=0x561ee64d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 -- 172.21.15.49:0/3870831544 shutdown_connections 2024-02-26T06:30:01.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.260+0000 7fe591d8b740 1 -- 172.21.15.49:0/3870831544 wait complete. 2024-02-26T06:30:01.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.261+0000 7fe591d8b740 1 -- start start 2024-02-26T06:30:01.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.261+0000 7fe591d8b740 1 Processor -- start 2024-02-26T06:30:01.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.261+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 0x561ee64e24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.261+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.261+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.261+0000 7fe591d8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee63e4120 con 0x561ee64d7900 2024-02-26T06:30:01.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.261+0000 7fe591d8b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee63e3dc0 con 0x561ee64d8220 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.261+0000 7fe591d8b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee61ad1c0 con 0x561ee64e5520 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe591d8b740 10 client.0 _ll_register_callbacks cb 0x561ee6433280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64e2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54838/0 (socket says 172.21.15.49:54838) 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 -- 172.21.15.49:0/1699850774 learned_addr learned my addr 172.21.15.49:0/1699850774 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587e00640 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 0x561ee64e24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe58ce00640 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 -- 172.21.15.49:0/1699850774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 msgr2=0x561ee64e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 -- 172.21.15.49:0/1699850774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 msgr2=0x561ee64e24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 0x561ee64e24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 -- 172.21.15.49:0/1699850774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58810e830 con 0x561ee64d8220 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe587400640 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64e2a30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe574097260 tx=0x7fe57409ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f5a25e79175c369 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe572000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:30:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.262+0000 7fe572000640 1 -- 172.21.15.49:0/1699850774 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57409d070 con 0x561ee64d8220 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe572000640 1 -- 172.21.15.49:0/1699850774 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe57409ae00 con 0x561ee64d8220 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe591d8b740 1 -- 172.21.15.49:0/1699850774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ee613e7e0 con 0x561ee64d8220 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe572000640 1 -- 172.21.15.49:0/1699850774 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe57409ef50 con 0x561ee64d8220 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe591d8b740 10 client.27086 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe591d8b740 1 -- 172.21.15.49:0/1699850774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561ee643d570 con 0x561ee64d8220 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe572000640 1 -- 172.21.15.49:0/1699850774 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5740b7020 con 0x561ee64d8220 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe572000640 1 -- 172.21.15.49:0/1699850774 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe5740cd070 con 0x561ee64d8220 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe591d8b740 10 client.27086 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe591d8b740 10 client.27086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.263+0000 7fe591d8b740 20 client.27086 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.264+0000 7fe591d8b740 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561ee64f55f0 0x561ee64f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.264+0000 7fe591d8b740 4 client.27086 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.264+0000 7fe591d8b740 1 -- 172.21.15.49:0/1699850774 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ee617b720 con 0x561ee64f55f0 2024-02-26T06:30:01.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.264+0000 7fe587e00640 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561ee64f55f0 0x561ee64f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.265+0000 7fe572000640 10 client.27086 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.265+0000 7fe587e00640 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561ee64f55f0 0x561ee64f7a30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9412039c07af5d server_cookie=92ab08b86ae950d2 in_seq=0 out_seq=0 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.265+0000 7fe572000640 1 -- 172.21.15.49:0/1699850774 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ee617b720 con 0x561ee64f55f0 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.265+0000 7fe572000640 10 client.27086 handle_command_reply: tid=0 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.265+0000 7fe591d8b740 1 client.27086 shutdown 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.265+0000 7fe591d8b740 1 -- 172.21.15.49:0/1699850774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 msgr2=0x561ee64e2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64e2a30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe574097260 tx=0x7fe57409ac90 comp rx=0 tx=0).stop 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 -- 172.21.15.49:0/1699850774 shutdown_connections 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561ee64f55f0 0x561ee64f7a30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64e2a30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe572000640 0 client.27086 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 --2- 172.21.15.49:0/1699850774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d7900 0x561ee64e24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 -- 172.21.15.49:0/1699850774 >> 172.21.15.49:0/1699850774 conn(0x561ee64d2340 msgr2=0x561ee64d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:01.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 -- 172.21.15.49:0/1699850774 shutdown_connections 2024-02-26T06:30:01.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 1 -- 172.21.15.49:0/1699850774 wait complete. 2024-02-26T06:30:01.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.266+0000 7fe591d8b740 20 client.27086 trim_cache size 0 max 16384 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.281+0000 7fe591d8b740 1 Processor -- start 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.281+0000 7fe591d8b740 1 -- start start 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.281+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d7900 0x561ee64e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.281+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64ec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.281+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64e5520 0x561ee64ec920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.281+0000 7fe591d8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee64f2080 con 0x561ee64e5520 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.281+0000 7fe591d8b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee64d1ea0 con 0x561ee64d8220 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.281+0000 7fe591d8b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee64f3de0 con 0x561ee64d7900 2024-02-26T06:30:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d7900 0x561ee64e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe58ce00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64e5520 0x561ee64ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64ec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d7900 0x561ee64e2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33958/0 (socket says 172.21.15.49:33958) 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 -- 172.21.15.49:0/2182146805 learned_addr learned my addr 172.21.15.49:0/2182146805 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 -- 172.21.15.49:0/2182146805 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 msgr2=0x561ee64ec3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 --2- 172.21.15.49:0/2182146805 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64ec3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 -- 172.21.15.49:0/2182146805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64e5520 msgr2=0x561ee64ec920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 --2- 172.21.15.49:0/2182146805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64e5520 0x561ee64ec920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 -- 172.21.15.49:0/2182146805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ee63d9a80 con 0x561ee64d7900 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587400640 1 --2- 172.21.15.49:0/2182146805 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64ec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.282+0000 7fe587e00640 1 --2- 172.21.15.49:0/2182146805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d7900 0x561ee64e2d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe57802f020 tx=0x7fe57804a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186401f939eb994b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.283+0000 7fe57cc00640 1 -- 172.21.15.49:0/2182146805 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57802c8a0 con 0x561ee64d7900 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.283+0000 7fe57cc00640 1 -- 172.21.15.49:0/2182146805 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe57802ca00 con 0x561ee64d7900 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.283+0000 7fe57cc00640 1 -- 172.21.15.49:0/2182146805 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5780046c0 con 0x561ee64d7900 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.283+0000 7fe591d8b740 1 -- 172.21.15.49:0/2182146805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d7900 msgr2=0x561ee64e2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.283+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2182146805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d7900 0x561ee64e2d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe57802f020 tx=0x7fe57804a040 comp rx=0 tx=0).stop 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 -- 172.21.15.49:0/2182146805 shutdown_connections 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2182146805 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d8220 0x561ee64ec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2182146805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64e5520 0x561ee64ec920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2182146805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64d7900 0x561ee64e2d90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 -- 172.21.15.49:0/2182146805 >> 172.21.15.49:0/2182146805 conn(0x561ee64d2340 msgr2=0x561ee64d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 -- 172.21.15.49:0/2182146805 shutdown_connections 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 -- 172.21.15.49:0/2182146805 wait complete. 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 -- start start 2024-02-26T06:30:01.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.284+0000 7fe591d8b740 1 Processor -- start 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d7900 0x561ee64eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d8220 0x561ee64eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe591d8b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe591d8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee654bc00 con 0x561ee64d8220 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe591d8b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee654bd70 con 0x561ee64d7900 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe591d8b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee654bf30 con 0x561ee64e5520 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe591d8b740 10 client.0 _ll_register_callbacks cb 0x7fe580009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe587e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d7900 0x561ee64eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe587e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d7900 0x561ee64eaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54848/0 (socket says 172.21.15.49:54848) 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe587e00640 1 -- 172.21.15.49:0/2437942444 learned_addr learned my addr 172.21.15.49:0/2437942444 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe587400640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d8220 0x561ee64eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe587e00640 1 -- 172.21.15.49:0/2437942444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 msgr2=0x561ee64eb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe58ce00640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.285+0000 7fe587e00640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64eb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe587e00640 1 -- 172.21.15.49:0/2437942444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d8220 msgr2=0x561ee64eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe587e00640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d8220 0x561ee64eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe587e00640 1 -- 172.21.15.49:0/2437942444 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ee643d570 con 0x561ee64d7900 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe58ce00640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe587400640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d8220 0x561ee64eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe587e00640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d7900 0x561ee64eaf70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe578032400 tx=0x7fe578000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f710baf57a857b07 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe572000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe572000640 1 -- 172.21.15.49:0/2437942444 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57802c860 con 0x561ee64d7900 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe572000640 1 -- 172.21.15.49:0/2437942444 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fe57804d040 con 0x561ee64d7900 2024-02-26T06:30:01.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.286+0000 7fe591d8b740 1 -- 172.21.15.49:0/2437942444 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe578034920 con 0x561ee64d7900 2024-02-26T06:30:01.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe572000640 1 -- 172.21.15.49:0/2437942444 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe578005040 con 0x561ee64d7900 2024-02-26T06:30:01.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe591d8b740 10 client.27092 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:01.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe591d8b740 1 -- 172.21.15.49:0/2437942444 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561ee6400dc0 con 0x561ee64d7900 2024-02-26T06:30:01.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe572000640 1 -- 172.21.15.49:0/2437942444 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe578034920 con 0x561ee64d7900 2024-02-26T06:30:01.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe572000640 1 -- 172.21.15.49:0/2437942444 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe578004ad0 con 0x561ee64d7900 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe591d8b740 10 client.27092 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe591d8b740 10 client.27092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe591d8b740 20 client.27092 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561ee654d350 0x561ee654f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe591d8b740 4 client.27092 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.287+0000 7fe591d8b740 1 -- 172.21.15.49:0/2437942444 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ee61ca890 con 0x561ee654d350 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.288+0000 7fe587400640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561ee654d350 0x561ee654f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.288+0000 7fe587400640 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561ee654d350 0x561ee654f770 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f784e2188c1fb49c server_cookie=a838a7d61e45bf9a in_seq=0 out_seq=0 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.288+0000 7fe572000640 10 client.27092 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.288+0000 7fe572000640 1 -- 172.21.15.49:0/2437942444 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ee61ca890 con 0x561ee654d350 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.288+0000 7fe572000640 10 client.27092 handle_command_reply: tid=0 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.288+0000 7fe591d8b740 1 client.27092 shutdown 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.288+0000 7fe591d8b740 1 -- 172.21.15.49:0/2437942444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d7900 msgr2=0x561ee64eaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.288+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d7900 0x561ee64eaf70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe578032400 tx=0x7fe578000bc0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 1 -- 172.21.15.49:0/2437942444 shutdown_connections 2024-02-26T06:30:01.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561ee654d350 0x561ee654f770 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ee64e5520 0x561ee64eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe572000640 0 client.27092 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561ee64d8220 0x561ee64eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 1 --2- 172.21.15.49:0/2437942444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561ee64d7900 0x561ee64eaf70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 1 -- 172.21.15.49:0/2437942444 >> 172.21.15.49:0/2437942444 conn(0x561ee64d2340 msgr2=0x561ee64d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 1 -- 172.21.15.49:0/2437942444 shutdown_connections 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 1 -- 172.21.15.49:0/2437942444 wait complete. 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.289+0000 7fe591d8b740 20 client.27092 trim_cache size 0 max 16384 2024-02-26T06:30:01.292 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2024-02-26T06:30:01.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 -- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe56c0a22b0 msgr2=0x7fe56c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe56c0a22b0 0x7fe56c0a4770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe580009fd0 tx=0x7fe580010040 comp rx=0 tx=0).stop 2024-02-26T06:30:01.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 -- 172.21.15.49:0/2120032337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe588106450 msgr2=0x7fe58819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:01.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe588106450 0x7fe58819d940 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe57400e9c0 tx=0x7fe57400ee80 comp rx=0 tx=0).stop 2024-02-26T06:30:01.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 -- 172.21.15.49:0/2120032337 shutdown_connections 2024-02-26T06:30:01.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe56c0a22b0 0x7fe56c0a4770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe588106450 0x7fe58819d940 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5881975b0 0x7fe588197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 --2- 172.21.15.49:0/2120032337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe588101380 0x7fe58819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:01.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 -- 172.21.15.49:0/2120032337 >> 172.21.15.49:0/2120032337 conn(0x7fe588078580 msgr2=0x7fe5880ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:01.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 -- 172.21.15.49:0/2120032337 shutdown_connections 2024-02-26T06:30:01.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:30:01.290+0000 7fe58ee00640 1 -- 172.21.15.49:0/2120032337 wait complete. 2024-02-26T06:30:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 370.0 s, 2068.6 tps, lat 46.419 ms stddev 54.080 2024-02-26T06:30:01.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:01 smithi049 ceph-mon[31696]: pgmap v201: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 665 op/s 2024-02-26T06:30:01.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:01 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1189055521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:30:01.306 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 313119143, 'ino': 2199023256084, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000032'}, {'damage_type': 'backtrace', 'id': 949296035, 'ino': 1099511629329, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000003B'}, {'damage_type': 'backtrace', 'id': 1288646021, 'ino': 2199023256081, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002F'}, {'damage_type': 'backtrace', 'id': 1562620774, 'ino': 2199023256076, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002B'}, {'damage_type': 'backtrace', 'id': 1909543629, 'ino': 3298534883888, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000034'}, {'damage_type': 'backtrace', 'id': 3078138514, 'ino': 1099511629285, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000053'}, {'damage_type': 'backtrace', 'id': 3113980660, 'ino': 2199023256079, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/00000001000000000000002D'}, {'damage_type': 'backtrace', 'id': 3184263012, 'ino': 2199023256082, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000030'}, {'damage_type': 'backtrace', 'id': 4012606509, 'ino': 2199023256088, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000037'}, {'damage_type': 'backtrace', 'id': 4279780572, 'ino': 2199023256068, 'path': '/volumes/qa/sv_1/f0ab4481-e268-4de4-aa7d-4cfde7b9cdf6/postgres/data/pg_wal/000000010000000000000061'}] 2024-02-26T06:30:01.306 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-02-26T06:30:01.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:01 smithi080 ceph-mon[36206]: pgmap v201: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 665 op/s 2024-02-26T06:30:01.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:01 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1189055521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:30:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:01 smithi115 ceph-mon[36426]: pgmap v201: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 665 op/s 2024-02-26T06:30:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:01 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1189055521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:30:03.382 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:03 smithi080 ceph-mon[36206]: pgmap v202: 97 pgs: 97 active+clean; 4.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 735 op/s 2024-02-26T06:30:03.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:03 smithi049 ceph-mon[31696]: pgmap v202: 97 pgs: 97 active+clean; 4.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 735 op/s 2024-02-26T06:30:03.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:03 smithi115 ceph-mon[36426]: pgmap v202: 97 pgs: 97 active+clean; 4.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 735 op/s 2024-02-26T06:30:05.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:05 smithi049 ceph-mon[31696]: pgmap v203: 97 pgs: 97 active+clean; 4.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 564 op/s 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 -- 172.21.15.49:0/10773530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 msgr2=0x7fd2581027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 --2- 172.21.15.49:0/10773530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 0x7fd2581027d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd240009900 tx=0x7fd24002bd60 comp rx=0 tx=0).stop 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 -- 172.21.15.49:0/10773530 shutdown_connections 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 --2- 172.21.15.49:0/10773530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd25810b730 0x7fd25810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 --2- 172.21.15.49:0/10773530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd258102d10 0x7fd25810b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 --2- 172.21.15.49:0/10773530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 0x7fd2581027d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 -- 172.21.15.49:0/10773530 >> 172.21.15.49:0/10773530 conn(0x7fd2580771a0 msgr2=0x7fd2580775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 -- 172.21.15.49:0/10773530 shutdown_connections 2024-02-26T06:30:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.507+0000 7fd25d200640 1 -- 172.21.15.49:0/10773530 wait complete. 2024-02-26T06:30:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.508+0000 7fd25d200640 1 Processor -- start 2024-02-26T06:30:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd25d200640 1 -- start start 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd25d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 0x7fd258100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd25d200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd258102d10 0x7fd258100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd25d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd25810b730 0x7fd258100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd25d200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd258101f90 con 0x7fd25810b730 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd25d200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2581aa0a0 con 0x7fd2581023f0 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd25d200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2581aa1e0 con 0x7fd258102d10 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd255e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 0x7fd258100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd255e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 0x7fd258100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54856/0 (socket says 172.21.15.49:54856) 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd255e00640 1 -- 172.21.15.49:0/2208372007 learned_addr learned my addr 172.21.15.49:0/2208372007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.509+0000 7fd255400640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd258102d10 0x7fd258100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd256800640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd25810b730 0x7fd258100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd255e00640 1 -- 172.21.15.49:0/2208372007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd258102d10 msgr2=0x7fd258100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd255e00640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd258102d10 0x7fd258100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd255e00640 1 -- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd25810b730 msgr2=0x7fd258100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd255e00640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd25810b730 0x7fd258100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd255e00640 1 -- 172.21.15.49:0/2208372007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240009560 con 0x7fd2581023f0 2024-02-26T06:30:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd255400640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd258102d10 0x7fd258100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd256800640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd25810b730 0x7fd258100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.510+0000 7fd255e00640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 0x7fd258100040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd24002c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.511+0000 7fd24ea00640 1 -- 172.21.15.49:0/2208372007 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24002dc70 con 0x7fd2581023f0 2024-02-26T06:30:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.511+0000 7fd24ea00640 1 -- 172.21.15.49:0/2208372007 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd24002ddd0 con 0x7fd2581023f0 2024-02-26T06:30:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.511+0000 7fd24ea00640 1 -- 172.21.15.49:0/2208372007 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240003bf0 con 0x7fd2581023f0 2024-02-26T06:30:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.511+0000 7fd25d200640 1 -- 172.21.15.49:0/2208372007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2581aa460 con 0x7fd2581023f0 2024-02-26T06:30:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.511+0000 7fd25d200640 1 -- 172.21.15.49:0/2208372007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2581aa920 con 0x7fd2581023f0 2024-02-26T06:30:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.514+0000 7fd24ea00640 1 -- 172.21.15.49:0/2208372007 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd240003d50 con 0x7fd2581023f0 2024-02-26T06:30:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.514+0000 7fd2601d0740 1 Processor -- start 2024-02-26T06:30:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.514+0000 7fd2601d0740 1 -- start start 2024-02-26T06:30:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.514+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b25867f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.514+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258680200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258689300 0x55b25868b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd2601d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25863b5f0 con 0x55b258689300 2024-02-26T06:30:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd2601d0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2585ed890 con 0x55b25867fd80 2024-02-26T06:30:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd2601d0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2585eadc0 con 0x55b258680af0 2024-02-26T06:30:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd256800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b25867f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd256800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b25867f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33982/0 (socket says 172.21.15.49:33982) 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd255e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258680200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd256800640 1 -- 172.21.15.49:0/2124121096 learned_addr learned my addr 172.21.15.49:0/2124121096 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd255400640 1 --2- 172.21.15.49:0/2124121096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258689300 0x55b25868b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd256800640 1 -- 172.21.15.49:0/2124121096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 msgr2=0x55b258680200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd256800640 1 --2- 172.21.15.49:0/2124121096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258680200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd256800640 1 -- 172.21.15.49:0/2124121096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258689300 msgr2=0x55b25868b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd256800640 1 --2- 172.21.15.49:0/2124121096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258689300 0x55b25868b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.515+0000 7fd256800640 1 -- 172.21.15.49:0/2124121096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b258567ad0 con 0x55b258680af0 2024-02-26T06:30:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.516+0000 7fd256800640 1 --2- 172.21.15.49:0/2124121096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b25867f840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd250009c20 tx=0x7fd2500117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af5a87040360864 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.516+0000 7fd247400640 1 -- 172.21.15.49:0/2124121096 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250009720 con 0x55b258680af0 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.516+0000 7fd247400640 1 -- 172.21.15.49:0/2124121096 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd250012930 con 0x55b258680af0 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.516+0000 7fd247400640 1 -- 172.21.15.49:0/2124121096 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25000fa70 con 0x55b258680af0 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.516+0000 7fd24ea00640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd23c0a2350 0x7fd23c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.516+0000 7fd24ea00640 1 -- 172.21.15.49:0/2208372007 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd2400f0610 con 0x7fd2581023f0 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd255e00640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd23c0a2350 0x7fd23c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 -- 172.21.15.49:0/2124121096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 msgr2=0x55b25867f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2124121096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b25867f840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd250009c20 tx=0x7fd2500117b0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 -- 172.21.15.49:0/2124121096 shutdown_connections 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2124121096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258680200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2124121096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258689300 0x55b25868b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2124121096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b25867f840 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 -- 172.21.15.49:0/2124121096 >> 172.21.15.49:0/2124121096 conn(0x55b258647480 msgr2=0x55b25867aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 -- 172.21.15.49:0/2124121096 shutdown_connections 2024-02-26T06:30:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd255e00640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd23c0a2350 0x7fd23c0a4810 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55b258681250 tx=0x7fd2400f60e0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.517+0000 7fd2601d0740 1 -- 172.21.15.49:0/2124121096 wait complete. 2024-02-26T06:30:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.518+0000 7fd2601d0740 1 -- start start 2024-02-26T06:30:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.518+0000 7fd2601d0740 1 Processor -- start 2024-02-26T06:30:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b25867fd80 0x55b258686330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b258686870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b258686db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd2601d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2585e65a0 con 0x55b25867fd80 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd2601d0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25858bb20 con 0x55b258689300 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd2601d0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25858b7c0 con 0x55b258680af0 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd2601d0740 10 client.0 _ll_register_callbacks cb 0x55b25859f520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd255400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b258686870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd256800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b25867fd80 0x55b258686330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd255400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b258686870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33984/0 (socket says 172.21.15.49:33984) 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd255400640 1 -- 172.21.15.49:0/1220401829 learned_addr learned my addr 172.21.15.49:0/1220401829 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.519+0000 7fd256800640 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b258686db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd255400640 1 -- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 msgr2=0x55b258686db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd255400640 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b258686db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd255400640 1 -- 172.21.15.49:0/1220401829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b25867fd80 msgr2=0x55b258686330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd255400640 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b25867fd80 0x55b258686330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd255400640 1 -- 172.21.15.49:0/1220401829 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd250009880 con 0x55b258680af0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd256800640 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b258686db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd255400640 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b258686870 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd248012d80 tx=0x7fd24800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fb22329d17f177e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd23be00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd23be00640 1 -- 172.21.15.49:0/1220401829 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248017070 con 0x55b258680af0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd23be00640 1 -- 172.21.15.49:0/1220401829 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd248003970 con 0x55b258680af0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd2601d0740 1 -- 172.21.15.49:0/1220401829 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2585b8ca0 con 0x55b258680af0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.520+0000 7fd23be00640 1 -- 172.21.15.49:0/1220401829 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd248003b30 con 0x55b258680af0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd2601d0740 10 client.27088 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd2601d0740 1 -- 172.21.15.49:0/1220401829 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b2585e4c20 con 0x55b258680af0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd23be00640 1 -- 172.21.15.49:0/1220401829 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b2585b8ca0 con 0x55b258680af0 2024-02-26T06:30:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd23be00640 1 -- 172.21.15.49:0/1220401829 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd248023070 con 0x55b258680af0 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd2601d0740 10 client.27088 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd2601d0740 10 client.27088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd2601d0740 20 client.27088 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd2601d0740 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2586993f0 0x55b25869b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd2601d0740 4 client.27088 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd2601d0740 1 -- 172.21.15.49:0/1220401829 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2585dccb0 con 0x55b2586993f0 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.521+0000 7fd256800640 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2586993f0 0x55b25869b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.522+0000 7fd256800640 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2586993f0 0x55b25869b810 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7013f6029caba96 server_cookie=8f1b82b7f1da4f86 in_seq=0 out_seq=0 2024-02-26T06:30:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.522+0000 7fd23be00640 10 client.27088 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.522+0000 7fd23be00640 1 -- 172.21.15.49:0/1220401829 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2585dccb0 con 0x55b2586993f0 2024-02-26T06:30:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.522+0000 7fd23be00640 10 client.27088 handle_command_reply: tid=0 2024-02-26T06:30:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 client.27088 shutdown 2024-02-26T06:30:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 -- 172.21.15.49:0/1220401829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 msgr2=0x55b258686870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b258686870 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd248012d80 tx=0x7fd24800f790 comp rx=0 tx=0).stop 2024-02-26T06:30:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 -- 172.21.15.49:0/1220401829 shutdown_connections 2024-02-26T06:30:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2586993f0 0x55b25869b810 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b258686db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258680af0 0x55b258686870 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 --2- 172.21.15.49:0/1220401829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b25867fd80 0x55b258686330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd23be00640 0 client.27088 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 -- 172.21.15.49:0/1220401829 >> 172.21.15.49:0/1220401829 conn(0x55b258647480 msgr2=0x55b25867c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 -- 172.21.15.49:0/1220401829 shutdown_connections 2024-02-26T06:30:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 1 -- 172.21.15.49:0/1220401829 wait complete. 2024-02-26T06:30:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.523+0000 7fd2601d0740 20 client.27088 trim_cache size 0 max 16384 2024-02-26T06:30:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.540+0000 7fd2601d0740 1 Processor -- start 2024-02-26T06:30:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.540+0000 7fd2601d0740 1 -- start start 2024-02-26T06:30:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258688a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b258686bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258689300 0x55b2586835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd2601d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25832ee60 con 0x55b258680af0 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd2601d0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b258687680 con 0x55b25867fd80 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd2601d0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25867f050 con 0x55b258689300 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258688a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258688a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:54894/0 (socket says 172.21.15.49:54894) 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 -- 172.21.15.49:0/3549574925 learned_addr learned my addr 172.21.15.49:0/3549574925 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd255400640 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b258686bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258689300 0x55b2586835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 -- 172.21.15.49:0/3549574925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258689300 msgr2=0x55b2586835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258689300 0x55b2586835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 -- 172.21.15.49:0/3549574925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 msgr2=0x55b258686bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b258686bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.541+0000 7fd256800640 1 -- 172.21.15.49:0/3549574925 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b258581500 con 0x55b25867fd80 2024-02-26T06:30:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.542+0000 7fd256800640 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258688a60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd25001c040 tx=0x7fd25000f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8b259652775033 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.542+0000 7fd246a00640 1 -- 172.21.15.49:0/3549574925 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25002eb00 con 0x55b25867fd80 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.542+0000 7fd246a00640 1 -- 172.21.15.49:0/3549574925 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd250034020 con 0x55b25867fd80 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.542+0000 7fd246a00640 1 -- 172.21.15.49:0/3549574925 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25000fcf0 con 0x55b25867fd80 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.543+0000 7fd2601d0740 1 -- 172.21.15.49:0/3549574925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 msgr2=0x55b258688a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.543+0000 7fd2601d0740 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258688a60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd25001c040 tx=0x7fd25000f670 comp rx=0 tx=0).stop 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.543+0000 7fd2601d0740 1 -- 172.21.15.49:0/3549574925 shutdown_connections 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.543+0000 7fd2601d0740 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b258689300 0x55b2586835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.543+0000 7fd2601d0740 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b258686bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.543+0000 7fd2601d0740 1 --2- 172.21.15.49:0/3549574925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b25867fd80 0x55b258688a60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.543+0000 7fd2601d0740 1 -- 172.21.15.49:0/3549574925 >> 172.21.15.49:0/3549574925 conn(0x55b258647480 msgr2=0x55b258684c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.544+0000 7fd2601d0740 1 -- 172.21.15.49:0/3549574925 shutdown_connections 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.544+0000 7fd2601d0740 1 -- 172.21.15.49:0/3549574925 wait complete. 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.544+0000 7fd2601d0740 1 -- start start 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.544+0000 7fd2601d0740 1 Processor -- start 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b25867fd80 0x55b25868ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b25868f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd2601d0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b2586ef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd2601d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25868fae0 con 0x55b258680af0 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd2601d0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25868fc50 con 0x55b258689300 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd2601d0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2586efe40 con 0x55b25867fd80 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd2601d0740 10 client.0 _ll_register_callbacks cb 0x7fd248012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd256800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b25867fd80 0x55b25868ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd256800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b25867fd80 0x55b25868ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34002/0 (socket says 172.21.15.49:34002) 2024-02-26T06:30:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd256800640 1 -- 172.21.15.49:0/2752132134 learned_addr learned my addr 172.21.15.49:0/2752132134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd255400640 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b25868f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.545+0000 7fd256800640 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b2586ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd255400640 1 -- 172.21.15.49:0/2752132134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b25867fd80 msgr2=0x55b25868ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd255400640 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b25867fd80 0x55b25868ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd255400640 1 -- 172.21.15.49:0/2752132134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 msgr2=0x55b2586ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd255400640 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b2586ef900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd255400640 1 -- 172.21.15.49:0/2752132134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b258567ad0 con 0x55b258680af0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd255400640 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b25868f380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd248010480 tx=0x7fd248009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6528bf8997cc6a41 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd23be00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd23be00640 1 -- 172.21.15.49:0/2752132134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248017030 con 0x55b258680af0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd2601d0740 1 -- 172.21.15.49:0/2752132134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b25860bd00 con 0x55b258680af0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.546+0000 7fd23be00640 1 -- 172.21.15.49:0/2752132134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8506+0+0 (secure 0 0 0) 0x7fd248010a00 con 0x55b258680af0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd23be00640 1 -- 172.21.15.49:0/2752132134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd248010bc0 con 0x55b258680af0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd2601d0740 10 client.16554 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd2601d0740 1 -- 172.21.15.49:0/2752132134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b2585a7290 con 0x55b258680af0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd23be00640 1 -- 172.21.15.49:0/2752132134 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b25860bd00 con 0x55b258680af0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd23be00640 1 -- 172.21.15.49:0/2752132134 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd248023070 con 0x55b258680af0 2024-02-26T06:30:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd2601d0740 10 client.16554 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd2601d0740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd2601d0740 20 client.16554 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2586f16b0 0x55b2586f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd2601d0740 4 client.16554 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:30:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.547+0000 7fd2601d0740 1 -- 172.21.15.49:0/2752132134 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b2582e6600 con 0x55b2586f16b0 2024-02-26T06:30:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.548+0000 7fd256800640 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2586f16b0 0x55b2586f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.548+0000 7fd256800640 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2586f16b0 0x55b2586f3ad0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5d31ebf97f934f server_cookie=3a52478ff6681ebf in_seq=0 out_seq=0 2024-02-26T06:30:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.548+0000 7fd23be00640 10 client.16554 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd23be00640 1 -- 172.21.15.49:0/2752132134 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b2582e6600 con 0x55b2586f16b0 2024-02-26T06:30:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd23be00640 10 client.16554 handle_command_reply: tid=0 2024-02-26T06:30:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 client.16554 shutdown 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 -- 172.21.15.49:0/2752132134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 msgr2=0x55b25868f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b25868f380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd248010480 tx=0x7fd248009f00 comp rx=0 tx=0).stop 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 -- 172.21.15.49:0/2752132134 shutdown_connections 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2586f16b0 0x55b2586f3ad0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b258689300 0x55b2586ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b258680af0 0x55b25868f380 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 --2- 172.21.15.49:0/2752132134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b25867fd80 0x55b25868ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd23be00640 0 client.16554 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.565+0000 7fd2601d0740 1 -- 172.21.15.49:0/2752132134 >> 172.21.15.49:0/2752132134 conn(0x55b258647480 msgr2=0x55b258679480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.566+0000 7fd2601d0740 1 -- 172.21.15.49:0/2752132134 shutdown_connections 2024-02-26T06:30:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.566+0000 7fd2601d0740 1 -- 172.21.15.49:0/2752132134 wait complete. 2024-02-26T06:30:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.567+0000 7fd2601d0740 20 client.16554 trim_cache size 0 max 16384 2024-02-26T06:30:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:30:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:30:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:30:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:30:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.569+0000 7fd25d200640 1 -- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd23c0a2350 msgr2=0x7fd23c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.569+0000 7fd25d200640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd23c0a2350 0x7fd23c0a4810 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55b258681250 tx=0x7fd2400f60e0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.569+0000 7fd25d200640 1 -- 172.21.15.49:0/2208372007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 msgr2=0x7fd258100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.569+0000 7fd25d200640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 0x7fd258100040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd24002c760 comp rx=0 tx=0).stop 2024-02-26T06:30:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.570+0000 7fd25d200640 1 -- 172.21.15.49:0/2208372007 shutdown_connections 2024-02-26T06:30:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.570+0000 7fd25d200640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd23c0a2350 0x7fd23c0a4810 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.570+0000 7fd25d200640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd25810b730 0x7fd258100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.570+0000 7fd25d200640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd258102d10 0x7fd258100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.570+0000 7fd25d200640 1 --2- 172.21.15.49:0/2208372007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2581023f0 0x7fd258100040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.570+0000 7fd25d200640 1 -- 172.21.15.49:0/2208372007 >> 172.21.15.49:0/2208372007 conn(0x7fd2580771a0 msgr2=0x7fd2580fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.570+0000 7fd25d200640 1 -- 172.21.15.49:0/2208372007 shutdown_connections 2024-02-26T06:30:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:05.570+0000 7fd25d200640 1 -- 172.21.15.49:0/2208372007 wait complete. 2024-02-26T06:30:05.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:05 smithi080 ceph-mon[36206]: pgmap v203: 97 pgs: 97 active+clean; 4.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 564 op/s 2024-02-26T06:30:05.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:05 smithi115 ceph-mon[36426]: pgmap v203: 97 pgs: 97 active+clean; 4.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 564 op/s 2024-02-26T06:30:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 375.0 s, 124.0 tps, lat 757.169 ms stddev 480.842 2024-02-26T06:30:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:07 smithi049 ceph-mon[31696]: pgmap v204: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.8 MiB/s wr, 797 op/s 2024-02-26T06:30:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:07 smithi049 ceph-mon[31696]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T06:30:07.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:07 smithi080 ceph-mon[36206]: pgmap v204: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.8 MiB/s wr, 797 op/s 2024-02-26T06:30:07.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:07 smithi080 ceph-mon[36206]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T06:30:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:07 smithi115 ceph-mon[36426]: pgmap v204: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.8 MiB/s wr, 797 op/s 2024-02-26T06:30:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:07 smithi115 ceph-mon[36426]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T06:30:09.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:09 smithi049 ceph-mon[31696]: pgmap v206: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.0 MiB/s wr, 623 op/s 2024-02-26T06:30:09.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:09 smithi080 ceph-mon[36206]: pgmap v206: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.0 MiB/s wr, 623 op/s 2024-02-26T06:30:09.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:09 smithi115 ceph-mon[36426]: pgmap v206: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.0 MiB/s wr, 623 op/s 2024-02-26T06:30:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 380.0 s, 158.8 tps, lat 675.452 ms stddev 1144.603 2024-02-26T06:30:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:11 smithi080 ceph-mon[36206]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.9 MiB/s wr, 521 op/s 2024-02-26T06:30:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:11 smithi115 ceph-mon[36426]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.9 MiB/s wr, 521 op/s 2024-02-26T06:30:11.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:11 smithi049 ceph-mon[31696]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.9 MiB/s wr, 521 op/s 2024-02-26T06:30:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:13 smithi080 ceph-mon[36206]: pgmap v208: 97 pgs: 97 active+clean; 3.7 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.9 MiB/s wr, 499 op/s 2024-02-26T06:30:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:13 smithi115 ceph-mon[36426]: pgmap v208: 97 pgs: 97 active+clean; 3.7 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.9 MiB/s wr, 499 op/s 2024-02-26T06:30:13.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:13 smithi049 ceph-mon[31696]: pgmap v208: 97 pgs: 97 active+clean; 3.7 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.9 MiB/s wr, 499 op/s 2024-02-26T06:30:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:15 smithi080 ceph-mon[36206]: pgmap v209: 97 pgs: 97 active+clean; 3.7 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.9 MiB/s wr, 499 op/s 2024-02-26T06:30:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:15 smithi080 ceph-mon[36206]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T06:30:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:15 smithi115 ceph-mon[36426]: pgmap v209: 97 pgs: 97 active+clean; 3.7 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.9 MiB/s wr, 499 op/s 2024-02-26T06:30:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:15 smithi115 ceph-mon[36426]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T06:30:15.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:15 smithi049 ceph-mon[31696]: pgmap v209: 97 pgs: 97 active+clean; 3.7 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.9 MiB/s wr, 499 op/s 2024-02-26T06:30:15.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:15 smithi049 ceph-mon[31696]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T06:30:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 385.0 s, 1749.8 tps, lat 58.574 ms stddev 93.374 2024-02-26T06:30:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:17 smithi080 ceph-mon[36206]: pgmap v211: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-02-26T06:30:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:17 smithi115 ceph-mon[36426]: pgmap v211: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-02-26T06:30:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:17 smithi049 ceph-mon[31696]: pgmap v211: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 -- 172.21.15.49:0/3294335661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2b0073e70 msgr2=0x7fa2b010d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 --2- 172.21.15.49:0/3294335661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2b0073e70 0x7fa2b010d2e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa298009ca0 tx=0x7fa29802c190 comp rx=0 tx=0).stop 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 -- 172.21.15.49:0/3294335661 shutdown_connections 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 --2- 172.21.15.49:0/3294335661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2b010d820 0x7fa2b010fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 --2- 172.21.15.49:0/3294335661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2b0073e70 0x7fa2b010d2e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 --2- 172.21.15.49:0/3294335661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0073550 0x7fa2b0073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 -- 172.21.15.49:0/3294335661 >> 172.21.15.49:0/3294335661 conn(0x7fa2b00fca90 msgr2=0x7fa2b00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 -- 172.21.15.49:0/3294335661 shutdown_connections 2024-02-26T06:30:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.719+0000 7fa2afe00640 1 -- 172.21.15.49:0/3294335661 wait complete. 2024-02-26T06:30:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.720+0000 7fa2afe00640 1 Processor -- start 2024-02-26T06:30:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2afe00640 1 -- start start 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2afe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0073550 0x7fa2b019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2afe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2b0073e70 0x7fa2b019fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2afe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2b010d820 0x7fa2b0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2afe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0199df0 con 0x7fa2b0073550 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2afe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0199f60 con 0x7fa2b010d820 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2afe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b019a120 con 0x7fa2b0073e70 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0073550 0x7fa2b019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2af400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2b010d820 0x7fa2b0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2b0073e70 0x7fa2b019fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0073550 0x7fa2b019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43870/0 (socket says 172.21.15.49:43870) 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.721+0000 7fa2aea00640 1 -- 172.21.15.49:0/460448254 learned_addr learned my addr 172.21.15.49:0/460448254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.722+0000 7fa2af400640 1 -- 172.21.15.49:0/460448254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2b0073e70 msgr2=0x7fa2b019fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.722+0000 7fa2af400640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2b0073e70 0x7fa2b019fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.722+0000 7fa2af400640 1 -- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0073550 msgr2=0x7fa2b019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.722+0000 7fa2af400640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0073550 0x7fa2b019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.722+0000 7fa2af400640 1 -- 172.21.15.49:0/460448254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa298009900 con 0x7fa2b010d820 2024-02-26T06:30:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.722+0000 7fa2aea00640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0073550 0x7fa2b019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.722+0000 7fa2af400640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2b010d820 0x7fa2b0199660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00744a0 tx=0x7fa2a800edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.723+0000 7fa2a7400640 1 -- 172.21.15.49:0/460448254 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8009620 con 0x7fa2b010d820 2024-02-26T06:30:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.723+0000 7fa2a7400640 1 -- 172.21.15.49:0/460448254 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a800ccc0 con 0x7fa2b010d820 2024-02-26T06:30:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.723+0000 7fa2afe00640 1 -- 172.21.15.49:0/460448254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b019a400 con 0x7fa2b010d820 2024-02-26T06:30:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.723+0000 7fa2a7400640 1 -- 172.21.15.49:0/460448254 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8004a00 con 0x7fa2b010d820 2024-02-26T06:30:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.723+0000 7fa2afe00640 1 -- 172.21.15.49:0/460448254 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b01a6520 con 0x7fa2b010d820 2024-02-26T06:30:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.726+0000 7fa2b8811740 1 Processor -- start 2024-02-26T06:30:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.726+0000 7fa2a7400640 1 -- 172.21.15.49:0/460448254 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa2a8004b60 con 0x7fa2b010d820 2024-02-26T06:30:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.726+0000 7fa2b8811740 1 -- start start 2024-02-26T06:30:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae45290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae45c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2b8811740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18adf85f0 con 0x55e18ae457d0 2024-02-26T06:30:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2b8811740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18adaa890 con 0x55e18ae4ed50 2024-02-26T06:30:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2b8811740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18ada7dc0 con 0x55e18ae46540 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae45c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae45290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae45290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48030/0 (socket says 172.21.15.49:48030) 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2ae000640 1 -- 172.21.15.49:0/4222664949 learned_addr learned my addr 172.21.15.49:0/4222664949 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.727+0000 7fa2ae000640 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2aea00640 1 -- 172.21.15.49:0/4222664949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 msgr2=0x55e18ae45290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2aea00640 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae45290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2aea00640 1 -- 172.21.15.49:0/4222664949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 msgr2=0x55e18ae51140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2aea00640 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae51140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2aea00640 1 -- 172.21.15.49:0/4222664949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e18ad24ad0 con 0x55e18ae457d0 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2ae000640 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae51140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2aea00640 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae45c50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00114f0 tx=0x7fa2a00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb115ad474a7c00 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2a7400640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa2900a2220 0x7fa2900a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa29fe00640 1 -- 172.21.15.49:0/4222664949 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0011250 con 0x55e18ae457d0 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2a7400640 1 -- 172.21.15.49:0/460448254 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa2a8014070 con 0x7fa2b010d820 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa29fe00640 1 -- 172.21.15.49:0/4222664949 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0004510 con 0x55e18ae457d0 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa29fe00640 1 -- 172.21.15.49:0/4222664949 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000f5c0 con 0x55e18ae457d0 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.728+0000 7fa2ae000640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa2900a2220 0x7fa2900a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 -- 172.21.15.49:0/4222664949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 msgr2=0x55e18ae45c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2ae000640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa2900a2220 0x7fa2900a46e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55e18ae41650 tx=0x7fa2980095c0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae45c50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00114f0 tx=0x7fa2a00119b0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 -- 172.21.15.49:0/4222664949 shutdown_connections 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae51140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae45c50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 --2- 172.21.15.49:0/4222664949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae45290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 -- 172.21.15.49:0/4222664949 >> 172.21.15.49:0/4222664949 conn(0x55e18ae04480 msgr2=0x55e18ae404c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 -- 172.21.15.49:0/4222664949 shutdown_connections 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.729+0000 7fa2b8811740 1 -- 172.21.15.49:0/4222664949 wait complete. 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.730+0000 7fa2b8811740 1 -- start start 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.730+0000 7fa2b8811740 1 Processor -- start 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae4c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2b8811740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18ada35a0 con 0x55e18ae457d0 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2b8811740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18ad48b20 con 0x55e18ae4ed50 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2b8811740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18ad487c0 con 0x55e18ae46540 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2b8811740 10 client.0 _ll_register_callbacks cb 0x55e18ad5c520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43896/0 (socket says 172.21.15.49:43896) 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2aea00640 1 -- 172.21.15.49:0/2145679776 learned_addr learned my addr 172.21.15.49:0/2145679776 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48036/0 (socket says 172.21.15.49:48036) 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2af400640 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae4c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2af400640 1 -- 172.21.15.49:0/2145679776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 msgr2=0x55e18ae4c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2af400640 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2af400640 1 -- 172.21.15.49:0/2145679776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 msgr2=0x55e18ae4bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2af400640 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.731+0000 7fa2af400640 1 -- 172.21.15.49:0/2145679776 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a0009cd0 con 0x55e18ae4ed50 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa2af400640 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae4c7e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80cb660 tx=0x7fa2a809a350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=464ef5d73aa163dd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa29cc00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa29cc00640 1 -- 172.21.15.49:0/2145679776 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80b3400 con 0x55e18ae4ed50 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa2b8811740 1 -- 172.21.15.49:0/2145679776 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e18ad75ca0 con 0x55e18ae4ed50 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa29cc00640 1 -- 172.21.15.49:0/2145679776 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa2a809ea80 con 0x55e18ae4ed50 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa29cc00640 1 -- 172.21.15.49:0/2145679776 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2a80a30d0 con 0x55e18ae4ed50 2024-02-26T06:30:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa2b8811740 10 client.27122 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa2b8811740 1 -- 172.21.15.49:0/2145679776 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e18ada1c20 con 0x55e18ae4ed50 2024-02-26T06:30:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.732+0000 7fa29cc00640 1 -- 172.21.15.49:0/2145679776 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a807d020 con 0x55e18ae4ed50 2024-02-26T06:30:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.733+0000 7fa29cc00640 1 -- 172.21.15.49:0/2145679776 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa2a80a2c40 con 0x55e18ae4ed50 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.733+0000 7fa2b8811740 10 client.27122 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.733+0000 7fa2b8811740 10 client.27122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.733+0000 7fa2b8811740 20 client.27122 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.733+0000 7fa2b8811740 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e18ae5edf0 0x55e18ae61210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.733+0000 7fa2b8811740 4 client.27122 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.733+0000 7fa2b8811740 1 -- 172.21.15.49:0/2145679776 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e18ad99cb0 con 0x55e18ae5edf0 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.733+0000 7fa2aea00640 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e18ae5edf0 0x55e18ae61210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.734+0000 7fa2aea00640 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e18ae5edf0 0x55e18ae61210 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e418ca9a9edb5c5 server_cookie=aaed6fbfe8d44a05 in_seq=0 out_seq=0 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.734+0000 7fa29cc00640 10 client.27122 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.734+0000 7fa29cc00640 1 -- 172.21.15.49:0/2145679776 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e18ad99cb0 con 0x55e18ae5edf0 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.734+0000 7fa29cc00640 10 client.27122 handle_command_reply: tid=0 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.734+0000 7fa2b8811740 1 client.27122 shutdown 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.734+0000 7fa2b8811740 1 -- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 msgr2=0x55e18ae4c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.734+0000 7fa2b8811740 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae4c7e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80cb660 tx=0x7fa2a809a350 comp rx=0 tx=0).stop 2024-02-26T06:30:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 1 -- 172.21.15.49:0/2145679776 shutdown_connections 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e18ae5edf0 0x55e18ae61210 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae4c7e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 1 --2- 172.21.15.49:0/2145679776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 1 -- 172.21.15.49:0/2145679776 >> 172.21.15.49:0/2145679776 conn(0x55e18ae04480 msgr2=0x55e18ae42210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa29cc00640 0 client.27122 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 1 -- 172.21.15.49:0/2145679776 shutdown_connections 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 1 -- 172.21.15.49:0/2145679776 wait complete. 2024-02-26T06:30:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.735+0000 7fa2b8811740 20 client.27122 trim_cache size 0 max 16384 2024-02-26T06:30:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.749+0000 7fa2b8811740 1 Processor -- start 2024-02-26T06:30:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.749+0000 7fa2b8811740 1 -- start start 2024-02-26T06:30:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.749+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.749+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.749+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae55b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.749+0000 7fa2b8811740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18aaebe60 con 0x55e18ae457d0 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.749+0000 7fa2b8811740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18ae479c0 con 0x55e18ae4ed50 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.749+0000 7fa2b8811740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18ae44aa0 con 0x55e18ae46540 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43904/0 (socket says 172.21.15.49:43904) 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2aea00640 1 -- 172.21.15.49:0/4210433933 learned_addr learned my addr 172.21.15.49:0/4210433933 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2af400640 1 --2- 172.21.15.49:0/4210433933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae55b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2aea00640 1 -- 172.21.15.49:0/4210433933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 msgr2=0x55e18ae4c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2aea00640 1 --2- 172.21.15.49:0/4210433933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2aea00640 1 -- 172.21.15.49:0/4210433933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 msgr2=0x55e18ae55b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2aea00640 1 --2- 172.21.15.49:0/4210433933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae55b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.750+0000 7fa2aea00640 1 -- 172.21.15.49:0/4210433933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e18ad3e500 con 0x55e18ae457d0 2024-02-26T06:30:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa2aea00640 1 --2- 172.21.15.49:0/4210433933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4e490 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa2a001b040 tx=0x7fa2a0009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbb178bab23ef31 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa29f400640 1 -- 172.21.15.49:0/4210433933 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a002a070 con 0x55e18ae457d0 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa29f400640 1 -- 172.21.15.49:0/4210433933 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0005070 con 0x55e18ae457d0 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa29f400640 1 -- 172.21.15.49:0/4210433933 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000f980 con 0x55e18ae457d0 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa2b8811740 1 -- 172.21.15.49:0/4210433933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 msgr2=0x55e18ae4e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa2b8811740 1 --2- 172.21.15.49:0/4210433933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4e490 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa2a001b040 tx=0x7fa2a0009c10 comp rx=0 tx=0).stop 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa2b8811740 1 -- 172.21.15.49:0/4210433933 shutdown_connections 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa2b8811740 1 --2- 172.21.15.49:0/4210433933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae4ed50 0x55e18ae55b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa2b8811740 1 --2- 172.21.15.49:0/4210433933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae4c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa2b8811740 1 --2- 172.21.15.49:0/4210433933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae457d0 0x55e18ae4e490 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.751+0000 7fa2b8811740 1 -- 172.21.15.49:0/4210433933 >> 172.21.15.49:0/4210433933 conn(0x55e18ae04480 msgr2=0x55e18ae65b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.752+0000 7fa2b8811740 1 -- 172.21.15.49:0/4210433933 shutdown_connections 2024-02-26T06:30:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.752+0000 7fa2b8811740 1 -- 172.21.15.49:0/4210433933 wait complete. 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.752+0000 7fa2b8811740 1 -- start start 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.752+0000 7fa2b8811740 1 Processor -- start 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae457d0 0x55e18ae48a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae48f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2b8811740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae4ed50 0x55e18ae494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2b8811740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18aeb52f0 con 0x55e18ae4ed50 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2b8811740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18aeb5430 con 0x55e18ae457d0 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2b8811740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18aeb5570 con 0x55e18ae46540 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2b8811740 10 client.0 _ll_register_callbacks cb 0x7fa2a8009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae457d0 0x55e18ae48a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2ae000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae48f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2aea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae457d0 0x55e18ae48a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57472/0 (socket says 172.21.15.49:57472) 2024-02-26T06:30:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2aea00640 1 -- 172.21.15.49:0/935617379 learned_addr learned my addr 172.21.15.49:0/935617379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2af400640 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae4ed50 0x55e18ae494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2aea00640 1 -- 172.21.15.49:0/935617379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 msgr2=0x55e18ae48f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2aea00640 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae48f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2aea00640 1 -- 172.21.15.49:0/935617379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae4ed50 msgr2=0x55e18ae494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2aea00640 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae4ed50 0x55e18ae494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.753+0000 7fa2aea00640 1 -- 172.21.15.49:0/935617379 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e18ad24ad0 con 0x55e18ae457d0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa2aea00640 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae457d0 0x55e18ae48a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0011230 tx=0x7fa2a00093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e969c83e1a0f5d7b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa29cc00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa29cc00640 1 -- 172.21.15.49:0/935617379 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a002a040 con 0x55e18ae457d0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa2b8811740 1 -- 172.21.15.49:0/935617379 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e18adc8d00 con 0x55e18ae457d0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa29cc00640 1 -- 172.21.15.49:0/935617379 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa2a0003a00 con 0x55e18ae457d0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa29cc00640 1 -- 172.21.15.49:0/935617379 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2a00053e0 con 0x55e18ae457d0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa2b8811740 10 client.27128 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa2b8811740 1 -- 172.21.15.49:0/935617379 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e18ad64290 con 0x55e18ae457d0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa29cc00640 1 -- 172.21.15.49:0/935617379 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0040080 con 0x55e18ae457d0 2024-02-26T06:30:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.754+0000 7fa29cc00640 1 -- 172.21.15.49:0/935617379 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa2a003c610 con 0x55e18ae457d0 2024-02-26T06:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa2b8811740 10 client.27128 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa2b8811740 10 client.27128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa2b8811740 20 client.27128 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa2b8811740 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e18aeb6a40 0x55e18aeb8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa2b8811740 4 client.27128 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa2b8811740 1 -- 172.21.15.49:0/935617379 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e18aaa3600 con 0x55e18aeb6a40 2024-02-26T06:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa2ae000640 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e18aeb6a40 0x55e18aeb8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa2ae000640 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e18aeb6a40 0x55e18aeb8e60 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d18fc3eb2628df server_cookie=d70c64d401310e5b in_seq=0 out_seq=0 2024-02-26T06:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.755+0000 7fa29cc00640 10 client.27128 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.780+0000 7fa29cc00640 1 -- 172.21.15.49:0/935617379 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e18aaa3600 con 0x55e18aeb6a40 2024-02-26T06:30:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.780+0000 7fa29cc00640 10 client.27128 handle_command_reply: tid=0 2024-02-26T06:30:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.781+0000 7fa2b8811740 1 client.27128 shutdown 2024-02-26T06:30:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.781+0000 7fa2b8811740 1 -- 172.21.15.49:0/935617379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae457d0 msgr2=0x55e18ae48a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.781+0000 7fa2b8811740 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae457d0 0x55e18ae48a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0011230 tx=0x7fa2a00093d0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa2b8811740 1 -- 172.21.15.49:0/935617379 shutdown_connections 2024-02-26T06:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa2b8811740 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55e18aeb6a40 0x55e18aeb8e60 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa2b8811740 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e18ae4ed50 0x55e18ae494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa2b8811740 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e18ae46540 0x55e18ae48f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa2b8811740 1 --2- 172.21.15.49:0/935617379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e18ae457d0 0x55e18ae48a30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa2b8811740 1 -- 172.21.15.49:0/935617379 >> 172.21.15.49:0/935617379 conn(0x55e18ae04480 msgr2=0x55e18ae4a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa29cc00640 0 client.27128 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa2b8811740 1 -- 172.21.15.49:0/935617379 shutdown_connections 2024-02-26T06:30:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.782+0000 7fa2b8811740 1 -- 172.21.15.49:0/935617379 wait complete. 2024-02-26T06:30:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.783+0000 7fa2b8811740 20 client.27128 trim_cache size 0 max 16384 2024-02-26T06:30:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:30:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:30:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:30:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:30:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.787+0000 7fa2afe00640 1 -- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa2900a2220 msgr2=0x7fa2900a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.787+0000 7fa2afe00640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa2900a2220 0x7fa2900a46e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55e18ae41650 tx=0x7fa2980095c0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.787+0000 7fa2afe00640 1 -- 172.21.15.49:0/460448254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2b010d820 msgr2=0x7fa2b0199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.787+0000 7fa2afe00640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2b010d820 0x7fa2b0199660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00744a0 tx=0x7fa2a800edd0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.788+0000 7fa2afe00640 1 -- 172.21.15.49:0/460448254 shutdown_connections 2024-02-26T06:30:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.788+0000 7fa2afe00640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa2900a2220 0x7fa2900a46e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.788+0000 7fa2afe00640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2b010d820 0x7fa2b0199660 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.788+0000 7fa2afe00640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2b0073e70 0x7fa2b019fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.788+0000 7fa2afe00640 1 --2- 172.21.15.49:0/460448254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0073550 0x7fa2b019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.788+0000 7fa2afe00640 1 -- 172.21.15.49:0/460448254 >> 172.21.15.49:0/460448254 conn(0x7fa2b00fca90 msgr2=0x7fa2b00fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.792+0000 7fa2afe00640 1 -- 172.21.15.49:0/460448254 shutdown_connections 2024-02-26T06:30:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:18.794+0000 7fa2afe00640 1 -- 172.21.15.49:0/460448254 wait complete. 2024-02-26T06:30:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:19 smithi080 ceph-mon[36206]: pgmap v212: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 1.37k op/s 2024-02-26T06:30:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:19 smithi115 ceph-mon[36426]: pgmap v212: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 1.37k op/s 2024-02-26T06:30:19.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:19 smithi049 ceph-mon[31696]: pgmap v212: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 1.37k op/s 2024-02-26T06:30:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 390.0 s, 1941.2 tps, lat 51.309 ms stddev 104.540 2024-02-26T06:30:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:21 smithi080 ceph-mon[36206]: pgmap v213: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-02-26T06:30:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:21 smithi115 ceph-mon[36426]: pgmap v213: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-02-26T06:30:21.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:21 smithi049 ceph-mon[31696]: pgmap v213: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-02-26T06:30:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:23 smithi080 ceph-mon[36206]: pgmap v214: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 1.44k op/s 2024-02-26T06:30:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:23 smithi115 ceph-mon[36426]: pgmap v214: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 1.44k op/s 2024-02-26T06:30:23.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:23 smithi049 ceph-mon[31696]: pgmap v214: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 1.44k op/s 2024-02-26T06:30:24.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:24 smithi080 ceph-mon[36206]: pgmap v215: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 1.44k op/s 2024-02-26T06:30:24.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:24 smithi115 ceph-mon[36426]: pgmap v215: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 1.44k op/s 2024-02-26T06:30:24.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:24 smithi049 ceph-mon[31696]: pgmap v215: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 1.44k op/s 2024-02-26T06:30:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 395.0 s, 2541.2 tps, lat 39.468 ms stddev 45.221 2024-02-26T06:30:26.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:26 smithi080 ceph-mon[36206]: pgmap v216: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2024-02-26T06:30:26.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:26 smithi115 ceph-mon[36426]: pgmap v216: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2024-02-26T06:30:26.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:26 smithi049 ceph-mon[31696]: pgmap v216: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2024-02-26T06:30:28.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:28 smithi115 ceph-mon[36426]: pgmap v217: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.1 MiB/s wr, 587 op/s 2024-02-26T06:30:28.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:28 smithi049 ceph-mon[31696]: pgmap v217: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.1 MiB/s wr, 587 op/s 2024-02-26T06:30:29.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:28 smithi080 ceph-mon[36206]: pgmap v217: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.1 MiB/s wr, 587 op/s 2024-02-26T06:30:30.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:30 smithi115 ceph-mon[36426]: pgmap v218: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-02-26T06:30:30.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:30 smithi049 ceph-mon[31696]: pgmap v218: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-02-26T06:30:31.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:30 smithi080 ceph-mon[36206]: pgmap v218: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-02-26T06:30:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 400.0 s, 2043.2 tps, lat 48.753 ms stddev 57.700 2024-02-26T06:30:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.992+0000 7fde06200640 1 -- 172.21.15.49:0/3814164316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde00100b00 msgr2=0x7fde00104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.992+0000 7fde06200640 1 --2- 172.21.15.49:0/3814164316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde00100b00 0x7fde00104fc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fddec009900 tx=0x7fddec02bd50 comp rx=0 tx=0).stop 2024-02-26T06:30:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.992+0000 7fde06200640 1 -- 172.21.15.49:0/3814164316 shutdown_connections 2024-02-26T06:30:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.992+0000 7fde06200640 1 --2- 172.21.15.49:0/3814164316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde00100b00 0x7fde00104fc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.992+0000 7fde06200640 1 --2- 172.21.15.49:0/3814164316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde00105500 0x7fde00105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.992+0000 7fde06200640 1 --2- 172.21.15.49:0/3814164316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde001001e0 0x7fde001005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.992+0000 7fde06200640 1 -- 172.21.15.49:0/3814164316 >> 172.21.15.49:0/3814164316 conn(0x7fde000771a0 msgr2=0x7fde000775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.993+0000 7fde06200640 1 -- 172.21.15.49:0/3814164316 shutdown_connections 2024-02-26T06:30:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.993+0000 7fde06200640 1 -- 172.21.15.49:0/3814164316 wait complete. 2024-02-26T06:30:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.994+0000 7fde06200640 1 Processor -- start 2024-02-26T06:30:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.994+0000 7fde06200640 1 -- start start 2024-02-26T06:30:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.995+0000 7fde06200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde001001e0 0x7fde001a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.995+0000 7fde06200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100b00 0x7fde001a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.995+0000 7fde06200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde00105500 0x7fde0019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.995+0000 7fde06200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019e110 con 0x7fde00100b00 2024-02-26T06:30:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.995+0000 7fde06200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019e280 con 0x7fde00105500 2024-02-26T06:30:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.995+0000 7fde06200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019e440 con 0x7fde001001e0 2024-02-26T06:30:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddff400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde001001e0 0x7fde001a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddff400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde001001e0 0x7fde001a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53242/0 (socket says 172.21.15.49:53242) 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddffe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde00105500 0x7fde0019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddff400640 1 -- 172.21.15.49:0/1351116313 learned_addr learned my addr 172.21.15.49:0/1351116313 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddfea00640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100b00 0x7fde001a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddff400640 1 -- 172.21.15.49:0/1351116313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde00105500 msgr2=0x7fde0019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddff400640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde00105500 0x7fde0019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddff400640 1 -- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100b00 msgr2=0x7fde001a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddff400640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100b00 0x7fde001a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.996+0000 7fddff400640 1 -- 172.21.15.49:0/1351116313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec009560 con 0x7fde001001e0 2024-02-26T06:30:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.997+0000 7fddff400640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde001001e0 0x7fde001a3910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e9f0 tx=0x7fddf000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.997+0000 7fddf7e00640 1 -- 172.21.15.49:0/1351116313 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0009700 con 0x7fde001001e0 2024-02-26T06:30:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.997+0000 7fde06200640 1 -- 172.21.15.49:0/1351116313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0019e720 con 0x7fde001001e0 2024-02-26T06:30:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.997+0000 7fddf7e00640 1 -- 172.21.15.49:0/1351116313 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddf000cd80 con 0x7fde001001e0 2024-02-26T06:30:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.997+0000 7fddf7e00640 1 -- 172.21.15.49:0/1351116313 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0018860 con 0x7fde001001e0 2024-02-26T06:30:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:31.997+0000 7fde06200640 1 -- 172.21.15.49:0/1351116313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde001aa9b0 con 0x7fde001001e0 2024-02-26T06:30:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.001+0000 7fde091ff740 1 Processor -- start 2024-02-26T06:30:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.001+0000 7fddf7e00640 1 -- 172.21.15.49:0/1351116313 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fddf00189e0 con 0x7fde001001e0 2024-02-26T06:30:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.001+0000 7fde091ff740 1 -- start start 2024-02-26T06:30:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.002+0000 7fde091ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3edfc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.002+0000 7fde091ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfcd80 0x560a3edfd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.002+0000 7fde091ff740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3ee06300 0x560a3ee086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.002+0000 7fde091ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3edb85f0 con 0x560a3edfdaf0 2024-02-26T06:30:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.002+0000 7fde091ff740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ed6a890 con 0x560a3ee06300 2024-02-26T06:30:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.002+0000 7fde091ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ed67dc0 con 0x560a3edfcd80 2024-02-26T06:30:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.003+0000 7fddffe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3edfc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddfea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3ee06300 0x560a3ee086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddffe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3edfc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42922/0 (socket says 172.21.15.49:42922) 2024-02-26T06:30:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddffe00640 1 -- 172.21.15.49:0/3525411947 learned_addr learned my addr 172.21.15.49:0/3525411947 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddff400640 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfcd80 0x560a3edfd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddfea00640 1 -- 172.21.15.49:0/3525411947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfcd80 msgr2=0x560a3edfd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddfea00640 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfcd80 0x560a3edfd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddfea00640 1 -- 172.21.15.49:0/3525411947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 msgr2=0x560a3edfc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddfea00640 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3edfc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.004+0000 7fddfea00640 1 -- 172.21.15.49:0/3525411947 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a3ece4ad0 con 0x560a3ee06300 2024-02-26T06:30:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.005+0000 7fddffe00640 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3edfc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.005+0000 7fddf7e00640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdde40a2350 0x7fdde40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.006+0000 7fddfea00640 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3ee06300 0x560a3ee086f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fddec02c290 tx=0x7fddec02c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a964ba98029c72e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.006+0000 7fddf7e00640 1 -- 172.21.15.49:0/1351116313 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fddf0014070 con 0x7fde001001e0 2024-02-26T06:30:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.006+0000 7fddebe00640 1 -- 172.21.15.49:0/3525411947 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec004900 con 0x560a3ee06300 2024-02-26T06:30:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.006+0000 7fddffe00640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdde40a2350 0x7fdde40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.006+0000 7fddebe00640 1 -- 172.21.15.49:0/3525411947 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddec004a60 con 0x560a3ee06300 2024-02-26T06:30:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.006+0000 7fddebe00640 1 -- 172.21.15.49:0/3525411947 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec005050 con 0x560a3ee06300 2024-02-26T06:30:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fde091ff740 1 -- 172.21.15.49:0/3525411947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3ee06300 msgr2=0x560a3ee086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fde091ff740 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3ee06300 0x560a3ee086f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fddec02c290 tx=0x7fddec02c860 comp rx=0 tx=0).stop 2024-02-26T06:30:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fddffe00640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdde40a2350 0x7fdde40a4810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fddf8005f40 tx=0x7fddf8009600 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fde091ff740 1 -- 172.21.15.49:0/3525411947 shutdown_connections 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fde091ff740 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3ee06300 0x560a3ee086f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fde091ff740 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfcd80 0x560a3edfd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fde091ff740 1 --2- 172.21.15.49:0/3525411947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3edfc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fde091ff740 1 -- 172.21.15.49:0/3525411947 >> 172.21.15.49:0/3525411947 conn(0x560a3edc4480 msgr2=0x560a3edf7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.007+0000 7fde091ff740 1 -- 172.21.15.49:0/3525411947 shutdown_connections 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.008+0000 7fde091ff740 1 -- 172.21.15.49:0/3525411947 wait complete. 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.008+0000 7fde091ff740 1 -- start start 2024-02-26T06:30:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.008+0000 7fde091ff740 1 Processor -- start 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fde091ff740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee03320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fde091ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fde091ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee03da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fde091ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ed635a0 con 0x560a3edfdaf0 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fddfea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fddfea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42934/0 (socket says 172.21.15.49:42934) 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fddfea00640 1 -- 172.21.15.49:0/130423379 learned_addr learned my addr 172.21.15.49:0/130423379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fddff400640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee03320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ed08b20 con 0x560a3edfcd80 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ed087c0 con 0x560a3ee06300 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.009+0000 7fde091ff740 10 client.0 _ll_register_callbacks cb 0x560a3ed1c520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddffe00640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee03da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddfea00640 1 -- 172.21.15.49:0/130423379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 msgr2=0x560a3ee03da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddfea00640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee03da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddfea00640 1 -- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 msgr2=0x560a3ee03320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddfea00640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee03320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddfea00640 1 -- 172.21.15.49:0/130423379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde00114e50 con 0x560a3edfdaf0 2024-02-26T06:30:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddffe00640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee03da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:30:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddff400640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee03320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fddfea00640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03860 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fddec036040 tx=0x7fddec02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3259bcf1f9a12ae6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.010+0000 7fdde8c00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:30:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.011+0000 7fdde8c00640 1 -- 172.21.15.49:0/130423379 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec004450 con 0x560a3edfdaf0 2024-02-26T06:30:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.011+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a3ed35ca0 con 0x560a3edfdaf0 2024-02-26T06:30:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.012+0000 7fdde8c00640 1 -- 172.21.15.49:0/130423379 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fddec004df0 con 0x560a3edfdaf0 2024-02-26T06:30:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.012+0000 7fdde8c00640 1 -- 172.21.15.49:0/130423379 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddec004fd0 con 0x560a3edfdaf0 2024-02-26T06:30:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.012+0000 7fde091ff740 10 client.16584 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.012+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560a3ed61c20 con 0x560a3edfdaf0 2024-02-26T06:30:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.013+0000 7fdde8c00640 1 -- 172.21.15.49:0/130423379 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560a3ed35ca0 con 0x560a3edfdaf0 2024-02-26T06:30:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.013+0000 7fdde8c00640 1 -- 172.21.15.49:0/130423379 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fddec03c050 con 0x560a3edfdaf0 2024-02-26T06:30:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.013+0000 7fde091ff740 10 client.16584 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.013+0000 7fde091ff740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.013+0000 7fde091ff740 20 client.16584 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.014+0000 7fde091ff740 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560a3ee160b0 0x560a3ee184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.014+0000 7fde091ff740 4 client.16584 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.014+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a3ed59cb0 con 0x560a3ee160b0 2024-02-26T06:30:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.014+0000 7fddff400640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560a3ee160b0 0x560a3ee184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.014+0000 7fddff400640 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560a3ee160b0 0x560a3ee184d0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc456ff7f484fad server_cookie=6ac06047cce220e9 in_seq=0 out_seq=0 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.014+0000 7fdde8c00640 10 client.16584 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fdde8c00640 1 -- 172.21.15.49:0/130423379 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a3ed59cb0 con 0x560a3ee160b0 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fdde8c00640 10 client.16584 handle_command_reply: tid=0 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 client.16584 shutdown 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 msgr2=0x560a3ee03860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03860 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fddec036040 tx=0x7fddec02cb30 comp rx=0 tx=0).stop 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 shutdown_connections 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560a3ee160b0 0x560a3ee184d0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee03da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03860 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 --2- 172.21.15.49:0/130423379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee03320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 >> 172.21.15.49:0/130423379 conn(0x560a3edc4480 msgr2=0x560a3edf97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.015+0000 7fdde8c00640 0 client.16584 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.016+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 shutdown_connections 2024-02-26T06:30:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.016+0000 7fde091ff740 1 -- 172.21.15.49:0/130423379 wait complete. 2024-02-26T06:30:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.016+0000 7fde091ff740 20 client.16584 trim_cache size 0 max 16384 2024-02-26T06:30:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.041+0000 7fde091ff740 1 Processor -- start 2024-02-26T06:30:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fde091ff740 1 -- start start 2024-02-26T06:30:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fde091ff740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee05a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fde091ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fde091ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee01bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fde091ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3eaabe60 con 0x560a3edfdaf0 2024-02-26T06:30:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fde091ff740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3edfc050 con 0x560a3edfcd80 2024-02-26T06:30:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fde091ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ee1d680 con 0x560a3ee06300 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fddfea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fddff400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee05a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fddff400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee05a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38340/0 (socket says 172.21.15.49:38340) 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fddff400640 1 -- 172.21.15.49:0/1096530996 learned_addr learned my addr 172.21.15.49:0/1096530996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.042+0000 7fddffe00640 1 --2- 172.21.15.49:0/1096530996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee01bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddff400640 1 -- 172.21.15.49:0/1096530996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 msgr2=0x560a3ee01bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddff400640 1 --2- 172.21.15.49:0/1096530996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee01bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddff400640 1 -- 172.21.15.49:0/1096530996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 msgr2=0x560a3ee03bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddff400640 1 --2- 172.21.15.49:0/1096530996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddff400640 1 -- 172.21.15.49:0/1096530996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a3ecfe500 con 0x560a3edfcd80 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddff400640 1 --2- 172.21.15.49:0/1096530996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee05a50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fddf009aba0 tx=0x7fddf00d1aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b54b13d7d2de0b97 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddeb400640 1 -- 172.21.15.49:0/1096530996 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf009a800 con 0x560a3edfcd80 2024-02-26T06:30:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddeb400640 1 -- 172.21.15.49:0/1096530996 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddf009aea0 con 0x560a3edfcd80 2024-02-26T06:30:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.043+0000 7fddeb400640 1 -- 172.21.15.49:0/1096530996 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf009a2f0 con 0x560a3edfcd80 2024-02-26T06:30:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.044+0000 7fde091ff740 1 -- 172.21.15.49:0/1096530996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 msgr2=0x560a3ee05a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.044+0000 7fde091ff740 1 --2- 172.21.15.49:0/1096530996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee05a50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fddf009aba0 tx=0x7fddf00d1aa0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.045+0000 7fde091ff740 1 -- 172.21.15.49:0/1096530996 shutdown_connections 2024-02-26T06:30:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.045+0000 7fde091ff740 1 --2- 172.21.15.49:0/1096530996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3ee06300 0x560a3ee01bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.045+0000 7fde091ff740 1 --2- 172.21.15.49:0/1096530996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3edfdaf0 0x560a3ee03bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.045+0000 7fde091ff740 1 --2- 172.21.15.49:0/1096530996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee05a50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.045+0000 7fde091ff740 1 -- 172.21.15.49:0/1096530996 >> 172.21.15.49:0/1096530996 conn(0x560a3edc4480 msgr2=0x560a3edfe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.046+0000 7fde091ff740 1 -- 172.21.15.49:0/1096530996 shutdown_connections 2024-02-26T06:30:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.046+0000 7fde091ff740 1 -- 172.21.15.49:0/1096530996 wait complete. 2024-02-26T06:30:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.047+0000 7fde091ff740 1 -- start start 2024-02-26T06:30:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.047+0000 7fde091ff740 1 Processor -- start 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.048+0000 7fde091ff740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee0d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.048+0000 7fde091ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfdaf0 0x560a3ee0b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.048+0000 7fde091ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3ee06300 0x560a3ee0bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.048+0000 7fde091ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ee0c1c0 con 0x560a3ee06300 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.048+0000 7fde091ff740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ee0c330 con 0x560a3edfcd80 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.048+0000 7fde091ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3ee0c4f0 con 0x560a3edfdaf0 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.048+0000 7fde091ff740 10 client.0 _ll_register_callbacks cb 0x7fddf8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.048+0000 7fddff400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee0d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddff400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee0d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38346/0 (socket says 172.21.15.49:38346) 2024-02-26T06:30:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddff400640 1 -- 172.21.15.49:0/2090290736 learned_addr learned my addr 172.21.15.49:0/2090290736 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddfea00640 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfdaf0 0x560a3ee0b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddff400640 1 -- 172.21.15.49:0/2090290736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfdaf0 msgr2=0x560a3ee0b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddff400640 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfdaf0 0x560a3ee0b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddff400640 1 -- 172.21.15.49:0/2090290736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3ee06300 msgr2=0x560a3ee0bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddff400640 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3ee06300 0x560a3ee0bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddff400640 1 -- 172.21.15.49:0/2090290736 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde001aa9b0 con 0x560a3edfcd80 2024-02-26T06:30:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddfea00640 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfdaf0 0x560a3ee0b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fddff400640 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee0d350 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fddf00a3060 tx=0x7fddf0005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406afd9a5eb46530 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.049+0000 7fdde8c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fdde8c00640 1 -- 172.21.15.49:0/2090290736 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf009ef40 con 0x560a3edfcd80 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fdde8c00640 1 -- 172.21.15.49:0/2090290736 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fddf00aee40 con 0x560a3edfcd80 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 1 -- 172.21.15.49:0/2090290736 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a3ed88d00 con 0x560a3edfcd80 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fdde8c00640 1 -- 172.21.15.49:0/2090290736 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddf0010b10 con 0x560a3edfcd80 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 10 client.27146 fetch_fsmap learned FSMap version 38 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 1 -- 172.21.15.49:0/2090290736 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560a3ed24290 con 0x560a3edfcd80 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fdde8c00640 1 -- 172.21.15.49:0/2090290736 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560a3ed88d00 con 0x560a3edfcd80 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fdde8c00640 1 -- 172.21.15.49:0/2090290736 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fddf009a3f0 con 0x560a3edfcd80 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 10 client.27146 fetch_fsmap finished waiting for FSMap version 38 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 10 client.27146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 20 client.27146 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560a3ee6d970 0x560a3ee6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 4 client.27146 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:30:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.050+0000 7fde091ff740 1 -- 172.21.15.49:0/2090290736 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a3ea63600 con 0x560a3ee6d970 2024-02-26T06:30:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.051+0000 7fddfea00640 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560a3ee6d970 0x560a3ee6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.051+0000 7fddfea00640 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560a3ee6d970 0x560a3ee6fd90 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860a047c5f3314e0 server_cookie=2352ce22156ba35c in_seq=0 out_seq=0 2024-02-26T06:30:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.051+0000 7fdde8c00640 10 client.27146 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.073+0000 7fdde8c00640 1 -- 172.21.15.49:0/2090290736 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a3ea63600 con 0x560a3ee6d970 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.073+0000 7fdde8c00640 10 client.27146 handle_command_reply: tid=0 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.073+0000 7fde091ff740 1 client.27146 shutdown 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 -- 172.21.15.49:0/2090290736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 msgr2=0x560a3ee0d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee0d350 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fddf00a3060 tx=0x7fddf0005c60 comp rx=0 tx=0).stop 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 -- 172.21.15.49:0/2090290736 shutdown_connections 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560a3ee6d970 0x560a3ee6fd90 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560a3ee06300 0x560a3ee0bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3edfdaf0 0x560a3ee0b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 --2- 172.21.15.49:0/2090290736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a3edfcd80 0x560a3ee0d350 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 -- 172.21.15.49:0/2090290736 >> 172.21.15.49:0/2090290736 conn(0x560a3edc4480 msgr2=0x560a3ee01b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fdde8c00640 0 client.27146 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 -- 172.21.15.49:0/2090290736 shutdown_connections 2024-02-26T06:30:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 1 -- 172.21.15.49:0/2090290736 wait complete. 2024-02-26T06:30:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.074+0000 7fde091ff740 20 client.27146 trim_cache size 0 max 16384 2024-02-26T06:30:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:30:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:30:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:30:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:30:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 -- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdde40a2350 msgr2=0x7fdde40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdde40a2350 0x7fdde40a4810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fddf8005f40 tx=0x7fddf8009600 comp rx=0 tx=0).stop 2024-02-26T06:30:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 -- 172.21.15.49:0/1351116313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde001001e0 msgr2=0x7fde001a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde001001e0 0x7fde001a3910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e9f0 tx=0x7fddf000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 -- 172.21.15.49:0/1351116313 shutdown_connections 2024-02-26T06:30:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fdde40a2350 0x7fdde40a4810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde00105500 0x7fde0019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100b00 0x7fde001a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 --2- 172.21.15.49:0/1351116313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde001001e0 0x7fde001a3910 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.075+0000 7fde06200640 1 -- 172.21.15.49:0/1351116313 >> 172.21.15.49:0/1351116313 conn(0x7fde000771a0 msgr2=0x7fde00102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.076+0000 7fde06200640 1 -- 172.21.15.49:0/1351116313 shutdown_connections 2024-02-26T06:30:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:32.076+0000 7fde06200640 1 -- 172.21.15.49:0/1351116313 wait complete. 2024-02-26T06:30:32.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:32 smithi115 ceph-mon[36426]: pgmap v219: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.7 MiB/s wr, 759 op/s 2024-02-26T06:30:32.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:32 smithi049 ceph-mon[31696]: pgmap v219: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.7 MiB/s wr, 759 op/s 2024-02-26T06:30:33.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:32 smithi080 ceph-mon[36206]: pgmap v219: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.7 MiB/s wr, 759 op/s 2024-02-26T06:30:34.884 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:34 smithi080 ceph-mon[36206]: pgmap v220: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 635 op/s 2024-02-26T06:30:34.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:34 smithi049 ceph-mon[31696]: pgmap v220: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 635 op/s 2024-02-26T06:30:35.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:34 smithi115 ceph-mon[36426]: pgmap v220: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 635 op/s 2024-02-26T06:30:35.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:35 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:30:36.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:35 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:30:36.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:35 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:30:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 405.0 s, 2156.0 tps, lat 46.665 ms stddev 56.788 2024-02-26T06:30:36.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:36 smithi049 ceph-mon[31696]: pgmap v221: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.0 MiB/s wr, 941 op/s 2024-02-26T06:30:37.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:36 smithi080 ceph-mon[36206]: pgmap v221: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.0 MiB/s wr, 941 op/s 2024-02-26T06:30:37.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:36 smithi115 ceph-mon[36426]: pgmap v221: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.0 MiB/s wr, 941 op/s 2024-02-26T06:30:38.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:38 smithi049 ceph-mon[31696]: pgmap v222: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-02-26T06:30:39.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:38 smithi080 ceph-mon[36206]: pgmap v222: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-02-26T06:30:39.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:38 smithi115 ceph-mon[36426]: pgmap v222: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-02-26T06:30:40.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:40 smithi049 ceph-mon[31696]: pgmap v223: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-02-26T06:30:41.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:40 smithi080 ceph-mon[36206]: pgmap v223: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-02-26T06:30:41.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:40 smithi115 ceph-mon[36426]: pgmap v223: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-02-26T06:30:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 410.0 s, 2419.2 tps, lat 40.850 ms stddev 49.827 2024-02-26T06:30:42.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:42 smithi049 ceph-mon[31696]: pgmap v224: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 801 op/s 2024-02-26T06:30:43.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:42 smithi080 ceph-mon[36206]: pgmap v224: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 801 op/s 2024-02-26T06:30:43.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:42 smithi115 ceph-mon[36426]: pgmap v224: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 801 op/s 2024-02-26T06:30:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:44 smithi049 ceph-mon[31696]: pgmap v225: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-02-26T06:30:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:44 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:30:45.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:44 smithi080 ceph-mon[36206]: pgmap v225: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-02-26T06:30:45.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:44 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:30:45.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:44 smithi115 ceph-mon[36426]: pgmap v225: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-02-26T06:30:45.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:44 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:30:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.283+0000 7f6d00e00640 1 -- 172.21.15.49:0/3724857588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc103f50 msgr2=0x7f6cfc107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.283+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3724857588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc103f50 0x7f6cfc107f90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009930 tx=0x7f6ce402bc20 comp rx=0 tx=0).stop 2024-02-26T06:30:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.284+0000 7f6d00e00640 1 -- 172.21.15.49:0/3724857588 shutdown_connections 2024-02-26T06:30:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.284+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3724857588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc103f50 0x7f6cfc107f90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.284+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3724857588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc108670 0x7f6cfc110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.284+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3724857588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc1035a0 0x7f6cfc103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.284+0000 7f6d00e00640 1 -- 172.21.15.49:0/3724857588 >> 172.21.15.49:0/3724857588 conn(0x7f6cfc0fee40 msgr2=0x7f6cfc101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.284+0000 7f6d00e00640 1 -- 172.21.15.49:0/3724857588 shutdown_connections 2024-02-26T06:30:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.284+0000 7f6d00e00640 1 -- 172.21.15.49:0/3724857588 wait complete. 2024-02-26T06:30:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.286+0000 7f6d00e00640 1 Processor -- start 2024-02-26T06:30:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.286+0000 7f6d00e00640 1 -- start start 2024-02-26T06:30:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.287+0000 7f6d00e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc1035a0 0x7f6cfc075690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.287+0000 7f6d00e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc108670 0x7f6cfc075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.287+0000 7f6d00e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc076110 0x7f6cfc1a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.287+0000 7f6d00e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc079870 con 0x7f6cfc108670 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.287+0000 7f6d00e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc1aa490 con 0x7f6cfc1035a0 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.287+0000 7f6d00e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc1aa650 con 0x7f6cfc076110 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.288+0000 7f6cf9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc1035a0 0x7f6cfc075690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.288+0000 7f6cf9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc1035a0 0x7f6cfc075690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:59830/0 (socket says 172.21.15.49:59830) 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.288+0000 7f6cf9e00640 1 -- 172.21.15.49:0/3387171197 learned_addr learned my addr 172.21.15.49:0/3387171197 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.288+0000 7f6cfa800640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc076110 0x7f6cfc1a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.289+0000 7f6cf9400640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc108670 0x7f6cfc075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.289+0000 7f6cfa800640 1 -- 172.21.15.49:0/3387171197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc1035a0 msgr2=0x7f6cfc075690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.289+0000 7f6cfa800640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc1035a0 0x7f6cfc075690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.289+0000 7f6cfa800640 1 -- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc108670 msgr2=0x7f6cfc075bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.289+0000 7f6cfa800640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc108670 0x7f6cfc075bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.289+0000 7f6cfa800640 1 -- 172.21.15.49:0/3387171197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf4009840 con 0x7f6cfc076110 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.289+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc1035a0 0x7f6cfc075690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.289+0000 7f6cf9400640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc108670 0x7f6cfc075bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.290+0000 7f6cfa800640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc076110 0x7f6cfc1a9f50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6cec01e6f0 tx=0x7f6cec01ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.290+0000 7f6cf2a00640 1 -- 172.21.15.49:0/3387171197 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec004220 con 0x7f6cfc076110 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.290+0000 7f6cf2a00640 1 -- 172.21.15.49:0/3387171197 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cec004380 con 0x7f6cfc076110 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.290+0000 7f6d00e00640 1 -- 172.21.15.49:0/3387171197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce4009560 con 0x7f6cfc076110 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.290+0000 7f6cf2a00640 1 -- 172.21.15.49:0/3387171197 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec003e70 con 0x7f6cfc076110 2024-02-26T06:30:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.291+0000 7f6d00e00640 1 -- 172.21.15.49:0/3387171197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc06af00 con 0x7f6cfc076110 2024-02-26T06:30:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.297+0000 7f6cf2a00640 1 -- 172.21.15.49:0/3387171197 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6cec009820 con 0x7f6cfc076110 2024-02-26T06:30:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.299+0000 7f6d03dc4740 1 Processor -- start 2024-02-26T06:30:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6d03dc4740 1 -- start start 2024-02-26T06:30:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c219af0 0x56169c218840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c219200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 0x56169c2246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c1d45f0 con 0x56169c219af0 2024-02-26T06:30:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c186890 con 0x56169c218d80 2024-02-26T06:30:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c183dc0 con 0x56169c222300 2024-02-26T06:30:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c219af0 0x56169c218840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.300+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c219af0 0x56169c218840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41182/0 (socket says 172.21.15.49:41182) 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf9400640 1 -- 172.21.15.49:0/3603393101 learned_addr learned my addr 172.21.15.49:0/3603393101 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c219200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf9400640 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 0x56169c2246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf2a00640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6ce00a2350 0x7f6ce00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf9400640 1 -- 172.21.15.49:0/3603393101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 msgr2=0x56169c2246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf9400640 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 0x56169c2246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf9400640 1 -- 172.21.15.49:0/3603393101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 msgr2=0x56169c219200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf2a00640 1 -- 172.21.15.49:0/3387171197 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6cec0cdde0 con 0x7f6cfc076110 2024-02-26T06:30:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.301+0000 7f6cf9400640 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c219200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.302+0000 7f6cf9400640 1 -- 172.21.15.49:0/3603393101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56169c100ad0 con 0x56169c219af0 2024-02-26T06:30:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.302+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c219200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.302+0000 7f6cfa800640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6ce00a2350 0x7f6ce00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.302+0000 7f6cf9400640 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c219af0 0x56169c218840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc06a900 tx=0x7f6ce4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d7757ce96a54de server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.303+0000 7f6ceb400640 1 -- 172.21.15.49:0/3603393101 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce402dea0 con 0x56169c219af0 2024-02-26T06:30:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.303+0000 7f6cfa800640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6ce00a2350 0x7f6ce00a4810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6cec09a740 tx=0x7f6cec009f20 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.303+0000 7f6ceb400640 1 -- 172.21.15.49:0/3603393101 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ce4004850 con 0x56169c219af0 2024-02-26T06:30:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.303+0000 7f6ceb400640 1 -- 172.21.15.49:0/3603393101 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce402dea0 con 0x56169c219af0 2024-02-26T06:30:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.304+0000 7f6d03dc4740 1 -- 172.21.15.49:0/3603393101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c219af0 msgr2=0x56169c218840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.304+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c219af0 0x56169c218840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc06a900 tx=0x7f6ce4005fd0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.305+0000 7f6d03dc4740 1 -- 172.21.15.49:0/3603393101 shutdown_connections 2024-02-26T06:30:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.305+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c219200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.305+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 0x56169c2246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.305+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/3603393101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c219af0 0x56169c218840 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.305+0000 7f6d03dc4740 1 -- 172.21.15.49:0/3603393101 >> 172.21.15.49:0/3603393101 conn(0x56169c1e0480 msgr2=0x56169c213a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.306+0000 7f6d03dc4740 1 -- 172.21.15.49:0/3603393101 shutdown_connections 2024-02-26T06:30:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.306+0000 7f6d03dc4740 1 -- 172.21.15.49:0/3603393101 wait complete. 2024-02-26T06:30:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.306+0000 7f6d03dc4740 1 -- start start 2024-02-26T06:30:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.307+0000 7f6d03dc4740 1 Processor -- start 2024-02-26T06:30:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.307+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c218d80 0x56169c21f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c218d80 0x56169c21f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c218d80 0x56169c21f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:39816/0 (socket says 172.21.15.49:39816) 2024-02-26T06:30:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6cf9400640 1 -- 172.21.15.49:0/2215392098 learned_addr learned my addr 172.21.15.49:0/2215392098 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c222300 0x56169c21fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c17f5a0 con 0x56169c222300 2024-02-26T06:30:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c124b20 con 0x56169c219af0 2024-02-26T06:30:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c1247c0 con 0x56169c218d80 2024-02-26T06:30:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.308+0000 7f6d03dc4740 10 client.0 _ll_register_callbacks cb 0x56169c138520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.309+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.310+0000 7f6cf9400640 1 -- 172.21.15.49:0/2215392098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 msgr2=0x56169c21f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.310+0000 7f6cf9400640 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.310+0000 7f6cf9400640 1 -- 172.21.15.49:0/2215392098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c222300 msgr2=0x56169c21fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.311+0000 7f6cf9400640 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c222300 0x56169c21fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.311+0000 7f6cf9400640 1 -- 172.21.15.49:0/2215392098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc110a50 con 0x56169c218d80 2024-02-26T06:30:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.311+0000 7f6cf9400640 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c218d80 0x56169c21f350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4004bf0 tx=0x7f6ce402cd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1162e0089f396f71 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.311+0000 7f6cdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:30:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.312+0000 7f6cdfe00640 1 -- 172.21.15.49:0/2215392098 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4005430 con 0x56169c218d80 2024-02-26T06:30:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.312+0000 7f6cdfe00640 1 -- 172.21.15.49:0/2215392098 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6ce4005590 con 0x56169c218d80 2024-02-26T06:30:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.312+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56169c151ca0 con 0x56169c218d80 2024-02-26T06:30:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.313+0000 7f6cdfe00640 1 -- 172.21.15.49:0/2215392098 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ce4004830 con 0x56169c218d80 2024-02-26T06:30:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.313+0000 7f6d03dc4740 10 client.27139 fetch_fsmap learned FSMap version 39 2024-02-26T06:30:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.313+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56169c17dc20 con 0x56169c218d80 2024-02-26T06:30:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.314+0000 7f6cdfe00640 1 -- 172.21.15.49:0/2215392098 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56169c151ca0 con 0x56169c218d80 2024-02-26T06:30:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.314+0000 7f6cdfe00640 1 -- 172.21.15.49:0/2215392098 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6ce4058070 con 0x56169c218d80 2024-02-26T06:30:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.315+0000 7f6d03dc4740 10 client.27139 fetch_fsmap finished waiting for FSMap version 39 2024-02-26T06:30:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.315+0000 7f6d03dc4740 10 client.27139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.315+0000 7f6d03dc4740 20 client.27139 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.315+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56169c231e80 0x56169c234270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.315+0000 7f6d03dc4740 4 client.27139 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.315+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56169c175cb0 con 0x56169c231e80 2024-02-26T06:30:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.315+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56169c231e80 0x56169c234270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.316+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56169c231e80 0x56169c234270 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42928008b5dea6d7 server_cookie=29660a5a2ecbb9c7 in_seq=0 out_seq=0 2024-02-26T06:30:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.316+0000 7f6cdfe00640 10 client.27139 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.317+0000 7f6cdfe00640 1 -- 172.21.15.49:0/2215392098 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56169c175cb0 con 0x56169c231e80 2024-02-26T06:30:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.317+0000 7f6cdfe00640 10 client.27139 handle_command_reply: tid=0 2024-02-26T06:30:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.318+0000 7f6d03dc4740 1 client.27139 shutdown 2024-02-26T06:30:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.319+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c218d80 msgr2=0x56169c21f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.319+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c218d80 0x56169c21f350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4004bf0 tx=0x7f6ce402cd40 comp rx=0 tx=0).stop 2024-02-26T06:30:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.319+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 shutdown_connections 2024-02-26T06:30:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.319+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56169c231e80 0x56169c234270 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.319+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.319+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c222300 0x56169c21fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.319+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/2215392098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c218d80 0x56169c21f350 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.319+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 >> 172.21.15.49:0/2215392098 conn(0x56169c1e0480 msgr2=0x56169c2157c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.320+0000 7f6cdfe00640 0 client.27139 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.320+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 shutdown_connections 2024-02-26T06:30:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.321+0000 7f6d03dc4740 1 -- 172.21.15.49:0/2215392098 wait complete. 2024-02-26T06:30:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.321+0000 7f6d03dc4740 20 client.27139 trim_cache size 0 max 16384 2024-02-26T06:30:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.352+0000 7f6d03dc4740 1 Processor -- start 2024-02-26T06:30:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.353+0000 7f6d03dc4740 1 -- start start 2024-02-26T06:30:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.353+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c218d80 0x56169c221a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.353+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.353+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 0x56169c21c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.353+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169bec7e60 con 0x56169c218d80 2024-02-26T06:30:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.353+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c22e780 con 0x56169c219af0 2024-02-26T06:30:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.353+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c21af90 con 0x56169c222300 2024-02-26T06:30:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.354+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c218d80 0x56169c221a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.354+0000 7f6cf9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.354+0000 7f6cf9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:59856/0 (socket says 172.21.15.49:59856) 2024-02-26T06:30:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.354+0000 7f6cf9e00640 1 -- 172.21.15.49:0/1424943016 learned_addr learned my addr 172.21.15.49:0/1424943016 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.355+0000 7f6cf9e00640 1 -- 172.21.15.49:0/1424943016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 msgr2=0x56169c21c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.355+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/1424943016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 0x56169c21c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.355+0000 7f6cf9e00640 1 -- 172.21.15.49:0/1424943016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c218d80 msgr2=0x56169c221a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.355+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/1424943016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c218d80 0x56169c221a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.355+0000 7f6cf9e00640 1 -- 172.21.15.49:0/1424943016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56169c11a500 con 0x56169c219af0 2024-02-26T06:30:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.355+0000 7f6cf9400640 1 --2- 172.21.15.49:0/1424943016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c218d80 0x56169c221a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.363+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/1424943016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21fbf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x56169c21a250 tx=0x7f6cf4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17176365a4dd90d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.364+0000 7f6ceaa00640 1 -- 172.21.15.49:0/1424943016 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4016040 con 0x56169c219af0 2024-02-26T06:30:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.364+0000 7f6ceaa00640 1 -- 172.21.15.49:0/1424943016 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cf4017040 con 0x56169c219af0 2024-02-26T06:30:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.364+0000 7f6ceaa00640 1 -- 172.21.15.49:0/1424943016 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4004440 con 0x56169c219af0 2024-02-26T06:30:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.366+0000 7f6d03dc4740 1 -- 172.21.15.49:0/1424943016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 msgr2=0x56169c21fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.366+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/1424943016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21fbf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x56169c21a250 tx=0x7f6cf4009500 comp rx=0 tx=0).stop 2024-02-26T06:30:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.367+0000 7f6d03dc4740 1 -- 172.21.15.49:0/1424943016 shutdown_connections 2024-02-26T06:30:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.367+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/1424943016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c219af0 0x56169c21fbf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.367+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/1424943016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c222300 0x56169c21c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.367+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/1424943016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c218d80 0x56169c221a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.367+0000 7f6d03dc4740 1 -- 172.21.15.49:0/1424943016 >> 172.21.15.49:0/1424943016 conn(0x56169c1e0480 msgr2=0x56169c21dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.367+0000 7f6d03dc4740 1 -- 172.21.15.49:0/1424943016 shutdown_connections 2024-02-26T06:30:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.367+0000 7f6d03dc4740 1 -- 172.21.15.49:0/1424943016 wait complete. 2024-02-26T06:30:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.368+0000 7f6d03dc4740 1 -- start start 2024-02-26T06:30:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.371+0000 7f6d03dc4740 1 Processor -- start 2024-02-26T06:30:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.372+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c227af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.372+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c219af0 0x56169c228030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.372+0000 7f6d03dc4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c222300 0x56169c228570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.372+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c288360 con 0x56169c222300 2024-02-26T06:30:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.372+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c2884a0 con 0x56169c218d80 2024-02-26T06:30:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.372+0000 7f6d03dc4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169c2885e0 con 0x56169c219af0 2024-02-26T06:30:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.372+0000 7f6d03dc4740 10 client.0 _ll_register_callbacks cb 0x7f6ce4048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c227af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c227af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:59864/0 (socket says 172.21.15.49:59864) 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9400640 1 -- 172.21.15.49:0/4061693029 learned_addr learned my addr 172.21.15.49:0/4061693029 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c219af0 0x56169c228030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9400640 1 -- 172.21.15.49:0/4061693029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c219af0 msgr2=0x56169c228030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9400640 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c219af0 0x56169c228030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9400640 1 -- 172.21.15.49:0/4061693029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c222300 msgr2=0x56169c228570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9400640 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c222300 0x56169c228570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.373+0000 7f6cf9400640 1 -- 172.21.15.49:0/4061693029 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56169c17dc20 con 0x56169c218d80 2024-02-26T06:30:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.374+0000 7f6cf9400640 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c227af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x56169c175df0 tx=0x7f6ce4040fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b48d382c791ab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.375+0000 7f6cdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:30:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.376+0000 7f6cdfe00640 1 -- 172.21.15.49:0/4061693029 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4043d60 con 0x56169c218d80 2024-02-26T06:30:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.376+0000 7f6cdfe00640 1 -- 172.21.15.49:0/4061693029 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6ce4005230 con 0x56169c218d80 2024-02-26T06:30:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.376+0000 7f6d03dc4740 1 -- 172.21.15.49:0/4061693029 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56169c1a4d00 con 0x56169c218d80 2024-02-26T06:30:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.377+0000 7f6cdfe00640 1 -- 172.21.15.49:0/4061693029 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ce40053f0 con 0x56169c218d80 2024-02-26T06:30:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.377+0000 7f6d03dc4740 10 client.27164 fetch_fsmap learned FSMap version 39 2024-02-26T06:30:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.377+0000 7f6d03dc4740 1 -- 172.21.15.49:0/4061693029 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f6cf4009c80 con 0x56169c218d80 2024-02-26T06:30:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.377+0000 7f6cdfe00640 1 -- 172.21.15.49:0/4061693029 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ce406b030 con 0x56169c218d80 2024-02-26T06:30:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.377+0000 7f6cdfe00640 1 -- 172.21.15.49:0/4061693029 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6ce4058070 con 0x56169c218d80 2024-02-26T06:30:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.379+0000 7f6d03dc4740 10 client.27164 fetch_fsmap finished waiting for FSMap version 39 2024-02-26T06:30:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.379+0000 7f6d03dc4740 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.379+0000 7f6d03dc4740 20 client.27164 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.379+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56169c289840 0x56169c28bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.379+0000 7f6d03dc4740 4 client.27164 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:30:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.379+0000 7f6d03dc4740 1 -- 172.21.15.49:0/4061693029 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56169be7f600 con 0x56169c289840 2024-02-26T06:30:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.379+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56169c289840 0x56169c28bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.380+0000 7f6cf9e00640 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56169c289840 0x56169c28bc30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4940fcdaee2e7a0b server_cookie=74b1991d1cb1d953 in_seq=0 out_seq=0 2024-02-26T06:30:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.380+0000 7f6cdfe00640 10 client.27164 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.441+0000 7f6cdfe00640 1 -- 172.21.15.49:0/4061693029 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56169be7f600 con 0x56169c289840 2024-02-26T06:30:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.441+0000 7f6cdfe00640 10 client.27164 handle_command_reply: tid=0 2024-02-26T06:30:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.441+0000 7f6d03dc4740 1 client.27164 shutdown 2024-02-26T06:30:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.441+0000 7f6d03dc4740 1 -- 172.21.15.49:0/4061693029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 msgr2=0x56169c227af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.441+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c227af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x56169c175df0 tx=0x7f6ce4040fc0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 1 -- 172.21.15.49:0/4061693029 shutdown_connections 2024-02-26T06:30:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56169c289840 0x56169c28bc30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56169c222300 0x56169c228570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56169c219af0 0x56169c228030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 1 --2- 172.21.15.49:0/4061693029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56169c218d80 0x56169c227af0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 1 -- 172.21.15.49:0/4061693029 >> 172.21.15.49:0/4061693029 conn(0x56169c1e0480 msgr2=0x56169c214dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6cdfe00640 0 client.27164 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 1 -- 172.21.15.49:0/4061693029 shutdown_connections 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 1 -- 172.21.15.49:0/4061693029 wait complete. 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.442+0000 7f6d03dc4740 20 client.27164 trim_cache size 0 max 16384 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:30:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:30:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.443+0000 7f6d00e00640 1 -- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6ce00a2350 msgr2=0x7f6ce00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.443+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6ce00a2350 0x7f6ce00a4810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6cec09a740 tx=0x7f6cec009f20 comp rx=0 tx=0).stop 2024-02-26T06:30:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.443+0000 7f6d00e00640 1 -- 172.21.15.49:0/3387171197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc076110 msgr2=0x7f6cfc1a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.443+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc076110 0x7f6cfc1a9f50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6cec01e6f0 tx=0x7f6cec01ebb0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.444+0000 7f6d00e00640 1 -- 172.21.15.49:0/3387171197 shutdown_connections 2024-02-26T06:30:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.444+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6ce00a2350 0x7f6ce00a4810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.444+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc076110 0x7f6cfc1a9f50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.444+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc108670 0x7f6cfc075bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.444+0000 7f6d00e00640 1 --2- 172.21.15.49:0/3387171197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cfc1035a0 0x7f6cfc075690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.444+0000 7f6d00e00640 1 -- 172.21.15.49:0/3387171197 >> 172.21.15.49:0/3387171197 conn(0x7f6cfc0fee40 msgr2=0x7f6cfc101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.445+0000 7f6d00e00640 1 -- 172.21.15.49:0/3387171197 shutdown_connections 2024-02-26T06:30:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:45.445+0000 7f6d00e00640 1 -- 172.21.15.49:0/3387171197 wait complete. 2024-02-26T06:30:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 415.0 s, 1626.6 tps, lat 61.799 ms stddev 70.041 2024-02-26T06:30:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:46 smithi049 ceph-mon[31696]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.3 MiB/s wr, 876 op/s 2024-02-26T06:30:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:46 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:30:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:46 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:30:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:46 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:30:47.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:46 smithi080 ceph-mon[36206]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.3 MiB/s wr, 876 op/s 2024-02-26T06:30:47.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:46 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:30:47.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:46 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:30:47.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:46 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:30:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:46 smithi115 ceph-mon[36426]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.3 MiB/s wr, 876 op/s 2024-02-26T06:30:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:46 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:30:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:46 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:30:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:46 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:30:48.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:48 smithi049 ceph-mon[31696]: pgmap v227: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-02-26T06:30:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:48 smithi080 ceph-mon[36206]: pgmap v227: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-02-26T06:30:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:48 smithi115 ceph-mon[36426]: pgmap v227: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-02-26T06:30:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:50 smithi080 ceph-mon[36206]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-02-26T06:30:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:50 smithi115 ceph-mon[36426]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-02-26T06:30:51.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:50 smithi049 ceph-mon[31696]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-02-26T06:30:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 420.0 s, 2367.8 tps, lat 42.435 ms stddev 52.213 2024-02-26T06:30:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:52 smithi080 ceph-mon[36206]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.9 MiB/s wr, 651 op/s 2024-02-26T06:30:53.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:52 smithi115 ceph-mon[36426]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.9 MiB/s wr, 651 op/s 2024-02-26T06:30:53.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:52 smithi049 ceph-mon[31696]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.9 MiB/s wr, 651 op/s 2024-02-26T06:30:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:54 smithi080 ceph-mon[36206]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2024-02-26T06:30:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:54 smithi115 ceph-mon[36426]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2024-02-26T06:30:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:54 smithi049 ceph-mon[31696]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2024-02-26T06:30:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 425.0 s, 1636.6 tps, lat 60.726 ms stddev 65.547 2024-02-26T06:30:57.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:57 smithi115 ceph-mon[36426]: pgmap v231: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.8 MiB/s wr, 829 op/s 2024-02-26T06:30:57.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:57 smithi049 ceph-mon[31696]: pgmap v231: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.8 MiB/s wr, 829 op/s 2024-02-26T06:30:57.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:57 smithi080 ceph-mon[36206]: pgmap v231: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.8 MiB/s wr, 829 op/s 2024-02-26T06:30:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.657+0000 7fcf89200640 1 -- 172.21.15.49:0/441286130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 msgr2=0x7fcf8410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.657+0000 7fcf89200640 1 --2- 172.21.15.49:0/441286130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 0x7fcf8410b1f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009900 tx=0x7fcf7002bd50 comp rx=0 tx=0).stop 2024-02-26T06:30:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.658+0000 7fcf89200640 1 -- 172.21.15.49:0/441286130 shutdown_connections 2024-02-26T06:30:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.658+0000 7fcf89200640 1 --2- 172.21.15.49:0/441286130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 0x7fcf8410b1f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.658+0000 7fcf89200640 1 --2- 172.21.15.49:0/441286130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf8410b730 0x7fcf8410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.658+0000 7fcf89200640 1 --2- 172.21.15.49:0/441286130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf841023f0 0x7fcf841027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.658+0000 7fcf89200640 1 -- 172.21.15.49:0/441286130 >> 172.21.15.49:0/441286130 conn(0x7fcf840771a0 msgr2=0x7fcf840775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.660+0000 7fcf89200640 1 -- 172.21.15.49:0/441286130 shutdown_connections 2024-02-26T06:30:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.661+0000 7fcf89200640 1 -- 172.21.15.49:0/441286130 wait complete. 2024-02-26T06:30:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.662+0000 7fcf89200640 1 Processor -- start 2024-02-26T06:30:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.664+0000 7fcf89200640 1 -- start start 2024-02-26T06:30:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.667+0000 7fcf89200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841023f0 0x7fcf8410aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.667+0000 7fcf89200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 0x7fcf84107f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.667+0000 7fcf81e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841023f0 0x7fcf8410aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.667+0000 7fcf89200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8410b730 0x7fcf84108480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.667+0000 7fcf89200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84108a50 con 0x7fcf8410b730 2024-02-26T06:30:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.667+0000 7fcf89200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84108bc0 con 0x7fcf84102d10 2024-02-26T06:30:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.667+0000 7fcf89200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84108d80 con 0x7fcf841023f0 2024-02-26T06:30:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 0x7fcf84107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 0x7fcf84107f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57942/0 (socket says 172.21.15.49:57942) 2024-02-26T06:30:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81400640 1 -- 172.21.15.49:0/3281285786 learned_addr learned my addr 172.21.15.49:0/3281285786 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81400640 1 -- 172.21.15.49:0/3281285786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841023f0 msgr2=0x7fcf8410aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81400640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841023f0 0x7fcf8410aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81400640 1 -- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8410b730 msgr2=0x7fcf84108480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81400640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8410b730 0x7fcf84108480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81400640 1 -- 172.21.15.49:0/3281285786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf70009560 con 0x7fcf84102d10 2024-02-26T06:30:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.668+0000 7fcf81e00640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841023f0 0x7fcf8410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:30:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.669+0000 7fcf81400640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 0x7fcf84107f40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcf7002c360 tx=0x7fcf7002db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.669+0000 7fcf7aa00640 1 -- 172.21.15.49:0/3281285786 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700044f0 con 0x7fcf84102d10 2024-02-26T06:30:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.669+0000 7fcf7aa00640 1 -- 172.21.15.49:0/3281285786 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf700054e0 con 0x7fcf84102d10 2024-02-26T06:30:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.669+0000 7fcf7aa00640 1 -- 172.21.15.49:0/3281285786 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7003e450 con 0x7fcf84102d10 2024-02-26T06:30:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.670+0000 7fcf89200640 1 -- 172.21.15.49:0/3281285786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8419fc20 con 0x7fcf84102d10 2024-02-26T06:30:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.670+0000 7fcf89200640 1 -- 172.21.15.49:0/3281285786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf841a00e0 con 0x7fcf84102d10 2024-02-26T06:30:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.677+0000 7fcf8bfed740 1 Processor -- start 2024-02-26T06:30:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.677+0000 7fcf7aa00640 1 -- 172.21.15.49:0/3281285786 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcf70003710 con 0x7fcf84102d10 2024-02-26T06:30:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.677+0000 7fcf8bfed740 1 -- start start 2024-02-26T06:30:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.678+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddfea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.678+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddfeb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.678+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 0x55fbddff66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.678+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddfa65f0 con 0x55fbddff4300 2024-02-26T06:30:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.678+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddf58890 con 0x55fbddfead80 2024-02-26T06:30:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.678+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddf55dc0 con 0x55fbddfebaf0 2024-02-26T06:30:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddfeb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 0x55fbddff66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddfeb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57946/0 (socket says 172.21.15.49:57946) 2024-02-26T06:30:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81e00640 1 -- 172.21.15.49:0/3973146555 learned_addr learned my addr 172.21.15.49:0/3973146555 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81e00640 1 -- 172.21.15.49:0/3973146555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 msgr2=0x55fbddfea840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:30:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81e00640 1 --2- 172.21.15.49:0/3973146555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddfea840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81e00640 1 -- 172.21.15.49:0/3973146555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 msgr2=0x55fbddff66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81e00640 1 --2- 172.21.15.49:0/3973146555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 0x55fbddff66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.679+0000 7fcf81e00640 1 -- 172.21.15.49:0/3973146555 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbdded2ad0 con 0x55fbddfead80 2024-02-26T06:30:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.680+0000 7fcf81e00640 1 --2- 172.21.15.49:0/3973146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddfeb200 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009c50 tx=0x7fcf7400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ac5cc2ed4e667a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.680+0000 7fcf7aa00640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fcf680a2500 0x7fcf680a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.680+0000 7fcf6f400640 1 -- 172.21.15.49:0/3973146555 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74005020 con 0x55fbddfead80 2024-02-26T06:30:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.680+0000 7fcf7aa00640 1 -- 172.21.15.49:0/3281285786 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fcf700f0730 con 0x7fcf84102d10 2024-02-26T06:30:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.680+0000 7fcf6f400640 1 -- 172.21.15.49:0/3973146555 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf74005660 con 0x55fbddfead80 2024-02-26T06:30:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.680+0000 7fcf6f400640 1 -- 172.21.15.49:0/3973146555 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74005020 con 0x55fbddfead80 2024-02-26T06:30:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.681+0000 7fcf82800640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fcf680a2500 0x7fcf680a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 -- 172.21.15.49:0/3973146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 msgr2=0x55fbddfeb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/3973146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddfeb200 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009c50 tx=0x7fcf7400e6c0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 -- 172.21.15.49:0/3973146555 shutdown_connections 2024-02-26T06:30:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/3973146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddfeb200 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/3973146555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 0x55fbddff66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/3973146555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddfea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 -- 172.21.15.49:0/3973146555 >> 172.21.15.49:0/3973146555 conn(0x55fbddfb2480 msgr2=0x55fbddfe5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 -- 172.21.15.49:0/3973146555 shutdown_connections 2024-02-26T06:30:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.682+0000 7fcf8bfed740 1 -- 172.21.15.49:0/3973146555 wait complete. 2024-02-26T06:30:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.683+0000 7fcf82800640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fcf680a2500 0x7fcf680a49c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0092f0 tx=0x7fcf7c009220 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.683+0000 7fcf8bfed740 1 -- start start 2024-02-26T06:30:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.683+0000 7fcf8bfed740 1 Processor -- start 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddff13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddf515a0 con 0x55fbddfead80 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddef6b20 con 0x55fbddff4300 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddef67c0 con 0x55fbddfebaf0 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf8bfed740 10 client.0 _ll_register_callbacks cb 0x55fbddf0a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf81400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf81e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddff13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf81400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58682/0 (socket says 172.21.15.49:58682) 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf81400640 1 -- 172.21.15.49:0/1614043218 learned_addr learned my addr 172.21.15.49:0/1614043218 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.684+0000 7fcf82800640 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf81400640 1 -- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 msgr2=0x55fbddff1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf81400640 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf81400640 1 -- 172.21.15.49:0/1614043218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 msgr2=0x55fbddff13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf81400640 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddff13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf81400640 1 -- 172.21.15.49:0/1614043218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf74009880 con 0x55fbddfebaf0 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf82800640 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf81400640 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff18e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcf700bd140 tx=0x7fcf7002bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32472fbcf62a3bb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf63e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf63e00640 1 -- 172.21.15.49:0/1614043218 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700c56c0 con 0x55fbddfebaf0 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1614043218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbddf23ca0 con 0x55fbddfebaf0 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf63e00640 1 -- 172.21.15.49:0/1614043218 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fcf7003de90 con 0x55fbddfebaf0 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf63e00640 1 -- 172.21.15.49:0/1614043218 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf700bd8b0 con 0x55fbddfebaf0 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf8bfed740 10 client.27148 fetch_fsmap learned FSMap version 39 2024-02-26T06:30:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.685+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1614043218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fbddf4fc20 con 0x55fbddfebaf0 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf63e00640 1 -- 172.21.15.49:0/1614043218 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fbddf23ca0 con 0x55fbddfebaf0 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf63e00640 1 -- 172.21.15.49:0/1614043218 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcf700b7d20 con 0x55fbddfebaf0 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf8bfed740 10 client.27148 fetch_fsmap finished waiting for FSMap version 39 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf8bfed740 10 client.27148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf8bfed740 20 client.27148 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55fbde004100 0x55fbde006520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf8bfed740 4 client.27148 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1614043218 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbddf47cb0 con 0x55fbde004100 2024-02-26T06:30:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.686+0000 7fcf81e00640 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55fbde004100 0x55fbde006520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.687+0000 7fcf81e00640 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55fbde004100 0x55fbde006520 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92bb3939b8699ad2 server_cookie=9b1d1c043bc2d174 in_seq=0 out_seq=0 2024-02-26T06:30:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.687+0000 7fcf63e00640 10 client.27148 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.688+0000 7fcf63e00640 1 -- 172.21.15.49:0/1614043218 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fbddf47cb0 con 0x55fbde004100 2024-02-26T06:30:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.688+0000 7fcf63e00640 10 client.27148 handle_command_reply: tid=0 2024-02-26T06:30:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.688+0000 7fcf8bfed740 1 client.27148 shutdown 2024-02-26T06:30:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.689+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1614043218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 msgr2=0x55fbddff18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.689+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff18e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcf700bd140 tx=0x7fcf7002bd50 comp rx=0 tx=0).stop 2024-02-26T06:30:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.690+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1614043218 shutdown_connections 2024-02-26T06:30:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.690+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55fbde004100 0x55fbde006520 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.690+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.690+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff18e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.690+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1614043218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddff13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.690+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1614043218 >> 172.21.15.49:0/1614043218 conn(0x55fbddfb2480 msgr2=0x55fbddfe77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.690+0000 7fcf63e00640 0 client.27148 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.691+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1614043218 shutdown_connections 2024-02-26T06:30:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.691+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1614043218 wait complete. 2024-02-26T06:30:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.691+0000 7fcf8bfed740 20 client.27148 trim_cache size 0 max 16384 2024-02-26T06:30:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.716+0000 7fcf8bfed740 1 Processor -- start 2024-02-26T06:30:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.716+0000 7fcf8bfed740 1 -- start start 2024-02-26T06:30:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.717+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddff3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 0x55fbddffb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddc99e60 con 0x55fbddff4300 2024-02-26T06:30:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddfd7670 con 0x55fbddfead80 2024-02-26T06:30:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbde003f00 con 0x55fbddfebaf0 2024-02-26T06:30:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf81e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddff3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf81400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf81e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddff3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57968/0 (socket says 172.21.15.49:57968) 2024-02-26T06:30:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf81e00640 1 -- 172.21.15.49:0/467174408 learned_addr learned my addr 172.21.15.49:0/467174408 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.718+0000 7fcf82800640 1 --2- 172.21.15.49:0/467174408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 0x55fbddffb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.719+0000 7fcf81e00640 1 -- 172.21.15.49:0/467174408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 msgr2=0x55fbddff1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.719+0000 7fcf81e00640 1 --2- 172.21.15.49:0/467174408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.719+0000 7fcf81e00640 1 -- 172.21.15.49:0/467174408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 msgr2=0x55fbddffb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.719+0000 7fcf81e00640 1 --2- 172.21.15.49:0/467174408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 0x55fbddffb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.719+0000 7fcf81e00640 1 -- 172.21.15.49:0/467174408 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbddeec500 con 0x55fbddfead80 2024-02-26T06:30:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.719+0000 7fcf81e00640 1 --2- 172.21.15.49:0/467174408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddff3ad0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009c50 tx=0x7fcf74004cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c294e71e715280c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.720+0000 7fcf6ea00640 1 -- 172.21.15.49:0/467174408 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74021050 con 0x55fbddfead80 2024-02-26T06:30:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.720+0000 7fcf6ea00640 1 -- 172.21.15.49:0/467174408 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf74022040 con 0x55fbddfead80 2024-02-26T06:30:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.720+0000 7fcf6ea00640 1 -- 172.21.15.49:0/467174408 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74005040 con 0x55fbddfead80 2024-02-26T06:30:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.722+0000 7fcf8bfed740 1 -- 172.21.15.49:0/467174408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 msgr2=0x55fbddff3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.722+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/467174408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddff3ad0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009c50 tx=0x7fcf74004cd0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.722+0000 7fcf8bfed740 1 -- 172.21.15.49:0/467174408 shutdown_connections 2024-02-26T06:30:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.722+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/467174408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddff4300 0x55fbddffb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.723+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/467174408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.723+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/467174408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddfead80 0x55fbddff3ad0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.723+0000 7fcf8bfed740 1 -- 172.21.15.49:0/467174408 >> 172.21.15.49:0/467174408 conn(0x55fbddfb2480 msgr2=0x55fbddfefc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.723+0000 7fcf8bfed740 1 -- 172.21.15.49:0/467174408 shutdown_connections 2024-02-26T06:30:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.723+0000 7fcf8bfed740 1 -- 172.21.15.49:0/467174408 wait complete. 2024-02-26T06:30:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.723+0000 7fcf8bfed740 1 -- start start 2024-02-26T06:30:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.724+0000 7fcf8bfed740 1 Processor -- start 2024-02-26T06:30:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddffb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf8bfed740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddff82b0 con 0x55fbddfead80 2024-02-26T06:30:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddff8420 con 0x55fbddff4300 2024-02-26T06:30:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf8bfed740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbddff85e0 con 0x55fbddfebaf0 2024-02-26T06:30:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf8bfed740 10 client.0 _ll_register_callbacks cb 0x7fcf700b9450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf81400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.725+0000 7fcf82800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf81400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58706/0 (socket says 172.21.15.49:58706) 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf81400640 1 -- 172.21.15.49:0/1121116798 learned_addr learned my addr 172.21.15.49:0/1121116798 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf81e00640 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddffb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf81e00640 1 -- 172.21.15.49:0/1121116798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 msgr2=0x55fbddff7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf81e00640 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf81e00640 1 -- 172.21.15.49:0/1121116798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 msgr2=0x55fbddff7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf81e00640 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf81e00640 1 -- 172.21.15.49:0/1121116798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbdded2ad0 con 0x55fbddfead80 2024-02-26T06:30:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.726+0000 7fcf82800640 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:30:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.727+0000 7fcf81e00640 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddffb430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400f010 tx=0x7fcf74009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa955f70df083cd6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.727+0000 7fcf63e00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:30:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.727+0000 7fcf63e00640 1 -- 172.21.15.49:0/1121116798 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74004890 con 0x55fbddfead80 2024-02-26T06:30:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.727+0000 7fcf63e00640 1 -- 172.21.15.49:0/1121116798 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fcf74004020 con 0x55fbddfead80 2024-02-26T06:30:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.727+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1121116798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbddf76d00 con 0x55fbddfead80 2024-02-26T06:30:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf63e00640 1 -- 172.21.15.49:0/1121116798 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf74021030 con 0x55fbddfead80 2024-02-26T06:30:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf8bfed740 10 client.16614 fetch_fsmap learned FSMap version 39 2024-02-26T06:30:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1121116798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fbddf12290 con 0x55fbddfead80 2024-02-26T06:30:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf63e00640 1 -- 172.21.15.49:0/1121116798 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fbddf76d00 con 0x55fbddfead80 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf63e00640 1 -- 172.21.15.49:0/1121116798 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcf74031070 con 0x55fbddfead80 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf8bfed740 10 client.16614 fetch_fsmap finished waiting for FSMap version 39 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf8bfed740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf8bfed740 20 client.16614 populate_metadata read hostname 'smithi049' 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55fbde05b790 0x55fbde05db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf8bfed740 4 client.16614 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.728+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1121116798 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fbddc51600 con 0x55fbde05b790 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.729+0000 7fcf81400640 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55fbde05b790 0x55fbde05db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.730+0000 7fcf81400640 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55fbde05b790 0x55fbde05db80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffff21198db3f9f server_cookie=896e8cdecc0cec54 in_seq=0 out_seq=0 2024-02-26T06:30:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.730+0000 7fcf63e00640 10 client.16614 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.755+0000 7fcf63e00640 1 -- 172.21.15.49:0/1121116798 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fbddc51600 con 0x55fbde05b790 2024-02-26T06:30:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.755+0000 7fcf63e00640 10 client.16614 handle_command_reply: tid=0 2024-02-26T06:30:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.755+0000 7fcf8bfed740 1 client.16614 shutdown 2024-02-26T06:30:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.756+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1121116798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 msgr2=0x55fbddffb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.756+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddffb430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400f010 tx=0x7fcf74009dc0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1121116798 shutdown_connections 2024-02-26T06:30:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55fbde05b790 0x55fbde05db80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fbddff4300 0x55fbddff7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fbddfebaf0 0x55fbddff7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf8bfed740 1 --2- 172.21.15.49:0/1121116798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fbddfead80 0x55fbddffb430 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1121116798 >> 172.21.15.49:0/1121116798 conn(0x55fbddfb2480 msgr2=0x55fbddfe4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf63e00640 0 client.16614 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:30:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1121116798 shutdown_connections 2024-02-26T06:30:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.757+0000 7fcf8bfed740 1 -- 172.21.15.49:0/1121116798 wait complete. 2024-02-26T06:30:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.758+0000 7fcf8bfed740 20 client.16614 trim_cache size 0 max 16384 2024-02-26T06:30:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:30:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:30:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:30:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.758+0000 7fcf89200640 1 -- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fcf680a2500 msgr2=0x7fcf680a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.758+0000 7fcf89200640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fcf680a2500 0x7fcf680a49c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0092f0 tx=0x7fcf7c009220 comp rx=0 tx=0).stop 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.758+0000 7fcf89200640 1 -- 172.21.15.49:0/3281285786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 msgr2=0x7fcf84107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.758+0000 7fcf89200640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 0x7fcf84107f40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcf7002c360 tx=0x7fcf7002db40 comp rx=0 tx=0).stop 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.759+0000 7fcf89200640 1 -- 172.21.15.49:0/3281285786 shutdown_connections 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.759+0000 7fcf89200640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fcf680a2500 0x7fcf680a49c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.759+0000 7fcf89200640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf84102d10 0x7fcf84107f40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.759+0000 7fcf89200640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8410b730 0x7fcf84108480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.759+0000 7fcf89200640 1 --2- 172.21.15.49:0/3281285786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841023f0 0x7fcf8410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.759+0000 7fcf89200640 1 -- 172.21.15.49:0/3281285786 >> 172.21.15.49:0/3281285786 conn(0x7fcf840771a0 msgr2=0x7fcf840fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.759+0000 7fcf89200640 1 -- 172.21.15.49:0/3281285786 shutdown_connections 2024-02-26T06:30:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:30:58.759+0000 7fcf89200640 1 -- 172.21.15.49:0/3281285786 wait complete. 2024-02-26T06:30:59.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:30:59 smithi049 ceph-mon[31696]: pgmap v232: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-02-26T06:30:59.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:30:59 smithi080 ceph-mon[36206]: pgmap v232: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-02-26T06:30:59.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:30:59 smithi115 ceph-mon[36426]: pgmap v232: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-02-26T06:31:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 430.0 s, 1647.2 tps, lat 57.105 ms stddev 93.658 2024-02-26T06:31:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:01 smithi049 ceph-mon[31696]: pgmap v233: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-02-26T06:31:01.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:01 smithi080 ceph-mon[36206]: pgmap v233: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-02-26T06:31:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:01 smithi115 ceph-mon[36426]: pgmap v233: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-02-26T06:31:03.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:03 smithi049 ceph-mon[31696]: pgmap v234: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.0 MiB/s wr, 726 op/s 2024-02-26T06:31:03.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:03 smithi080 ceph-mon[36206]: pgmap v234: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.0 MiB/s wr, 726 op/s 2024-02-26T06:31:03.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:03 smithi115 ceph-mon[36426]: pgmap v234: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.0 MiB/s wr, 726 op/s 2024-02-26T06:31:05.381 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:05 smithi080 ceph-mon[36206]: pgmap v235: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-02-26T06:31:05.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:05 smithi049 ceph-mon[31696]: pgmap v235: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-02-26T06:31:05.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:05 smithi115 ceph-mon[36426]: pgmap v235: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-02-26T06:31:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 435.0 s, 157.4 tps, lat 646.266 ms stddev 879.665 2024-02-26T06:31:07.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:07 smithi049 ceph-mon[31696]: pgmap v236: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 928 op/s 2024-02-26T06:31:07.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:07 smithi049 ceph-mon[31696]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T06:31:07.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:07 smithi080 ceph-mon[36206]: pgmap v236: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 928 op/s 2024-02-26T06:31:07.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:07 smithi080 ceph-mon[36206]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T06:31:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:07 smithi115 ceph-mon[36426]: pgmap v236: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 928 op/s 2024-02-26T06:31:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:07 smithi115 ceph-mon[36426]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T06:31:09.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:09 smithi049 ceph-mon[31696]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 880 op/s 2024-02-26T06:31:09.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:09 smithi080 ceph-mon[36206]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 880 op/s 2024-02-26T06:31:09.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:09 smithi115 ceph-mon[36426]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 880 op/s 2024-02-26T06:31:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 440.0 s, 1355.9 tps, lat 76.584 ms stddev 111.478 2024-02-26T06:31:11.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:11 smithi049 ceph-mon[31696]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.36k op/s 2024-02-26T06:31:11.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:11 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:31:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:11 smithi080 ceph-mon[36206]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.36k op/s 2024-02-26T06:31:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:11 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:31:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:11 smithi115 ceph-mon[36426]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.36k op/s 2024-02-26T06:31:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:11 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:31:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.952+0000 7fc65be00640 1 -- 172.21.15.49:0/513323053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 msgr2=0x7fc65c1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.952+0000 7fc65be00640 1 --2- 172.21.15.49:0/513323053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 0x7fc65c1004b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc654009ca0 tx=0x7fc65402c100 comp rx=0 tx=0).stop 2024-02-26T06:31:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.952+0000 7fc65be00640 1 -- 172.21.15.49:0/513323053 shutdown_connections 2024-02-26T06:31:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.952+0000 7fc65be00640 1 --2- 172.21.15.49:0/513323053 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65c100a80 0x7fc65c104ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.952+0000 7fc65be00640 1 --2- 172.21.15.49:0/513323053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc65c1051a0 0x7fc65c10cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.952+0000 7fc65be00640 1 --2- 172.21.15.49:0/513323053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 0x7fc65c1004b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.952+0000 7fc65be00640 1 -- 172.21.15.49:0/513323053 >> 172.21.15.49:0/513323053 conn(0x7fc65c0fb990 msgr2=0x7fc65c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.952+0000 7fc65be00640 1 -- 172.21.15.49:0/513323053 shutdown_connections 2024-02-26T06:31:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.953+0000 7fc65be00640 1 -- 172.21.15.49:0/513323053 wait complete. 2024-02-26T06:31:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.954+0000 7fc65be00640 1 Processor -- start 2024-02-26T06:31:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.954+0000 7fc65be00640 1 -- start start 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 0x7fc65c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65c1051a0 0x7fc65c19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc65c199a20 0x7fc65c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c19a4d0 con 0x7fc65c199a20 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c19a640 con 0x7fc65c1051a0 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65be00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c19a800 con 0x7fc65c1000d0 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65c1051a0 0x7fc65c19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65c1051a0 0x7fc65c19fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39918/0 (socket says 172.21.15.49:39918) 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65a000640 1 -- 172.21.15.49:0/62994147 learned_addr learned my addr 172.21.15.49:0/62994147 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65b400640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc65c199a20 0x7fc65c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65aa00640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 0x7fc65c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65a000640 1 -- 172.21.15.49:0/62994147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 msgr2=0x7fc65c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65a000640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 0x7fc65c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65a000640 1 -- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc65c199a20 msgr2=0x7fc65c199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65a000640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc65c199a20 0x7fc65c199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65a000640 1 -- 172.21.15.49:0/62994147 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654009900 con 0x7fc65c1051a0 2024-02-26T06:31:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.955+0000 7fc65aa00640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 0x7fc65c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:31:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.956+0000 7fc65a000640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65c1051a0 0x7fc65c19fa80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e9c0 tx=0x7fc64400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.956+0000 7fc653400640 1 -- 172.21.15.49:0/62994147 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644009700 con 0x7fc65c1051a0 2024-02-26T06:31:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.956+0000 7fc65be00640 1 -- 172.21.15.49:0/62994147 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c1a6360 con 0x7fc65c1051a0 2024-02-26T06:31:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.956+0000 7fc653400640 1 -- 172.21.15.49:0/62994147 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc64400cd80 con 0x7fc65c1051a0 2024-02-26T06:31:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.956+0000 7fc653400640 1 -- 172.21.15.49:0/62994147 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644010430 con 0x7fc65c1051a0 2024-02-26T06:31:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.956+0000 7fc65be00640 1 -- 172.21.15.49:0/62994147 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c1a68c0 con 0x7fc65c1051a0 2024-02-26T06:31:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.963+0000 7fc6649ca740 1 Processor -- start 2024-02-26T06:31:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.963+0000 7fc653400640 1 -- 172.21.15.49:0/62994147 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc6440105f0 con 0x7fc65c1051a0 2024-02-26T06:31:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.963+0000 7fc6649ca740 1 -- start start 2024-02-26T06:31:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.963+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 0x55d007e3da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.963+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e3e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.963+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e47540 0x55d007e49930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.963+0000 7fc6649ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007df95f0 con 0x55d007e47540 2024-02-26T06:31:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.964+0000 7fc6649ca740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007dab890 con 0x55d007e3ed30 2024-02-26T06:31:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.964+0000 7fc6649ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007da8dc0 con 0x55d007e3dfc0 2024-02-26T06:31:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.964+0000 7fc65aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e3e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.964+0000 7fc65a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e47540 0x55d007e49930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.965+0000 7fc65a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e47540 0x55d007e49930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59174/0 (socket says 172.21.15.49:59174) 2024-02-26T06:31:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.965+0000 7fc65a000640 1 -- 172.21.15.49:0/3423062998 learned_addr learned my addr 172.21.15.49:0/3423062998 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.965+0000 7fc65a000640 1 -- 172.21.15.49:0/3423062998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 msgr2=0x55d007e3e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.965+0000 7fc65a000640 1 --2- 172.21.15.49:0/3423062998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e3e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.965+0000 7fc65a000640 1 -- 172.21.15.49:0/3423062998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 msgr2=0x55d007e3da80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:31:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.965+0000 7fc65a000640 1 --2- 172.21.15.49:0/3423062998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 0x55d007e3da80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.965+0000 7fc65a000640 1 -- 172.21.15.49:0/3423062998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d007d25ad0 con 0x55d007e47540 2024-02-26T06:31:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.965+0000 7fc653400640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc63c0a2220 0x7fc63c0a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.966+0000 7fc65a000640 1 --2- 172.21.15.49:0/3423062998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e47540 0x55d007e49930 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc6440cbd20 tx=0x7fc6440d45b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff82ed475b480565 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.966+0000 7fc653400640 1 -- 172.21.15.49:0/62994147 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fc644014070 con 0x7fc65c1051a0 2024-02-26T06:31:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.966+0000 7fc65b400640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc63c0a2220 0x7fc63c0a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.966+0000 7fc64be00640 1 -- 172.21.15.49:0/3423062998 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644005500 con 0x55d007e47540 2024-02-26T06:31:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.966+0000 7fc64be00640 1 -- 172.21.15.49:0/3423062998 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc64409aa20 con 0x55d007e47540 2024-02-26T06:31:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.967+0000 7fc64be00640 1 -- 172.21.15.49:0/3423062998 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64409b780 con 0x55d007e47540 2024-02-26T06:31:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 -- 172.21.15.49:0/3423062998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e47540 msgr2=0x55d007e49930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc65b400640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc63c0a2220 0x7fc63c0a46e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc64c005e00 tx=0x7fc64c005c50 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 --2- 172.21.15.49:0/3423062998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e47540 0x55d007e49930 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc6440cbd20 tx=0x7fc6440d45b0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 -- 172.21.15.49:0/3423062998 shutdown_connections 2024-02-26T06:31:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 --2- 172.21.15.49:0/3423062998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e47540 0x55d007e49930 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 --2- 172.21.15.49:0/3423062998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e3e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 --2- 172.21.15.49:0/3423062998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 0x55d007e3da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 -- 172.21.15.49:0/3423062998 >> 172.21.15.49:0/3423062998 conn(0x55d007e05480 msgr2=0x55d007e38cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 -- 172.21.15.49:0/3423062998 shutdown_connections 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.968+0000 7fc6649ca740 1 -- 172.21.15.49:0/3423062998 wait complete. 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 1 -- start start 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 1 Processor -- start 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e44520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e44a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e47540 0x55d007e44fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007da45a0 con 0x55d007e3ed30 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007d49b20 con 0x55d007e3dfc0 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007d497c0 con 0x55d007e47540 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.969+0000 7fc6649ca740 10 client.0 _ll_register_callbacks cb 0x55d007d5d520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e44a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e44a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59178/0 (socket says 172.21.15.49:59178) 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65a000640 1 -- 172.21.15.49:0/4217774809 learned_addr learned my addr 172.21.15.49:0/4217774809 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e47540 0x55d007e44fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65aa00640 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e44520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65a000640 1 -- 172.21.15.49:0/4217774809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e47540 msgr2=0x55d007e44fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65a000640 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e47540 0x55d007e44fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65a000640 1 -- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 msgr2=0x55d007e44520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65a000640 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e44520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65a000640 1 -- 172.21.15.49:0/4217774809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c1a68c0 con 0x55d007e3ed30 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.970+0000 7fc65aa00640 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e44520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.971+0000 7fc65a000640 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e44a60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc64409b010 tx=0x7fc64400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9552feaaa600f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.971+0000 7fc648c00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.971+0000 7fc648c00640 1 -- 172.21.15.49:0/4217774809 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440d1b20 con 0x55d007e3ed30 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.971+0000 7fc648c00640 1 -- 172.21.15.49:0/4217774809 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fc644097030 con 0x55d007e3ed30 2024-02-26T06:31:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.971+0000 7fc6649ca740 1 -- 172.21.15.49:0/4217774809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d007d76ca0 con 0x55d007e3ed30 2024-02-26T06:31:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.972+0000 7fc648c00640 1 -- 172.21.15.49:0/4217774809 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc64409aa20 con 0x55d007e3ed30 2024-02-26T06:31:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.973+0000 7fc6649ca740 10 client.16626 fetch_fsmap learned FSMap version 40 2024-02-26T06:31:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.973+0000 7fc6649ca740 1 -- 172.21.15.49:0/4217774809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d007da2c20 con 0x55d007e3ed30 2024-02-26T06:31:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.974+0000 7fc648c00640 1 -- 172.21.15.49:0/4217774809 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d007d76ca0 con 0x55d007e3ed30 2024-02-26T06:31:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.974+0000 7fc648c00640 1 -- 172.21.15.49:0/4217774809 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc64401f060 con 0x55d007e3ed30 2024-02-26T06:31:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.975+0000 7fc6649ca740 10 client.16626 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:31:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.975+0000 7fc6649ca740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:31:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.975+0000 7fc6649ca740 20 client.16626 populate_metadata read hostname 'smithi049' 2024-02-26T06:31:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.975+0000 7fc6649ca740 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d007e68190 0x55d007e6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.976+0000 7fc6649ca740 4 client.16626 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:31:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.976+0000 7fc6649ca740 1 -- 172.21.15.49:0/4217774809 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d007d9acb0 con 0x55d007e68190 2024-02-26T06:31:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.976+0000 7fc65aa00640 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d007e68190 0x55d007e6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.977+0000 7fc65aa00640 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d007e68190 0x55d007e6a5b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e22557787cf507 server_cookie=a2c362da0812d529 in_seq=0 out_seq=0 2024-02-26T06:31:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.977+0000 7fc648c00640 10 client.16626 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.978+0000 7fc648c00640 1 -- 172.21.15.49:0/4217774809 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d007d9acb0 con 0x55d007e68190 2024-02-26T06:31:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.978+0000 7fc648c00640 10 client.16626 handle_command_reply: tid=0 2024-02-26T06:31:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.978+0000 7fc6649ca740 1 client.16626 shutdown 2024-02-26T06:31:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.979+0000 7fc6649ca740 1 -- 172.21.15.49:0/4217774809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 msgr2=0x55d007e44a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.979+0000 7fc6649ca740 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e44a60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc64409b010 tx=0x7fc64400cef0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.979+0000 7fc6649ca740 1 -- 172.21.15.49:0/4217774809 shutdown_connections 2024-02-26T06:31:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.980+0000 7fc6649ca740 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d007e68190 0x55d007e6a5b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.980+0000 7fc6649ca740 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e47540 0x55d007e44fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.980+0000 7fc6649ca740 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e44a60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.980+0000 7fc6649ca740 1 --2- 172.21.15.49:0/4217774809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e44520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.980+0000 7fc6649ca740 1 -- 172.21.15.49:0/4217774809 >> 172.21.15.49:0/4217774809 conn(0x55d007e05480 msgr2=0x55d007e3aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.980+0000 7fc648c00640 0 client.16626 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.980+0000 7fc6649ca740 1 -- 172.21.15.49:0/4217774809 shutdown_connections 2024-02-26T06:31:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.980+0000 7fc6649ca740 1 -- 172.21.15.49:0/4217774809 wait complete. 2024-02-26T06:31:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:11.981+0000 7fc6649ca740 20 client.16626 trim_cache size 0 max 16384 2024-02-26T06:31:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.037+0000 7fc6649ca740 1 Processor -- start 2024-02-26T06:31:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.037+0000 7fc6649ca740 1 -- start start 2024-02-26T06:31:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.038+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e05480 0x55d007e46c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.038+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.038+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e42e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.038+0000 7fc6649ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007aece60 con 0x55d007e3ed30 2024-02-26T06:31:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.038+0000 7fc6649ca740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007e44350 con 0x55d007e3dfc0 2024-02-26T06:31:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.038+0000 7fc6649ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007e38520 con 0x55d007e05480 2024-02-26T06:31:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.039+0000 7fc65a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.039+0000 7fc65a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e6eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39954/0 (socket says 172.21.15.49:39954) 2024-02-26T06:31:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.039+0000 7fc65a000640 1 -- 172.21.15.49:0/3174235194 learned_addr learned my addr 172.21.15.49:0/3174235194 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.040+0000 7fc65b400640 1 --2- 172.21.15.49:0/3174235194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e42e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.040+0000 7fc65a000640 1 -- 172.21.15.49:0/3174235194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e05480 msgr2=0x55d007e46c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:31:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.040+0000 7fc65a000640 1 --2- 172.21.15.49:0/3174235194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e05480 0x55d007e46c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.040+0000 7fc65a000640 1 -- 172.21.15.49:0/3174235194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 msgr2=0x55d007e42e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.040+0000 7fc65a000640 1 --2- 172.21.15.49:0/3174235194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e42e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.040+0000 7fc65a000640 1 -- 172.21.15.49:0/3174235194 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d007d3f500 con 0x55d007e3dfc0 2024-02-26T06:31:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.041+0000 7fc65a000640 1 --2- 172.21.15.49:0/3174235194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e6eed0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc6440a32f0 tx=0x7fc644097560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe7c1c4c9284dd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.042+0000 7fc64b400640 1 -- 172.21.15.49:0/3174235194 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644005500 con 0x55d007e3dfc0 2024-02-26T06:31:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.042+0000 7fc64b400640 1 -- 172.21.15.49:0/3174235194 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc64409aa20 con 0x55d007e3dfc0 2024-02-26T06:31:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.042+0000 7fc64b400640 1 -- 172.21.15.49:0/3174235194 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440d1b20 con 0x55d007e3dfc0 2024-02-26T06:31:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.043+0000 7fc6649ca740 1 -- 172.21.15.49:0/3174235194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 msgr2=0x55d007e6eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.043+0000 7fc6649ca740 1 --2- 172.21.15.49:0/3174235194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e6eed0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc6440a32f0 tx=0x7fc644097560 comp rx=0 tx=0).stop 2024-02-26T06:31:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.044+0000 7fc6649ca740 1 -- 172.21.15.49:0/3174235194 shutdown_connections 2024-02-26T06:31:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.044+0000 7fc6649ca740 1 --2- 172.21.15.49:0/3174235194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3dfc0 0x55d007e6eed0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.044+0000 7fc6649ca740 1 --2- 172.21.15.49:0/3174235194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e3ed30 0x55d007e42e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.044+0000 7fc6649ca740 1 --2- 172.21.15.49:0/3174235194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e05480 0x55d007e46c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.044+0000 7fc6649ca740 1 -- 172.21.15.49:0/3174235194 >> 172.21.15.49:0/3174235194 conn(0x55d007e3db20 msgr2=0x55d007e3f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.045+0000 7fc6649ca740 1 -- 172.21.15.49:0/3174235194 shutdown_connections 2024-02-26T06:31:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.045+0000 7fc6649ca740 1 -- 172.21.15.49:0/3174235194 wait complete. 2024-02-26T06:31:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.047+0000 7fc6649ca740 1 -- start start 2024-02-26T06:31:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.047+0000 7fc6649ca740 1 Processor -- start 2024-02-26T06:31:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.048+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e05480 0x55d007e5d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.048+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e5d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.048+0000 7fc6649ca740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 0x55d007e5de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.048+0000 7fc6649ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007ec5f30 con 0x55d007e05480 2024-02-26T06:31:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.048+0000 7fc6649ca740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007ec60a0 con 0x55d007e3ed30 2024-02-26T06:31:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.048+0000 7fc6649ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d007ec6260 con 0x55d007e3dfc0 2024-02-26T06:31:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.048+0000 7fc6649ca740 10 client.0 _ll_register_callbacks cb 0x7fc6440d42f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e5d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e5d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:37952/0 (socket says 172.21.15.49:37952) 2024-02-26T06:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65a000640 1 -- 172.21.15.49:0/897476228 learned_addr learned my addr 172.21.15.49:0/897476228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65aa00640 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e05480 0x55d007e5d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65b400640 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 0x55d007e5de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65a000640 1 -- 172.21.15.49:0/897476228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 msgr2=0x55d007e5de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65a000640 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 0x55d007e5de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65a000640 1 -- 172.21.15.49:0/897476228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e05480 msgr2=0x55d007e5d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65a000640 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e05480 0x55d007e5d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65a000640 1 -- 172.21.15.49:0/897476228 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d007da2c20 con 0x55d007e3dfc0 2024-02-26T06:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.049+0000 7fc65aa00640 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e05480 0x55d007e5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.050+0000 7fc65a000640 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e5d940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc6440048f0 tx=0x7fc644005430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dd021a99f662690 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.050+0000 7fc648c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.051+0000 7fc648c00640 1 -- 172.21.15.49:0/897476228 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644097270 con 0x55d007e3dfc0 2024-02-26T06:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.051+0000 7fc648c00640 1 -- 172.21.15.49:0/897476228 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fc644097040 con 0x55d007e3dfc0 2024-02-26T06:31:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.053+0000 7fc6649ca740 1 -- 172.21.15.49:0/897476228 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d007dc9d00 con 0x55d007e3dfc0 2024-02-26T06:31:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.053+0000 7fc648c00640 1 -- 172.21.15.49:0/897476228 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc644005500 con 0x55d007e3dfc0 2024-02-26T06:31:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.053+0000 7fc6649ca740 10 client.27169 fetch_fsmap learned FSMap version 40 2024-02-26T06:31:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.053+0000 7fc6649ca740 1 -- 172.21.15.49:0/897476228 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d007d65290 con 0x55d007e3dfc0 2024-02-26T06:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.054+0000 7fc648c00640 1 -- 172.21.15.49:0/897476228 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d007dc9d00 con 0x55d007e3dfc0 2024-02-26T06:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.054+0000 7fc648c00640 1 -- 172.21.15.49:0/897476228 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc64401f070 con 0x55d007e3dfc0 2024-02-26T06:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.054+0000 7fc6649ca740 10 client.27169 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.054+0000 7fc6649ca740 10 client.27169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.054+0000 7fc6649ca740 20 client.27169 populate_metadata read hostname 'smithi049' 2024-02-26T06:31:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.054+0000 7fc6649ca740 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d007e47540 0x55d007ec9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.054+0000 7fc6649ca740 4 client.27169 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:31:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.054+0000 7fc6649ca740 1 -- 172.21.15.49:0/897476228 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d007aa4600 con 0x55d007e47540 2024-02-26T06:31:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.055+0000 7fc65aa00640 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d007e47540 0x55d007ec9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.055+0000 7fc65aa00640 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d007e47540 0x55d007ec9540 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf959f2591b396c2 server_cookie=65ba0f741db2841f in_seq=0 out_seq=0 2024-02-26T06:31:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.056+0000 7fc648c00640 10 client.27169 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.101+0000 7fc648c00640 1 -- 172.21.15.49:0/897476228 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d007aa4600 con 0x55d007e47540 2024-02-26T06:31:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.101+0000 7fc648c00640 10 client.27169 handle_command_reply: tid=0 2024-02-26T06:31:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.101+0000 7fc6649ca740 1 client.27169 shutdown 2024-02-26T06:31:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 -- 172.21.15.49:0/897476228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 msgr2=0x55d007e5d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e5d940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc6440048f0 tx=0x7fc644005430 comp rx=0 tx=0).stop 2024-02-26T06:31:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 -- 172.21.15.49:0/897476228 shutdown_connections 2024-02-26T06:31:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d007e47540 0x55d007ec9540 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d007e3ed30 0x55d007e5de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d007e3dfc0 0x55d007e5d940 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 --2- 172.21.15.49:0/897476228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d007e05480 0x55d007e5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 -- 172.21.15.49:0/897476228 >> 172.21.15.49:0/897476228 conn(0x55d007e3db20 msgr2=0x55d007e6eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc648c00640 0 client.27169 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 -- 172.21.15.49:0/897476228 shutdown_connections 2024-02-26T06:31:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 1 -- 172.21.15.49:0/897476228 wait complete. 2024-02-26T06:31:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.102+0000 7fc6649ca740 20 client.27169 trim_cache size 0 max 16384 2024-02-26T06:31:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:31:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:31:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:31:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:31:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 -- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc63c0a2220 msgr2=0x7fc63c0a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc63c0a2220 0x7fc63c0a46e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc64c005e00 tx=0x7fc64c005c50 comp rx=0 tx=0).stop 2024-02-26T06:31:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 -- 172.21.15.49:0/62994147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65c1051a0 msgr2=0x7fc65c19fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65c1051a0 0x7fc65c19fa80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e9c0 tx=0x7fc64400ee80 comp rx=0 tx=0).stop 2024-02-26T06:31:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 -- 172.21.15.49:0/62994147 shutdown_connections 2024-02-26T06:31:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc63c0a2220 0x7fc63c0a46e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65c1051a0 0x7fc65c19fa80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc65c199a20 0x7fc65c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 --2- 172.21.15.49:0/62994147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc65c1000d0 0x7fc65c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 -- 172.21.15.49:0/62994147 >> 172.21.15.49:0/62994147 conn(0x7fc65c0fb990 msgr2=0x7fc65c0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 -- 172.21.15.49:0/62994147 shutdown_connections 2024-02-26T06:31:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:12.103+0000 7fc65be00640 1 -- 172.21.15.49:0/62994147 wait complete. 2024-02-26T06:31:13.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:13 smithi049 ceph-mon[31696]: pgmap v240: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2024-02-26T06:31:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:13 smithi080 ceph-mon[36206]: pgmap v240: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2024-02-26T06:31:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:13 smithi115 ceph-mon[36426]: pgmap v240: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2024-02-26T06:31:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:15 smithi049 ceph-mon[31696]: pgmap v241: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2024-02-26T06:31:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:15 smithi049 ceph-mon[31696]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T06:31:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:15 smithi080 ceph-mon[36206]: pgmap v241: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2024-02-26T06:31:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:15 smithi080 ceph-mon[36206]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T06:31:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:15 smithi115 ceph-mon[36426]: pgmap v241: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2024-02-26T06:31:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:15 smithi115 ceph-mon[36426]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T06:31:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 445.0 s, 1571.2 tps, lat 63.865 ms stddev 74.550 2024-02-26T06:31:17.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:17 smithi049 ceph-mon[31696]: pgmap v243: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-02-26T06:31:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:17 smithi080 ceph-mon[36206]: pgmap v243: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-02-26T06:31:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:17 smithi115 ceph-mon[36426]: pgmap v243: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-02-26T06:31:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:19 smithi080 ceph-mon[36206]: pgmap v244: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-02-26T06:31:19.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:19 smithi115 ceph-mon[36426]: pgmap v244: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-02-26T06:31:19.674 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:19 smithi049 ceph-mon[31696]: pgmap v244: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-02-26T06:31:21.304 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 450.0 s, 1507.8 tps, lat 66.327 ms stddev 72.014 2024-02-26T06:31:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:21 smithi080 ceph-mon[36206]: pgmap v245: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-02-26T06:31:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:21 smithi115 ceph-mon[36426]: pgmap v245: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-02-26T06:31:21.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:21 smithi049 ceph-mon[31696]: pgmap v245: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-02-26T06:31:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:23 smithi080 ceph-mon[36206]: pgmap v246: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-02-26T06:31:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:23 smithi115 ceph-mon[36426]: pgmap v246: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-02-26T06:31:23.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:23 smithi049 ceph-mon[31696]: pgmap v246: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-02-26T06:31:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.295+0000 7f79abe00640 1 -- 172.21.15.49:0/1964927090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a41013a0 msgr2=0x7f79a4101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.295+0000 7f79abe00640 1 --2- 172.21.15.49:0/1964927090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a41013a0 0x7f79a4101780 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7998009900 tx=0x7f799802bc30 comp rx=0 tx=0).stop 2024-02-26T06:31:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.295+0000 7f79abe00640 1 -- 172.21.15.49:0/1964927090 shutdown_connections 2024-02-26T06:31:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.295+0000 7f79abe00640 1 --2- 172.21.15.49:0/1964927090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a4106470 0x7f79a410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.295+0000 7f79abe00640 1 --2- 172.21.15.49:0/1964927090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a4101d50 0x7f79a4105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.295+0000 7f79abe00640 1 --2- 172.21.15.49:0/1964927090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a41013a0 0x7f79a4101780 secure :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7998009900 tx=0x7f799802bc30 comp rx=0 tx=0).stop 2024-02-26T06:31:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.295+0000 7f79abe00640 1 -- 172.21.15.49:0/1964927090 >> 172.21.15.49:0/1964927090 conn(0x7f79a40fcbe0 msgr2=0x7f79a40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.296+0000 7f79abe00640 1 -- 172.21.15.49:0/1964927090 shutdown_connections 2024-02-26T06:31:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.296+0000 7f79abe00640 1 -- 172.21.15.49:0/1964927090 wait complete. 2024-02-26T06:31:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.297+0000 7f79abe00640 1 Processor -- start 2024-02-26T06:31:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.297+0000 7f79abe00640 1 -- start start 2024-02-26T06:31:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.297+0000 7f79abe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a4106470 0x7f79a4110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.297+0000 7f79abe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a4114060 0x7f79a4110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.297+0000 7f79abe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a4110cd0 0x7f79a41aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.297+0000 7f79abe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41aa800 con 0x7f79a4106470 2024-02-26T06:31:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.297+0000 7f79abe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41aa970 con 0x7f79a4114060 2024-02-26T06:31:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.297+0000 7f79abe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41aab30 con 0x7f79a4110cd0 2024-02-26T06:31:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.298+0000 7f79a8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a4114060 0x7f79a4110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.298+0000 7f79a8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a4114060 0x7f79a4110680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47210/0 (socket says 172.21.15.49:47210) 2024-02-26T06:31:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.298+0000 7f79a8a00640 1 -- 172.21.15.49:0/1326158152 learned_addr learned my addr 172.21.15.49:0/1326158152 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.298+0000 7f79a9e00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a4110cd0 0x7f79a41aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.298+0000 7f79a9400640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a4106470 0x7f79a4110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a9e00640 1 -- 172.21.15.49:0/1326158152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a4114060 msgr2=0x7f79a4110680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a9e00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a4114060 0x7f79a4110680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a9e00640 1 -- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a4106470 msgr2=0x7f79a4110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a9e00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a4106470 0x7f79a4110140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a9e00640 1 -- 172.21.15.49:0/1326158152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998009560 con 0x7f79a4110cd0 2024-02-26T06:31:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a9e00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a4110cd0 0x7f79a41aa0f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f799c00be20 tx=0x7f799c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a2000640 1 -- 172.21.15.49:0/1326158152 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c00ce70 con 0x7f79a4110cd0 2024-02-26T06:31:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a2000640 1 -- 172.21.15.49:0/1326158152 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799c014920 con 0x7f79a4110cd0 2024-02-26T06:31:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79a2000640 1 -- 172.21.15.49:0/1326158152 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c0129e0 con 0x7f79a4110cd0 2024-02-26T06:31:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.299+0000 7f79abe00640 1 -- 172.21.15.49:0/1326158152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a41aae10 con 0x7f79a4110cd0 2024-02-26T06:31:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.300+0000 7f79abe00640 1 -- 172.21.15.49:0/1326158152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41ab2a0 con 0x7f79a4110cd0 2024-02-26T06:31:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.303+0000 7f79aebcb740 1 Processor -- start 2024-02-26T06:31:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.303+0000 7f79a2000640 1 -- 172.21.15.49:0/1326158152 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f799c014a90 con 0x7f79a4110cd0 2024-02-26T06:31:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.304+0000 7f79aebcb740 1 -- start start 2024-02-26T06:31:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.304+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.304+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff6b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.304+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.304+0000 7f79aebcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591ff265f0 con 0x55591ff6ad80 2024-02-26T06:31:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.304+0000 7f79aebcb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591fed8890 con 0x55591ff74300 2024-02-26T06:31:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.304+0000 7f79aebcb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591fed5dc0 con 0x55591ff6baf0 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33642/0 (socket says 172.21.15.49:33642) 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 -- 172.21.15.49:0/3433391518 learned_addr learned my addr 172.21.15.49:0/3433391518 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 -- 172.21.15.49:0/3433391518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 msgr2=0x55591ff766f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a9400640 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff6b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff766f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 -- 172.21.15.49:0/3433391518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 msgr2=0x55591ff6b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff6b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a8a00640 1 -- 172.21.15.49:0/3433391518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55591fe52ad0 con 0x55591ff6baf0 2024-02-26T06:31:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.305+0000 7f79a9400640 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff6b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:31:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.306+0000 7f79a2000640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f798c0a2380 0x7f798c0a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.306+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6a840 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f799000a630 tx=0x7f799000a7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89892ec1f49722b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.306+0000 7f79a2000640 1 -- 172.21.15.49:0/1326158152 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f799c0cd9d0 con 0x7f79a4110cd0 2024-02-26T06:31:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.306+0000 7f7996600640 1 -- 172.21.15.49:0/3433391518 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7990004050 con 0x55591ff6baf0 2024-02-26T06:31:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.306+0000 7f79a9400640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f798c0a2380 0x7f798c0a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.306+0000 7f7996600640 1 -- 172.21.15.49:0/3433391518 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7990005500 con 0x55591ff6baf0 2024-02-26T06:31:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.306+0000 7f7996600640 1 -- 172.21.15.49:0/3433391518 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7990004c20 con 0x55591ff6baf0 2024-02-26T06:31:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79aebcb740 1 -- 172.21.15.49:0/3433391518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 msgr2=0x55591ff6a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79a9400640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f798c0a2380 0x7f798c0a4840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7998036040 tx=0x7f7998005b50 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6a840 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f799000a630 tx=0x7f799000a7d0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79aebcb740 1 -- 172.21.15.49:0/3433391518 shutdown_connections 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff766f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff6b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3433391518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6a840 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79aebcb740 1 -- 172.21.15.49:0/3433391518 >> 172.21.15.49:0/3433391518 conn(0x55591ff32480 msgr2=0x55591ff65a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.307+0000 7f79aebcb740 1 -- 172.21.15.49:0/3433391518 shutdown_connections 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.308+0000 7f79aebcb740 1 -- 172.21.15.49:0/3433391518 wait complete. 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.308+0000 7f79aebcb740 1 -- start start 2024-02-26T06:31:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.308+0000 7f79aebcb740 1 Processor -- start 2024-02-26T06:31:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6baf0 0x55591ff718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff74300 0x55591ff71e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79aebcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591fed15a0 con 0x55591ff6baf0 2024-02-26T06:31:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79aebcb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591fe76b20 con 0x55591ff6ad80 2024-02-26T06:31:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79aebcb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591fe767c0 con 0x55591ff74300 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79aebcb740 10 client.0 _ll_register_callbacks cb 0x55591fe8a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79a9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79a9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47226/0 (socket says 172.21.15.49:47226) 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79a9400640 1 -- 172.21.15.49:0/3535729376 learned_addr learned my addr 172.21.15.49:0/3535729376 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79a9e00640 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff74300 0x55591ff71e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.309+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6baf0 0x55591ff718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f79a9400640 1 -- 172.21.15.49:0/3535729376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff74300 msgr2=0x55591ff71e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f79a9400640 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff74300 0x55591ff71e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f79a9400640 1 -- 172.21.15.49:0/3535729376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6baf0 msgr2=0x55591ff718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f79a9400640 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6baf0 0x55591ff718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f79a9400640 1 -- 172.21.15.49:0/3535729376 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7990019040 con 0x55591ff6ad80 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6baf0 0x55591ff718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f79a9400640 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff713b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7998005c40 tx=0x7f799802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4dd1e168e53906d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f798aa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:31:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.310+0000 7f798aa00640 1 -- 172.21.15.49:0/3535729376 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998004ff0 con 0x55591ff6ad80 2024-02-26T06:31:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.311+0000 7f798aa00640 1 -- 172.21.15.49:0/3535729376 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f7998046070 con 0x55591ff6ad80 2024-02-26T06:31:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.311+0000 7f79aebcb740 1 -- 172.21.15.49:0/3535729376 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55591fea3ca0 con 0x55591ff6ad80 2024-02-26T06:31:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.311+0000 7f798aa00640 1 -- 172.21.15.49:0/3535729376 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f799802ce40 con 0x55591ff6ad80 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.311+0000 7f79aebcb740 10 client.27206 fetch_fsmap learned FSMap version 40 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.311+0000 7f79aebcb740 1 -- 172.21.15.49:0/3535729376 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55591fecfc20 con 0x55591ff6ad80 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.312+0000 7f798aa00640 1 -- 172.21.15.49:0/3535729376 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55591fea3ca0 con 0x55591ff6ad80 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.313+0000 7f798aa00640 1 -- 172.21.15.49:0/3535729376 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f799804b460 con 0x55591ff6ad80 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.313+0000 7f79aebcb740 10 client.27206 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.313+0000 7f79aebcb740 10 client.27206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.313+0000 7f79aebcb740 20 client.27206 populate_metadata read hostname 'smithi049' 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.313+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55591ff843e0 0x55591ff86820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.313+0000 7f79aebcb740 4 client.27206 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:31:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.313+0000 7f79aebcb740 1 -- 172.21.15.49:0/3535729376 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55591fec7cb0 con 0x55591ff843e0 2024-02-26T06:31:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.314+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55591ff843e0 0x55591ff86820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.315+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55591ff843e0 0x55591ff86820 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d016557d95e7f2 server_cookie=47adbe6a20aa98c3 in_seq=0 out_seq=0 2024-02-26T06:31:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.315+0000 7f798aa00640 10 client.27206 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.316+0000 7f798aa00640 1 -- 172.21.15.49:0/3535729376 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55591fec7cb0 con 0x55591ff843e0 2024-02-26T06:31:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.316+0000 7f798aa00640 10 client.27206 handle_command_reply: tid=0 2024-02-26T06:31:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.316+0000 7f79aebcb740 1 client.27206 shutdown 2024-02-26T06:31:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 -- 172.21.15.49:0/3535729376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 msgr2=0x55591ff713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff713b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7998005c40 tx=0x7f799802bc30 comp rx=0 tx=0).stop 2024-02-26T06:31:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 -- 172.21.15.49:0/3535729376 shutdown_connections 2024-02-26T06:31:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55591ff843e0 0x55591ff86820 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff74300 0x55591ff71e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6baf0 0x55591ff718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3535729376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff713b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 -- 172.21.15.49:0/3535729376 >> 172.21.15.49:0/3535729376 conn(0x55591ff32480 msgr2=0x55591ff677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f798aa00640 0 client.27206 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 -- 172.21.15.49:0/3535729376 shutdown_connections 2024-02-26T06:31:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 1 -- 172.21.15.49:0/3535729376 wait complete. 2024-02-26T06:31:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.317+0000 7f79aebcb740 20 client.27206 trim_cache size 0 max 16384 2024-02-26T06:31:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.340+0000 7f79aebcb740 1 Processor -- start 2024-02-26T06:31:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.340+0000 7f79aebcb740 1 -- start start 2024-02-26T06:31:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.341+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff73ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.341+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.341+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.341+0000 7f79aebcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591fc19e60 con 0x55591ff6ad80 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.341+0000 7f79aebcb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591ff66870 con 0x55591ff74300 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.341+0000 7f79aebcb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591ff72380 con 0x55591ff6baf0 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.342+0000 7f79a9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.342+0000 7f79a9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47240/0 (socket says 172.21.15.49:47240) 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.342+0000 7f79a9e00640 1 -- 172.21.15.49:0/3894014608 learned_addr learned my addr 172.21.15.49:0/3894014608 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.342+0000 7f79a8a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.342+0000 7f79a9400640 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff73ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.342+0000 7f79a8a00640 1 -- 172.21.15.49:0/3894014608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 msgr2=0x55591ff701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.343+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.343+0000 7f79a8a00640 1 -- 172.21.15.49:0/3894014608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 msgr2=0x55591ff73ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.343+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff73ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.343+0000 7f79a8a00640 1 -- 172.21.15.49:0/3894014608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55591fe6c500 con 0x55591ff6baf0 2024-02-26T06:31:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.343+0000 7f79a9e00640 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:31:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.343+0000 7f79a8a00640 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6fc80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f799000a5c0 tx=0x7f7990004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6378d26795f740 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.344+0000 7f7995c00640 1 -- 172.21.15.49:0/3894014608 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7990023040 con 0x55591ff6baf0 2024-02-26T06:31:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.344+0000 7f7995c00640 1 -- 172.21.15.49:0/3894014608 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799002a070 con 0x55591ff6baf0 2024-02-26T06:31:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.344+0000 7f7995c00640 1 -- 172.21.15.49:0/3894014608 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79900045c0 con 0x55591ff6baf0 2024-02-26T06:31:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 -- 172.21.15.49:0/3894014608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 msgr2=0x55591ff6fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6fc80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f799000a5c0 tx=0x7f7990004840 comp rx=0 tx=0).stop 2024-02-26T06:31:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 -- 172.21.15.49:0/3894014608 shutdown_connections 2024-02-26T06:31:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff74300 0x55591ff701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff6fc80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 --2- 172.21.15.49:0/3894014608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff6ad80 0x55591ff73ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 -- 172.21.15.49:0/3894014608 >> 172.21.15.49:0/3894014608 conn(0x55591ff32480 msgr2=0x55591ff6c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 -- 172.21.15.49:0/3894014608 shutdown_connections 2024-02-26T06:31:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.345+0000 7f79aebcb740 1 -- 172.21.15.49:0/3894014608 wait complete. 2024-02-26T06:31:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.346+0000 7f79aebcb740 1 -- start start 2024-02-26T06:31:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.346+0000 7f79aebcb740 1 Processor -- start 2024-02-26T06:31:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.346+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff7b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.346+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.346+0000 7f79aebcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff74300 0x55591ff79cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.346+0000 7f79aebcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591ff7a300 con 0x55591ff74300 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.346+0000 7f79aebcb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591ff7a470 con 0x55591ff6ad80 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.346+0000 7f79aebcb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55591ff7a630 con 0x55591ff6baf0 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79aebcb740 10 client.0 _ll_register_callbacks cb 0x7f7998009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff7b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff7b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:47250/0 (socket says 172.21.15.49:47250) 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 -- 172.21.15.49:0/232654103 learned_addr learned my addr 172.21.15.49:0/232654103 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a8a00640 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9e00640 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff74300 0x55591ff79cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 -- 172.21.15.49:0/232654103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 msgr2=0x55591ff797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 -- 172.21.15.49:0/232654103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff74300 msgr2=0x55591ff79cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff74300 0x55591ff79cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 -- 172.21.15.49:0/232654103 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41ab2a0 con 0x55591ff6ad80 2024-02-26T06:31:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f79a9400640 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff7b3c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f799804b810 tx=0x7f799804b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353b0264154896ce server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.347+0000 7f798aa00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:31:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.348+0000 7f798aa00640 1 -- 172.21.15.49:0/232654103 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998003b40 con 0x55591ff6ad80 2024-02-26T06:31:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.348+0000 7f798aa00640 1 -- 172.21.15.49:0/232654103 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f799802ce40 con 0x55591ff6ad80 2024-02-26T06:31:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.349+0000 7f79aebcb740 1 -- 172.21.15.49:0/232654103 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55591ff711e0 con 0x55591ff6ad80 2024-02-26T06:31:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.349+0000 7f798aa00640 1 -- 172.21.15.49:0/232654103 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7998003ca0 con 0x55591ff6ad80 2024-02-26T06:31:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.349+0000 7f79aebcb740 10 client.27212 fetch_fsmap learned FSMap version 40 2024-02-26T06:31:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.349+0000 7f79aebcb740 1 -- 172.21.15.49:0/232654103 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55591fe52ad0 con 0x55591ff6ad80 2024-02-26T06:31:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f798aa00640 1 -- 172.21.15.49:0/232654103 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799806b020 con 0x55591ff6ad80 2024-02-26T06:31:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f798aa00640 1 -- 172.21.15.49:0/232654103 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79980555f0 con 0x55591ff6ad80 2024-02-26T06:31:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f79aebcb740 10 client.27212 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:31:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f79aebcb740 10 client.27212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:31:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f79aebcb740 20 client.27212 populate_metadata read hostname 'smithi049' 2024-02-26T06:31:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f79aebcb740 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55591ffdbca0 0x55591ffde0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f79aebcb740 4 client.27212 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:31:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f79aebcb740 1 -- 172.21.15.49:0/232654103 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55591fef6d00 con 0x55591ffdbca0 2024-02-26T06:31:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.350+0000 7f79a8a00640 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55591ffdbca0 0x55591ffde0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.351+0000 7f79a8a00640 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55591ffdbca0 0x55591ffde0c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e0193c3fb408bd server_cookie=5394238bf70446cc in_seq=0 out_seq=0 2024-02-26T06:31:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.351+0000 7f798aa00640 10 client.27212 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.370+0000 7f798aa00640 1 -- 172.21.15.49:0/232654103 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55591fef6d00 con 0x55591ffdbca0 2024-02-26T06:31:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.371+0000 7f798aa00640 10 client.27212 handle_command_reply: tid=0 2024-02-26T06:31:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.371+0000 7f79aebcb740 1 client.27212 shutdown 2024-02-26T06:31:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.371+0000 7f79aebcb740 1 -- 172.21.15.49:0/232654103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 msgr2=0x55591ff7b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.371+0000 7f79aebcb740 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff7b3c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f799804b810 tx=0x7f799804b5a0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 1 -- 172.21.15.49:0/232654103 shutdown_connections 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55591ffdbca0 0x55591ffde0c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55591ff74300 0x55591ff79cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f798aa00640 0 client.27212 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55591ff6baf0 0x55591ff797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 1 --2- 172.21.15.49:0/232654103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55591ff6ad80 0x55591ff7b3c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 1 -- 172.21.15.49:0/232654103 >> 172.21.15.49:0/232654103 conn(0x55591ff32480 msgr2=0x55591ff6fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 1 -- 172.21.15.49:0/232654103 shutdown_connections 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 1 -- 172.21.15.49:0/232654103 wait complete. 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.372+0000 7f79aebcb740 20 client.27212 trim_cache size 0 max 16384 2024-02-26T06:31:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:31:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:31:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:31:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:31:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 -- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f798c0a2380 msgr2=0x7f798c0a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f798c0a2380 0x7f798c0a4840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7998036040 tx=0x7f7998005b50 comp rx=0 tx=0).stop 2024-02-26T06:31:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 -- 172.21.15.49:0/1326158152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a4110cd0 msgr2=0x7f79a41aa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a4110cd0 0x7f79a41aa0f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f799c00be20 tx=0x7f799c00bf10 comp rx=0 tx=0).stop 2024-02-26T06:31:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 -- 172.21.15.49:0/1326158152 shutdown_connections 2024-02-26T06:31:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f798c0a2380 0x7f798c0a4840 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a4114060 0x7f79a4110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a4110cd0 0x7f79a41aa0f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 --2- 172.21.15.49:0/1326158152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a4106470 0x7f79a4110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 -- 172.21.15.49:0/1326158152 >> 172.21.15.49:0/1326158152 conn(0x7f79a40fcbe0 msgr2=0x7f79a40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 -- 172.21.15.49:0/1326158152 shutdown_connections 2024-02-26T06:31:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:25.373+0000 7f79abe00640 1 -- 172.21.15.49:0/1326158152 wait complete. 2024-02-26T06:31:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:25 smithi080 ceph-mon[36206]: pgmap v247: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-02-26T06:31:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:25 smithi115 ceph-mon[36426]: pgmap v247: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-02-26T06:31:25.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:25 smithi049 ceph-mon[31696]: pgmap v247: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-02-26T06:31:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 455.0 s, 1485.6 tps, lat 66.461 ms stddev 72.686 2024-02-26T06:31:27.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:27 smithi080 ceph-mon[36206]: pgmap v248: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-02-26T06:31:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:27 smithi115 ceph-mon[36426]: pgmap v248: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-02-26T06:31:27.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:27 smithi049 ceph-mon[31696]: pgmap v248: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-02-26T06:31:29.476 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:29 smithi049 ceph-mon[31696]: pgmap v249: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 600 op/s 2024-02-26T06:31:29.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:29 smithi080 ceph-mon[36206]: pgmap v249: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 600 op/s 2024-02-26T06:31:29.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:29 smithi115 ceph-mon[36426]: pgmap v249: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 600 op/s 2024-02-26T06:31:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 460.0 s, 1661.2 tps, lat 60.953 ms stddev 77.877 2024-02-26T06:31:31.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:31 smithi080 ceph-mon[36206]: pgmap v250: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 660 op/s 2024-02-26T06:31:31.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:31 smithi115 ceph-mon[36426]: pgmap v250: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 660 op/s 2024-02-26T06:31:31.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:31 smithi049 ceph-mon[31696]: pgmap v250: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 660 op/s 2024-02-26T06:31:33.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:33 smithi080 ceph-mon[36206]: pgmap v251: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-02-26T06:31:33.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:33 smithi115 ceph-mon[36426]: pgmap v251: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-02-26T06:31:33.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:33 smithi049 ceph-mon[31696]: pgmap v251: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-02-26T06:31:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:35 smithi080 ceph-mon[36206]: pgmap v252: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.0 MiB/s wr, 553 op/s 2024-02-26T06:31:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:35 smithi115 ceph-mon[36426]: pgmap v252: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.0 MiB/s wr, 553 op/s 2024-02-26T06:31:35.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:35 smithi049 ceph-mon[31696]: pgmap v252: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.0 MiB/s wr, 553 op/s 2024-02-26T06:31:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 465.0 s, 1545.8 tps, lat 64.330 ms stddev 72.619 2024-02-26T06:31:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:37 smithi080 ceph-mon[36206]: pgmap v253: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 816 op/s 2024-02-26T06:31:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:37 smithi115 ceph-mon[36426]: pgmap v253: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 816 op/s 2024-02-26T06:31:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:37 smithi049 ceph-mon[31696]: pgmap v253: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 816 op/s 2024-02-26T06:31:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.592+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1736032473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc101d50 msgr2=0x7fc8cc105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.592+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1736032473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc101d50 0x7fc8cc105d90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009900 tx=0x7fc8b802bc30 comp rx=0 tx=0).stop 2024-02-26T06:31:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.593+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1736032473 shutdown_connections 2024-02-26T06:31:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.593+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1736032473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8cc106470 0x7fc8cc10e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.593+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1736032473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc101d50 0x7fc8cc105d90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.593+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1736032473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8cc1013a0 0x7fc8cc101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.593+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1736032473 >> 172.21.15.49:0/1736032473 conn(0x7fc8cc0fcc40 msgr2=0x7fc8cc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.594+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1736032473 shutdown_connections 2024-02-26T06:31:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.594+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1736032473 wait complete. 2024-02-26T06:31:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.595+0000 7fc8d2a00640 1 Processor -- start 2024-02-26T06:31:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.596+0000 7fc8d2a00640 1 -- start start 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.596+0000 7fc8d2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8cc1013a0 0x7fc8cc115470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.596+0000 7fc8d2a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc106470 0x7fc8cc11b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.596+0000 7fc8d2a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8cc115b10 0x7fc8cc110620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.596+0000 7fc8d2a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc110c70 con 0x7fc8cc115b10 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.596+0000 7fc8d2a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc110de0 con 0x7fc8cc1013a0 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.596+0000 7fc8d2a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc110fa0 con 0x7fc8cc106470 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8cc1013a0 0x7fc8cc115470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc106470 0x7fc8cc11b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc106470 0x7fc8cc11b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:50890/0 (socket says 172.21.15.49:50890) 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8cc1013a0 0x7fc8cc115470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39072/0 (socket says 172.21.15.49:39072) 2024-02-26T06:31:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cb400640 1 -- 172.21.15.49:0/1172852713 learned_addr learned my addr 172.21.15.49:0/1172852713 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cbe00640 1 -- 172.21.15.49:0/1172852713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc106470 msgr2=0x7fc8cc11b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc106470 0x7fc8cc11b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cbe00640 1 -- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8cc115b10 msgr2=0x7fc8cc110620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8cc115b10 0x7fc8cc110620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.597+0000 7fc8cbe00640 1 -- 172.21.15.49:0/1172852713 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8009560 con 0x7fc8cc1013a0 2024-02-26T06:31:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.598+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8cc1013a0 0x7fc8cc115470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00c970 tx=0x7fc8bc00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.598+0000 7fc8c8c00640 1 -- 172.21.15.49:0/1172852713 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc007bb0 con 0x7fc8cc1013a0 2024-02-26T06:31:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.599+0000 7fc8c8c00640 1 -- 172.21.15.49:0/1172852713 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8bc007d10 con 0x7fc8cc1013a0 2024-02-26T06:31:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.599+0000 7fc8c8c00640 1 -- 172.21.15.49:0/1172852713 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc007bb0 con 0x7fc8cc1013a0 2024-02-26T06:31:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.599+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1172852713 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8cc111280 con 0x7fc8cc1013a0 2024-02-26T06:31:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.600+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1172852713 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc1b35d0 con 0x7fc8cc1013a0 2024-02-26T06:31:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.608+0000 7fc8c8c00640 1 -- 172.21.15.49:0/1172852713 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc8bc004c80 con 0x7fc8cc1013a0 2024-02-26T06:31:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.610+0000 7fc8c8c00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc8b00a2430 0x7fc8b00a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.610+0000 7fc8c8c00640 1 -- 172.21.15.49:0/1172852713 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fc8bc0cd920 con 0x7fc8cc1013a0 2024-02-26T06:31:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.610+0000 7fc8cb400640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc8b00a2430 0x7fc8b00a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.613+0000 7fc8d596a740 1 Processor -- start 2024-02-26T06:31:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.613+0000 7fc8d596a740 1 -- start start 2024-02-26T06:31:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.613+0000 7fc8cb400640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc8b00a2430 0x7fc8b00a48f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8006010 tx=0x7fc8b8005a20 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.613+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a7b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.613+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.613+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.614+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.614+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a7b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.614+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:50902/0 (socket says 172.21.15.49:50902) 2024-02-26T06:31:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.614+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a7b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39082/0 (socket says 172.21.15.49:39082) 2024-02-26T06:31:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.615+0000 7fc8d596a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e2a375f0 con 0x5599e2a85300 2024-02-26T06:31:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.615+0000 7fc8d0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.615+0000 7fc8cb400640 1 -- 172.21.15.49:0/952772351 learned_addr learned my addr 172.21.15.49:0/952772351 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.616+0000 7fc8d596a740 1 -- 172.21.15.49:0/952772351 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e29e9890 con 0x5599e2a7caf0 2024-02-26T06:31:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.616+0000 7fc8d596a740 1 -- 172.21.15.49:0/952772351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e29e6dc0 con 0x5599e2a7bd80 2024-02-26T06:31:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.619+0000 7fc8cb400640 1 -- 172.21.15.49:0/952772351 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 msgr2=0x5599e2a7b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.619+0000 7fc8cb400640 1 --2- 172.21.15.49:0/952772351 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a7b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.619+0000 7fc8cb400640 1 -- 172.21.15.49:0/952772351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 msgr2=0x5599e2a876f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.619+0000 7fc8cb400640 1 --2- 172.21.15.49:0/952772351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a876f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.619+0000 7fc8cb400640 1 -- 172.21.15.49:0/952772351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e2963ad0 con 0x5599e2a7bd80 2024-02-26T06:31:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.619+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/952772351 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a7b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:31:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.621+0000 7fc8cb400640 1 --2- 172.21.15.49:0/952772351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7c200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc8b802c170 tx=0x7fc8b803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed46032d6a0b8ad server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.621+0000 7fc8c1200640 1 -- 172.21.15.49:0/952772351 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b802dea0 con 0x5599e2a7bd80 2024-02-26T06:31:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.622+0000 7fc8c1200640 1 -- 172.21.15.49:0/952772351 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8b802cc10 con 0x5599e2a7bd80 2024-02-26T06:31:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.622+0000 7fc8c1200640 1 -- 172.21.15.49:0/952772351 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80035f0 con 0x5599e2a7bd80 2024-02-26T06:31:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.623+0000 7fc8d596a740 1 -- 172.21.15.49:0/952772351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 msgr2=0x5599e2a7c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.623+0000 7fc8d596a740 1 --2- 172.21.15.49:0/952772351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7c200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc8b802c170 tx=0x7fc8b803f000 comp rx=0 tx=0).stop 2024-02-26T06:31:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.623+0000 7fc8d596a740 1 -- 172.21.15.49:0/952772351 shutdown_connections 2024-02-26T06:31:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.623+0000 7fc8d596a740 1 --2- 172.21.15.49:0/952772351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.623+0000 7fc8d596a740 1 --2- 172.21.15.49:0/952772351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7c200 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.624+0000 7fc8d596a740 1 --2- 172.21.15.49:0/952772351 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a7b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.624+0000 7fc8d596a740 1 -- 172.21.15.49:0/952772351 >> 172.21.15.49:0/952772351 conn(0x5599e2a43480 msgr2=0x5599e2a76a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.626+0000 7fc8d596a740 1 -- 172.21.15.49:0/952772351 shutdown_connections 2024-02-26T06:31:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.626+0000 7fc8d596a740 1 -- 172.21.15.49:0/952772351 wait complete. 2024-02-26T06:31:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.627+0000 7fc8d596a740 1 -- start start 2024-02-26T06:31:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.628+0000 7fc8d596a740 1 Processor -- start 2024-02-26T06:31:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a82910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a82e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d596a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e29e25a0 con 0x5599e2a85300 2024-02-26T06:31:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d596a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e2987b20 con 0x5599e2a7caf0 2024-02-26T06:31:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d596a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e29877c0 con 0x5599e2a7bd80 2024-02-26T06:31:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d596a740 10 client.0 _ll_register_callbacks cb 0x5599e299b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:31:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a82e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a82e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57690/0 (socket says 172.21.15.49:57690) 2024-02-26T06:31:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.629+0000 7fc8d0a00640 1 -- 172.21.15.49:0/3173653727 learned_addr learned my addr 172.21.15.49:0/3173653727 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.630+0000 7fc8cb400640 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a82910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.630+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.630+0000 7fc8cb400640 1 -- 172.21.15.49:0/3173653727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 msgr2=0x5599e2a823d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.630+0000 7fc8cb400640 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a823d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.630+0000 7fc8cb400640 1 -- 172.21.15.49:0/3173653727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 msgr2=0x5599e2a82e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.630+0000 7fc8cb400640 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a82e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.630+0000 7fc8cb400640 1 -- 172.21.15.49:0/3173653727 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc10e850 con 0x5599e2a7caf0 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.631+0000 7fc8cb400640 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a82910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc8b802c240 tx=0x7fc8b802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad88d1112c8d71a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.631+0000 7fc8b5600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.631+0000 7fc8b5600640 1 -- 172.21.15.49:0/3173653727 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b802dea0 con 0x5599e2a7caf0 2024-02-26T06:31:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.631+0000 7fc8b5600640 1 -- 172.21.15.49:0/3173653727 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fc8b8047050 con 0x5599e2a7caf0 2024-02-26T06:31:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.632+0000 7fc8d596a740 1 -- 172.21.15.49:0/3173653727 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599e29b4ca0 con 0x5599e2a7caf0 2024-02-26T06:31:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.633+0000 7fc8b5600640 1 -- 172.21.15.49:0/3173653727 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b802cc10 con 0x5599e2a7caf0 2024-02-26T06:31:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.635+0000 7fc8d596a740 10 client.27221 fetch_fsmap learned FSMap version 40 2024-02-26T06:31:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.635+0000 7fc8d596a740 1 -- 172.21.15.49:0/3173653727 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5599e29e0c20 con 0x5599e2a7caf0 2024-02-26T06:31:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.636+0000 7fc8b5600640 1 -- 172.21.15.49:0/3173653727 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5599e29b4ca0 con 0x5599e2a7caf0 2024-02-26T06:31:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.636+0000 7fc8b5600640 1 -- 172.21.15.49:0/3173653727 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc8b803f5e0 con 0x5599e2a7caf0 2024-02-26T06:31:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.636+0000 7fc8d596a740 10 client.27221 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:31:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.636+0000 7fc8d596a740 10 client.27221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:31:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.636+0000 7fc8d596a740 20 client.27221 populate_metadata read hostname 'smithi049' 2024-02-26T06:31:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.636+0000 7fc8d596a740 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599e2aa6020 0x5599e2aa8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.636+0000 7fc8d596a740 4 client.27221 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:31:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.636+0000 7fc8d596a740 1 -- 172.21.15.49:0/3173653727 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599e29d8cb0 con 0x5599e2aa6020 2024-02-26T06:31:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.637+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599e2aa6020 0x5599e2aa8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.637+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599e2aa6020 0x5599e2aa8440 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a289d1d40238d732 server_cookie=8a1e94ac12ed56f0 in_seq=0 out_seq=0 2024-02-26T06:31:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.638+0000 7fc8b5600640 10 client.27221 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.638+0000 7fc8b5600640 1 -- 172.21.15.49:0/3173653727 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599e29d8cb0 con 0x5599e2aa6020 2024-02-26T06:31:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.639+0000 7fc8b5600640 10 client.27221 handle_command_reply: tid=0 2024-02-26T06:31:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.639+0000 7fc8d596a740 1 client.27221 shutdown 2024-02-26T06:31:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.639+0000 7fc8d596a740 1 -- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 msgr2=0x5599e2a82910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.639+0000 7fc8d596a740 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a82910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc8b802c240 tx=0x7fc8b802bc30 comp rx=0 tx=0).stop 2024-02-26T06:31:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.641+0000 7fc8d596a740 1 -- 172.21.15.49:0/3173653727 shutdown_connections 2024-02-26T06:31:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.641+0000 7fc8d596a740 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599e2aa6020 0x5599e2aa8440 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.641+0000 7fc8d596a740 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a7caf0 0x5599e2a82910 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.641+0000 7fc8d596a740 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a85300 0x5599e2a82e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.641+0000 7fc8d596a740 1 --2- 172.21.15.49:0/3173653727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a823d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.641+0000 7fc8b5600640 0 client.27221 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.641+0000 7fc8d596a740 1 -- 172.21.15.49:0/3173653727 >> 172.21.15.49:0/3173653727 conn(0x5599e2a43480 msgr2=0x5599e2a787c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.642+0000 7fc8d596a740 1 -- 172.21.15.49:0/3173653727 shutdown_connections 2024-02-26T06:31:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.642+0000 7fc8d596a740 1 -- 172.21.15.49:0/3173653727 wait complete. 2024-02-26T06:31:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.642+0000 7fc8d596a740 20 client.27221 trim_cache size 0 max 16384 2024-02-26T06:31:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.681+0000 7fc8d596a740 1 Processor -- start 2024-02-26T06:31:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.682+0000 7fc8d596a740 1 -- start start 2024-02-26T06:31:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.682+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a84b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.682+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a7caf0 0x5599e2a8c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.682+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a8c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.682+0000 7fc8d596a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e272ae60 con 0x5599e2a7caf0 2024-02-26T06:31:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.682+0000 7fc8d596a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e2a68670 con 0x5599e2a85300 2024-02-26T06:31:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.682+0000 7fc8d596a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e2a726c0 con 0x5599e2a7bd80 2024-02-26T06:31:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.683+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.683+0000 7fc8d0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a8c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.684+0000 7fc8d0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a8c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39096/0 (socket says 172.21.15.49:39096) 2024-02-26T06:31:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.684+0000 7fc8d0a00640 1 -- 172.21.15.49:0/1365940844 learned_addr learned my addr 172.21.15.49:0/1365940844 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.688+0000 7fc8cbe00640 1 -- 172.21.15.49:0/1365940844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 msgr2=0x5599e2a8c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.688+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/1365940844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a8c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.688+0000 7fc8cbe00640 1 -- 172.21.15.49:0/1365940844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a7caf0 msgr2=0x5599e2a8c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.688+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/1365940844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a7caf0 0x5599e2a8c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.688+0000 7fc8cbe00640 1 -- 172.21.15.49:0/1365940844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e297d500 con 0x5599e2a7bd80 2024-02-26T06:31:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.690+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/1365940844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a84b00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0caff0 tx=0x7fc8bc0cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3907e519868503ab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.690+0000 7fc8b7e00640 1 -- 172.21.15.49:0/1365940844 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc09e3e0 con 0x5599e2a7bd80 2024-02-26T06:31:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.691+0000 7fc8b7e00640 1 -- 172.21.15.49:0/1365940844 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8bc0a2a30 con 0x5599e2a7bd80 2024-02-26T06:31:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.691+0000 7fc8b7e00640 1 -- 172.21.15.49:0/1365940844 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc09e0b0 con 0x5599e2a7bd80 2024-02-26T06:31:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.692+0000 7fc8d596a740 1 -- 172.21.15.49:0/1365940844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 msgr2=0x5599e2a84b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.693+0000 7fc8d596a740 1 --2- 172.21.15.49:0/1365940844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a84b00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0caff0 tx=0x7fc8bc0cd040 comp rx=0 tx=0).stop 2024-02-26T06:31:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.693+0000 7fc8d596a740 1 -- 172.21.15.49:0/1365940844 shutdown_connections 2024-02-26T06:31:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.693+0000 7fc8d596a740 1 --2- 172.21.15.49:0/1365940844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a8c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.693+0000 7fc8d596a740 1 --2- 172.21.15.49:0/1365940844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a7caf0 0x5599e2a8c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.693+0000 7fc8d596a740 1 --2- 172.21.15.49:0/1365940844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a84b00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.693+0000 7fc8d596a740 1 -- 172.21.15.49:0/1365940844 >> 172.21.15.49:0/1365940844 conn(0x5599e2a43480 msgr2=0x5599e2aacd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.693+0000 7fc8d596a740 1 -- 172.21.15.49:0/1365940844 shutdown_connections 2024-02-26T06:31:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.695+0000 7fc8d596a740 1 -- 172.21.15.49:0/1365940844 wait complete. 2024-02-26T06:31:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.695+0000 7fc8d596a740 1 -- start start 2024-02-26T06:31:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.696+0000 7fc8d596a740 1 Processor -- start 2024-02-26T06:31:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.697+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.697+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a7caf0 0x5599e2a7f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.697+0000 7fc8d596a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a7fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.697+0000 7fc8d596a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e2afc560 con 0x5599e2a7caf0 2024-02-26T06:31:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.697+0000 7fc8d596a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e2afc6d0 con 0x5599e2a85300 2024-02-26T06:31:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.697+0000 7fc8d596a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e2afc890 con 0x5599e2a7bd80 2024-02-26T06:31:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.698+0000 7fc8d596a740 10 client.0 _ll_register_callbacks cb 0x7fc8c4007340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:31:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.698+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.698+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:50930/0 (socket says 172.21.15.49:50930) 2024-02-26T06:31:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.698+0000 7fc8cbe00640 1 -- 172.21.15.49:0/141491676 learned_addr learned my addr 172.21.15.49:0/141491676 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.698+0000 7fc8d0a00640 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a7fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.699+0000 7fc8cbe00640 1 -- 172.21.15.49:0/141491676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 msgr2=0x5599e2a7fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.699+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a7fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.699+0000 7fc8cbe00640 1 -- 172.21.15.49:0/141491676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a7caf0 msgr2=0x5599e2a7f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.699+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a7caf0 0x5599e2a7f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.699+0000 7fc8cbe00640 1 -- 172.21.15.49:0/141491676 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc111280 con 0x5599e2a7bd80 2024-02-26T06:31:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.699+0000 7fc8b5600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:31:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.699+0000 7fc8cbe00640 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7f260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00c940 tx=0x7fc8bc09aa60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=985f55a654655d78 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.703+0000 7fc8b5600640 1 -- 172.21.15.49:0/141491676 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc0a6070 con 0x5599e2a7bd80 2024-02-26T06:31:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.704+0000 7fc8b5600640 1 -- 172.21.15.49:0/141491676 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fc8bc098330 con 0x5599e2a7bd80 2024-02-26T06:31:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.705+0000 7fc8d596a740 1 -- 172.21.15.49:0/141491676 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599e2a07d00 con 0x5599e2a7bd80 2024-02-26T06:31:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.705+0000 7fc8b5600640 1 -- 172.21.15.49:0/141491676 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8bc09e140 con 0x5599e2a7bd80 2024-02-26T06:31:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.706+0000 7fc8d596a740 10 client.27205 fetch_fsmap learned FSMap version 40 2024-02-26T06:31:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.706+0000 7fc8d596a740 1 -- 172.21.15.49:0/141491676 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5599e29a3290 con 0x5599e2a7bd80 2024-02-26T06:31:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.706+0000 7fc8b5600640 1 -- 172.21.15.49:0/141491676 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5599e2a07d00 con 0x5599e2a7bd80 2024-02-26T06:31:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.707+0000 7fc8b5600640 1 -- 172.21.15.49:0/141491676 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc8bc0a9030 con 0x5599e2a7bd80 2024-02-26T06:31:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.708+0000 7fc8d596a740 10 client.27205 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:31:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.708+0000 7fc8d596a740 10 client.27205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:31:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.708+0000 7fc8d596a740 20 client.27205 populate_metadata read hostname 'smithi049' 2024-02-26T06:31:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.708+0000 7fc8d596a740 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599e2afddf0 0x5599e2b00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.708+0000 7fc8d596a740 4 client.27205 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:31:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.708+0000 7fc8d596a740 1 -- 172.21.15.49:0/141491676 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599e26e2600 con 0x5599e2afddf0 2024-02-26T06:31:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.708+0000 7fc8cb400640 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599e2afddf0 0x5599e2b00210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.712+0000 7fc8b5600640 10 client.27205 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:38.712+0000 7fc8cb400640 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599e2afddf0 0x5599e2b00210 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc22c5772d1c0937 server_cookie=21a39482ee485b06 in_seq=0 out_seq=0 2024-02-26T06:31:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.123+0000 7fc8b5600640 1 -- 172.21.15.49:0/141491676 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599e26e2600 con 0x5599e2afddf0 2024-02-26T06:31:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.123+0000 7fc8b5600640 10 client.27205 handle_command_reply: tid=0 2024-02-26T06:31:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.124+0000 7fc8d596a740 1 client.27205 shutdown 2024-02-26T06:31:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.124+0000 7fc8d596a740 1 -- 172.21.15.49:0/141491676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 msgr2=0x5599e2a7f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.124+0000 7fc8d596a740 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7f260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00c940 tx=0x7fc8bc09aa60 comp rx=0 tx=0).stop 2024-02-26T06:31:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.125+0000 7fc8d596a740 1 -- 172.21.15.49:0/141491676 shutdown_connections 2024-02-26T06:31:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.125+0000 7fc8d596a740 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5599e2afddf0 0x5599e2b00210 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.125+0000 7fc8d596a740 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5599e2a85300 0x5599e2a7fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.125+0000 7fc8b5600640 0 client.27205 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.125+0000 7fc8d596a740 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599e2a7caf0 0x5599e2a7f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.125+0000 7fc8d596a740 1 --2- 172.21.15.49:0/141491676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5599e2a7bd80 0x5599e2a7f260 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.125+0000 7fc8d596a740 1 -- 172.21.15.49:0/141491676 >> 172.21.15.49:0/141491676 conn(0x5599e2a43480 msgr2=0x5599e2a80c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.126+0000 7fc8d596a740 1 -- 172.21.15.49:0/141491676 shutdown_connections 2024-02-26T06:31:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.126+0000 7fc8d596a740 1 -- 172.21.15.49:0/141491676 wait complete. 2024-02-26T06:31:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.126+0000 7fc8d596a740 20 client.27205 trim_cache size 0 max 16384 2024-02-26T06:31:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:31:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:31:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:31:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:31:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.127+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc8b00a2430 msgr2=0x7fc8b00a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.127+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc8b00a2430 0x7fc8b00a48f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8006010 tx=0x7fc8b8005a20 comp rx=0 tx=0).stop 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.127+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1172852713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8cc1013a0 msgr2=0x7fc8cc115470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.127+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8cc1013a0 0x7fc8cc115470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00c970 tx=0x7fc8bc00ce30 comp rx=0 tx=0).stop 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.128+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1172852713 shutdown_connections 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.128+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc8b00a2430 0x7fc8b00a48f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.128+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8cc115b10 0x7fc8cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.128+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8cc106470 0x7fc8cc11b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.128+0000 7fc8d2a00640 1 --2- 172.21.15.49:0/1172852713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8cc1013a0 0x7fc8cc115470 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.128+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1172852713 >> 172.21.15.49:0/1172852713 conn(0x7fc8cc0fcc40 msgr2=0x7fc8cc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.128+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1172852713 shutdown_connections 2024-02-26T06:31:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:39.129+0000 7fc8d2a00640 1 -- 172.21.15.49:0/1172852713 wait complete. 2024-02-26T06:31:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:39 smithi080 ceph-mon[36206]: pgmap v254: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 596 op/s 2024-02-26T06:31:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:39 smithi115 ceph-mon[36426]: pgmap v254: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 596 op/s 2024-02-26T06:31:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:39 smithi049 ceph-mon[31696]: pgmap v254: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 596 op/s 2024-02-26T06:31:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 470.0 s, 1313.1 tps, lat 76.809 ms stddev 134.170 2024-02-26T06:31:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:41 smithi080 ceph-mon[36206]: pgmap v255: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-02-26T06:31:41.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:41 smithi115 ceph-mon[36426]: pgmap v255: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-02-26T06:31:41.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:41 smithi049 ceph-mon[31696]: pgmap v255: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-02-26T06:31:43.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:43 smithi080 ceph-mon[36206]: pgmap v256: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.70k op/s 2024-02-26T06:31:43.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:43 smithi115 ceph-mon[36426]: pgmap v256: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.70k op/s 2024-02-26T06:31:43.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:43 smithi049 ceph-mon[31696]: pgmap v256: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.70k op/s 2024-02-26T06:31:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:44 smithi080 ceph-mon[36206]: pgmap v257: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 18 MiB/s wr, 1.55k op/s 2024-02-26T06:31:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:44 smithi115 ceph-mon[36426]: pgmap v257: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 18 MiB/s wr, 1.55k op/s 2024-02-26T06:31:44.672 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:44 smithi049 ceph-mon[31696]: pgmap v257: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 18 MiB/s wr, 1.55k op/s 2024-02-26T06:31:46.304 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 475.0 s, 1773.5 tps, lat 56.243 ms stddev 64.803 2024-02-26T06:31:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:46 smithi049 ceph-mon[31696]: pgmap v258: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-26T06:31:47.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:46 smithi080 ceph-mon[36206]: pgmap v258: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-26T06:31:47.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:46 smithi115 ceph-mon[36426]: pgmap v258: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-26T06:31:47.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:47 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:31:48.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:47 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:31:48.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:47 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:31:48.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:48 smithi049 ceph-mon[31696]: pgmap v259: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-02-26T06:31:48.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:48 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:31:48.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:48 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:31:48.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:48 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:31:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:48 smithi080 ceph-mon[36206]: pgmap v259: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-02-26T06:31:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:48 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:31:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:48 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:31:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:48 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:31:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:48 smithi115 ceph-mon[36426]: pgmap v259: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-02-26T06:31:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:48 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:31:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:48 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:31:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:48 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:31:50.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:50 smithi049 ceph-mon[31696]: pgmap v260: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2024-02-26T06:31:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:50 smithi080 ceph-mon[36206]: pgmap v260: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2024-02-26T06:31:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:50 smithi115 ceph-mon[36426]: pgmap v260: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2024-02-26T06:31:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 480.0 s, 1904.4 tps, lat 52.335 ms stddev 61.432 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 -- 172.21.15.49:0/3392369624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65501035a0 msgr2=0x7f6550103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 --2- 172.21.15.49:0/3392369624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65501035a0 0x7f6550103980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6540009890 tx=0x7f654002bb70 comp rx=0 tx=0).stop 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 -- 172.21.15.49:0/3392369624 shutdown_connections 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 --2- 172.21.15.49:0/3392369624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6550108670 0x7f6550110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 --2- 172.21.15.49:0/3392369624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6550103f50 0x7f6550107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 --2- 172.21.15.49:0/3392369624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65501035a0 0x7f6550103980 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 -- 172.21.15.49:0/3392369624 >> 172.21.15.49:0/3392369624 conn(0x7f65500fee00 msgr2=0x7f6550101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 -- 172.21.15.49:0/3392369624 shutdown_connections 2024-02-26T06:31:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.305+0000 7f6557200640 1 -- 172.21.15.49:0/3392369624 wait complete. 2024-02-26T06:31:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.306+0000 7f6557200640 1 Processor -- start 2024-02-26T06:31:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.306+0000 7f6557200640 1 -- start start 2024-02-26T06:31:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.307+0000 7f6557200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65501035a0 0x7f655019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.307+0000 7f6557200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6550108670 0x7f655019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.307+0000 7f6557200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550199790 0x7f6550199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.307+0000 7f6557200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655019a240 con 0x7f6550199790 2024-02-26T06:31:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.307+0000 7f6557200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655019a380 con 0x7f65501035a0 2024-02-26T06:31:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.307+0000 7f6557200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655019a540 con 0x7f6550108670 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.308+0000 7f654fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65501035a0 0x7f655019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.308+0000 7f654fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65501035a0 0x7f655019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49302/0 (socket says 172.21.15.49:49302) 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.308+0000 7f654fe00640 1 -- 172.21.15.49:0/3607557475 learned_addr learned my addr 172.21.15.49:0/3607557475 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.308+0000 7f6555200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550199790 0x7f6550199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.309+0000 7f654fe00640 1 -- 172.21.15.49:0/3607557475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6550108670 msgr2=0x7f655019fb60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.309+0000 7f654fe00640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6550108670 0x7f655019fb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.309+0000 7f654fe00640 1 -- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550199790 msgr2=0x7f6550199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.309+0000 7f654fe00640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550199790 0x7f6550199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.309+0000 7f654fe00640 1 -- 172.21.15.49:0/3607557475 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540009490 con 0x7f65501035a0 2024-02-26T06:31:52.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.314+0000 7f654fe00640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65501035a0 0x7f655019f620 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6540000c00 tx=0x7f6540004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:52.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.315+0000 7f654d600640 1 -- 172.21.15.49:0/3607557475 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540034950 con 0x7f65501035a0 2024-02-26T06:31:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.315+0000 7f654d600640 1 -- 172.21.15.49:0/3607557475 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f654002c9f0 con 0x7f65501035a0 2024-02-26T06:31:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.315+0000 7f654d600640 1 -- 172.21.15.49:0/3607557475 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540032600 con 0x7f65501035a0 2024-02-26T06:31:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.315+0000 7f6557200640 1 -- 172.21.15.49:0/3607557475 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655006aba0 con 0x7f65501035a0 2024-02-26T06:31:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.316+0000 7f6557200640 1 -- 172.21.15.49:0/3607557475 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655006afd0 con 0x7f65501035a0 2024-02-26T06:31:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.323+0000 7f654d600640 1 -- 172.21.15.49:0/3607557475 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f65400035f0 con 0x7f65501035a0 2024-02-26T06:31:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.327+0000 7f655a180740 1 Processor -- start 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.328+0000 7f655a180740 1 -- start start 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.328+0000 7f655a180740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c8300 0x55b2268c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.328+0000 7f655a180740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c7590 0x55b2268c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.328+0000 7f655a180740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.329+0000 7f655a180740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22687a5f0 con 0x55b2268d0b10 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.329+0000 7f655a180740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22682c890 con 0x55b2268c8300 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.329+0000 7f655a180740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226829dc0 con 0x55b2268c7590 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.329+0000 7f654d600640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65340a23d0 0x7f65340a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.329+0000 7f6547e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.330+0000 7f654d600640 1 -- 172.21.15.49:0/3607557475 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6540111db0 con 0x7f65501035a0 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.331+0000 7f6555200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c8300 0x55b2268c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.331+0000 7f6555200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c8300 0x55b2268c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49314/0 (socket says 172.21.15.49:49314) 2024-02-26T06:31:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.331+0000 7f6555200640 1 -- 172.21.15.49:0/2237694384 learned_addr learned my addr 172.21.15.49:0/2237694384 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.331+0000 7f6555200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65340a23d0 0x7f65340a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.332+0000 7f6555200640 1 -- 172.21.15.49:0/2237694384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c7590 msgr2=0x55b2268c7a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:31:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.332+0000 7f6555200640 1 --2- 172.21.15.49:0/2237694384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c7590 0x55b2268c7a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.332+0000 7f6555200640 1 -- 172.21.15.49:0/2237694384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 msgr2=0x55b2268d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.332+0000 7f6555200640 1 --2- 172.21.15.49:0/2237694384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.332+0000 7f6555200640 1 -- 172.21.15.49:0/2237694384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2267a6ad0 con 0x55b2268c8300 2024-02-26T06:31:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.333+0000 7f6547e00640 1 --2- 172.21.15.49:0/2237694384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:31:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.334+0000 7f6555200640 1 --2- 172.21.15.49:0/2237694384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c8300 0x55b2268c7050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6548005f40 tx=0x7f654800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c83e39edbd8a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.334+0000 7f6555200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65340a23d0 0x7f65340a4890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f654800fc90 tx=0x7f6548018000 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.338+0000 7f6545600640 1 -- 172.21.15.49:0/2237694384 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654800ec50 con 0x55b2268c8300 2024-02-26T06:31:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.338+0000 7f6545600640 1 -- 172.21.15.49:0/2237694384 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6548009dd0 con 0x55b2268c8300 2024-02-26T06:31:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.338+0000 7f6545600640 1 -- 172.21.15.49:0/2237694384 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65480042b0 con 0x55b2268c8300 2024-02-26T06:31:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.339+0000 7f655a180740 1 -- 172.21.15.49:0/2237694384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c8300 msgr2=0x55b2268c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.339+0000 7f655a180740 1 --2- 172.21.15.49:0/2237694384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c8300 0x55b2268c7050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6548005f40 tx=0x7f654800f790 comp rx=0 tx=0).stop 2024-02-26T06:31:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.340+0000 7f655a180740 1 -- 172.21.15.49:0/2237694384 shutdown_connections 2024-02-26T06:31:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.340+0000 7f655a180740 1 --2- 172.21.15.49:0/2237694384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.340+0000 7f655a180740 1 --2- 172.21.15.49:0/2237694384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c7590 0x55b2268c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.340+0000 7f655a180740 1 --2- 172.21.15.49:0/2237694384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c8300 0x55b2268c7050 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.340+0000 7f655a180740 1 -- 172.21.15.49:0/2237694384 >> 172.21.15.49:0/2237694384 conn(0x55b226886480 msgr2=0x55b2268c2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.341+0000 7f655a180740 1 -- 172.21.15.49:0/2237694384 shutdown_connections 2024-02-26T06:31:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.344+0000 7f655a180740 1 -- 172.21.15.49:0/2237694384 wait complete. 2024-02-26T06:31:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.345+0000 7f655a180740 1 -- start start 2024-02-26T06:31:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.345+0000 7f655a180740 1 Processor -- start 2024-02-26T06:31:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.347+0000 7f655a180740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.347+0000 7f655a180740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.347+0000 7f655a180740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f655a180740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2268255a0 con 0x55b2268c8300 2024-02-26T06:31:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f655a180740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2267cab20 con 0x55b2268c7590 2024-02-26T06:31:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f655a180740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2267ca7c0 con 0x55b2268d0b10 2024-02-26T06:31:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f6555200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f6547e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f654fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f654fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49316/0 (socket says 172.21.15.49:49316) 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f654fe00640 1 -- 172.21.15.49:0/945500951 learned_addr learned my addr 172.21.15.49:0/945500951 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f6547e00640 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60076/0 (socket says 172.21.15.49:60076) 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f6555200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49190/0 (socket says 172.21.15.49:49190) 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.348+0000 7f655a180740 10 client.0 _ll_register_callbacks cb 0x55b2267de520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.349+0000 7f654fe00640 1 -- 172.21.15.49:0/945500951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 msgr2=0x55b2268ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.349+0000 7f654fe00640 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.349+0000 7f654fe00640 1 -- 172.21.15.49:0/945500951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 msgr2=0x55b2268ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.349+0000 7f654fe00640 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.349+0000 7f654fe00640 1 -- 172.21.15.49:0/945500951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6548019040 con 0x55b2268c7590 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.349+0000 7f6547e00640 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:31:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.349+0000 7f655a180740 1 -- 172.21.15.49:0/945500951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2267f7ca0 con 0x55b2268c7590 2024-02-26T06:31:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.350+0000 7f654fe00640 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268cdb80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f65400de530 tx=0x7f65400daad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5475a0dd1b52f41d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.350+0000 7f653e000640 10 client.27239 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:31:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.351+0000 7f653e000640 1 -- 172.21.15.49:0/945500951 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654010f900 con 0x55b2268c7590 2024-02-26T06:31:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.351+0000 7f653e000640 1 -- 172.21.15.49:0/945500951 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f65400def30 con 0x55b2268c7590 2024-02-26T06:31:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.352+0000 7f653e000640 1 -- 172.21.15.49:0/945500951 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65400de830 con 0x55b2268c7590 2024-02-26T06:31:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.352+0000 7f655a180740 10 client.27239 fetch_fsmap learned FSMap version 40 2024-02-26T06:31:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.352+0000 7f655a180740 1 -- 172.21.15.49:0/945500951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({fsmap=40}) v3 -- 0x55b226823c20 con 0x55b2268c7590 2024-02-26T06:31:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.353+0000 7f653e000640 1 -- 172.21.15.49:0/945500951 <== mon.2 v2:172.21.15.80:3300/0 4 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f65400dea90 con 0x55b2268c7590 2024-02-26T06:31:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.353+0000 7f655a180740 10 client.27239 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:31:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.353+0000 7f655a180740 10 client.27239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:31:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.353+0000 7f655a180740 20 client.27239 populate_metadata read hostname 'smithi049' 2024-02-26T06:31:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.353+0000 7f655a180740 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2268e05b0 0x55b2268e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.353+0000 7f655a180740 4 client.27239 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:31:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.353+0000 7f655a180740 1 -- 172.21.15.49:0/945500951 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b22681bcb0 con 0x55b2268e05b0 2024-02-26T06:31:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.356+0000 7f6547e00640 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2268e05b0 0x55b2268e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.359+0000 7f6547e00640 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2268e05b0 0x55b2268e29c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ce3dde1548d40b server_cookie=bedcb21f9af4ae38 in_seq=0 out_seq=0 2024-02-26T06:31:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.359+0000 7f653e000640 10 client.27239 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.360+0000 7f653e000640 1 -- 172.21.15.49:0/945500951 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b22681bcb0 con 0x55b2268e05b0 2024-02-26T06:31:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.360+0000 7f653e000640 10 client.27239 handle_command_reply: tid=0 2024-02-26T06:31:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.360+0000 7f655a180740 1 client.27239 shutdown 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f655a180740 1 -- 172.21.15.49:0/945500951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 msgr2=0x55b2268cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f655a180740 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268cdb80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f65400de530 tx=0x7f65400daad0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f655a180740 1 -- 172.21.15.49:0/945500951 shutdown_connections 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f655a180740 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b2268e05b0 0x55b2268e29c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f655a180740 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f655a180740 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f655a180740 1 --2- 172.21.15.49:0/945500951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268cdb80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f655a180740 1 -- 172.21.15.49:0/945500951 >> 172.21.15.49:0/945500951 conn(0x55b226886480 msgr2=0x55b2268c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.361+0000 7f653e000640 0 client.27239 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.362+0000 7f655a180740 1 -- 172.21.15.49:0/945500951 shutdown_connections 2024-02-26T06:31:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.362+0000 7f655a180740 1 -- 172.21.15.49:0/945500951 wait complete. 2024-02-26T06:31:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.362+0000 7f655a180740 20 client.27239 trim_cache size 0 max 16384 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f655a180740 1 Processor -- start 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f655a180740 1 -- start start 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f655a180740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f655a180740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f655a180740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f655a180740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22656de60 con 0x55b2268c8300 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f655a180740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2268c8a60 con 0x55b2268c7590 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f655a180740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2268dd670 con 0x55b2268d0b10 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.398+0000 7f6547e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6555200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f654fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6547e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268d7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60086/0 (socket says 172.21.15.49:60086) 2024-02-26T06:31:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6547e00640 1 -- 172.21.15.49:0/820307605 learned_addr learned my addr 172.21.15.49:0/820307605 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6547e00640 1 -- 172.21.15.49:0/820307605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 msgr2=0x55b2268cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6547e00640 1 --2- 172.21.15.49:0/820307605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6547e00640 1 -- 172.21.15.49:0/820307605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 msgr2=0x55b2268d7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6547e00640 1 --2- 172.21.15.49:0/820307605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268d7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6547e00640 1 -- 172.21.15.49:0/820307605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2267c0500 con 0x55b2268c8300 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6547e00640 1 --2- 172.21.15.49:0/820307605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268d7ec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f655019aae0 tx=0x7f65380056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0fd02dbe1d8d03 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6544c00640 1 -- 172.21.15.49:0/820307605 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538016040 con 0x55b2268c8300 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6544c00640 1 -- 172.21.15.49:0/820307605 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6538020070 con 0x55b2268c8300 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.399+0000 7f6544c00640 1 -- 172.21.15.49:0/820307605 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538003850 con 0x55b2268c8300 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 -- 172.21.15.49:0/820307605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 msgr2=0x55b2268d7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 --2- 172.21.15.49:0/820307605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268d7ec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f655019aae0 tx=0x7f65380056e0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 -- 172.21.15.49:0/820307605 shutdown_connections 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 --2- 172.21.15.49:0/820307605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268d0b10 0x55b2268cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 --2- 172.21.15.49:0/820307605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268c8300 0x55b2268d7ec0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 --2- 172.21.15.49:0/820307605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 -- 172.21.15.49:0/820307605 >> 172.21.15.49:0/820307605 conn(0x55b226886480 msgr2=0x55b2268c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 -- 172.21.15.49:0/820307605 shutdown_connections 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.400+0000 7f655a180740 1 -- 172.21.15.49:0/820307605 wait complete. 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 1 -- start start 2024-02-26T06:31:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 1 Processor -- start 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c8300 0x55b2268d6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226936af0 con 0x55b2268d0b10 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226936c30 con 0x55b2268c7590 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226936da0 con 0x55b2268c8300 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.401+0000 7f655a180740 10 client.0 _ll_register_callbacks cb 0x7f65400038a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6547e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c8300 0x55b2268d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6547e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c8300 0x55b2268d6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49208/0 (socket says 172.21.15.49:49208) 2024-02-26T06:31:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6547e00640 1 -- 172.21.15.49:0/3476792849 learned_addr learned my addr 172.21.15.49:0/3476792849 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6555200640 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6547e00640 1 -- 172.21.15.49:0/3476792849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 msgr2=0x55b2268d63e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6547e00640 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268d63e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6547e00640 1 -- 172.21.15.49:0/3476792849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 msgr2=0x55b2268d6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6547e00640 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6547e00640 1 -- 172.21.15.49:0/3476792849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6538005040 con 0x55b2268c8300 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.402+0000 7f6555200640 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.403+0000 7f6547e00640 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c8300 0x55b2268d6920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6538005e90 tx=0x7f6538004e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9adf54840cd0648f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:31:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.403+0000 7f653e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:31:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.404+0000 7f653e000640 1 -- 172.21.15.49:0/3476792849 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538016040 con 0x55b2268c8300 2024-02-26T06:31:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.404+0000 7f653e000640 1 -- 172.21.15.49:0/3476792849 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f653801c690 con 0x55b2268c8300 2024-02-26T06:31:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.404+0000 7f655a180740 1 -- 172.21.15.49:0/3476792849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b22684ad00 con 0x55b2268c8300 2024-02-26T06:31:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.404+0000 7f653e000640 1 -- 172.21.15.49:0/3476792849 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f653801c850 con 0x55b2268c8300 2024-02-26T06:31:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f655a180740 10 client.27217 fetch_fsmap learned FSMap version 40 2024-02-26T06:31:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f655a180740 1 -- 172.21.15.49:0/3476792849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b2267e6290 con 0x55b2268c8300 2024-02-26T06:31:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f653e000640 1 -- 172.21.15.49:0/3476792849 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f653803a080 con 0x55b2268c8300 2024-02-26T06:31:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f653e000640 1 -- 172.21.15.49:0/3476792849 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6538011440 con 0x55b2268c8300 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f655a180740 10 client.27217 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f655a180740 10 client.27217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f655a180740 20 client.27217 populate_metadata read hostname 'smithi049' 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f655a180740 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b226938530 0x55b22693a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f655a180740 4 client.27217 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.405+0000 7f655a180740 1 -- 172.21.15.49:0/3476792849 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b226525600 con 0x55b226938530 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.406+0000 7f654fe00640 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b226938530 0x55b22693a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.406+0000 7f654fe00640 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b226938530 0x55b22693a950 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29bd541bf2fe403 server_cookie=f8b6975b543f6434 in_seq=0 out_seq=0 2024-02-26T06:31:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.406+0000 7f653e000640 10 client.27217 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.420+0000 7f653e000640 1 -- 172.21.15.49:0/3476792849 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b226525600 con 0x55b226938530 2024-02-26T06:31:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.421+0000 7f653e000640 10 client.27217 handle_command_reply: tid=0 2024-02-26T06:31:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.421+0000 7f655a180740 1 client.27217 shutdown 2024-02-26T06:31:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.422+0000 7f655a180740 1 -- 172.21.15.49:0/3476792849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c8300 msgr2=0x55b2268d6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.422+0000 7f655a180740 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c8300 0x55b2268d6920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6538005e90 tx=0x7f6538004e70 comp rx=0 tx=0).stop 2024-02-26T06:31:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.423+0000 7f655a180740 1 -- 172.21.15.49:0/3476792849 shutdown_connections 2024-02-26T06:31:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.423+0000 7f655a180740 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b226938530 0x55b22693a950 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.424+0000 7f655a180740 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2268d0b10 0x55b2268d6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.424+0000 7f655a180740 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2268c8300 0x55b2268d6920 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.424+0000 7f655a180740 1 --2- 172.21.15.49:0/3476792849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b2268c7590 0x55b2268d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.424+0000 7f655a180740 1 -- 172.21.15.49:0/3476792849 >> 172.21.15.49:0/3476792849 conn(0x55b226886480 msgr2=0x55b2268cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.424+0000 7f653e000640 0 client.27217 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:31:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.424+0000 7f655a180740 1 -- 172.21.15.49:0/3476792849 shutdown_connections 2024-02-26T06:31:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.424+0000 7f655a180740 1 -- 172.21.15.49:0/3476792849 wait complete. 2024-02-26T06:31:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.424+0000 7f655a180740 20 client.27217 trim_cache size 0 max 16384 2024-02-26T06:31:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:31:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:31:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:31:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:31:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.426+0000 7f6557200640 1 -- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65340a23d0 msgr2=0x7f65340a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.426+0000 7f6557200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65340a23d0 0x7f65340a4890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f654800fc90 tx=0x7f6548018000 comp rx=0 tx=0).stop 2024-02-26T06:31:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.426+0000 7f6557200640 1 -- 172.21.15.49:0/3607557475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65501035a0 msgr2=0x7f655019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:31:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.426+0000 7f6557200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65501035a0 0x7f655019f620 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6540000c00 tx=0x7f6540004530 comp rx=0 tx=0).stop 2024-02-26T06:31:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.427+0000 7f6557200640 1 -- 172.21.15.49:0/3607557475 shutdown_connections 2024-02-26T06:31:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.427+0000 7f6557200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65340a23d0 0x7f65340a4890 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.427+0000 7f6557200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550199790 0x7f6550199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.427+0000 7f6557200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6550108670 0x7f655019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.427+0000 7f6557200640 1 --2- 172.21.15.49:0/3607557475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65501035a0 0x7f655019f620 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:31:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.427+0000 7f6557200640 1 -- 172.21.15.49:0/3607557475 >> 172.21.15.49:0/3607557475 conn(0x7f65500fee00 msgr2=0x7f6550101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:31:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.427+0000 7f6557200640 1 -- 172.21.15.49:0/3607557475 shutdown_connections 2024-02-26T06:31:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:31:52.427+0000 7f6557200640 1 -- 172.21.15.49:0/3607557475 wait complete. 2024-02-26T06:31:52.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:52 smithi115 ceph-mon[36426]: pgmap v261: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.24k op/s 2024-02-26T06:31:52.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:52 smithi049 ceph-mon[31696]: pgmap v261: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.24k op/s 2024-02-26T06:31:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:52 smithi080 ceph-mon[36206]: pgmap v261: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.24k op/s 2024-02-26T06:31:54.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:54 smithi049 ceph-mon[31696]: pgmap v262: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 516 op/s 2024-02-26T06:31:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:54 smithi080 ceph-mon[36206]: pgmap v262: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 516 op/s 2024-02-26T06:31:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:54 smithi115 ceph-mon[36426]: pgmap v262: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 516 op/s 2024-02-26T06:31:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 485.0 s, 1910.0 tps, lat 52.131 ms stddev 71.679 2024-02-26T06:31:56.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:56 smithi049 ceph-mon[31696]: pgmap v263: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.6 MiB/s wr, 770 op/s 2024-02-26T06:31:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:56 smithi080 ceph-mon[36206]: pgmap v263: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.6 MiB/s wr, 770 op/s 2024-02-26T06:31:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:56 smithi115 ceph-mon[36426]: pgmap v263: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.6 MiB/s wr, 770 op/s 2024-02-26T06:31:58.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:31:58 smithi049 ceph-mon[31696]: pgmap v264: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 610 op/s 2024-02-26T06:31:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:31:58 smithi080 ceph-mon[36206]: pgmap v264: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 610 op/s 2024-02-26T06:31:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:31:58 smithi115 ceph-mon[36426]: pgmap v264: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 610 op/s 2024-02-26T06:32:00.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:00 smithi049 ceph-mon[31696]: pgmap v265: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 677 op/s 2024-02-26T06:32:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:00 smithi080 ceph-mon[36206]: pgmap v265: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 677 op/s 2024-02-26T06:32:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:00 smithi115 ceph-mon[36426]: pgmap v265: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 677 op/s 2024-02-26T06:32:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 490.0 s, 1202.8 tps, lat 79.469 ms stddev 93.465 2024-02-26T06:32:02.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:02 smithi049 ceph-mon[31696]: pgmap v266: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 687 op/s 2024-02-26T06:32:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:02 smithi080 ceph-mon[36206]: pgmap v266: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 687 op/s 2024-02-26T06:32:03.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:02 smithi115 ceph-mon[36426]: pgmap v266: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 687 op/s 2024-02-26T06:32:04.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:04 smithi049 ceph-mon[31696]: pgmap v267: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 571 op/s 2024-02-26T06:32:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:04 smithi080 ceph-mon[36206]: pgmap v267: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 571 op/s 2024-02-26T06:32:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:04 smithi115 ceph-mon[36426]: pgmap v267: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 571 op/s 2024-02-26T06:32:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.613+0000 7fc144400640 1 -- 172.21.15.49:0/574572716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c1001e0 msgr2=0x7fc13c1005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.613+0000 7fc144400640 1 --2- 172.21.15.49:0/574572716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c1001e0 0x7fc13c1005c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc130009900 tx=0x7fc13002bd60 comp rx=0 tx=0).stop 2024-02-26T06:32:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.613+0000 7fc144400640 1 -- 172.21.15.49:0/574572716 shutdown_connections 2024-02-26T06:32:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.613+0000 7fc144400640 1 --2- 172.21.15.49:0/574572716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc13c100b00 0x7fc13c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.613+0000 7fc144400640 1 --2- 172.21.15.49:0/574572716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc13c105500 0x7fc13c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.613+0000 7fc144400640 1 --2- 172.21.15.49:0/574572716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c1001e0 0x7fc13c1005c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.613+0000 7fc144400640 1 -- 172.21.15.49:0/574572716 >> 172.21.15.49:0/574572716 conn(0x7fc13c0771a0 msgr2=0x7fc13c0775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.613+0000 7fc144400640 1 -- 172.21.15.49:0/574572716 shutdown_connections 2024-02-26T06:32:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.614+0000 7fc144400640 1 -- 172.21.15.49:0/574572716 wait complete. 2024-02-26T06:32:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.615+0000 7fc144400640 1 Processor -- start 2024-02-26T06:32:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.616+0000 7fc144400640 1 -- start start 2024-02-26T06:32:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc144400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc13c1001e0 0x7fc13c1a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc144400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc13c100b00 0x7fc13c1a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc144400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c105500 0x7fc13c19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc144400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c19e160 con 0x7fc13c1001e0 2024-02-26T06:32:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc144400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c19e2d0 con 0x7fc13c100b00 2024-02-26T06:32:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc144400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c19e490 con 0x7fc13c105500 2024-02-26T06:32:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc142400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c105500 0x7fc13c19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc141000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc13c100b00 0x7fc13c1a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc142400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c105500 0x7fc13c19db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47568/0 (socket says 172.21.15.49:47568) 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc142400640 1 -- 172.21.15.49:0/1854514246 learned_addr learned my addr 172.21.15.49:0/1854514246 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.617+0000 7fc141a00640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc13c1001e0 0x7fc13c1a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc141000640 1 -- 172.21.15.49:0/1854514246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c105500 msgr2=0x7fc13c19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc141000640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c105500 0x7fc13c19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc141000640 1 -- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc13c1001e0 msgr2=0x7fc13c1a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc141000640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc13c1001e0 0x7fc13c1a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc141000640 1 -- 172.21.15.49:0/1854514246 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc130009560 con 0x7fc13c100b00 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc142400640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c105500 0x7fc13c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc141000640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc13c100b00 0x7fc13c1a3e50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc12800e9f0 tx=0x7fc12800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc13a000640 1 -- 172.21.15.49:0/1854514246 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12800ccc0 con 0x7fc13c100b00 2024-02-26T06:32:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.618+0000 7fc144400640 1 -- 172.21.15.49:0/1854514246 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c19e770 con 0x7fc13c100b00 2024-02-26T06:32:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.619+0000 7fc144400640 1 -- 172.21.15.49:0/1854514246 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c1aa9b0 con 0x7fc13c100b00 2024-02-26T06:32:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.619+0000 7fc13a000640 1 -- 172.21.15.49:0/1854514246 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc12800ce20 con 0x7fc13c100b00 2024-02-26T06:32:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.619+0000 7fc13a000640 1 -- 172.21.15.49:0/1854514246 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128018920 con 0x7fc13c100b00 2024-02-26T06:32:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.623+0000 7fc147201740 1 Processor -- start 2024-02-26T06:32:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.623+0000 7fc147201740 1 -- start start 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc147201740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebc9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc147201740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc147201740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd3300 0x55f1aebd56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc147201740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aeb855f0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc147201740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aeb37890 con 0x55f1aebc9d80 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc147201740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aeb34dc0 con 0x55f1aebd3300 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc142400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebc9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc142400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebc9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34700/0 (socket says 172.21.15.49:34700) 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc142400640 1 -- 172.21.15.49:0/4110883162 learned_addr learned my addr 172.21.15.49:0/4110883162 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.624+0000 7fc141a00640 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.625+0000 7fc13a000640 1 -- 172.21.15.49:0/1854514246 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc128018a80 con 0x7fc13c100b00 2024-02-26T06:32:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.625+0000 7fc142400640 1 -- 172.21.15.49:0/4110883162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd3300 msgr2=0x55f1aebd56f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:32:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.625+0000 7fc142400640 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd3300 0x55f1aebd56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.625+0000 7fc142400640 1 -- 172.21.15.49:0/4110883162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 msgr2=0x55f1aebca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.625+0000 7fc142400640 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.625+0000 7fc142400640 1 -- 172.21.15.49:0/4110883162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1aeab1ad0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.625+0000 7fc141a00640 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:32:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.625+0000 7fc142400640 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebc9840 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc134011460 tx=0x7fc134011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04219bda64a1ada server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.626+0000 7fc12ea00640 1 -- 172.21.15.49:0/4110883162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134012770 con 0x55f1aebcaaf0 2024-02-26T06:32:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.626+0000 7fc12ea00640 1 -- 172.21.15.49:0/4110883162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc134012db0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.626+0000 7fc12ea00640 1 -- 172.21.15.49:0/4110883162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13400f3c0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.627+0000 7fc13a000640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc1240a2350 0x7fc1240a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc147201740 1 -- 172.21.15.49:0/4110883162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 msgr2=0x55f1aebc9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc147201740 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebc9840 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc134011460 tx=0x7fc134011920 comp rx=0 tx=0).stop 2024-02-26T06:32:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc147201740 1 -- 172.21.15.49:0/4110883162 shutdown_connections 2024-02-26T06:32:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc147201740 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc147201740 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd3300 0x55f1aebd56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc147201740 1 --2- 172.21.15.49:0/4110883162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebc9840 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc147201740 1 -- 172.21.15.49:0/4110883162 >> 172.21.15.49:0/4110883162 conn(0x55f1aeb91480 msgr2=0x55f1aebc4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc13a000640 1 -- 172.21.15.49:0/1854514246 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fc128014070 con 0x7fc13c100b00 2024-02-26T06:32:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc147201740 1 -- 172.21.15.49:0/4110883162 shutdown_connections 2024-02-26T06:32:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.628+0000 7fc141a00640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc1240a2350 0x7fc1240a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.629+0000 7fc147201740 1 -- 172.21.15.49:0/4110883162 wait complete. 2024-02-26T06:32:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.629+0000 7fc147201740 1 -- start start 2024-02-26T06:32:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc147201740 1 Processor -- start 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc141a00640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc1240a2350 0x7fc1240a4810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc13c077640 tx=0x7fc13002e040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc147201740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc147201740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc147201740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebd0eb0 0x55f1aebdf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc147201740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aeb305a0 con 0x55f1aebd0eb0 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc147201740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aead5b20 con 0x55f1aebcaaf0 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc147201740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aead57c0 con 0x55f1aebc9d80 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.630+0000 7fc147201740 10 client.0 _ll_register_callbacks cb 0x55f1aeae9520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.631+0000 7fc141000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.631+0000 7fc141000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34098/0 (socket says 172.21.15.49:34098) 2024-02-26T06:32:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.631+0000 7fc141000640 1 -- 172.21.15.49:0/2750507814 learned_addr learned my addr 172.21.15.49:0/2750507814 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.631+0000 7fc142400640 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebd0eb0 0x55f1aebdf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.631+0000 7fc141a00640 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.632+0000 7fc141000640 1 -- 172.21.15.49:0/2750507814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 msgr2=0x55f1aebd0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.632+0000 7fc141000640 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.632+0000 7fc141000640 1 -- 172.21.15.49:0/2750507814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebd0eb0 msgr2=0x55f1aebdf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.632+0000 7fc141000640 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebd0eb0 0x55f1aebdf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.632+0000 7fc141000640 1 -- 172.21.15.49:0/2750507814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc134009c90 con 0x55f1aebcaaf0 2024-02-26T06:32:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.633+0000 7fc141000640 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd0970 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc12800e9c0 tx=0x7fc12809a790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f0c898da435f0e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.634+0000 7fc123400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:32:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.634+0000 7fc123400640 1 -- 172.21.15.49:0/2750507814 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12809a390 con 0x55f1aebcaaf0 2024-02-26T06:32:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.634+0000 7fc123400640 1 -- 172.21.15.49:0/2750507814 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fc1280a34e0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.634+0000 7fc147201740 1 -- 172.21.15.49:0/2750507814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1aeb02ca0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.635+0000 7fc123400640 1 -- 172.21.15.49:0/2750507814 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc12809a4f0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.635+0000 7fc147201740 10 client.27254 fetch_fsmap learned FSMap version 40 2024-02-26T06:32:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.635+0000 7fc147201740 1 -- 172.21.15.49:0/2750507814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f1aeb2ec20 con 0x55f1aebcaaf0 2024-02-26T06:32:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.635+0000 7fc123400640 1 -- 172.21.15.49:0/2750507814 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f1aeb02ca0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.636+0000 7fc123400640 1 -- 172.21.15.49:0/2750507814 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc12809fdb0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.636+0000 7fc147201740 10 client.27254 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:32:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.636+0000 7fc147201740 10 client.27254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:32:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.636+0000 7fc147201740 20 client.27254 populate_metadata read hostname 'smithi049' 2024-02-26T06:32:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.636+0000 7fc147201740 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f1aebe3990 0x55f1aebe5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.636+0000 7fc147201740 4 client.27254 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:32:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.636+0000 7fc147201740 1 -- 172.21.15.49:0/2750507814 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1aeb26cb0 con 0x55f1aebe3990 2024-02-26T06:32:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.637+0000 7fc141a00640 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f1aebe3990 0x55f1aebe5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.637+0000 7fc141a00640 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f1aebe3990 0x55f1aebe5db0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b796051055f7b1c6 server_cookie=5501d123808cfc8e in_seq=0 out_seq=0 2024-02-26T06:32:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.637+0000 7fc123400640 10 client.27254 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.638+0000 7fc123400640 1 -- 172.21.15.49:0/2750507814 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1aeb26cb0 con 0x55f1aebe3990 2024-02-26T06:32:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.638+0000 7fc123400640 10 client.27254 handle_command_reply: tid=0 2024-02-26T06:32:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.638+0000 7fc147201740 1 client.27254 shutdown 2024-02-26T06:32:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.638+0000 7fc147201740 1 -- 172.21.15.49:0/2750507814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebcaaf0 msgr2=0x55f1aebd0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.638+0000 7fc147201740 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd0970 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc12800e9c0 tx=0x7fc12809a790 comp rx=0 tx=0).stop 2024-02-26T06:32:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 1 -- 172.21.15.49:0/2750507814 shutdown_connections 2024-02-26T06:32:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f1aebe3990 0x55f1aebe5db0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd0970 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebd0eb0 0x55f1aebdf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc123400640 0 client.27254 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 1 --2- 172.21.15.49:0/2750507814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 1 -- 172.21.15.49:0/2750507814 >> 172.21.15.49:0/2750507814 conn(0x55f1aeb91480 msgr2=0x55f1aebc5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 1 -- 172.21.15.49:0/2750507814 shutdown_connections 2024-02-26T06:32:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 1 -- 172.21.15.49:0/2750507814 wait complete. 2024-02-26T06:32:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.639+0000 7fc147201740 20 client.27254 trim_cache size 0 max 16384 2024-02-26T06:32:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc147201740 1 Processor -- start 2024-02-26T06:32:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc147201740 1 -- start start 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc147201740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebc3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc147201740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebcd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc147201740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd0eb0 0x55f1aebcdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc147201740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ae878e60 con 0x55f1aebcaaf0 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc147201740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aebe2130 con 0x55f1aebc9d80 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc147201740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aebd5f10 con 0x55f1aebd0eb0 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc141a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebc3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.659+0000 7fc141a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebc3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34114/0 (socket says 172.21.15.49:34114) 2024-02-26T06:32:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.660+0000 7fc141a00640 1 -- 172.21.15.49:0/899692598 learned_addr learned my addr 172.21.15.49:0/899692598 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.660+0000 7fc142400640 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd0eb0 0x55f1aebcdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.660+0000 7fc141a00640 1 -- 172.21.15.49:0/899692598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd0eb0 msgr2=0x55f1aebcdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.660+0000 7fc141a00640 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd0eb0 0x55f1aebcdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.661+0000 7fc141a00640 1 -- 172.21.15.49:0/899692598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 msgr2=0x55f1aebcd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.660+0000 7fc141000640 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebcd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.661+0000 7fc141a00640 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebcd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.661+0000 7fc141a00640 1 -- 172.21.15.49:0/899692598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1aeacb500 con 0x55f1aebc9d80 2024-02-26T06:32:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.661+0000 7fc141000640 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebcd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:32:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.662+0000 7fc141a00640 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebc3b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc130006010 tx=0x7fc130040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=323512f22c0a06c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.663+0000 7fc12e000640 1 -- 172.21.15.49:0/899692598 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130047040 con 0x55f1aebc9d80 2024-02-26T06:32:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.663+0000 7fc12e000640 1 -- 172.21.15.49:0/899692598 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc13002cdf0 con 0x55f1aebc9d80 2024-02-26T06:32:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc12e000640 1 -- 172.21.15.49:0/899692598 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1300548c0 con 0x55f1aebc9d80 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 -- 172.21.15.49:0/899692598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 msgr2=0x55f1aebc3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebc3b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc130006010 tx=0x7fc130040000 comp rx=0 tx=0).stop 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 -- 172.21.15.49:0/899692598 shutdown_connections 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebd0eb0 0x55f1aebcdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebcd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 --2- 172.21.15.49:0/899692598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebc9d80 0x55f1aebc3b90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 -- 172.21.15.49:0/899692598 >> 172.21.15.49:0/899692598 conn(0x55f1aeb91480 msgr2=0x55f1aebc3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 -- 172.21.15.49:0/899692598 shutdown_connections 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.664+0000 7fc147201740 1 -- 172.21.15.49:0/899692598 wait complete. 2024-02-26T06:32:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 1 -- start start 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 1 Processor -- start 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebd0eb0 0x55f1aec39ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aebd9db0 con 0x55f1aebcaaf0 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aec3a3e0 con 0x55f1aebd0eb0 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aec3a520 con 0x55f1aebc9d80 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.665+0000 7fc147201740 10 client.0 _ll_register_callbacks cb 0x7fc1280968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:32:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47598/0 (socket says 172.21.15.49:47598) 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141a00640 1 -- 172.21.15.49:0/4030968255 learned_addr learned my addr 172.21.15.49:0/4030968255 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc142400640 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebd0eb0 0x55f1aec39ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141a00640 1 -- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebd0eb0 msgr2=0x55f1aec39ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141a00640 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebd0eb0 0x55f1aec39ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141a00640 1 -- 172.21.15.49:0/4030968255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 msgr2=0x55f1aebd94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141a00640 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc141a00640 1 -- 172.21.15.49:0/4030968255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc130030070 con 0x55f1aebc9d80 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.666+0000 7fc142400640 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebd0eb0 0x55f1aec39ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.667+0000 7fc141a00640 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd8f70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc130009850 tx=0x7fc130054250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5c7d7afaf25de78 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.667+0000 7fc123400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:32:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.667+0000 7fc123400640 1 -- 172.21.15.49:0/4030968255 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130054dc0 con 0x55f1aebc9d80 2024-02-26T06:32:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.667+0000 7fc123400640 1 -- 172.21.15.49:0/4030968255 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fc130047030 con 0x55f1aebc9d80 2024-02-26T06:32:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.667+0000 7fc147201740 1 -- 172.21.15.49:0/4030968255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1aeb55d00 con 0x55f1aebc9d80 2024-02-26T06:32:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.675+0000 7fc123400640 1 -- 172.21.15.49:0/4030968255 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc13002c910 con 0x55f1aebc9d80 2024-02-26T06:32:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.675+0000 7fc147201740 10 client.27232 fetch_fsmap learned FSMap version 40 2024-02-26T06:32:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.675+0000 7fc147201740 1 -- 172.21.15.49:0/4030968255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f1aeaf1290 con 0x55f1aebc9d80 2024-02-26T06:32:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.676+0000 7fc123400640 1 -- 172.21.15.49:0/4030968255 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc13006b020 con 0x55f1aebc9d80 2024-02-26T06:32:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.676+0000 7fc123400640 1 -- 172.21.15.49:0/4030968255 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1300035f0 con 0x55f1aebc9d80 2024-02-26T06:32:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.678+0000 7fc147201740 10 client.27232 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:32:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.678+0000 7fc147201740 10 client.27232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:32:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.678+0000 7fc147201740 20 client.27232 populate_metadata read hostname 'smithi049' 2024-02-26T06:32:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.678+0000 7fc147201740 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f1aec3b820 0x55f1aec3dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.678+0000 7fc147201740 4 client.27232 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:32:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.678+0000 7fc147201740 1 -- 172.21.15.49:0/4030968255 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f1ae830600 con 0x55f1aec3b820 2024-02-26T06:32:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.678+0000 7fc141000640 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f1aec3b820 0x55f1aec3dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.679+0000 7fc123400640 10 client.27232 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:05.679+0000 7fc141000640 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f1aec3b820 0x55f1aec3dc40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=378b041eb14bca3c server_cookie=d9195302b6cff70c in_seq=0 out_seq=0 2024-02-26T06:32:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 495.0 s, 395.6 tps, lat 253.457 ms stddev 297.650 2024-02-26T06:32:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:06 smithi080 ceph-mon[36206]: pgmap v268: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-02-26T06:32:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:06 smithi080 ceph-mon[36206]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T06:32:07.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:06 smithi115 ceph-mon[36426]: pgmap v268: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-02-26T06:32:07.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:06 smithi115 ceph-mon[36426]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T06:32:07.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:06 smithi049 ceph-mon[31696]: pgmap v268: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-02-26T06:32:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:06 smithi049 ceph-mon[31696]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T06:32:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc123400640 1 -- 172.21.15.49:0/4030968255 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f1ae830600 con 0x55f1aec3b820 2024-02-26T06:32:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc123400640 10 client.27232 handle_command_reply: tid=0 2024-02-26T06:32:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 client.27232 shutdown 2024-02-26T06:32:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 -- 172.21.15.49:0/4030968255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 msgr2=0x55f1aebd8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd8f70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc130009850 tx=0x7fc130054250 comp rx=0 tx=0).stop 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 -- 172.21.15.49:0/4030968255 shutdown_connections 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f1aec3b820 0x55f1aec3dc40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f1aebd0eb0 0x55f1aec39ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f1aebcaaf0 0x55f1aebd94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 --2- 172.21.15.49:0/4030968255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1aebc9d80 0x55f1aebd8f70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc147201740 1 -- 172.21.15.49:0/4030968255 >> 172.21.15.49:0/4030968255 conn(0x55f1aeb91480 msgr2=0x55f1aebc3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.250+0000 7fc123400640 0 client.27232 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.251+0000 7fc147201740 1 -- 172.21.15.49:0/4030968255 shutdown_connections 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.251+0000 7fc147201740 1 -- 172.21.15.49:0/4030968255 wait complete. 2024-02-26T06:32:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.251+0000 7fc147201740 20 client.27232 trim_cache size 0 max 16384 2024-02-26T06:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:32:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 -- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc1240a2350 msgr2=0x7fc1240a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc1240a2350 0x7fc1240a4810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc13c077640 tx=0x7fc13002e040 comp rx=0 tx=0).stop 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 -- 172.21.15.49:0/1854514246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc13c100b00 msgr2=0x7fc13c1a3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc13c100b00 0x7fc13c1a3e50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc12800e9f0 tx=0x7fc12800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 -- 172.21.15.49:0/1854514246 shutdown_connections 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc1240a2350 0x7fc1240a4810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc13c100b00 0x7fc13c1a3e50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13c105500 0x7fc13c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 --2- 172.21.15.49:0/1854514246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc13c1001e0 0x7fc13c1a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 -- 172.21.15.49:0/1854514246 >> 172.21.15.49:0/1854514246 conn(0x7fc13c0771a0 msgr2=0x7fc13c102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 -- 172.21.15.49:0/1854514246 shutdown_connections 2024-02-26T06:32:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:08.252+0000 7fc144400640 1 -- 172.21.15.49:0/1854514246 wait complete. 2024-02-26T06:32:09.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:08 smithi080 ceph-mon[36206]: pgmap v270: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-02-26T06:32:09.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:08 smithi115 ceph-mon[36426]: pgmap v270: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-02-26T06:32:09.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:08 smithi049 ceph-mon[31696]: pgmap v270: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-02-26T06:32:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:10 smithi080 ceph-mon[36206]: pgmap v271: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.76k op/s 2024-02-26T06:32:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:10 smithi115 ceph-mon[36426]: pgmap v271: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.76k op/s 2024-02-26T06:32:11.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:10 smithi049 ceph-mon[31696]: pgmap v271: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.76k op/s 2024-02-26T06:32:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 500.0 s, 2426.2 tps, lat 43.215 ms stddev 67.959 2024-02-26T06:32:13.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:12 smithi080 ceph-mon[36206]: pgmap v272: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-02-26T06:32:13.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:12 smithi115 ceph-mon[36426]: pgmap v272: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-02-26T06:32:13.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:12 smithi049 ceph-mon[31696]: pgmap v272: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-02-26T06:32:15.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:14 smithi080 ceph-mon[36206]: pgmap v273: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-02-26T06:32:15.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:14 smithi115 ceph-mon[36426]: pgmap v273: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-02-26T06:32:15.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:14 smithi049 ceph-mon[31696]: pgmap v273: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-02-26T06:32:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 505.0 s, 2412.8 tps, lat 41.453 ms stddev 52.735 2024-02-26T06:32:16.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:16 smithi049 ceph-mon[31696]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T06:32:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:16 smithi080 ceph-mon[36206]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T06:32:16.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:16 smithi115 ceph-mon[36426]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T06:32:17.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:17 smithi049 ceph-mon[31696]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.9 MiB/s wr, 795 op/s 2024-02-26T06:32:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:17 smithi080 ceph-mon[36206]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.9 MiB/s wr, 795 op/s 2024-02-26T06:32:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:17 smithi115 ceph-mon[36426]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.9 MiB/s wr, 795 op/s 2024-02-26T06:32:19.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:19 smithi049 ceph-mon[31696]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.3 MiB/s wr, 663 op/s 2024-02-26T06:32:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:19 smithi080 ceph-mon[36206]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.3 MiB/s wr, 663 op/s 2024-02-26T06:32:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:19 smithi115 ceph-mon[36426]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.3 MiB/s wr, 663 op/s 2024-02-26T06:32:21.304 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 510.0 s, 2011.2 tps, lat 49.526 ms stddev 56.467 2024-02-26T06:32:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.417+0000 7f02c6c00640 1 -- 172.21.15.49:0/3326820084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c0103f70 msgr2=0x7f02c0107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.417+0000 7f02c6c00640 1 --2- 172.21.15.49:0/3326820084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c0103f70 0x7f02c0107fb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009930 tx=0x7f02ac02bc30 comp rx=0 tx=0).stop 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.417+0000 7f02c6c00640 1 -- 172.21.15.49:0/3326820084 shutdown_connections 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.417+0000 7f02c6c00640 1 --2- 172.21.15.49:0/3326820084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c0103f70 0x7f02c0107fb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.417+0000 7f02c6c00640 1 --2- 172.21.15.49:0/3326820084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c0108690 0x7f02c0110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.417+0000 7f02c6c00640 1 --2- 172.21.15.49:0/3326820084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02c01035c0 0x7f02c01039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.417+0000 7f02c6c00640 1 -- 172.21.15.49:0/3326820084 >> 172.21.15.49:0/3326820084 conn(0x7f02c00fee40 msgr2=0x7f02c0101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.418+0000 7f02c6c00640 1 -- 172.21.15.49:0/3326820084 shutdown_connections 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.418+0000 7f02c6c00640 1 -- 172.21.15.49:0/3326820084 wait complete. 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.418+0000 7f02c6c00640 1 Processor -- start 2024-02-26T06:32:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.419+0000 7f02c6c00640 1 -- start start 2024-02-26T06:32:21.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:21 smithi049 ceph-mon[31696]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.419+0000 7f02c6c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02c01035c0 0x7f02c019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.419+0000 7f02c6c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c0108690 0x7f02c019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.419+0000 7f02c6c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c01998a0 0x7f02c0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.419+0000 7f02c6c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c019a350 con 0x7f02c0108690 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.419+0000 7f02c6c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c019a4c0 con 0x7f02c01998a0 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.419+0000 7f02c6c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c019a630 con 0x7f02c01035c0 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.420+0000 7f02bf400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c0108690 0x7f02c019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.420+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02c01035c0 0x7f02c019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.420+0000 7f02c4c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c01998a0 0x7f02c0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.420+0000 7f02c4c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c01998a0 0x7f02c0199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36034/0 (socket says 172.21.15.49:36034) 2024-02-26T06:32:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.420+0000 7f02c4c00640 1 -- 172.21.15.49:0/1932035813 learned_addr learned my addr 172.21.15.49:0/1932035813 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.421+0000 7f02c4c00640 1 -- 172.21.15.49:0/1932035813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02c01035c0 msgr2=0x7f02c019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.421+0000 7f02c4c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02c01035c0 0x7f02c019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.421+0000 7f02c4c00640 1 -- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c0108690 msgr2=0x7f02c019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.421+0000 7f02c4c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c0108690 0x7f02c019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.421+0000 7f02c4c00640 1 -- 172.21.15.49:0/1932035813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac009560 con 0x7f02c01998a0 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.421+0000 7f02bfe00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02c01035c0 0x7f02c019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.422+0000 7f02c4c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c01998a0 0x7f02c0199d50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f02b800eb10 tx=0x7f02b800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.422+0000 7f02bcc00640 1 -- 172.21.15.49:0/1932035813 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8004b20 con 0x7f02c01998a0 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.422+0000 7f02bcc00640 1 -- 172.21.15.49:0/1932035813 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02b8004c80 con 0x7f02c01998a0 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.422+0000 7f02c6c00640 1 -- 172.21.15.49:0/1932035813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c01a6590 con 0x7f02c01998a0 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.422+0000 7f02bcc00640 1 -- 172.21.15.49:0/1932035813 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8010650 con 0x7f02c01998a0 2024-02-26T06:32:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.422+0000 7f02c6c00640 1 -- 172.21.15.49:0/1932035813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c01a6a20 con 0x7f02c01998a0 2024-02-26T06:32:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.427+0000 7f02c9bb3740 1 Processor -- start 2024-02-26T06:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.428+0000 7f02bcc00640 1 -- 172.21.15.49:0/1932035813 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f02b8005020 con 0x7f02c01998a0 2024-02-26T06:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.428+0000 7f02c9bb3740 1 -- start start 2024-02-26T06:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.429+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5467a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.429+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a5467b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.429+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 0x556a5468ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.429+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a546365f0 con 0x556a5467baf0 2024-02-26T06:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.429+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a545e8890 con 0x556a5467ad80 2024-02-26T06:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.429+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a545e5dc0 con 0x556a5468cb10 2024-02-26T06:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.430+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a5467b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.430+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a5467b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36048/0 (socket says 172.21.15.49:36048) 2024-02-26T06:32:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.430+0000 7f02bfe00640 1 -- 172.21.15.49:0/3081725057 learned_addr learned my addr 172.21.15.49:0/3081725057 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.430+0000 7f02bf400640 1 --2- 172.21.15.49:0/3081725057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5467a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.431+0000 7f02bfe00640 1 -- 172.21.15.49:0/3081725057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 msgr2=0x556a5468ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.431+0000 7f02bfe00640 1 --2- 172.21.15.49:0/3081725057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 0x556a5468ef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.431+0000 7f02bfe00640 1 -- 172.21.15.49:0/3081725057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 msgr2=0x556a5467a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.431+0000 7f02bfe00640 1 --2- 172.21.15.49:0/3081725057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5467a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.431+0000 7f02bfe00640 1 -- 172.21.15.49:0/3081725057 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a54562ad0 con 0x556a5467ad80 2024-02-26T06:32:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.431+0000 7f02bfe00640 1 --2- 172.21.15.49:0/3081725057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a5467b200 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f02b00115c0 tx=0x7f02b0011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb982504f4f0063e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.432+0000 7f02b5200640 1 -- 172.21.15.49:0/3081725057 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000f640 con 0x556a5467ad80 2024-02-26T06:32:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.432+0000 7f02b5200640 1 -- 172.21.15.49:0/3081725057 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02b000fc80 con 0x556a5467ad80 2024-02-26T06:32:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.432+0000 7f02b5200640 1 -- 172.21.15.49:0/3081725057 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0019d20 con 0x556a5467ad80 2024-02-26T06:32:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.432+0000 7f02bcc00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f02a40a2350 0x7f02a40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.432+0000 7f02bcc00640 1 -- 172.21.15.49:0/1932035813 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f02b8014070 con 0x7f02c01998a0 2024-02-26T06:32:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.432+0000 7f02bf400640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f02a40a2350 0x7f02a40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 -- 172.21.15.49:0/3081725057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 msgr2=0x556a5467b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/3081725057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a5467b200 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f02b00115c0 tx=0x7f02b0011a80 comp rx=0 tx=0).stop 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 -- 172.21.15.49:0/3081725057 shutdown_connections 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/3081725057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a5467b200 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/3081725057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 0x556a5468ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/3081725057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5467a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 -- 172.21.15.49:0/3081725057 >> 172.21.15.49:0/3081725057 conn(0x556a54642480 msgr2=0x556a54675a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 -- 172.21.15.49:0/3081725057 shutdown_connections 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02c9bb3740 1 -- 172.21.15.49:0/3081725057 wait complete. 2024-02-26T06:32:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.433+0000 7f02bf400640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f02a40a2350 0x7f02a40a4810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f02ac02c170 tx=0x7f02ac000bc0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.434+0000 7f02c9bb3740 1 -- start start 2024-02-26T06:32:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.434+0000 7f02c9bb3740 1 Processor -- start 2024-02-26T06:32:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a54689bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5468a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 0x556a5468a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a545e15a0 con 0x556a5467baf0 2024-02-26T06:32:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a54586b20 con 0x556a5467ad80 2024-02-26T06:32:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a545867c0 con 0x556a5468cb10 2024-02-26T06:32:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02c9bb3740 10 client.0 _ll_register_callbacks cb 0x556a5459a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a54689bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a54689bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36064/0 (socket says 172.21.15.49:36064) 2024-02-26T06:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02bfe00640 1 -- 172.21.15.49:0/1263946281 learned_addr learned my addr 172.21.15.49:0/1263946281 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02c4c00640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 0x556a5468a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02bf400640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5468a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02bfe00640 1 -- 172.21.15.49:0/1263946281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 msgr2=0x556a5468a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02bfe00640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 0x556a5468a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02bfe00640 1 -- 172.21.15.49:0/1263946281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 msgr2=0x556a5468a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.436+0000 7f02bfe00640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5468a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.437+0000 7f02bfe00640 1 -- 172.21.15.49:0/1263946281 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b0009d40 con 0x556a5467ad80 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.437+0000 7f02c4c00640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 0x556a5468a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.437+0000 7f02bf400640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5468a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.437+0000 7f02bfe00640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a54689bc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009380 tx=0x7f02b0019c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163cc71f0776c46c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.437+0000 7f02a9600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.437+0000 7f02a9600640 1 -- 172.21.15.49:0/1263946281 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0004250 con 0x556a5467ad80 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.437+0000 7f02a9600640 1 -- 172.21.15.49:0/1263946281 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f02b00043b0 con 0x556a5467ad80 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.437+0000 7f02c9bb3740 1 -- 172.21.15.49:0/1263946281 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a545b3ca0 con 0x556a5467ad80 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.438+0000 7f02a9600640 1 -- 172.21.15.49:0/1263946281 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b000f640 con 0x556a5467ad80 2024-02-26T06:32:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.438+0000 7f02c9bb3740 10 client.27272 fetch_fsmap learned FSMap version 40 2024-02-26T06:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.438+0000 7f02c9bb3740 1 -- 172.21.15.49:0/1263946281 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556a545dfc20 con 0x556a5467ad80 2024-02-26T06:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.438+0000 7f02a9600640 1 -- 172.21.15.49:0/1263946281 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a545b3ca0 con 0x556a5467ad80 2024-02-26T06:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.439+0000 7f02a9600640 1 -- 172.21.15.49:0/1263946281 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f02b000f7a0 con 0x556a5467ad80 2024-02-26T06:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.439+0000 7f02c9bb3740 10 client.27272 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.439+0000 7f02c9bb3740 10 client.27272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:32:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.439+0000 7f02c9bb3740 20 client.27272 populate_metadata read hostname 'smithi049' 2024-02-26T06:32:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.439+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556a5469ca50 0x556a5469ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.439+0000 7f02c9bb3740 4 client.27272 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:32:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.439+0000 7f02c9bb3740 1 -- 172.21.15.49:0/1263946281 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a545d7cb0 con 0x556a5469ca50 2024-02-26T06:32:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.442+0000 7f02bf400640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556a5469ca50 0x556a5469ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.447+0000 7f02bf400640 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556a5469ca50 0x556a5469ee90 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51115d1851a9cc56 server_cookie=89034769c87ff914 in_seq=0 out_seq=0 2024-02-26T06:32:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.447+0000 7f02a9600640 10 client.27272 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.448+0000 7f02a9600640 1 -- 172.21.15.49:0/1263946281 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a545d7cb0 con 0x556a5469ca50 2024-02-26T06:32:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.448+0000 7f02a9600640 10 client.27272 handle_command_reply: tid=0 2024-02-26T06:32:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.448+0000 7f02c9bb3740 1 client.27272 shutdown 2024-02-26T06:32:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.451+0000 7f02c9bb3740 1 -- 172.21.15.49:0/1263946281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 msgr2=0x556a54689bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.451+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a54689bc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009380 tx=0x7f02b0019c50 comp rx=0 tx=0).stop 2024-02-26T06:32:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.452+0000 7f02c9bb3740 1 -- 172.21.15.49:0/1263946281 shutdown_connections 2024-02-26T06:32:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.452+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556a5469ca50 0x556a5469ee90 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.452+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5468cb10 0x556a5468a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.452+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467baf0 0x556a5468a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.452+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/1263946281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5467ad80 0x556a54689bc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.452+0000 7f02a9600640 0 client.27272 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.452+0000 7f02c9bb3740 1 -- 172.21.15.49:0/1263946281 >> 172.21.15.49:0/1263946281 conn(0x556a54642480 msgr2=0x556a546777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.454+0000 7f02c9bb3740 1 -- 172.21.15.49:0/1263946281 shutdown_connections 2024-02-26T06:32:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.454+0000 7f02c9bb3740 1 -- 172.21.15.49:0/1263946281 wait complete. 2024-02-26T06:32:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.454+0000 7f02c9bb3740 20 client.27272 trim_cache size 0 max 16384 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.505+0000 7f02c9bb3740 1 Processor -- start 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.505+0000 7f02c9bb3740 1 -- start start 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.506+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467ad80 0x556a5468c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.506+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a54693930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.506+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 0x556a54693e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.506+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a54329e60 con 0x556a5467ad80 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.506+0000 7f02bf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a54693930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5468f340 con 0x556a5468cb10 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a546a40e0 con 0x556a5467baf0 2024-02-26T06:32:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02bf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a54693930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:54790/0 (socket says 172.21.15.49:54790) 2024-02-26T06:32:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02bf400640 1 -- 172.21.15.49:0/2344560600 learned_addr learned my addr 172.21.15.49:0/2344560600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02bf400640 1 -- 172.21.15.49:0/2344560600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 msgr2=0x556a54693e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02bf400640 1 --2- 172.21.15.49:0/2344560600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 0x556a54693e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02bf400640 1 -- 172.21.15.49:0/2344560600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467ad80 msgr2=0x556a5468c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02bf400640 1 --2- 172.21.15.49:0/2344560600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467ad80 0x556a5468c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.507+0000 7f02bf400640 1 -- 172.21.15.49:0/2344560600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a5457c500 con 0x556a5467baf0 2024-02-26T06:32:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.508+0000 7f02bf400640 1 --2- 172.21.15.49:0/2344560600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a54693930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009930 tx=0x7f02ac041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8538716e645e67fd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.508+0000 7f02abe00640 1 -- 172.21.15.49:0/2344560600 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac02c9e0 con 0x556a5467baf0 2024-02-26T06:32:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.508+0000 7f02abe00640 1 -- 172.21.15.49:0/2344560600 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02ac050040 con 0x556a5467baf0 2024-02-26T06:32:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.509+0000 7f02abe00640 1 -- 172.21.15.49:0/2344560600 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac052530 con 0x556a5467baf0 2024-02-26T06:32:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.512+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2344560600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 msgr2=0x556a54693930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.513+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2344560600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a54693930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009930 tx=0x7f02ac041000 comp rx=0 tx=0).stop 2024-02-26T06:32:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.513+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2344560600 shutdown_connections 2024-02-26T06:32:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.513+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2344560600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 0x556a54693e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.513+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2344560600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a54693930 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.513+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2344560600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467ad80 0x556a5468c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.513+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2344560600 >> 172.21.15.49:0/2344560600 conn(0x556a54642480 msgr2=0x556a54676640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.513+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2344560600 shutdown_connections 2024-02-26T06:32:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.514+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2344560600 wait complete. 2024-02-26T06:32:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.514+0000 7f02c9bb3740 1 -- start start 2024-02-26T06:32:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.514+0000 7f02c9bb3740 1 Processor -- start 2024-02-26T06:32:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.516+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467ad80 0x556a546ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.516+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a546ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.517+0000 7f02c9bb3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 0x556a546ec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.518+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a546ec6d0 con 0x556a5467ad80 2024-02-26T06:32:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.518+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a546ec840 con 0x556a5468cb10 2024-02-26T06:32:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.518+0000 7f02c9bb3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a546eca00 con 0x556a5467baf0 2024-02-26T06:32:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02bf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a546ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02bf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a546ebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:54792/0 (socket says 172.21.15.49:54792) 2024-02-26T06:32:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02bf400640 1 -- 172.21.15.49:0/2747798028 learned_addr learned my addr 172.21.15.49:0/2747798028 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02c4c00640 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 0x556a546ec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02c9bb3740 10 client.0 _ll_register_callbacks cb 0x7f02b0011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:32:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02bf400640 1 -- 172.21.15.49:0/2747798028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 msgr2=0x556a546ec0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02bf400640 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 0x556a546ec0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02bf400640 1 -- 172.21.15.49:0/2747798028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467ad80 msgr2=0x556a546ee7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02bf400640 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467ad80 0x556a546ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.519+0000 7f02bf400640 1 -- 172.21.15.49:0/2747798028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c0110a70 con 0x556a5467baf0 2024-02-26T06:32:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.520+0000 7f02bf400640 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a546ebb80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009900 tx=0x7f02ac02cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9487c2d1e7f22a9a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.520+0000 7f02a9600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:32:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.520+0000 7f02a9600640 1 -- 172.21.15.49:0/2747798028 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac005280 con 0x556a5467baf0 2024-02-26T06:32:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.520+0000 7f02a9600640 1 -- 172.21.15.49:0/2747798028 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f02ac0053e0 con 0x556a5467baf0 2024-02-26T06:32:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.520+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2747798028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a546899f0 con 0x556a5467baf0 2024-02-26T06:32:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.521+0000 7f02a9600640 1 -- 172.21.15.49:0/2747798028 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02ac0055c0 con 0x556a5467baf0 2024-02-26T06:32:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.522+0000 7f02c9bb3740 10 client.27244 fetch_fsmap learned FSMap version 40 2024-02-26T06:32:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.522+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2747798028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556a545a2290 con 0x556a5467baf0 2024-02-26T06:32:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.522+0000 7f02a9600640 1 -- 172.21.15.49:0/2747798028 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a546899f0 con 0x556a5467baf0 2024-02-26T06:32:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.523+0000 7f02a9600640 1 -- 172.21.15.49:0/2747798028 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f02ac059070 con 0x556a5467baf0 2024-02-26T06:32:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.523+0000 7f02c9bb3740 10 client.27244 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:32:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.523+0000 7f02c9bb3740 10 client.27244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:32:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.523+0000 7f02c9bb3740 20 client.27244 populate_metadata read hostname 'smithi049' 2024-02-26T06:32:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.523+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556a54693b90 0x556a54707fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.523+0000 7f02c9bb3740 4 client.27244 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:32:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.523+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2747798028 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a54606d00 con 0x556a54693b90 2024-02-26T06:32:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.524+0000 7f02bfe00640 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556a54693b90 0x556a54707fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.526+0000 7f02bfe00640 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556a54693b90 0x556a54707fd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a79ff1f24d0d7f15 server_cookie=6e36d8a8ecbad526 in_seq=0 out_seq=0 2024-02-26T06:32:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.526+0000 7f02a9600640 10 client.27244 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.566+0000 7f02a9600640 1 -- 172.21.15.49:0/2747798028 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a54606d00 con 0x556a54693b90 2024-02-26T06:32:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.567+0000 7f02a9600640 10 client.27244 handle_command_reply: tid=0 2024-02-26T06:32:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.567+0000 7f02c9bb3740 1 client.27244 shutdown 2024-02-26T06:32:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2747798028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 msgr2=0x556a546ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a546ebb80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009900 tx=0x7f02ac02cc40 comp rx=0 tx=0).stop 2024-02-26T06:32:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2747798028 shutdown_connections 2024-02-26T06:32:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x556a54693b90 0x556a54707fd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556a5468cb10 0x556a546ec0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a5467baf0 0x556a546ebb80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 --2- 172.21.15.49:0/2747798028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556a5467ad80 0x556a546ee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2747798028 >> 172.21.15.49:0/2747798028 conn(0x556a54642480 msgr2=0x556a546755a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02a9600640 0 client.27244 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2747798028 shutdown_connections 2024-02-26T06:32:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.568+0000 7f02c9bb3740 1 -- 172.21.15.49:0/2747798028 wait complete. 2024-02-26T06:32:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.569+0000 7f02c9bb3740 20 client.27244 trim_cache size 0 max 16384 2024-02-26T06:32:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:32:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:32:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:32:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:32:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.569+0000 7f02c6c00640 1 -- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f02a40a2350 msgr2=0x7f02a40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.569+0000 7f02c6c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f02a40a2350 0x7f02a40a4810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f02ac02c170 tx=0x7f02ac000bc0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.569+0000 7f02c6c00640 1 -- 172.21.15.49:0/1932035813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c01998a0 msgr2=0x7f02c0199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.569+0000 7f02c6c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c01998a0 0x7f02c0199d50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f02b800eb10 tx=0x7f02b800c2d0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.570+0000 7f02c6c00640 1 -- 172.21.15.49:0/1932035813 shutdown_connections 2024-02-26T06:32:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.570+0000 7f02c6c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f02a40a2350 0x7f02a40a4810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.570+0000 7f02c6c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c01998a0 0x7f02c0199d50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.570+0000 7f02c6c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c0108690 0x7f02c019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.570+0000 7f02c6c00640 1 --2- 172.21.15.49:0/1932035813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02c01035c0 0x7f02c019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.570+0000 7f02c6c00640 1 -- 172.21.15.49:0/1932035813 >> 172.21.15.49:0/1932035813 conn(0x7f02c00fee40 msgr2=0x7f02c0101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.570+0000 7f02c6c00640 1 -- 172.21.15.49:0/1932035813 shutdown_connections 2024-02-26T06:32:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:21.570+0000 7f02c6c00640 1 -- 172.21.15.49:0/1932035813 wait complete. 2024-02-26T06:32:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:21 smithi080 ceph-mon[36206]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-02-26T06:32:21.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:21 smithi115 ceph-mon[36426]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-02-26T06:32:23.391 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:23 smithi115 ceph-mon[36426]: pgmap v278: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-02-26T06:32:23.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:23 smithi049 ceph-mon[31696]: pgmap v278: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-02-26T06:32:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:23 smithi080 ceph-mon[36206]: pgmap v278: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-02-26T06:32:25.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:25 smithi049 ceph-mon[31696]: pgmap v279: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-02-26T06:32:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:25 smithi080 ceph-mon[36206]: pgmap v279: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-02-26T06:32:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:25 smithi115 ceph-mon[36426]: pgmap v279: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-02-26T06:32:26.304 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 515.0 s, 1938.2 tps, lat 51.375 ms stddev 60.905 2024-02-26T06:32:27.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:27 smithi049 ceph-mon[31696]: pgmap v280: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-02-26T06:32:27.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:27 smithi080 ceph-mon[36206]: pgmap v280: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-02-26T06:32:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:27 smithi115 ceph-mon[36426]: pgmap v280: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-02-26T06:32:29.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:29 smithi049 ceph-mon[31696]: pgmap v281: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 583 op/s 2024-02-26T06:32:29.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:29 smithi080 ceph-mon[36206]: pgmap v281: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 583 op/s 2024-02-26T06:32:29.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:29 smithi115 ceph-mon[36426]: pgmap v281: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 583 op/s 2024-02-26T06:32:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 520.0 s, 2297.1 tps, lat 43.913 ms stddev 52.532 2024-02-26T06:32:31.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:31 smithi080 ceph-mon[36206]: pgmap v282: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-02-26T06:32:31.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:31 smithi115 ceph-mon[36426]: pgmap v282: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-02-26T06:32:31.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:31 smithi049 ceph-mon[31696]: pgmap v282: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-02-26T06:32:33.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:33 smithi080 ceph-mon[36206]: pgmap v283: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-02-26T06:32:33.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:33 smithi115 ceph-mon[36426]: pgmap v283: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-02-26T06:32:33.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:33 smithi049 ceph-mon[31696]: pgmap v283: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-02-26T06:32:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.775+0000 7f6c2d000640 1 -- 172.21.15.49:0/74498632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c281006e0 msgr2=0x7f6c2810d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.775+0000 7f6c2d000640 1 --2- 172.21.15.49:0/74498632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c281006e0 0x7f6c2810d3a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009900 tx=0x7f6c1002bc40 comp rx=0 tx=0).stop 2024-02-26T06:32:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.775+0000 7f6c2d000640 1 -- 172.21.15.49:0/74498632 shutdown_connections 2024-02-26T06:32:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.775+0000 7f6c2d000640 1 --2- 172.21.15.49:0/74498632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c281006e0 0x7f6c2810d3a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.775+0000 7f6c2d000640 1 --2- 172.21.15.49:0/74498632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c2810d8e0 0x7f6c2810fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.775+0000 7f6c2d000640 1 --2- 172.21.15.49:0/74498632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c280ffd30 0x7f6c28100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.775+0000 7f6c2d000640 1 -- 172.21.15.49:0/74498632 >> 172.21.15.49:0/74498632 conn(0x7f6c280fb990 msgr2=0x7f6c280fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.775+0000 7f6c2d000640 1 -- 172.21.15.49:0/74498632 shutdown_connections 2024-02-26T06:32:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.776+0000 7f6c2d000640 1 -- 172.21.15.49:0/74498632 wait complete. 2024-02-26T06:32:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.777+0000 7f6c2d000640 1 Processor -- start 2024-02-26T06:32:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.777+0000 7f6c2d000640 1 -- start start 2024-02-26T06:32:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.777+0000 7f6c2d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c280ffd30 0x7f6c2819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.777+0000 7f6c2d000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c281006e0 0x7f6c2819fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.777+0000 7f6c2d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c2810d8e0 0x7f6c281996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.777+0000 7f6c2d000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28199e50 con 0x7f6c2810d8e0 2024-02-26T06:32:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.777+0000 7f6c2d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28199fc0 con 0x7f6c280ffd30 2024-02-26T06:32:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.777+0000 7f6c2d000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2819a180 con 0x7f6c281006e0 2024-02-26T06:32:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.778+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c281006e0 0x7f6c2819fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.778+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c281006e0 0x7f6c2819fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:37742/0 (socket says 172.21.15.49:37742) 2024-02-26T06:32:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.778+0000 7f6c1fe00640 1 -- 172.21.15.49:0/3364507065 learned_addr learned my addr 172.21.15.49:0/3364507065 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.778+0000 7f6c27e00640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c2810d8e0 0x7f6c281996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.778+0000 7f6c27400640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c280ffd30 0x7f6c2819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.778+0000 7f6c1fe00640 1 -- 172.21.15.49:0/3364507065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c280ffd30 msgr2=0x7f6c2819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.778+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c280ffd30 0x7f6c2819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.779+0000 7f6c1fe00640 1 -- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c2810d8e0 msgr2=0x7f6c281996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.779+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c2810d8e0 0x7f6c281996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.779+0000 7f6c1fe00640 1 -- 172.21.15.49:0/3364507065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c10009560 con 0x7f6c281006e0 2024-02-26T06:32:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.779+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c281006e0 0x7f6c2819fab0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6c10000c00 tx=0x7f6c1002c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.780+0000 7f6c24c00640 1 -- 172.21.15.49:0/3364507065 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10004390 con 0x7f6c281006e0 2024-02-26T06:32:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.780+0000 7f6c2d000640 1 -- 172.21.15.49:0/3364507065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2819a400 con 0x7f6c281006e0 2024-02-26T06:32:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.780+0000 7f6c2d000640 1 -- 172.21.15.49:0/3364507065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c281a6560 con 0x7f6c281006e0 2024-02-26T06:32:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.780+0000 7f6c24c00640 1 -- 172.21.15.49:0/3364507065 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c100044f0 con 0x7f6c281006e0 2024-02-26T06:32:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.780+0000 7f6c24c00640 1 -- 172.21.15.49:0/3364507065 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10003c20 con 0x7f6c281006e0 2024-02-26T06:32:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.785+0000 7f6c31612740 1 Processor -- start 2024-02-26T06:32:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.786+0000 7f6c31612740 1 -- start start 2024-02-26T06:32:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.786+0000 7f6c31612740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.786+0000 7f6c31612740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.786+0000 7f6c31612740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109ccd50 0x555b109cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.786+0000 7f6c31612740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b109765f0 con 0x555b109ccd50 2024-02-26T06:32:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.786+0000 7f6c31612740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b10928890 con 0x555b109c4540 2024-02-26T06:32:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.786+0000 7f6c31612740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b10925dc0 con 0x555b109c37d0 2024-02-26T06:32:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45594/0 (socket says 172.21.15.49:45594) 2024-02-26T06:32:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27e00640 1 -- 172.21.15.49:0/1018798635 learned_addr learned my addr 172.21.15.49:0/1018798635 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27400640 1 --2- 172.21.15.49:0/1018798635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c24c00640 1 -- 172.21.15.49:0/3364507065 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6c10004df0 con 0x7f6c281006e0 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27e00640 1 -- 172.21.15.49:0/1018798635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 msgr2=0x555b109c3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27e00640 1 --2- 172.21.15.49:0/1018798635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27e00640 1 -- 172.21.15.49:0/1018798635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109ccd50 msgr2=0x555b109cf140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27e00640 1 --2- 172.21.15.49:0/1018798635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109ccd50 0x555b109cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.787+0000 7f6c27e00640 1 -- 172.21.15.49:0/1018798635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b108a2ad0 con 0x555b109c4540 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.788+0000 7f6c27e00640 1 --2- 172.21.15.49:0/1018798635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c3290 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009f30 tx=0x7f6c200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a964787f631443 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.788+0000 7f6c1cc00640 1 -- 172.21.15.49:0/1018798635 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004d20 con 0x555b109c4540 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.788+0000 7f6c1cc00640 1 -- 172.21.15.49:0/1018798635 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c20004e80 con 0x555b109c4540 2024-02-26T06:32:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.788+0000 7f6c1cc00640 1 -- 172.21.15.49:0/1018798635 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20005280 con 0x555b109c4540 2024-02-26T06:32:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 -- 172.21.15.49:0/1018798635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 msgr2=0x555b109c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 --2- 172.21.15.49:0/1018798635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c3290 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009f30 tx=0x7f6c200118d0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 -- 172.21.15.49:0/1018798635 shutdown_connections 2024-02-26T06:32:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 --2- 172.21.15.49:0/1018798635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109ccd50 0x555b109cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 --2- 172.21.15.49:0/1018798635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 --2- 172.21.15.49:0/1018798635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c3290 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 -- 172.21.15.49:0/1018798635 >> 172.21.15.49:0/1018798635 conn(0x555b10982480 msgr2=0x555b109be4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 -- 172.21.15.49:0/1018798635 shutdown_connections 2024-02-26T06:32:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.789+0000 7f6c31612740 1 -- 172.21.15.49:0/1018798635 wait complete. 2024-02-26T06:32:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.790+0000 7f6c31612740 1 -- start start 2024-02-26T06:32:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.790+0000 7f6c31612740 1 Processor -- start 2024-02-26T06:32:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.791+0000 7f6c31612740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.791+0000 7f6c31612740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109c4540 0x555b109ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.791+0000 7f6c31612740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109ccd50 0x555b109ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.791+0000 7f6c31612740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b109215a0 con 0x555b109c4540 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.791+0000 7f6c31612740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b108c6b20 con 0x555b109c37d0 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.791+0000 7f6c31612740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b108c67c0 con 0x555b109ccd50 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.792+0000 7f6c24c00640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c0c0a21d0 0x7f6c0c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.792+0000 7f6c31612740 10 client.0 _ll_register_callbacks cb 0x555b108da520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.792+0000 7f6c24c00640 1 -- 172.21.15.49:0/3364507065 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6c100f0520 con 0x7f6c281006e0 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.792+0000 7f6c27e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.792+0000 7f6c27e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45596/0 (socket says 172.21.15.49:45596) 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.792+0000 7f6c27e00640 1 -- 172.21.15.49:0/2698807928 learned_addr learned my addr 172.21.15.49:0/2698807928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.792+0000 7f6c27e00640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c0c0a21d0 0x7f6c0c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.793+0000 7f6c27400640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109c4540 0x555b109ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.793+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109ccd50 0x555b109ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.793+0000 7f6c27e00640 1 -- 172.21.15.49:0/2698807928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109ccd50 msgr2=0x555b109ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.793+0000 7f6c27e00640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109ccd50 0x555b109ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.793+0000 7f6c27e00640 1 -- 172.21.15.49:0/2698807928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109c4540 msgr2=0x555b109ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.793+0000 7f6c27e00640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109c4540 0x555b109ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.793+0000 7f6c27e00640 1 -- 172.21.15.49:0/2698807928 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c20009be0 con 0x555b109c37d0 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.794+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109ccd50 0x555b109ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.794+0000 7f6c27400640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109c4540 0x555b109ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.794+0000 7f6c27e00640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109c9d90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6c20004620 tx=0x7f6c20005080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ce60e19d3fa8f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.794+0000 7f6c15600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:32:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.795+0000 7f6c27e00640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c0c0a21d0 0x7f6c0c0a4690 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009f00 tx=0x7f6c2002d000 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.795+0000 7f6c15600640 1 -- 172.21.15.49:0/2698807928 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20029a30 con 0x555b109c37d0 2024-02-26T06:32:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.796+0000 7f6c15600640 1 -- 172.21.15.49:0/2698807928 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6c20029b90 con 0x555b109c37d0 2024-02-26T06:32:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.796+0000 7f6c31612740 1 -- 172.21.15.49:0/2698807928 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b108f3ca0 con 0x555b109c37d0 2024-02-26T06:32:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.796+0000 7f6c15600640 1 -- 172.21.15.49:0/2698807928 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c20029a30 con 0x555b109c37d0 2024-02-26T06:32:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.796+0000 7f6c31612740 10 client.27287 fetch_fsmap learned FSMap version 40 2024-02-26T06:32:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.796+0000 7f6c31612740 1 -- 172.21.15.49:0/2698807928 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555b1091fc20 con 0x555b109c37d0 2024-02-26T06:32:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.798+0000 7f6c15600640 1 -- 172.21.15.49:0/2698807928 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555b108f3ca0 con 0x555b109c37d0 2024-02-26T06:32:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.798+0000 7f6c15600640 1 -- 172.21.15.49:0/2698807928 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c2000fc70 con 0x555b109c37d0 2024-02-26T06:32:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.799+0000 7f6c31612740 10 client.27287 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:32:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.799+0000 7f6c31612740 10 client.27287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:32:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.799+0000 7f6c31612740 20 client.27287 populate_metadata read hostname 'smithi049' 2024-02-26T06:32:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.799+0000 7f6c31612740 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555b109b96d0 0x555b109bbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.799+0000 7f6c31612740 4 client.27287 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:32:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.799+0000 7f6c27400640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555b109b96d0 0x555b109bbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.799+0000 7f6c31612740 1 -- 172.21.15.49:0/2698807928 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b10917cb0 con 0x555b109b96d0 2024-02-26T06:32:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.800+0000 7f6c27400640 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555b109b96d0 0x555b109bbaf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27bc7aa76f3fd6fa server_cookie=4bf0ad1c9c28f43b in_seq=0 out_seq=0 2024-02-26T06:32:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.800+0000 7f6c15600640 10 client.27287 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.801+0000 7f6c15600640 1 -- 172.21.15.49:0/2698807928 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b10917cb0 con 0x555b109b96d0 2024-02-26T06:32:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.801+0000 7f6c15600640 10 client.27287 handle_command_reply: tid=0 2024-02-26T06:32:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.802+0000 7f6c31612740 1 client.27287 shutdown 2024-02-26T06:32:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c31612740 1 -- 172.21.15.49:0/2698807928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 msgr2=0x555b109c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c31612740 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109c9d90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6c20004620 tx=0x7f6c20005080 comp rx=0 tx=0).stop 2024-02-26T06:32:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c31612740 1 -- 172.21.15.49:0/2698807928 shutdown_connections 2024-02-26T06:32:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c31612740 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555b109b96d0 0x555b109bbaf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c31612740 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109ccd50 0x555b109ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c15600640 0 client.27287 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c31612740 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109c4540 0x555b109ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c31612740 1 --2- 172.21.15.49:0/2698807928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109c9d90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.803+0000 7f6c31612740 1 -- 172.21.15.49:0/2698807928 >> 172.21.15.49:0/2698807928 conn(0x555b10982480 msgr2=0x555b109c0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.804+0000 7f6c31612740 1 -- 172.21.15.49:0/2698807928 shutdown_connections 2024-02-26T06:32:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.804+0000 7f6c31612740 1 -- 172.21.15.49:0/2698807928 wait complete. 2024-02-26T06:32:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.804+0000 7f6c31612740 20 client.27287 trim_cache size 0 max 16384 2024-02-26T06:32:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.827+0000 7f6c31612740 1 Processor -- start 2024-02-26T06:32:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.828+0000 7f6c31612740 1 -- start start 2024-02-26T06:32:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.828+0000 7f6c31612740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.828+0000 7f6c31612740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109d4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.828+0000 7f6c31612740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c4540 0x555b109bd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.828+0000 7f6c31612740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b10669e60 con 0x555b109b96d0 2024-02-26T06:32:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.828+0000 7f6c31612740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b108f3ca0 con 0x555b109c37d0 2024-02-26T06:32:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.828+0000 7f6c31612740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b109b01d0 con 0x555b109c4540 2024-02-26T06:32:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c27e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c4540 0x555b109bd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c27400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109d4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c27e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c4540 0x555b109bd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:37770/0 (socket says 172.21.15.49:37770) 2024-02-26T06:32:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c27400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109d3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46184/0 (socket says 172.21.15.49:46184) 2024-02-26T06:32:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109d4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:45612/0 (socket says 172.21.15.49:45612) 2024-02-26T06:32:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c27e00640 1 -- 172.21.15.49:0/3239089934 learned_addr learned my addr 172.21.15.49:0/3239089934 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c1fe00640 1 -- 172.21.15.49:0/3239089934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c4540 msgr2=0x555b109bd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/3239089934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c4540 0x555b109bd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c1fe00640 1 -- 172.21.15.49:0/3239089934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 msgr2=0x555b109d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/3239089934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109d3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.829+0000 7f6c1fe00640 1 -- 172.21.15.49:0/3239089934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b108bc500 con 0x555b109c37d0 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.830+0000 7f6c27400640 1 --2- 172.21.15.49:0/3239089934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.830+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/3239089934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109d4110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6c100edd00 tx=0x7f6c100bcc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b62a787466b6e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.830+0000 7f6c17e00640 1 -- 172.21.15.49:0/3239089934 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c100bd030 con 0x555b109c37d0 2024-02-26T06:32:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.831+0000 7f6c17e00640 1 -- 172.21.15.49:0/3239089934 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c100c1180 con 0x555b109c37d0 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.831+0000 7f6c17e00640 1 -- 172.21.15.49:0/3239089934 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c100ede70 con 0x555b109c37d0 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 -- 172.21.15.49:0/3239089934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 msgr2=0x555b109d4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 --2- 172.21.15.49:0/3239089934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109d4110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6c100edd00 tx=0x7f6c100bcc30 comp rx=0 tx=0).stop 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 -- 172.21.15.49:0/3239089934 shutdown_connections 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 --2- 172.21.15.49:0/3239089934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c37d0 0x555b109d4110 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 --2- 172.21.15.49:0/3239089934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c4540 0x555b109bd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 --2- 172.21.15.49:0/3239089934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 -- 172.21.15.49:0/3239089934 >> 172.21.15.49:0/3239089934 conn(0x555b10982480 msgr2=0x555b109c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 -- 172.21.15.49:0/3239089934 shutdown_connections 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 -- 172.21.15.49:0/3239089934 wait complete. 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 -- start start 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.832+0000 7f6c31612740 1 Processor -- start 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c31612740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c31612740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c31612740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c31612740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b10a3b4e0 con 0x555b109b96d0 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c31612740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b10a3b620 con 0x555b109c4540 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c31612740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b10a3b790 con 0x555b109c37d0 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:37782/0 (socket says 172.21.15.49:37782) 2024-02-26T06:32:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c1fe00640 1 -- 172.21.15.49:0/4052517181 learned_addr learned my addr 172.21.15.49:0/4052517181 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.833+0000 7f6c27e00640 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c1fe00640 1 -- 172.21.15.49:0/4052517181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 msgr2=0x555b109c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c27400640 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c1fe00640 1 -- 172.21.15.49:0/4052517181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 msgr2=0x555b109c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c1fe00640 1 -- 172.21.15.49:0/4052517181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c18009d90 con 0x555b109c37d0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c27400640 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c31612740 10 client.0 _ll_register_callbacks cb 0x7f6c20009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.834+0000 7f6c31612740 1 -- 172.21.15.49:0/4052517181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b10946d00 con 0x555b109c37d0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.835+0000 7f6c1fe00640 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c7000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6c100b9a00 tx=0x7f6c100bcca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52fbbcf74e78a1c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.835+0000 7f6c15600640 10 client.27262 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.835+0000 7f6c15600640 1 -- 172.21.15.49:0/4052517181 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c100bd030 con 0x555b109c37d0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.835+0000 7f6c15600640 1 -- 172.21.15.49:0/4052517181 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6c100db2c0 con 0x555b109c37d0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.836+0000 7f6c15600640 1 -- 172.21.15.49:0/4052517181 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c100db4c0 con 0x555b109c37d0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.836+0000 7f6c31612740 10 client.27262 fetch_fsmap learned FSMap version 40 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.836+0000 7f6c31612740 1 -- 172.21.15.49:0/4052517181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=40}) v3 -- 0x7f6c281a6560 con 0x555b109c37d0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.837+0000 7f6c15600640 1 -- 172.21.15.49:0/4052517181 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c1003c070 con 0x555b109c37d0 2024-02-26T06:32:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.837+0000 7f6c31612740 10 client.27262 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.837+0000 7f6c31612740 10 client.27262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.837+0000 7f6c31612740 20 client.27262 populate_metadata read hostname 'smithi049' 2024-02-26T06:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.837+0000 7f6c31612740 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555b109ccd50 0x555b10a3e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.837+0000 7f6c31612740 4 client.27262 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.837+0000 7f6c31612740 1 -- 172.21.15.49:0/4052517181 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555b10621600 con 0x555b109ccd50 2024-02-26T06:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.837+0000 7f6c27400640 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555b109ccd50 0x555b10a3e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.838+0000 7f6c27400640 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555b109ccd50 0x555b10a3e9a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc05dc5e5979e377 server_cookie=fd82374de9f4bcb8 in_seq=0 out_seq=0 2024-02-26T06:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.838+0000 7f6c15600640 10 client.27262 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.854+0000 7f6c15600640 1 -- 172.21.15.49:0/4052517181 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555b10621600 con 0x555b109ccd50 2024-02-26T06:32:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.854+0000 7f6c15600640 10 client.27262 handle_command_reply: tid=0 2024-02-26T06:32:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.854+0000 7f6c31612740 1 client.27262 shutdown 2024-02-26T06:32:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.854+0000 7f6c31612740 1 -- 172.21.15.49:0/4052517181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 msgr2=0x555b109c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.854+0000 7f6c31612740 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c7000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6c100b9a00 tx=0x7f6c100bcca0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 1 -- 172.21.15.49:0/4052517181 shutdown_connections 2024-02-26T06:32:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x555b109ccd50 0x555b10a3e9a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555b109c4540 0x555b109c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b109c37d0 0x555b109c7000 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 1 --2- 172.21.15.49:0/4052517181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b109b96d0 0x555b109c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 1 -- 172.21.15.49:0/4052517181 >> 172.21.15.49:0/4052517181 conn(0x555b10982480 msgr2=0x555b109b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c15600640 0 client.27262 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 1 -- 172.21.15.49:0/4052517181 shutdown_connections 2024-02-26T06:32:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 1 -- 172.21.15.49:0/4052517181 wait complete. 2024-02-26T06:32:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.855+0000 7f6c31612740 20 client.27262 trim_cache size 0 max 16384 2024-02-26T06:32:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:32:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:32:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:32:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 -- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c0c0a21d0 msgr2=0x7f6c0c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c0c0a21d0 0x7f6c0c0a4690 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009f00 tx=0x7f6c2002d000 comp rx=0 tx=0).stop 2024-02-26T06:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 -- 172.21.15.49:0/3364507065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c281006e0 msgr2=0x7f6c2819fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c281006e0 0x7f6c2819fab0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6c10000c00 tx=0x7f6c1002c640 comp rx=0 tx=0).stop 2024-02-26T06:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 -- 172.21.15.49:0/3364507065 shutdown_connections 2024-02-26T06:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6c0c0a21d0 0x7f6c0c0a4690 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c2810d8e0 0x7f6c281996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c281006e0 0x7f6c2819fab0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 --2- 172.21.15.49:0/3364507065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c280ffd30 0x7f6c2819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 -- 172.21.15.49:0/3364507065 >> 172.21.15.49:0/3364507065 conn(0x7f6c280fb990 msgr2=0x7f6c280fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 -- 172.21.15.49:0/3364507065 shutdown_connections 2024-02-26T06:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:34.856+0000 7f6c2d000640 1 -- 172.21.15.49:0/3364507065 wait complete. 2024-02-26T06:32:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:35 smithi080 ceph-mon[36206]: pgmap v284: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.6 MiB/s wr, 573 op/s 2024-02-26T06:32:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:35 smithi115 ceph-mon[36426]: pgmap v284: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.6 MiB/s wr, 573 op/s 2024-02-26T06:32:35.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:35 smithi049 ceph-mon[31696]: pgmap v284: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.6 MiB/s wr, 573 op/s 2024-02-26T06:32:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 525.0 s, 1871.8 tps, lat 53.194 ms stddev 75.302 2024-02-26T06:32:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:37 smithi080 ceph-mon[36206]: pgmap v285: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 854 op/s 2024-02-26T06:32:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:37 smithi115 ceph-mon[36426]: pgmap v285: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 854 op/s 2024-02-26T06:32:37.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:37 smithi049 ceph-mon[31696]: pgmap v285: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 854 op/s 2024-02-26T06:32:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:39 smithi080 ceph-mon[36206]: pgmap v286: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 633 op/s 2024-02-26T06:32:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:39 smithi115 ceph-mon[36426]: pgmap v286: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 633 op/s 2024-02-26T06:32:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:39 smithi049 ceph-mon[31696]: pgmap v286: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 633 op/s 2024-02-26T06:32:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 530.0 s, 1228.0 tps, lat 79.690 ms stddev 171.205 2024-02-26T06:32:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:41 smithi080 ceph-mon[36206]: pgmap v287: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-02-26T06:32:41.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:41 smithi115 ceph-mon[36426]: pgmap v287: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-02-26T06:32:41.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:41 smithi049 ceph-mon[31696]: pgmap v287: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-02-26T06:32:43.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:43 smithi080 ceph-mon[36206]: pgmap v288: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-02-26T06:32:43.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:43 smithi115 ceph-mon[36426]: pgmap v288: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-02-26T06:32:43.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:43 smithi049 ceph-mon[31696]: pgmap v288: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-02-26T06:32:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:45 smithi080 ceph-mon[36206]: pgmap v289: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2024-02-26T06:32:45.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:45 smithi115 ceph-mon[36426]: pgmap v289: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2024-02-26T06:32:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:45 smithi049 ceph-mon[31696]: pgmap v289: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2024-02-26T06:32:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 535.0 s, 1305.0 tps, lat 76.450 ms stddev 123.906 2024-02-26T06:32:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:47 smithi080 ceph-mon[36206]: pgmap v290: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 1.98k op/s 2024-02-26T06:32:47.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:47 smithi115 ceph-mon[36426]: pgmap v290: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 1.98k op/s 2024-02-26T06:32:47.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:47 smithi049 ceph-mon[31696]: pgmap v290: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 1.98k op/s 2024-02-26T06:32:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.061+0000 7f2a02600640 1 -- 172.21.15.49:0/2781074655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1142d0 msgr2=0x7f29fc1166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.061+0000 7f2a02600640 1 --2- 172.21.15.49:0/2781074655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1142d0 0x7f29fc1166c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009c10 tx=0x7f29f402bee0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.062+0000 7f2a02600640 1 -- 172.21.15.49:0/2781074655 shutdown_connections 2024-02-26T06:32:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.062+0000 7f2a02600640 1 --2- 172.21.15.49:0/2781074655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1142d0 0x7f29fc1166c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.062+0000 7f2a02600640 1 --2- 172.21.15.49:0/2781074655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29fc108cf0 0x7f29fc113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.062+0000 7f2a02600640 1 --2- 172.21.15.49:0/2781074655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29fc1083d0 0x7f29fc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.062+0000 7f2a02600640 1 -- 172.21.15.49:0/2781074655 >> 172.21.15.49:0/2781074655 conn(0x7f29fc06c220 msgr2=0x7f29fc06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.063+0000 7f2a02600640 1 -- 172.21.15.49:0/2781074655 shutdown_connections 2024-02-26T06:32:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.063+0000 7f2a02600640 1 -- 172.21.15.49:0/2781074655 wait complete. 2024-02-26T06:32:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.065+0000 7f2a02600640 1 Processor -- start 2024-02-26T06:32:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.065+0000 7f2a02600640 1 -- start start 2024-02-26T06:32:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f2a02600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1083d0 0x7f29fc1a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f2a02600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29fc108cf0 0x7f29fc1a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f2a02600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29fc1142d0 0x7f29fc19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f2a02600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc19e080 con 0x7f29fc108cf0 2024-02-26T06:32:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f2a02600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc19e1f0 con 0x7f29fc1083d0 2024-02-26T06:32:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f2a02600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc19e3b0 con 0x7f29fc1142d0 2024-02-26T06:32:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f29fb400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1083d0 0x7f29fc1a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f29fb400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1083d0 0x7f29fc1a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57118/0 (socket says 172.21.15.49:57118) 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f29faa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29fc108cf0 0x7f29fc1a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f29fb400640 1 -- 172.21.15.49:0/490711260 learned_addr learned my addr 172.21.15.49:0/490711260 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.066+0000 7f29fbe00640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29fc1142d0 0x7f29fc19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.067+0000 7f29fb400640 1 -- 172.21.15.49:0/490711260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29fc1142d0 msgr2=0x7f29fc19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.067+0000 7f29fb400640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29fc1142d0 0x7f29fc19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.067+0000 7f29fb400640 1 -- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29fc108cf0 msgr2=0x7f29fc1a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.067+0000 7f29fb400640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29fc108cf0 0x7f29fc1a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.067+0000 7f29fb400640 1 -- 172.21.15.49:0/490711260 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f4009840 con 0x7f29fc1083d0 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.067+0000 7f29faa00640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29fc108cf0 0x7f29fc1a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:32:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.067+0000 7f29fb400640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1083d0 0x7f29fc1a3880 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00c920 tx=0x7f29ec00cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.068+0000 7f29f3e00640 1 -- 172.21.15.49:0/490711260 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec007bb0 con 0x7f29fc1083d0 2024-02-26T06:32:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.068+0000 7f29f3e00640 1 -- 172.21.15.49:0/490711260 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29ec007d10 con 0x7f29fc1083d0 2024-02-26T06:32:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.068+0000 7f2a02600640 1 -- 172.21.15.49:0/490711260 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc19e690 con 0x7f29fc1083d0 2024-02-26T06:32:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.068+0000 7f29f3e00640 1 -- 172.21.15.49:0/490711260 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0187c0 con 0x7f29fc1083d0 2024-02-26T06:32:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.068+0000 7f2a02600640 1 -- 172.21.15.49:0/490711260 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc1aa920 con 0x7f29fc1083d0 2024-02-26T06:32:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.076+0000 7f29f3e00640 1 -- 172.21.15.49:0/490711260 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f29ec018920 con 0x7f29fc1083d0 2024-02-26T06:32:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.076+0000 7f2a05459740 1 Processor -- start 2024-02-26T06:32:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.077+0000 7f2a05459740 1 -- start start 2024-02-26T06:32:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.077+0000 7f2a05459740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec46d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.077+0000 7f2a05459740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec46dd80 0x5592ec46e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.077+0000 7f2a05459740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec477300 0x5592ec4796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.077+0000 7f2a05459740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec4295f0 con 0x5592ec46eaf0 2024-02-26T06:32:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.077+0000 7f2a05459740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec3db890 con 0x5592ec477300 2024-02-26T06:32:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.077+0000 7f2a05459740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec3d8dc0 con 0x5592ec46dd80 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29fbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec46d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29faa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec477300 0x5592ec4796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29fbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec46d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42530/0 (socket says 172.21.15.49:42530) 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29fbe00640 1 -- 172.21.15.49:0/3529409951 learned_addr learned my addr 172.21.15.49:0/3529409951 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29fbe00640 1 -- 172.21.15.49:0/3529409951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec46dd80 msgr2=0x5592ec46e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29fbe00640 1 --2- 172.21.15.49:0/3529409951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec46dd80 0x5592ec46e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29fbe00640 1 -- 172.21.15.49:0/3529409951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec477300 msgr2=0x5592ec4796f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29fbe00640 1 --2- 172.21.15.49:0/3529409951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec477300 0x5592ec4796f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.078+0000 7f29fbe00640 1 -- 172.21.15.49:0/3529409951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592ec355ad0 con 0x5592ec46eaf0 2024-02-26T06:32:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.079+0000 7f29fbe00640 1 --2- 172.21.15.49:0/3529409951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec46d840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f29f40096f0 tx=0x7f29f402cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44e16a4942c85c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.079+0000 7f29f3e00640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f29e00a6840 0x7f29e00a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.080+0000 7f29f3e00640 1 -- 172.21.15.49:0/490711260 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f29ec0ce280 con 0x7f29fc1083d0 2024-02-26T06:32:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.081+0000 7f29f0c00640 1 -- 172.21.15.49:0/3529409951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4033ba0 con 0x5592ec46eaf0 2024-02-26T06:32:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.081+0000 7f29f0c00640 1 -- 172.21.15.49:0/3529409951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29f4033d00 con 0x5592ec46eaf0 2024-02-26T06:32:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.081+0000 7f29f0c00640 1 -- 172.21.15.49:0/3529409951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40042a0 con 0x5592ec46eaf0 2024-02-26T06:32:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.081+0000 7f29fb400640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f29e00a6840 0x7f29e00a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.082+0000 7f2a05459740 1 -- 172.21.15.49:0/3529409951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 msgr2=0x5592ec46d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.082+0000 7f2a05459740 1 --2- 172.21.15.49:0/3529409951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec46d840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f29f40096f0 tx=0x7f29f402cb40 comp rx=0 tx=0).stop 2024-02-26T06:32:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.083+0000 7f29fb400640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f29e00a6840 0x7f29e00a8d00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5592ec46f250 tx=0x7f29ec013000 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.083+0000 7f2a05459740 1 -- 172.21.15.49:0/3529409951 shutdown_connections 2024-02-26T06:32:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.083+0000 7f2a05459740 1 --2- 172.21.15.49:0/3529409951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec477300 0x5592ec4796f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.083+0000 7f2a05459740 1 --2- 172.21.15.49:0/3529409951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec46dd80 0x5592ec46e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.083+0000 7f2a05459740 1 --2- 172.21.15.49:0/3529409951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec46d840 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.083+0000 7f2a05459740 1 -- 172.21.15.49:0/3529409951 >> 172.21.15.49:0/3529409951 conn(0x5592ec435480 msgr2=0x5592ec468a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.083+0000 7f2a05459740 1 -- 172.21.15.49:0/3529409951 shutdown_connections 2024-02-26T06:32:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.083+0000 7f2a05459740 1 -- 172.21.15.49:0/3529409951 wait complete. 2024-02-26T06:32:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.084+0000 7f2a05459740 1 -- start start 2024-02-26T06:32:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.084+0000 7f2a05459740 1 Processor -- start 2024-02-26T06:32:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.086+0000 7f2a05459740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec474370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.086+0000 7f2a05459740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec4748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.086+0000 7f2a05459740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec474df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.086+0000 7f2a05459740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec3d45a0 con 0x5592ec46dd80 2024-02-26T06:32:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.086+0000 7f2a05459740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec379b20 con 0x5592ec46eaf0 2024-02-26T06:32:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.086+0000 7f2a05459740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec3797c0 con 0x5592ec477300 2024-02-26T06:32:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.086+0000 7f2a05459740 10 client.0 _ll_register_callbacks cb 0x5592ec38d520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:32:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.087+0000 7f29faa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec4748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.087+0000 7f29faa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec4748b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57132/0 (socket says 172.21.15.49:57132) 2024-02-26T06:32:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.087+0000 7f29faa00640 1 -- 172.21.15.49:0/2397552899 learned_addr learned my addr 172.21.15.49:0/2397552899 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.087+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec474df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.087+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec474370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.091+0000 7f29fbe00640 1 -- 172.21.15.49:0/2397552899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 msgr2=0x5592ec474df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.091+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec474df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.091+0000 7f29fbe00640 1 -- 172.21.15.49:0/2397552899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 msgr2=0x5592ec4748b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.091+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec4748b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.093+0000 7f29fbe00640 1 -- 172.21.15.49:0/2397552899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc117050 con 0x5592ec46dd80 2024-02-26T06:32:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.094+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec474370 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f29f4003f50 tx=0x7f29f4003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbebc2269683344a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.095+0000 7f29e5600640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:32:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.095+0000 7f29e5600640 1 -- 172.21.15.49:0/2397552899 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004440 con 0x5592ec46dd80 2024-02-26T06:32:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.095+0000 7f29e5600640 1 -- 172.21.15.49:0/2397552899 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f29f40045a0 con 0x5592ec46dd80 2024-02-26T06:32:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.095+0000 7f2a05459740 1 -- 172.21.15.49:0/2397552899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592ec3a6ca0 con 0x5592ec46dd80 2024-02-26T06:32:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.096+0000 7f29e5600640 1 -- 172.21.15.49:0/2397552899 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f4004760 con 0x5592ec46dd80 2024-02-26T06:32:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.097+0000 7f2a05459740 10 client.16704 fetch_fsmap learned FSMap version 40 2024-02-26T06:32:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.097+0000 7f2a05459740 1 -- 172.21.15.49:0/2397552899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5592ec3d2c20 con 0x5592ec46dd80 2024-02-26T06:32:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.099+0000 7f29e5600640 1 -- 172.21.15.49:0/2397552899 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5592ec3a6ca0 con 0x5592ec46dd80 2024-02-26T06:32:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.099+0000 7f29e5600640 1 -- 172.21.15.49:0/2397552899 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f29f403c050 con 0x5592ec46dd80 2024-02-26T06:32:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.099+0000 7f2a05459740 10 client.16704 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:32:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.099+0000 7f2a05459740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:32:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.099+0000 7f2a05459740 20 client.16704 populate_metadata read hostname 'smithi049' 2024-02-26T06:32:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.099+0000 7f2a05459740 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5592ec48f5c0 0x5592ec4919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.099+0000 7f2a05459740 4 client.16704 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:32:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.099+0000 7f2a05459740 1 -- 172.21.15.49:0/2397552899 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592ec3cacb0 con 0x5592ec48f5c0 2024-02-26T06:32:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.101+0000 7f29faa00640 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5592ec48f5c0 0x5592ec4919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.105+0000 7f29faa00640 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5592ec48f5c0 0x5592ec4919e0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ea1f54edb5e6ba server_cookie=3ccaefc5eb5c62e4 in_seq=0 out_seq=0 2024-02-26T06:32:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.106+0000 7f29e5600640 10 client.16704 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.107+0000 7f29e5600640 1 -- 172.21.15.49:0/2397552899 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592ec3cacb0 con 0x5592ec48f5c0 2024-02-26T06:32:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.107+0000 7f29e5600640 10 client.16704 handle_command_reply: tid=0 2024-02-26T06:32:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.107+0000 7f2a05459740 1 client.16704 shutdown 2024-02-26T06:32:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f2a05459740 1 -- 172.21.15.49:0/2397552899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 msgr2=0x5592ec474370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f2a05459740 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec474370 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f29f4003f50 tx=0x7f29f4003e50 comp rx=0 tx=0).stop 2024-02-26T06:32:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f2a05459740 1 -- 172.21.15.49:0/2397552899 shutdown_connections 2024-02-26T06:32:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f2a05459740 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5592ec48f5c0 0x5592ec4919e0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f2a05459740 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec4748b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f2a05459740 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec474df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f29e5600640 0 client.16704 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f2a05459740 1 --2- 172.21.15.49:0/2397552899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec474370 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.110+0000 7f2a05459740 1 -- 172.21.15.49:0/2397552899 >> 172.21.15.49:0/2397552899 conn(0x5592ec435480 msgr2=0x5592ec46a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.111+0000 7f2a05459740 1 -- 172.21.15.49:0/2397552899 shutdown_connections 2024-02-26T06:32:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.112+0000 7f2a05459740 1 -- 172.21.15.49:0/2397552899 wait complete. 2024-02-26T06:32:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.115+0000 7f2a05459740 20 client.16704 trim_cache size 0 max 16384 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.145+0000 7f2a05459740 1 Processor -- start 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.145+0000 7f2a05459740 1 -- start start 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f2a05459740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46dd80 0x5592ec4715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f2a05459740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec471b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f2a05459740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec474330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f2a05459740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec11ce60 con 0x5592ec46eaf0 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f2a05459740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec3a6ca0 con 0x5592ec46dd80 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f2a05459740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec46f0e0 con 0x5592ec477300 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f29fbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46dd80 0x5592ec4715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f29fbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46dd80 0x5592ec4715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57136/0 (socket says 172.21.15.49:57136) 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f29fbe00640 1 -- 172.21.15.49:0/2043257378 learned_addr learned my addr 172.21.15.49:0/2043257378 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.146+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2043257378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec474330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.147+0000 7f29fbe00640 1 -- 172.21.15.49:0/2043257378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 msgr2=0x5592ec474330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.147+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2043257378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec474330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.147+0000 7f29fbe00640 1 -- 172.21.15.49:0/2043257378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 msgr2=0x5592ec471b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.147+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2043257378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec471b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.147+0000 7f29fbe00640 1 -- 172.21.15.49:0/2043257378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592ec36f500 con 0x5592ec46dd80 2024-02-26T06:32:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.147+0000 7f29fbe00640 1 --2- 172.21.15.49:0/2043257378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46dd80 0x5592ec4715d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29f4033770 tx=0x7f29f402bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b65dc3c6b4d70d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.147+0000 7f29e7e00640 1 -- 172.21.15.49:0/2043257378 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4033c50 con 0x5592ec46dd80 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.148+0000 7f29e7e00640 1 -- 172.21.15.49:0/2043257378 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29f4033810 con 0x5592ec46dd80 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.148+0000 7f29e7e00640 1 -- 172.21.15.49:0/2043257378 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f404acb0 con 0x5592ec46dd80 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.149+0000 7f2a05459740 1 -- 172.21.15.49:0/2043257378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46dd80 msgr2=0x5592ec4715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.149+0000 7f2a05459740 1 --2- 172.21.15.49:0/2043257378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46dd80 0x5592ec4715d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29f4033770 tx=0x7f29f402bee0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.149+0000 7f2a05459740 1 -- 172.21.15.49:0/2043257378 shutdown_connections 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.149+0000 7f2a05459740 1 --2- 172.21.15.49:0/2043257378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec474330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.149+0000 7f2a05459740 1 --2- 172.21.15.49:0/2043257378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46eaf0 0x5592ec471b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.149+0000 7f2a05459740 1 --2- 172.21.15.49:0/2043257378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46dd80 0x5592ec4715d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.149+0000 7f2a05459740 1 -- 172.21.15.49:0/2043257378 >> 172.21.15.49:0/2043257378 conn(0x5592ec435480 msgr2=0x5592ec472c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.150+0000 7f2a05459740 1 -- 172.21.15.49:0/2043257378 shutdown_connections 2024-02-26T06:32:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.150+0000 7f2a05459740 1 -- 172.21.15.49:0/2043257378 wait complete. 2024-02-26T06:32:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.150+0000 7f2a05459740 1 -- start start 2024-02-26T06:32:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.151+0000 7f2a05459740 1 Processor -- start 2024-02-26T06:32:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.151+0000 7f2a05459740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec47cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f2a05459740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec47d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f2a05459740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec47d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f2a05459740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec4e5a90 con 0x5592ec46dd80 2024-02-26T06:32:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f2a05459740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec4e5bd0 con 0x5592ec46eaf0 2024-02-26T06:32:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f29faa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec47d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f2a05459740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ec4e5d60 con 0x5592ec477300 2024-02-26T06:32:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f29faa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec47d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:57152/0 (socket says 172.21.15.49:57152) 2024-02-26T06:32:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f29faa00640 1 -- 172.21.15.49:0/1975380341 learned_addr learned my addr 172.21.15.49:0/1975380341 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f2a05459740 10 client.0 _ll_register_callbacks cb 0x7f29f4033ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.152+0000 7f29fbe00640 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec47d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.153+0000 7f29fbe00640 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec47cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.153+0000 7f29faa00640 1 -- 172.21.15.49:0/1975380341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 msgr2=0x5592ec47d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.153+0000 7f29faa00640 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec47d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.153+0000 7f29faa00640 1 -- 172.21.15.49:0/1975380341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 msgr2=0x5592ec47cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.153+0000 7f29faa00640 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec47cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.153+0000 7f29faa00640 1 -- 172.21.15.49:0/1975380341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592ec3d2c20 con 0x5592ec46eaf0 2024-02-26T06:32:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.153+0000 7f29fbe00640 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec47d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:32:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.154+0000 7f29faa00640 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec47d0f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5592ec4922b0 tx=0x7f29e8009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209cbac9a0e3bf7f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:32:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.154+0000 7f29e5600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:32:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.154+0000 7f29e5600640 1 -- 172.21.15.49:0/1975380341 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8017070 con 0x5592ec46eaf0 2024-02-26T06:32:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.154+0000 7f29e5600640 1 -- 172.21.15.49:0/1975380341 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f29e8012e20 con 0x5592ec46eaf0 2024-02-26T06:32:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.154+0000 7f2a05459740 1 -- 172.21.15.49:0/1975380341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592ec3f9d00 con 0x5592ec46eaf0 2024-02-26T06:32:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.155+0000 7f29e5600640 1 -- 172.21.15.49:0/1975380341 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e80122e0 con 0x5592ec46eaf0 2024-02-26T06:32:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.155+0000 7f2a05459740 10 client.27311 fetch_fsmap learned FSMap version 40 2024-02-26T06:32:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.155+0000 7f2a05459740 1 -- 172.21.15.49:0/1975380341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5592ec395290 con 0x5592ec46eaf0 2024-02-26T06:32:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.155+0000 7f29e5600640 1 -- 172.21.15.49:0/1975380341 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5592ec3f9d00 con 0x5592ec46eaf0 2024-02-26T06:32:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.156+0000 7f29e5600640 1 -- 172.21.15.49:0/1975380341 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f29e80216f0 con 0x5592ec46eaf0 2024-02-26T06:32:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.156+0000 7f2a05459740 10 client.27311 fetch_fsmap finished waiting for FSMap version 40 2024-02-26T06:32:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.156+0000 7f2a05459740 10 client.27311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:32:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.156+0000 7f2a05459740 20 client.27311 populate_metadata read hostname 'smithi049' 2024-02-26T06:32:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.156+0000 7f2a05459740 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5592ec4e7640 0x5592ec4e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:32:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.156+0000 7f2a05459740 4 client.27311 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:32:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.156+0000 7f2a05459740 1 -- 172.21.15.49:0/1975380341 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5592ec0d4600 con 0x5592ec4e7640 2024-02-26T06:32:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.156+0000 7f29fbe00640 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5592ec4e7640 0x5592ec4e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:32:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.157+0000 7f29fbe00640 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5592ec4e7640 0x5592ec4e9a60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a36c2f179ab000 server_cookie=964f63ba92fda922 in_seq=0 out_seq=0 2024-02-26T06:32:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.157+0000 7f29e5600640 10 client.27311 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.209+0000 7f29e5600640 1 -- 172.21.15.49:0/1975380341 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5592ec0d4600 con 0x5592ec4e7640 2024-02-26T06:32:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.209+0000 7f29e5600640 10 client.27311 handle_command_reply: tid=0 2024-02-26T06:32:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.209+0000 7f2a05459740 1 client.27311 shutdown 2024-02-26T06:32:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.210+0000 7f2a05459740 1 -- 172.21.15.49:0/1975380341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 msgr2=0x5592ec47d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.210+0000 7f2a05459740 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec47d0f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5592ec4922b0 tx=0x7f29e8009bd0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.210+0000 7f2a05459740 1 -- 172.21.15.49:0/1975380341 shutdown_connections 2024-02-26T06:32:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.210+0000 7f2a05459740 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5592ec4e7640 0x5592ec4e9a60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.210+0000 7f2a05459740 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592ec46eaf0 0x5592ec47d0f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.210+0000 7f29e5600640 0 client.27311 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:32:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.211+0000 7f2a05459740 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592ec477300 0x5592ec47d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.211+0000 7f2a05459740 1 --2- 172.21.15.49:0/1975380341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592ec46dd80 0x5592ec47cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.211+0000 7f2a05459740 1 -- 172.21.15.49:0/1975380341 >> 172.21.15.49:0/1975380341 conn(0x5592ec435480 msgr2=0x5592ec467d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.211+0000 7f2a05459740 1 -- 172.21.15.49:0/1975380341 shutdown_connections 2024-02-26T06:32:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.211+0000 7f2a05459740 1 -- 172.21.15.49:0/1975380341 wait complete. 2024-02-26T06:32:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.212+0000 7f2a05459740 20 client.27311 trim_cache size 0 max 16384 2024-02-26T06:32:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:32:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:32:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:32:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:32:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 -- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f29e00a6840 msgr2=0x7f29e00a8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f29e00a6840 0x7f29e00a8d00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5592ec46f250 tx=0x7f29ec013000 comp rx=0 tx=0).stop 2024-02-26T06:32:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 -- 172.21.15.49:0/490711260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1083d0 msgr2=0x7f29fc1a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:32:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1083d0 0x7f29fc1a3880 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00c920 tx=0x7f29ec00cd20 comp rx=0 tx=0).stop 2024-02-26T06:32:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 -- 172.21.15.49:0/490711260 shutdown_connections 2024-02-26T06:32:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f29e00a6840 0x7f29e00a8d00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29fc1142d0 0x7f29fc19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29fc108cf0 0x7f29fc1a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 --2- 172.21.15.49:0/490711260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc1083d0 0x7f29fc1a3880 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:32:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 -- 172.21.15.49:0/490711260 >> 172.21.15.49:0/490711260 conn(0x7f29fc06c220 msgr2=0x7f29fc06fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:32:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 -- 172.21.15.49:0/490711260 shutdown_connections 2024-02-26T06:32:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:32:48.213+0000 7f2a02600640 1 -- 172.21.15.49:0/490711260 wait complete. 2024-02-26T06:32:49.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:49 smithi080 ceph-mon[36206]: pgmap v291: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 30 MiB/s wr, 1.77k op/s 2024-02-26T06:32:49.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:49 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:32:49.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:49 smithi115 ceph-mon[36426]: pgmap v291: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 30 MiB/s wr, 1.77k op/s 2024-02-26T06:32:49.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:49 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:32:49.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:49 smithi049 ceph-mon[31696]: pgmap v291: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 30 MiB/s wr, 1.77k op/s 2024-02-26T06:32:49.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:49 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:32:50.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:50 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:32:50.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:50 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:32:50.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:50 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:32:50.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:50 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:32:50.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:50 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:32:50.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:50 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:32:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 540.0 s, 1588.4 tps, lat 64.499 ms stddev 79.549 2024-02-26T06:32:51.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:51 smithi080 ceph-mon[36206]: pgmap v292: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 32 MiB/s wr, 1.83k op/s 2024-02-26T06:32:51.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:51 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:32:51.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:51 smithi115 ceph-mon[36426]: pgmap v292: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 32 MiB/s wr, 1.83k op/s 2024-02-26T06:32:51.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:51 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:32:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:51 smithi049 ceph-mon[31696]: pgmap v292: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 32 MiB/s wr, 1.83k op/s 2024-02-26T06:32:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:51 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:32:53.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:53 smithi080 ceph-mon[36206]: pgmap v293: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2024-02-26T06:32:53.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:53 smithi115 ceph-mon[36426]: pgmap v293: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2024-02-26T06:32:53.672 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:53 smithi049 ceph-mon[31696]: pgmap v293: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2024-02-26T06:32:55.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:55 smithi080 ceph-mon[36206]: pgmap v294: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 588 op/s 2024-02-26T06:32:55.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:55 smithi115 ceph-mon[36426]: pgmap v294: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 588 op/s 2024-02-26T06:32:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:55 smithi049 ceph-mon[31696]: pgmap v294: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 588 op/s 2024-02-26T06:32:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 545.0 s, 1880.6 tps, lat 52.987 ms stddev 59.665 2024-02-26T06:32:57.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:57 smithi080 ceph-mon[36206]: pgmap v295: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 764 op/s 2024-02-26T06:32:57.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:57 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:32:57.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:57 smithi115 ceph-mon[36426]: pgmap v295: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 764 op/s 2024-02-26T06:32:57.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:57 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:32:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:57 smithi049 ceph-mon[31696]: pgmap v295: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 764 op/s 2024-02-26T06:32:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:57 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:32:59.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:32:59 smithi080 ceph-mon[36206]: pgmap v296: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 616 op/s 2024-02-26T06:32:59.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:32:59 smithi115 ceph-mon[36426]: pgmap v296: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 616 op/s 2024-02-26T06:32:59.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:32:59 smithi049 ceph-mon[31696]: pgmap v296: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 616 op/s 2024-02-26T06:33:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 550.0 s, 1634.0 tps, lat 60.627 ms stddev 75.014 2024-02-26T06:33:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 -- 172.21.15.49:0/3549540908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424103f70 msgr2=0x7fa424107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 --2- 172.21.15.49:0/3549540908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424103f70 0x7fa424107fb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa410009900 tx=0x7fa41002bc30 comp rx=0 tx=0).stop 2024-02-26T06:33:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 -- 172.21.15.49:0/3549540908 shutdown_connections 2024-02-26T06:33:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 --2- 172.21.15.49:0/3549540908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa424108690 0x7fa424110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 --2- 172.21.15.49:0/3549540908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424103f70 0x7fa424107fb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 --2- 172.21.15.49:0/3549540908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4241035c0 0x7fa4241039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 -- 172.21.15.49:0/3549540908 >> 172.21.15.49:0/3549540908 conn(0x7fa4240fee40 msgr2=0x7fa424101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 -- 172.21.15.49:0/3549540908 shutdown_connections 2024-02-26T06:33:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.371+0000 7fa42ae00640 1 -- 172.21.15.49:0/3549540908 wait complete. 2024-02-26T06:33:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.372+0000 7fa42ae00640 1 Processor -- start 2024-02-26T06:33:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.372+0000 7fa42ae00640 1 -- start start 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa42ae00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4241035c0 0x7fa42419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa42ae00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424108690 0x7fa42419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa42ae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa424199740 0x7fa424199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa42ae00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42419a210 con 0x7fa4241035c0 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa42ae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42419a380 con 0x7fa424199740 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa42ae00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42419a540 con 0x7fa424108690 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa423400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424108690 0x7fa42419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa423400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424108690 0x7fa42419fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58232/0 (socket says 172.21.15.49:58232) 2024-02-26T06:33:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa423400640 1 -- 172.21.15.49:0/787110586 learned_addr learned my addr 172.21.15.49:0/787110586 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa428e00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa424199740 0x7fa424199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.373+0000 7fa423e00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4241035c0 0x7fa42419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.374+0000 7fa423400640 1 -- 172.21.15.49:0/787110586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa424199740 msgr2=0x7fa424199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.374+0000 7fa423400640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa424199740 0x7fa424199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.374+0000 7fa423400640 1 -- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4241035c0 msgr2=0x7fa42419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.374+0000 7fa423400640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4241035c0 0x7fa42419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.374+0000 7fa423400640 1 -- 172.21.15.49:0/787110586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c009840 con 0x7fa424108690 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.374+0000 7fa428e00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa424199740 0x7fa424199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.374+0000 7fa423400640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424108690 0x7fa42419fbc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa410000c00 tx=0x7fa41002cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.374+0000 7fa423e00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4241035c0 0x7fa42419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.375+0000 7fa420c00640 1 -- 172.21.15.49:0/787110586 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41002dea0 con 0x7fa424108690 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.375+0000 7fa420c00640 1 -- 172.21.15.49:0/787110586 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa410004350 con 0x7fa424108690 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.375+0000 7fa420c00640 1 -- 172.21.15.49:0/787110586 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410003b20 con 0x7fa424108690 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.375+0000 7fa42ae00640 1 -- 172.21.15.49:0/787110586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa410009560 con 0x7fa424108690 2024-02-26T06:33:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.375+0000 7fa42ae00640 1 -- 172.21.15.49:0/787110586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4241a6780 con 0x7fa424108690 2024-02-26T06:33:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.378+0000 7fa420c00640 1 -- 172.21.15.49:0/787110586 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa410003c80 con 0x7fa424108690 2024-02-26T06:33:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.380+0000 7fa42dc29740 1 Processor -- start 2024-02-26T06:33:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.380+0000 7fa42dc29740 1 -- start start 2024-02-26T06:33:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd78af0 0x55ec2cd77840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd78200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd81300 0x55ec2cd836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa42dc29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cd335f0 con 0x55ec2cd78af0 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa42dc29740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cce5890 con 0x55ec2cd77d80 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa42dc29740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cce2dc0 con 0x55ec2cd81300 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa420c00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa4080a2350 0x7fa4080a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa420c00640 1 -- 172.21.15.49:0/787110586 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa4100f0650 con 0x7fa424108690 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa423e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd78200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa423e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd78200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34808/0 (socket says 172.21.15.49:34808) 2024-02-26T06:33:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.381+0000 7fa423e00640 1 -- 172.21.15.49:0/1234274965 learned_addr learned my addr 172.21.15.49:0/1234274965 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.382+0000 7fa423400640 1 --2- 172.21.15.49:0/1234274965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd81300 0x55ec2cd836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.382+0000 7fa428e00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa4080a2350 0x7fa4080a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.382+0000 7fa423e00640 1 -- 172.21.15.49:0/1234274965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd81300 msgr2=0x55ec2cd836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.382+0000 7fa423e00640 1 --2- 172.21.15.49:0/1234274965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd81300 0x55ec2cd836f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.382+0000 7fa423e00640 1 -- 172.21.15.49:0/1234274965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd78af0 msgr2=0x55ec2cd77840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.382+0000 7fa423e00640 1 --2- 172.21.15.49:0/1234274965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd78af0 0x55ec2cd77840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.382+0000 7fa423e00640 1 -- 172.21.15.49:0/1234274965 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2cc5fad0 con 0x55ec2cd77d80 2024-02-26T06:33:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.382+0000 7fa423e00640 1 --2- 172.21.15.49:0/1234274965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd78200 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa41400bcc0 tx=0x7fa41400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4b9bcf6896faa7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.383+0000 7fa428e00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa4080a2350 0x7fa4080a4810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009270 tx=0x7fa41c0091e0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.383+0000 7fa419600640 1 -- 172.21.15.49:0/1234274965 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414005020 con 0x55ec2cd77d80 2024-02-26T06:33:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.383+0000 7fa419600640 1 -- 172.21.15.49:0/1234274965 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa414005660 con 0x55ec2cd77d80 2024-02-26T06:33:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.383+0000 7fa419600640 1 -- 172.21.15.49:0/1234274965 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414011730 con 0x55ec2cd77d80 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 -- 172.21.15.49:0/1234274965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 msgr2=0x55ec2cd78200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 --2- 172.21.15.49:0/1234274965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd78200 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa41400bcc0 tx=0x7fa41400f730 comp rx=0 tx=0).stop 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 -- 172.21.15.49:0/1234274965 shutdown_connections 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 --2- 172.21.15.49:0/1234274965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd78200 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 --2- 172.21.15.49:0/1234274965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd81300 0x55ec2cd836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 --2- 172.21.15.49:0/1234274965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd78af0 0x55ec2cd77840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 -- 172.21.15.49:0/1234274965 >> 172.21.15.49:0/1234274965 conn(0x55ec2cd3f480 msgr2=0x55ec2cd72a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 -- 172.21.15.49:0/1234274965 shutdown_connections 2024-02-26T06:33:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.384+0000 7fa42dc29740 1 -- 172.21.15.49:0/1234274965 wait complete. 2024-02-26T06:33:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.385+0000 7fa42dc29740 1 -- start start 2024-02-26T06:33:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.385+0000 7fa42dc29740 1 Processor -- start 2024-02-26T06:33:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd7e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa42dc29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2ccde5a0 con 0x55ec2cd81300 2024-02-26T06:33:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa42dc29740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cc83b20 con 0x55ec2cd78af0 2024-02-26T06:33:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa42dc29740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cc837c0 con 0x55ec2cd77d80 2024-02-26T06:33:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa42dc29740 10 client.0 _ll_register_callbacks cb 0x55ec2cc97520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa423400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa423400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34810/0 (socket says 172.21.15.49:34810) 2024-02-26T06:33:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.386+0000 7fa423400640 1 -- 172.21.15.49:0/2779766867 learned_addr learned my addr 172.21.15.49:0/2779766867 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa423e00640 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd7e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa428e00640 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa423400640 1 -- 172.21.15.49:0/2779766867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 msgr2=0x55ec2cd7e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa423400640 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd7e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa423400640 1 -- 172.21.15.49:0/2779766867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 msgr2=0x55ec2cd7edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa423400640 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa423400640 1 -- 172.21.15.49:0/2779766867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41400b920 con 0x55ec2cd78af0 2024-02-26T06:33:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa428e00640 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:33:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa423400640 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7e870 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa41002c170 tx=0x7fa410004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5a08048222fd63 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.387+0000 7fa40e000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:33:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.388+0000 7fa40e000640 1 -- 172.21.15.49:0/2779766867 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100bd160 con 0x55ec2cd78af0 2024-02-26T06:33:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.388+0000 7fa40e000640 1 -- 172.21.15.49:0/2779766867 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa4100c12b0 con 0x55ec2cd78af0 2024-02-26T06:33:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.388+0000 7fa42dc29740 1 -- 172.21.15.49:0/2779766867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec2ccb0ca0 con 0x55ec2cd78af0 2024-02-26T06:33:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.388+0000 7fa40e000640 1 -- 172.21.15.49:0/2779766867 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4100c5870 con 0x55ec2cd78af0 2024-02-26T06:33:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa42dc29740 10 client.27320 fetch_fsmap learned FSMap version 41 2024-02-26T06:33:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa42dc29740 1 -- 172.21.15.49:0/2779766867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ec2ccdcc20 con 0x55ec2cd78af0 2024-02-26T06:33:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa40e000640 1 -- 172.21.15.49:0/2779766867 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ec2ccb0ca0 con 0x55ec2cd78af0 2024-02-26T06:33:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa40e000640 1 -- 172.21.15.49:0/2779766867 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa41003c070 con 0x55ec2cd78af0 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa42dc29740 10 client.27320 fetch_fsmap finished waiting for FSMap version 41 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa42dc29740 10 client.27320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa42dc29740 20 client.27320 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2cd99a80 0x55ec2cd9bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa42dc29740 4 client.27320 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.389+0000 7fa42dc29740 1 -- 172.21.15.49:0/2779766867 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec2ccd4cb0 con 0x55ec2cd99a80 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.390+0000 7fa423e00640 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2cd99a80 0x55ec2cd9bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.390+0000 7fa40e000640 10 client.27320 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.391+0000 7fa423e00640 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2cd99a80 0x55ec2cd9bea0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8223537331ec4946 server_cookie=7eb46f53f521f47e in_seq=0 out_seq=0 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.391+0000 7fa40e000640 1 -- 172.21.15.49:0/2779766867 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec2ccd4cb0 con 0x55ec2cd99a80 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.391+0000 7fa40e000640 10 client.27320 handle_command_reply: tid=0 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 client.27320 shutdown 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 -- 172.21.15.49:0/2779766867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 msgr2=0x55ec2cd7e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7e870 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa41002c170 tx=0x7fa410004580 comp rx=0 tx=0).stop 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 -- 172.21.15.49:0/2779766867 shutdown_connections 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2cd99a80 0x55ec2cd9bea0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7e870 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2779766867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd7e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 -- 172.21.15.49:0/2779766867 >> 172.21.15.49:0/2779766867 conn(0x55ec2cd3f480 msgr2=0x55ec2cd747c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa40e000640 0 client.27320 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 -- 172.21.15.49:0/2779766867 shutdown_connections 2024-02-26T06:33:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 1 -- 172.21.15.49:0/2779766867 wait complete. 2024-02-26T06:33:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.392+0000 7fa42dc29740 20 client.27320 trim_cache size 0 max 16384 2024-02-26T06:33:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.414+0000 7fa42dc29740 1 Processor -- start 2024-02-26T06:33:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.414+0000 7fa42dc29740 1 -- start start 2024-02-26T06:33:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.414+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd80a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.414+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.414+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd90a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.414+0000 7fa42dc29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2ca26e60 con 0x55ec2cd81300 2024-02-26T06:33:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.414+0000 7fa42dc29740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cd64670 con 0x55ec2cd78af0 2024-02-26T06:33:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.414+0000 7fa42dc29740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cd83f10 con 0x55ec2cd77d80 2024-02-26T06:33:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd80a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd80a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58266/0 (socket says 172.21.15.49:58266) 2024-02-26T06:33:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 -- 172.21.15.49:0/1359316665 learned_addr learned my addr 172.21.15.49:0/1359316665 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa428e00640 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd90a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 -- 172.21.15.49:0/1359316665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 msgr2=0x55ec2cd7ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 -- 172.21.15.49:0/1359316665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 msgr2=0x55ec2cd90a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd90a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 -- 172.21.15.49:0/1359316665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2cc79500 con 0x55ec2cd77d80 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423400640 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.415+0000 7fa423e00640 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd80a60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa41400bcc0 tx=0x7fa41400b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f1cfeb094cab53 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.416+0000 7fa418c00640 1 -- 172.21.15.49:0/1359316665 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414027460 con 0x55ec2cd77d80 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.416+0000 7fa418c00640 1 -- 172.21.15.49:0/1359316665 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa414022040 con 0x55ec2cd77d80 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.416+0000 7fa418c00640 1 -- 172.21.15.49:0/1359316665 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414003850 con 0x55ec2cd77d80 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.416+0000 7fa42dc29740 1 -- 172.21.15.49:0/1359316665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 msgr2=0x55ec2cd80a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.416+0000 7fa42dc29740 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd80a60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa41400bcc0 tx=0x7fa41400b5e0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 -- 172.21.15.49:0/1359316665 shutdown_connections 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd90a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 --2- 172.21.15.49:0/1359316665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd77d80 0x55ec2cd80a60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 -- 172.21.15.49:0/1359316665 >> 172.21.15.49:0/1359316665 conn(0x55ec2cd3f480 msgr2=0x55ec2cd725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 -- 172.21.15.49:0/1359316665 shutdown_connections 2024-02-26T06:33:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 -- 172.21.15.49:0/1359316665 wait complete. 2024-02-26T06:33:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 -- start start 2024-02-26T06:33:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.417+0000 7fa42dc29740 1 Processor -- start 2024-02-26T06:33:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd90c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa42dc29740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa42dc29740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cd7b5e0 con 0x55ec2cd81300 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa42dc29740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cd7b750 con 0x55ec2cd77d80 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa42dc29740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2cd7b8c0 con 0x55ec2cd78af0 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa42dc29740 10 client.0 _ll_register_callbacks cb 0x7fa410005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd90c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd90c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34830/0 (socket says 172.21.15.49:34830) 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423e00640 1 -- 172.21.15.49:0/2289968819 learned_addr learned my addr 172.21.15.49:0/2289968819 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423400640 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa428e00640 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423e00640 1 -- 172.21.15.49:0/2289968819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd78af0 msgr2=0x55ec2cd7ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423e00640 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423e00640 1 -- 172.21.15.49:0/2289968819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 msgr2=0x55ec2cd7b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423e00640 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa423e00640 1 -- 172.21.15.49:0/2289968819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec2ccdcc20 con 0x55ec2cd77d80 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.418+0000 7fa428e00640 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa423e00640 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd90c00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa4140277a0 tx=0x7fa414027a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b4902f07f9cab21 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa40e000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa40e000640 1 -- 172.21.15.49:0/2289968819 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140037f0 con 0x55ec2cd77d80 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa40e000640 1 -- 172.21.15.49:0/2289968819 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa414003950 con 0x55ec2cd77d80 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa42dc29740 1 -- 172.21.15.49:0/2289968819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec2cd03d00 con 0x55ec2cd77d80 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa40e000640 1 -- 172.21.15.49:0/2289968819 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa414003b10 con 0x55ec2cd77d80 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa42dc29740 10 client.27326 fetch_fsmap learned FSMap version 41 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa42dc29740 1 -- 172.21.15.49:0/2289968819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ec2cc5fad0 con 0x55ec2cd77d80 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa40e000640 1 -- 172.21.15.49:0/2289968819 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ec2cd03d00 con 0x55ec2cd77d80 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.419+0000 7fa40e000640 1 -- 172.21.15.49:0/2289968819 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa414011740 con 0x55ec2cd77d80 2024-02-26T06:33:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa42dc29740 10 client.27326 fetch_fsmap finished waiting for FSMap version 41 2024-02-26T06:33:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa42dc29740 10 client.27326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa42dc29740 20 client.27326 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2cdf10d0 0x55ec2cdf34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa42dc29740 4 client.27326 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:33:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa42dc29740 1 -- 172.21.15.49:0/2289968819 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ec2c9de600 con 0x55ec2cdf10d0 2024-02-26T06:33:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa423400640 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2cdf10d0 0x55ec2cdf34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa423400640 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2cdf10d0 0x55ec2cdf34c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd026951de9a36c5 server_cookie=2046f14ba1fc06bb in_seq=0 out_seq=0 2024-02-26T06:33:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.420+0000 7fa40e000640 10 client.27326 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.435+0000 7fa40e000640 1 -- 172.21.15.49:0/2289968819 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ec2c9de600 con 0x55ec2cdf10d0 2024-02-26T06:33:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.435+0000 7fa40e000640 10 client.27326 handle_command_reply: tid=0 2024-02-26T06:33:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.435+0000 7fa42dc29740 1 client.27326 shutdown 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 -- 172.21.15.49:0/2289968819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 msgr2=0x55ec2cd90c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd90c00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa4140277a0 tx=0x7fa414027a20 comp rx=0 tx=0).stop 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 -- 172.21.15.49:0/2289968819 shutdown_connections 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2cdf10d0 0x55ec2cdf34c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2cd81300 0x55ec2cd7b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2cd78af0 0x55ec2cd7ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa40e000640 0 client.27326 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 --2- 172.21.15.49:0/2289968819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2cd77d80 0x55ec2cd90c00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 -- 172.21.15.49:0/2289968819 >> 172.21.15.49:0/2289968819 conn(0x55ec2cd3f480 msgr2=0x55ec2cd7cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.436+0000 7fa42dc29740 1 -- 172.21.15.49:0/2289968819 shutdown_connections 2024-02-26T06:33:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.437+0000 7fa42dc29740 1 -- 172.21.15.49:0/2289968819 wait complete. 2024-02-26T06:33:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.437+0000 7fa42dc29740 20 client.27326 trim_cache size 0 max 16384 2024-02-26T06:33:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:33:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:33:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:33:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:33:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 -- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa4080a2350 msgr2=0x7fa4080a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa4080a2350 0x7fa4080a4810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009270 tx=0x7fa41c0091e0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 -- 172.21.15.49:0/787110586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424108690 msgr2=0x7fa42419fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424108690 0x7fa42419fbc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa410000c00 tx=0x7fa41002cd80 comp rx=0 tx=0).stop 2024-02-26T06:33:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 -- 172.21.15.49:0/787110586 shutdown_connections 2024-02-26T06:33:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa4080a2350 0x7fa4080a4810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa424199740 0x7fa424199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa424108690 0x7fa42419fbc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 --2- 172.21.15.49:0/787110586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4241035c0 0x7fa42419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.438+0000 7fa42ae00640 1 -- 172.21.15.49:0/787110586 >> 172.21.15.49:0/787110586 conn(0x7fa4240fee40 msgr2=0x7fa4240ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.439+0000 7fa42ae00640 1 -- 172.21.15.49:0/787110586 shutdown_connections 2024-02-26T06:33:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:01.439+0000 7fa42ae00640 1 -- 172.21.15.49:0/787110586 wait complete. 2024-02-26T06:33:01.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:01 smithi080 ceph-mon[36206]: pgmap v297: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 679 op/s 2024-02-26T06:33:01.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:01 smithi115 ceph-mon[36426]: pgmap v297: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 679 op/s 2024-02-26T06:33:01.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:01 smithi049 ceph-mon[31696]: pgmap v297: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 679 op/s 2024-02-26T06:33:03.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:03 smithi080 ceph-mon[36206]: pgmap v298: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 781 op/s 2024-02-26T06:33:03.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:03 smithi115 ceph-mon[36426]: pgmap v298: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 781 op/s 2024-02-26T06:33:03.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:03 smithi049 ceph-mon[31696]: pgmap v298: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 781 op/s 2024-02-26T06:33:04.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:04 smithi080 ceph-mon[36206]: pgmap v299: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 597 op/s 2024-02-26T06:33:04.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:04 smithi115 ceph-mon[36426]: pgmap v299: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 597 op/s 2024-02-26T06:33:04.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:04 smithi049 ceph-mon[31696]: pgmap v299: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 597 op/s 2024-02-26T06:33:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 555.0 s, 800.8 tps, lat 123.034 ms stddev 214.343 2024-02-26T06:33:06.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:06 smithi080 ceph-mon[36206]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T06:33:06.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:06 smithi080 ceph-mon[36206]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-02-26T06:33:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:06 smithi115 ceph-mon[36426]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T06:33:06.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:06 smithi115 ceph-mon[36426]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-02-26T06:33:06.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:06 smithi049 ceph-mon[31696]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T06:33:06.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:06 smithi049 ceph-mon[31696]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-02-26T06:33:08.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:08 smithi049 ceph-mon[31696]: pgmap v302: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2024-02-26T06:33:09.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:08 smithi080 ceph-mon[36206]: pgmap v302: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2024-02-26T06:33:09.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:08 smithi115 ceph-mon[36426]: pgmap v302: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2024-02-26T06:33:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 560.0 s, 1479.2 tps, lat 68.328 ms stddev 81.127 2024-02-26T06:33:11.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:11 smithi049 ceph-mon[31696]: pgmap v303: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.66k op/s 2024-02-26T06:33:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:11 smithi080 ceph-mon[36206]: pgmap v303: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.66k op/s 2024-02-26T06:33:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:11 smithi115 ceph-mon[36426]: pgmap v303: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.66k op/s 2024-02-26T06:33:13.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:13 smithi049 ceph-mon[31696]: pgmap v304: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-02-26T06:33:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:13 smithi080 ceph-mon[36206]: pgmap v304: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-02-26T06:33:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:13 smithi115 ceph-mon[36426]: pgmap v304: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-02-26T06:33:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.620+0000 7f231c400640 1 -- 172.21.15.49:0/1117905112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231410d760 msgr2=0x7f231410db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.620+0000 7f231c400640 1 --2- 172.21.15.49:0/1117905112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231410d760 0x7f231410db40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2308009930 tx=0x7f230802bc20 comp rx=0 tx=0).stop 2024-02-26T06:33:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.620+0000 7f231c400640 1 -- 172.21.15.49:0/1117905112 shutdown_connections 2024-02-26T06:33:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.621+0000 7f231c400640 1 --2- 172.21.15.49:0/1117905112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2314108110 0x7f2314114620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.621+0000 7f231c400640 1 --2- 172.21.15.49:0/1117905112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2314107750 0x7f2314107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.621+0000 7f231c400640 1 --2- 172.21.15.49:0/1117905112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231410d760 0x7f231410db40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.621+0000 7f231c400640 1 -- 172.21.15.49:0/1117905112 >> 172.21.15.49:0/1117905112 conn(0x7f23140756f0 msgr2=0x7f2314077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.621+0000 7f231c400640 1 -- 172.21.15.49:0/1117905112 shutdown_connections 2024-02-26T06:33:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.621+0000 7f231c400640 1 -- 172.21.15.49:0/1117905112 wait complete. 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.622+0000 7f231c400640 1 Processor -- start 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.622+0000 7f231c400640 1 -- start start 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.622+0000 7f231c400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2314107750 0x7f231419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.622+0000 7f231c400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2314108110 0x7f231419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.622+0000 7f231c400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f231410d760 0x7f23141a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.622+0000 7f231c400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141a2be0 con 0x7f231410d760 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.622+0000 7f231c400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141a2d50 con 0x7f2314107750 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.622+0000 7f231c400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141a2f10 con 0x7f2314108110 2024-02-26T06:33:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2314108110 0x7f231419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2314107750 0x7f231419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f231a400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f231410d760 0x7f23141a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2314107750 0x7f231419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35432/0 (socket says 172.21.15.49:35432) 2024-02-26T06:33:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 -- 172.21.15.49:0/2248409905 learned_addr learned my addr 172.21.15.49:0/2248409905 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2314108110 0x7f231419e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57940/0 (socket says 172.21.15.49:57940) 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 -- 172.21.15.49:0/2248409905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2314108110 msgr2=0x7f231419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2314108110 0x7f231419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 -- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f231410d760 msgr2=0x7f23141a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f231410d760 0x7f23141a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 -- 172.21.15.49:0/2248409905 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2308009560 con 0x7f2314107750 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.623+0000 7f2319a00640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2314107750 0x7f231419dd20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f230802cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.624+0000 7f2312000640 1 -- 172.21.15.49:0/2248409905 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230802dea0 con 0x7f2314107750 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.624+0000 7f2312000640 1 -- 172.21.15.49:0/2248409905 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2308004350 con 0x7f2314107750 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.624+0000 7f2312000640 1 -- 172.21.15.49:0/2248409905 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308003b50 con 0x7f2314107750 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.624+0000 7f231c400640 1 -- 172.21.15.49:0/2248409905 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23141a3190 con 0x7f2314107750 2024-02-26T06:33:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.624+0000 7f231c400640 1 -- 172.21.15.49:0/2248409905 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23141aaa70 con 0x7f2314107750 2024-02-26T06:33:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.631+0000 7f2312000640 1 -- 172.21.15.49:0/2248409905 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2308003cb0 con 0x7f2314107750 2024-02-26T06:33:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.631+0000 7f231f432740 1 Processor -- start 2024-02-26T06:33:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.632+0000 7f231f432740 1 -- start start 2024-02-26T06:33:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.633+0000 7f231f432740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a083c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.633+0000 7f231f432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a083d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.633+0000 7f231f432740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a0846300 0x5583a08486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.633+0000 7f231f432740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a07f85f0 con 0x5583a0846300 2024-02-26T06:33:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.633+0000 7f231f432740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a07aa890 con 0x5583a083daf0 2024-02-26T06:33:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.633+0000 7f231f432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a07a7dc0 con 0x5583a083cd80 2024-02-26T06:33:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.633+0000 7f2312000640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22fc0a2350 0x7f22fc0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.634+0000 7f231a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a083c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.634+0000 7f2312000640 1 -- 172.21.15.49:0/2248409905 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f23080f04a0 con 0x7f2314107750 2024-02-26T06:33:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.634+0000 7f231a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a083c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35444/0 (socket says 172.21.15.49:35444) 2024-02-26T06:33:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.634+0000 7f231a400640 1 -- 172.21.15.49:0/2894607855 learned_addr learned my addr 172.21.15.49:0/2894607855 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.634+0000 7f2319000640 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a0846300 0x5583a08486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.634+0000 7f2319a00640 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a083d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.634+0000 7f231a400640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22fc0a2350 0x7f22fc0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.635+0000 7f2319a00640 1 -- 172.21.15.49:0/2894607855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 msgr2=0x5583a083c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.635+0000 7f2319a00640 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a083c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.635+0000 7f2319a00640 1 -- 172.21.15.49:0/2894607855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a0846300 msgr2=0x5583a08486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.635+0000 7f2319a00640 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a0846300 0x5583a08486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.635+0000 7f2319a00640 1 -- 172.21.15.49:0/2894607855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583a0724ad0 con 0x5583a083cd80 2024-02-26T06:33:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.635+0000 7f2319a00640 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a083d200 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f23080edcb0 tx=0x7f23080bca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146260233ba24b53 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.635+0000 7f2306a00640 1 -- 172.21.15.49:0/2894607855 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080c1130 con 0x5583a083cd80 2024-02-26T06:33:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.636+0000 7f2306a00640 1 -- 172.21.15.49:0/2894607855 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23080c56f0 con 0x5583a083cd80 2024-02-26T06:33:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.636+0000 7f2306a00640 1 -- 172.21.15.49:0/2894607855 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080c54c0 con 0x5583a083cd80 2024-02-26T06:33:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.636+0000 7f231f432740 1 -- 172.21.15.49:0/2894607855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 msgr2=0x5583a083d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.636+0000 7f231f432740 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a083d200 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f23080edcb0 tx=0x7f23080bca00 comp rx=0 tx=0).stop 2024-02-26T06:33:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.636+0000 7f231a400640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22fc0a2350 0x7f22fc0a4810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f230c009fd0 tx=0x7f230c0096a0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 -- 172.21.15.49:0/2894607855 shutdown_connections 2024-02-26T06:33:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a0846300 0x5583a08486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a083d200 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 --2- 172.21.15.49:0/2894607855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a083c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 -- 172.21.15.49:0/2894607855 >> 172.21.15.49:0/2894607855 conn(0x5583a0804480 msgr2=0x5583a0837a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 -- 172.21.15.49:0/2894607855 shutdown_connections 2024-02-26T06:33:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 -- 172.21.15.49:0/2894607855 wait complete. 2024-02-26T06:33:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 -- start start 2024-02-26T06:33:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.637+0000 7f231f432740 1 Processor -- start 2024-02-26T06:33:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f231f432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a08433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f231f432740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a08438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f231f432740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a0846300 0x5583a0843e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f231f432740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a07a35a0 con 0x5583a083daf0 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f231f432740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a0748b20 con 0x5583a0846300 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f231f432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a07487c0 con 0x5583a083cd80 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f231f432740 10 client.0 _ll_register_callbacks cb 0x5583a075c520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f2319a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a08433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f2319a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a08433b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57946/0 (socket says 172.21.15.49:57946) 2024-02-26T06:33:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f2319a00640 1 -- 172.21.15.49:0/3519152448 learned_addr learned my addr 172.21.15.49:0/3519152448 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.639+0000 7f231a400640 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a0846300 0x5583a0843e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.640+0000 7f2319a00640 1 -- 172.21.15.49:0/3519152448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a0846300 msgr2=0x5583a0843e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.640+0000 7f2319a00640 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a0846300 0x5583a0843e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.640+0000 7f2319a00640 1 -- 172.21.15.49:0/3519152448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 msgr2=0x5583a08438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.640+0000 7f2319000640 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a08438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.640+0000 7f2319a00640 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a08438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.640+0000 7f2319a00640 1 -- 172.21.15.49:0/3519152448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23141aaa70 con 0x5583a083cd80 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.640+0000 7f2319000640 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a08438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f2319a00640 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a08433b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f23080b9440 tx=0x7f230802bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8086094dbdba891 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f22fb400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f22fb400640 1 -- 172.21.15.49:0/3519152448 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080c1130 con 0x5583a083cd80 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f22fb400640 1 -- 172.21.15.49:0/3519152448 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f23080bb2e0 con 0x5583a083cd80 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f231f432740 1 -- 172.21.15.49:0/3519152448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583a0775ca0 con 0x5583a083cd80 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f22fb400640 1 -- 172.21.15.49:0/3519152448 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23080c5470 con 0x5583a083cd80 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f231f432740 10 client.27304 fetch_fsmap learned FSMap version 41 2024-02-26T06:33:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f231f432740 1 -- 172.21.15.49:0/3519152448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5583a07a1c20 con 0x5583a083cd80 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f22fb400640 1 -- 172.21.15.49:0/3519152448 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583a0775ca0 con 0x5583a083cd80 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.641+0000 7f22fb400640 1 -- 172.21.15.49:0/3519152448 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f230803c070 con 0x5583a083cd80 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f231f432740 10 client.27304 fetch_fsmap finished waiting for FSMap version 41 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f231f432740 10 client.27304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f231f432740 20 client.27304 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f231f432740 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583a0855cc0 0x5583a08580d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f231f432740 4 client.27304 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f231f432740 1 -- 172.21.15.49:0/3519152448 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583a0799cb0 con 0x5583a0855cc0 2024-02-26T06:33:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f2319000640 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583a0855cc0 0x5583a08580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f2319000640 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583a0855cc0 0x5583a08580d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca82f94e0f725ac3 server_cookie=ad5af1f3261c0840 in_seq=0 out_seq=0 2024-02-26T06:33:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.642+0000 7f22fb400640 10 client.27304 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.643+0000 7f22fb400640 1 -- 172.21.15.49:0/3519152448 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583a0799cb0 con 0x5583a0855cc0 2024-02-26T06:33:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.643+0000 7f22fb400640 10 client.27304 handle_command_reply: tid=0 2024-02-26T06:33:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.644+0000 7f231f432740 1 client.27304 shutdown 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f231f432740 1 -- 172.21.15.49:0/3519152448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 msgr2=0x5583a08433b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f231f432740 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a08433b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f23080b9440 tx=0x7f230802bc20 comp rx=0 tx=0).stop 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f231f432740 1 -- 172.21.15.49:0/3519152448 shutdown_connections 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f231f432740 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583a0855cc0 0x5583a08580d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f231f432740 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a0846300 0x5583a0843e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f231f432740 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a08438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f231f432740 1 --2- 172.21.15.49:0/3519152448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a083cd80 0x5583a08433b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f231f432740 1 -- 172.21.15.49:0/3519152448 >> 172.21.15.49:0/3519152448 conn(0x5583a0804480 msgr2=0x5583a08397c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.645+0000 7f22fb400640 0 client.27304 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.646+0000 7f231f432740 1 -- 172.21.15.49:0/3519152448 shutdown_connections 2024-02-26T06:33:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.646+0000 7f231f432740 1 -- 172.21.15.49:0/3519152448 wait complete. 2024-02-26T06:33:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.646+0000 7f231f432740 20 client.27304 trim_cache size 0 max 16384 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.667+0000 7f231f432740 1 Processor -- start 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.667+0000 7f231f432740 1 -- start start 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231f432740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083cd80 0x5583a0845ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231f432740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a0843c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231f432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231f432740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a04ebe60 con 0x5583a083cd80 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231f432740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a0799cb0 con 0x5583a083daf0 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231f432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a0775ca0 con 0x5583a0846300 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f2319a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083cd80 0x5583a0845ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f2319000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a0843c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231a400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57960/0 (socket says 172.21.15.49:57960) 2024-02-26T06:33:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f2319000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a0843c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35460/0 (socket says 172.21.15.49:35460) 2024-02-26T06:33:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f2319a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083cd80 0x5583a0845ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32946/0 (socket says 172.21.15.49:32946) 2024-02-26T06:33:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f231a400640 1 -- 172.21.15.49:0/27533689 learned_addr learned my addr 172.21.15.49:0/27533689 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f2319000640 1 -- 172.21.15.49:0/27533689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 msgr2=0x5583a084cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f2319000640 1 --2- 172.21.15.49:0/27533689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.668+0000 7f2319000640 1 -- 172.21.15.49:0/27533689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083cd80 msgr2=0x5583a0845ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.669+0000 7f2319000640 1 --2- 172.21.15.49:0/27533689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083cd80 0x5583a0845ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.669+0000 7f2319000640 1 -- 172.21.15.49:0/27533689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583a073e500 con 0x5583a083daf0 2024-02-26T06:33:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.669+0000 7f2319a00640 1 --2- 172.21.15.49:0/27533689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083cd80 0x5583a0845ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:33:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.670+0000 7f2319000640 1 --2- 172.21.15.49:0/27533689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a0843c50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5583a0858a20 tx=0x7f2300012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6ecf5a2e234d711 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.670+0000 7f2306000640 1 -- 172.21.15.49:0/27533689 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300012ec0 con 0x5583a083daf0 2024-02-26T06:33:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.670+0000 7f2306000640 1 -- 172.21.15.49:0/27533689 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23000122d0 con 0x5583a083daf0 2024-02-26T06:33:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.670+0000 7f2306000640 1 -- 172.21.15.49:0/27533689 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230001e070 con 0x5583a083daf0 2024-02-26T06:33:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.671+0000 7f231f432740 1 -- 172.21.15.49:0/27533689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 msgr2=0x5583a0843c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.671+0000 7f231f432740 1 --2- 172.21.15.49:0/27533689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a0843c50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5583a0858a20 tx=0x7f2300012820 comp rx=0 tx=0).stop 2024-02-26T06:33:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.671+0000 7f231f432740 1 -- 172.21.15.49:0/27533689 shutdown_connections 2024-02-26T06:33:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.671+0000 7f231f432740 1 --2- 172.21.15.49:0/27533689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083daf0 0x5583a0843c50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.671+0000 7f231f432740 1 --2- 172.21.15.49:0/27533689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.671+0000 7f231f432740 1 --2- 172.21.15.49:0/27533689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083cd80 0x5583a0845ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.671+0000 7f231f432740 1 -- 172.21.15.49:0/27533689 >> 172.21.15.49:0/27533689 conn(0x5583a0804480 msgr2=0x5583a08375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.672+0000 7f231f432740 1 -- 172.21.15.49:0/27533689 shutdown_connections 2024-02-26T06:33:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.672+0000 7f231f432740 1 -- 172.21.15.49:0/27533689 wait complete. 2024-02-26T06:33:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.672+0000 7f231f432740 1 -- start start 2024-02-26T06:33:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.672+0000 7f231f432740 1 Processor -- start 2024-02-26T06:33:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f231f432740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083cd80 0x5583a0849960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f231f432740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a0849ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f231f432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f231f432740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a08ac1f0 con 0x5583a083daf0 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f231f432740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a08ac330 con 0x5583a083cd80 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f231f432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583a08ac470 con 0x5583a0846300 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f231f432740 10 client.0 _ll_register_callbacks cb 0x7f23080d3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083cd80 0x5583a0849960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083cd80 0x5583a0849960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:35472/0 (socket says 172.21.15.49:35472) 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319a00640 1 -- 172.21.15.49:0/365150036 learned_addr learned my addr 172.21.15.49:0/365150036 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319000640 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a0849ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f231a400640 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319a00640 1 -- 172.21.15.49:0/365150036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 msgr2=0x5583a084a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319a00640 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319a00640 1 -- 172.21.15.49:0/365150036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 msgr2=0x5583a0849ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319a00640 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a0849ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.673+0000 7f2319a00640 1 -- 172.21.15.49:0/365150036 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583a07a1c20 con 0x5583a083cd80 2024-02-26T06:33:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.674+0000 7f2319a00640 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083cd80 0x5583a0849960 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f23080ee1a0 tx=0x7f2308003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba04ca3c39122c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.674+0000 7f22fb400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:33:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.674+0000 7f22fb400640 1 -- 172.21.15.49:0/365150036 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080c9070 con 0x5583a083cd80 2024-02-26T06:33:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.674+0000 7f22fb400640 1 -- 172.21.15.49:0/365150036 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f23080c1130 con 0x5583a083cd80 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.674+0000 7f231f432740 1 -- 172.21.15.49:0/365150036 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583a04a3600 con 0x5583a083cd80 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.675+0000 7f22fb400640 1 -- 172.21.15.49:0/365150036 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23080a6310 con 0x5583a083cd80 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.675+0000 7f231f432740 10 client.27344 fetch_fsmap learned FSMap version 41 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.675+0000 7f231f432740 1 -- 172.21.15.49:0/365150036 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f23000096a0 con 0x5583a083cd80 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.675+0000 7f22fb400640 1 -- 172.21.15.49:0/365150036 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583a04a3600 con 0x5583a083cd80 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.676+0000 7f22fb400640 1 -- 172.21.15.49:0/365150036 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f23080cc020 con 0x5583a083cd80 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.676+0000 7f231f432740 10 client.27344 fetch_fsmap finished waiting for FSMap version 41 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.676+0000 7f231f432740 10 client.27344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.676+0000 7f231f432740 20 client.27344 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.676+0000 7f231f432740 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583a08ad7f0 0x5583a08afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.676+0000 7f231f432740 4 client.27344 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.676+0000 7f231f432740 1 -- 172.21.15.49:0/365150036 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5583a04f41b0 con 0x5583a08ad7f0 2024-02-26T06:33:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.676+0000 7f2319000640 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583a08ad7f0 0x5583a08afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.677+0000 7f2319000640 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583a08ad7f0 0x5583a08afc10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ee6e720f71a0ab server_cookie=673705f3fdf546a6 in_seq=0 out_seq=0 2024-02-26T06:33:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.677+0000 7f22fb400640 10 client.27344 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.695+0000 7f22fb400640 1 -- 172.21.15.49:0/365150036 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5583a04f41b0 con 0x5583a08ad7f0 2024-02-26T06:33:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.695+0000 7f22fb400640 10 client.27344 handle_command_reply: tid=0 2024-02-26T06:33:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.695+0000 7f231f432740 1 client.27344 shutdown 2024-02-26T06:33:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f231f432740 1 -- 172.21.15.49:0/365150036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083cd80 msgr2=0x5583a0849960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f231f432740 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083cd80 0x5583a0849960 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f23080ee1a0 tx=0x7f2308003f60 comp rx=0 tx=0).stop 2024-02-26T06:33:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f231f432740 1 -- 172.21.15.49:0/365150036 shutdown_connections 2024-02-26T06:33:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f231f432740 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5583a08ad7f0 0x5583a08afc10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f231f432740 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583a0846300 0x5583a084a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f231f432740 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583a083daf0 0x5583a0849ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f231f432740 1 --2- 172.21.15.49:0/365150036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5583a083cd80 0x5583a0849960 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f231f432740 1 -- 172.21.15.49:0/365150036 >> 172.21.15.49:0/365150036 conn(0x5583a0804480 msgr2=0x5583a0841bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.696+0000 7f22fb400640 0 client.27344 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.697+0000 7f231f432740 1 -- 172.21.15.49:0/365150036 shutdown_connections 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.697+0000 7f231f432740 1 -- 172.21.15.49:0/365150036 wait complete. 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.697+0000 7f231f432740 20 client.27344 trim_cache size 0 max 16384 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:33:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.697+0000 7f231c400640 1 -- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22fc0a2350 msgr2=0x7f22fc0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.697+0000 7f231c400640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22fc0a2350 0x7f22fc0a4810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f230c009fd0 tx=0x7f230c0096a0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.697+0000 7f231c400640 1 -- 172.21.15.49:0/2248409905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2314107750 msgr2=0x7f231419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.697+0000 7f231c400640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2314107750 0x7f231419dd20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f230802cd70 comp rx=0 tx=0).stop 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.698+0000 7f231c400640 1 -- 172.21.15.49:0/2248409905 shutdown_connections 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.698+0000 7f231c400640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f22fc0a2350 0x7f22fc0a4810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.698+0000 7f231c400640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f231410d760 0x7f23141a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.698+0000 7f231c400640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2314108110 0x7f231419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.698+0000 7f231c400640 1 --2- 172.21.15.49:0/2248409905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2314107750 0x7f231419dd20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.698+0000 7f231c400640 1 -- 172.21.15.49:0/2248409905 >> 172.21.15.49:0/2248409905 conn(0x7f23140756f0 msgr2=0x7f2314075c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.698+0000 7f231c400640 1 -- 172.21.15.49:0/2248409905 shutdown_connections 2024-02-26T06:33:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:14.698+0000 7f231c400640 1 -- 172.21.15.49:0/2248409905 wait complete. 2024-02-26T06:33:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:15 smithi049 ceph-mon[31696]: pgmap v305: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-02-26T06:33:15.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:15 smithi049 ceph-mon[31696]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T06:33:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:15 smithi080 ceph-mon[36206]: pgmap v305: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-02-26T06:33:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:15 smithi080 ceph-mon[36206]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T06:33:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:15 smithi115 ceph-mon[36426]: pgmap v305: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-02-26T06:33:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:15 smithi115 ceph-mon[36426]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T06:33:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 565.0 s, 1507.6 tps, lat 66.757 ms stddev 83.628 2024-02-26T06:33:17.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:17 smithi049 ceph-mon[31696]: pgmap v307: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 957 op/s 2024-02-26T06:33:17.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:17 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:17 smithi080 ceph-mon[36206]: pgmap v307: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 957 op/s 2024-02-26T06:33:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:17 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:17 smithi115 ceph-mon[36426]: pgmap v307: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 957 op/s 2024-02-26T06:33:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:17 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:19.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:19 smithi049 ceph-mon[31696]: pgmap v308: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 630 op/s 2024-02-26T06:33:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:19 smithi080 ceph-mon[36206]: pgmap v308: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 630 op/s 2024-02-26T06:33:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:19 smithi115 ceph-mon[36426]: pgmap v308: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 630 op/s 2024-02-26T06:33:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 570.0 s, 1950.0 tps, lat 51.886 ms stddev 64.141 2024-02-26T06:33:21.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:21 smithi049 ceph-mon[31696]: pgmap v309: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 629 op/s 2024-02-26T06:33:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:21 smithi080 ceph-mon[36206]: pgmap v309: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 629 op/s 2024-02-26T06:33:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:21 smithi115 ceph-mon[36426]: pgmap v309: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 629 op/s 2024-02-26T06:33:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:23 smithi080 ceph-mon[36206]: pgmap v310: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 672 op/s 2024-02-26T06:33:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:23 smithi115 ceph-mon[36426]: pgmap v310: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 672 op/s 2024-02-26T06:33:23.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:23 smithi049 ceph-mon[31696]: pgmap v310: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 672 op/s 2024-02-26T06:33:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:25 smithi080 ceph-mon[36206]: pgmap v311: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 672 op/s 2024-02-26T06:33:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:25 smithi115 ceph-mon[36426]: pgmap v311: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 672 op/s 2024-02-26T06:33:25.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:25 smithi049 ceph-mon[31696]: pgmap v311: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 672 op/s 2024-02-26T06:33:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 575.0 s, 1512.0 tps, lat 65.902 ms stddev 138.466 2024-02-26T06:33:27.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:27 smithi080 ceph-mon[36206]: pgmap v312: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-02-26T06:33:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:27 smithi115 ceph-mon[36426]: pgmap v312: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-02-26T06:33:27.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:27 smithi049 ceph-mon[31696]: pgmap v312: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-02-26T06:33:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.869+0000 7f3371800640 1 -- 172.21.15.49:0/2949640328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c0756b0 msgr2=0x7f336c075a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.869+0000 7f3371800640 1 --2- 172.21.15.49:0/2949640328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c0756b0 0x7f336c075a90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f335c009930 tx=0x7f335c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:33:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.869+0000 7f3371800640 1 -- 172.21.15.49:0/2949640328 shutdown_connections 2024-02-26T06:33:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.869+0000 7f3371800640 1 --2- 172.21.15.49:0/2949640328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336c111fd0 0x7f336c1143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.869+0000 7f3371800640 1 --2- 172.21.15.49:0/2949640328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c075fd0 0x7f336c111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.869+0000 7f3371800640 1 --2- 172.21.15.49:0/2949640328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c0756b0 0x7f336c075a90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.870+0000 7f3371800640 1 -- 172.21.15.49:0/2949640328 >> 172.21.15.49:0/2949640328 conn(0x7f336c0fee00 msgr2=0x7f336c101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.870+0000 7f3371800640 1 -- 172.21.15.49:0/2949640328 shutdown_connections 2024-02-26T06:33:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.870+0000 7f3371800640 1 -- 172.21.15.49:0/2949640328 wait complete. 2024-02-26T06:33:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f3371800640 1 Processor -- start 2024-02-26T06:33:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f3371800640 1 -- start start 2024-02-26T06:33:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f3371800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c0756b0 0x7f336c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f3371800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c075fd0 0x7f336c19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f3371800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336c111fd0 0x7f336c1a2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f3371800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c1a2c70 con 0x7f336c111fd0 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f3371800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c1a2de0 con 0x7f336c075fd0 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f3371800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c1a2fa0 con 0x7f336c0756b0 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f336a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c075fd0 0x7f336c19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f336a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c075fd0 0x7f336c19e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60050/0 (socket says 172.21.15.49:60050) 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f336a000640 1 -- 172.21.15.49:0/1856009034 learned_addr learned my addr 172.21.15.49:0/1856009034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f336b400640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336c111fd0 0x7f336c1a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.871+0000 7f336aa00640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c0756b0 0x7f336c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f336a000640 1 -- 172.21.15.49:0/1856009034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c0756b0 msgr2=0x7f336c19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f336a000640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c0756b0 0x7f336c19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f336a000640 1 -- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336c111fd0 msgr2=0x7f336c1a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f336a000640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336c111fd0 0x7f336c1a2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f336a000640 1 -- 172.21.15.49:0/1856009034 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c009560 con 0x7f336c075fd0 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f336b400640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336c111fd0 0x7f336c1a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:33:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f336a000640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c075fd0 0x7f336c19e340 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f335800e9f0 tx=0x7f335800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f3363400640 1 -- 172.21.15.49:0/1856009034 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358009700 con 0x7f336c075fd0 2024-02-26T06:33:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f3363400640 1 -- 172.21.15.49:0/1856009034 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f335800cd80 con 0x7f336c075fd0 2024-02-26T06:33:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f3363400640 1 -- 172.21.15.49:0/1856009034 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358018920 con 0x7f336c075fd0 2024-02-26T06:33:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.872+0000 7f3371800640 1 -- 172.21.15.49:0/1856009034 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c1a3280 con 0x7f336c075fd0 2024-02-26T06:33:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.873+0000 7f3371800640 1 -- 172.21.15.49:0/1856009034 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c1aab00 con 0x7f336c075fd0 2024-02-26T06:33:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.877+0000 7f3363400640 1 -- 172.21.15.49:0/1856009034 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3358010430 con 0x7f336c075fd0 2024-02-26T06:33:27.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.879+0000 7f3363400640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f33500a2350 0x7f33500a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.879+0000 7f3363400640 1 -- 172.21.15.49:0/1856009034 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f3358014070 con 0x7f336c075fd0 2024-02-26T06:33:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.879+0000 7f336aa00640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f33500a2350 0x7f33500a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.880+0000 7f336aa00640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f33500a2350 0x7f33500a4810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f335c009900 tx=0x7f335c005a20 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.880+0000 7f33745f0740 1 Processor -- start 2024-02-26T06:33:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f33745f0740 1 -- start start 2024-02-26T06:33:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f33745f0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d2200840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f33745f0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2201200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f33745f0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f33745f0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d21bc5f0 con 0x5556d220a300 2024-02-26T06:33:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f33745f0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d216e890 con 0x5556d2201af0 2024-02-26T06:33:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f33745f0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d216bdc0 con 0x5556d2200d80 2024-02-26T06:33:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f336b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f336b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53410/0 (socket says 172.21.15.49:53410) 2024-02-26T06:33:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f336b400640 1 -- 172.21.15.49:0/4105970294 learned_addr learned my addr 172.21.15.49:0/4105970294 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.881+0000 7f336aa00640 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d2200840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.882+0000 7f336a000640 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2201200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.882+0000 7f336b400640 1 -- 172.21.15.49:0/4105970294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 msgr2=0x5556d2201200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.882+0000 7f336b400640 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2201200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.882+0000 7f336b400640 1 -- 172.21.15.49:0/4105970294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 msgr2=0x5556d2200840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.882+0000 7f336b400640 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d2200840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.882+0000 7f336b400640 1 -- 172.21.15.49:0/4105970294 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556d20e8ad0 con 0x5556d220a300 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.882+0000 7f336a000640 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2201200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.882+0000 7f336b400640 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220c6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3364009f80 tx=0x7f336400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0b4b27acbca4ca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.883+0000 7f3357e00640 1 -- 172.21.15.49:0/4105970294 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364011070 con 0x5556d220a300 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.883+0000 7f3357e00640 1 -- 172.21.15.49:0/4105970294 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33640158f0 con 0x5556d220a300 2024-02-26T06:33:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.883+0000 7f3357e00640 1 -- 172.21.15.49:0/4105970294 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336400a900 con 0x5556d220a300 2024-02-26T06:33:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.884+0000 7f33745f0740 1 -- 172.21.15.49:0/4105970294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 msgr2=0x5556d220c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.884+0000 7f33745f0740 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220c6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3364009f80 tx=0x7f336400c8b0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.884+0000 7f33745f0740 1 -- 172.21.15.49:0/4105970294 shutdown_connections 2024-02-26T06:33:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.884+0000 7f33745f0740 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220c6f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.884+0000 7f33745f0740 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2201200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.884+0000 7f33745f0740 1 --2- 172.21.15.49:0/4105970294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d2200840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.884+0000 7f33745f0740 1 -- 172.21.15.49:0/4105970294 >> 172.21.15.49:0/4105970294 conn(0x5556d21c8480 msgr2=0x5556d21fba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.886+0000 7f33745f0740 1 -- 172.21.15.49:0/4105970294 shutdown_connections 2024-02-26T06:33:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.886+0000 7f33745f0740 1 -- 172.21.15.49:0/4105970294 wait complete. 2024-02-26T06:33:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.886+0000 7f33745f0740 1 -- start start 2024-02-26T06:33:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.887+0000 7f33745f0740 1 Processor -- start 2024-02-26T06:33:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.887+0000 7f33745f0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2200d80 0x5556d2207370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.887+0000 7f33745f0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2201af0 0x5556d22078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.887+0000 7f33745f0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d2207df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.887+0000 7f33745f0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d21675a0 con 0x5556d220a300 2024-02-26T06:33:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.887+0000 7f33745f0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d210cb20 con 0x5556d2200d80 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.887+0000 7f33745f0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d210c7c0 con 0x5556d2201af0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.887+0000 7f33745f0740 10 client.0 _ll_register_callbacks cb 0x5556d2120520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2201af0 0x5556d22078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2201af0 0x5556d22078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49792/0 (socket says 172.21.15.49:49792) 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 -- 172.21.15.49:0/3504409962 learned_addr learned my addr 172.21.15.49:0/3504409962 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336aa00640 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2200d80 0x5556d2207370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336b400640 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d2207df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 -- 172.21.15.49:0/3504409962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2200d80 msgr2=0x5556d2207370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2200d80 0x5556d2207370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 -- 172.21.15.49:0/3504409962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 msgr2=0x5556d2207df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d2207df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 -- 172.21.15.49:0/3504409962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3364009c30 con 0x5556d2201af0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f336a000640 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2201af0 0x5556d22078b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f33580cbc30 tx=0x7f335809a870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ddae36223a4547 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.888+0000 7f3354c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.889+0000 7f3354c00640 1 -- 172.21.15.49:0/3504409962 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580a35c0 con 0x5556d2201af0 2024-02-26T06:33:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.889+0000 7f3354c00640 1 -- 172.21.15.49:0/3504409962 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f33580106e0 con 0x5556d2201af0 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.889+0000 7f33745f0740 1 -- 172.21.15.49:0/3504409962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556d2139ca0 con 0x5556d2201af0 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.889+0000 7f3354c00640 1 -- 172.21.15.49:0/3504409962 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33580108a0 con 0x5556d2201af0 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.889+0000 7f33745f0740 10 client.27322 fetch_fsmap learned FSMap version 42 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.889+0000 7f33745f0740 1 -- 172.21.15.49:0/3504409962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5556d2165c20 con 0x5556d2201af0 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.889+0000 7f3354c00640 1 -- 172.21.15.49:0/3504409962 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5556d2139ca0 con 0x5556d2201af0 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.889+0000 7f3354c00640 1 -- 172.21.15.49:0/3504409962 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f33580b3c70 con 0x5556d2201af0 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.890+0000 7f33745f0740 10 client.27322 fetch_fsmap finished waiting for FSMap version 42 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.890+0000 7f33745f0740 10 client.27322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.890+0000 7f33745f0740 20 client.27322 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.890+0000 7f33745f0740 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5556d221a3a0 0x5556d221c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.890+0000 7f33745f0740 4 client.27322 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.890+0000 7f33745f0740 1 -- 172.21.15.49:0/3504409962 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556d215dcb0 con 0x5556d221a3a0 2024-02-26T06:33:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.890+0000 7f336aa00640 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5556d221a3a0 0x5556d221c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.893+0000 7f336aa00640 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5556d221a3a0 0x5556d221c7c0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ff92afabf28e53 server_cookie=76d88c73aa79b7af in_seq=0 out_seq=0 2024-02-26T06:33:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.893+0000 7f3354c00640 10 client.27322 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.894+0000 7f3354c00640 1 -- 172.21.15.49:0/3504409962 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556d215dcb0 con 0x5556d221a3a0 2024-02-26T06:33:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.894+0000 7f3354c00640 10 client.27322 handle_command_reply: tid=0 2024-02-26T06:33:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 client.27322 shutdown 2024-02-26T06:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 -- 172.21.15.49:0/3504409962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2201af0 msgr2=0x5556d22078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2201af0 0x5556d22078b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f33580cbc30 tx=0x7f335809a870 comp rx=0 tx=0).stop 2024-02-26T06:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 -- 172.21.15.49:0/3504409962 shutdown_connections 2024-02-26T06:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5556d221a3a0 0x5556d221c7c0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d2207df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2201af0 0x5556d22078b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 --2- 172.21.15.49:0/3504409962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2200d80 0x5556d2207370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 -- 172.21.15.49:0/3504409962 >> 172.21.15.49:0/3504409962 conn(0x5556d21c8480 msgr2=0x5556d21fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f3354c00640 0 client.27322 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 -- 172.21.15.49:0/3504409962 shutdown_connections 2024-02-26T06:33:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.895+0000 7f33745f0740 1 -- 172.21.15.49:0/3504409962 wait complete. 2024-02-26T06:33:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.896+0000 7f33745f0740 20 client.27322 trim_cache size 0 max 16384 2024-02-26T06:33:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.924+0000 7f33745f0740 1 Processor -- start 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f33745f0740 1 -- start start 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f33745f0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2209aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f33745f0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d2201af0 0x5556d2211120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f33745f0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d220a300 0x5556d2211660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f33745f0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1eafe60 con 0x5556d2201af0 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f33745f0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d22167e0 con 0x5556d220a300 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f33745f0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d2208340 con 0x5556d2200d80 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f336aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2209aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f336a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d2201af0 0x5556d2211120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f336aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2209aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49802/0 (socket says 172.21.15.49:49802) 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f336aa00640 1 -- 172.21.15.49:0/2424156007 learned_addr learned my addr 172.21.15.49:0/2424156007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.925+0000 7f336b400640 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d220a300 0x5556d2211660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f336aa00640 1 -- 172.21.15.49:0/2424156007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d220a300 msgr2=0x5556d2211660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f336aa00640 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d220a300 0x5556d2211660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f336aa00640 1 -- 172.21.15.49:0/2424156007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d2201af0 msgr2=0x5556d2211120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f336aa00640 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d2201af0 0x5556d2211120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f336aa00640 1 -- 172.21.15.49:0/2424156007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556d2102500 con 0x5556d2200d80 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f336b400640 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d220a300 0x5556d2211660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f336aa00640 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2209aa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f335c005ab0 tx=0x7f335c043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21897d2c0ccc586b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f3357400640 1 -- 172.21.15.49:0/2424156007 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c054b00 con 0x5556d2200d80 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f3357400640 1 -- 172.21.15.49:0/2424156007 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f335c054c60 con 0x5556d2200d80 2024-02-26T06:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.926+0000 7f3357400640 1 -- 172.21.15.49:0/2424156007 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c0035f0 con 0x5556d2200d80 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 -- 172.21.15.49:0/2424156007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 msgr2=0x5556d2209aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2209aa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f335c005ab0 tx=0x7f335c043000 comp rx=0 tx=0).stop 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 -- 172.21.15.49:0/2424156007 shutdown_connections 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d220a300 0x5556d2211660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d2201af0 0x5556d2211120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 --2- 172.21.15.49:0/2424156007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2209aa0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 -- 172.21.15.49:0/2424156007 >> 172.21.15.49:0/2424156007 conn(0x5556d21c8480 msgr2=0x5556d2205ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 -- 172.21.15.49:0/2424156007 shutdown_connections 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 -- 172.21.15.49:0/2424156007 wait complete. 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.927+0000 7f33745f0740 1 -- start start 2024-02-26T06:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f33745f0740 1 Processor -- start 2024-02-26T06:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f33745f0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2211130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f33745f0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d220f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f33745f0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f33745f0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d2211870 con 0x5556d220a300 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f33745f0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d22119e0 con 0x5556d2201af0 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f33745f0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d220fb90 con 0x5556d2200d80 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f33745f0740 10 client.0 _ll_register_callbacks cb 0x7f335800cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f336aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2211130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f336a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d220f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f336b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f336aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2211130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49812/0 (socket says 172.21.15.49:49812) 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.928+0000 7f336aa00640 1 -- 172.21.15.49:0/3073098433 learned_addr learned my addr 172.21.15.49:0/3073098433 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f336a000640 1 -- 172.21.15.49:0/3073098433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 msgr2=0x5556d2211130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f336a000640 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2211130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f336a000640 1 -- 172.21.15.49:0/3073098433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 msgr2=0x5556d220f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f336a000640 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f336a000640 1 -- 172.21.15.49:0/3073098433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556d20e8ad0 con 0x5556d2201af0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f336a000640 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d220f110 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f335800cfa0 tx=0x7f33580ce3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb82fd22db6f9df server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f3354c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f3354c00640 1 -- 172.21.15.49:0/3073098433 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580108e0 con 0x5556d2201af0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f33745f0740 1 -- 172.21.15.49:0/3073098433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556d218cd00 con 0x5556d2201af0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f3354c00640 1 -- 172.21.15.49:0/3073098433 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f33580a35c0 con 0x5556d2201af0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f3354c00640 1 -- 172.21.15.49:0/3073098433 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335809aea0 con 0x5556d2201af0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f33745f0740 10 client.27362 fetch_fsmap learned FSMap version 42 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f33745f0740 1 -- 172.21.15.49:0/3073098433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f336c114e50 con 0x5556d2201af0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.929+0000 7f3354c00640 1 -- 172.21.15.49:0/3073098433 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5556d218cd00 con 0x5556d2201af0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.930+0000 7f3354c00640 1 -- 172.21.15.49:0/3073098433 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3358085c70 con 0x5556d2201af0 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.930+0000 7f33745f0740 10 client.27362 fetch_fsmap finished waiting for FSMap version 42 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.930+0000 7f33745f0740 10 client.27362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.930+0000 7f33745f0740 20 client.27362 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.930+0000 7f33745f0740 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5556d2272730 0x5556d2274b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.930+0000 7f33745f0740 4 client.27362 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:33:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.930+0000 7f33745f0740 1 -- 172.21.15.49:0/3073098433 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5556d1e67600 con 0x5556d2272730 2024-02-26T06:33:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.930+0000 7f336aa00640 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5556d2272730 0x5556d2274b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.931+0000 7f336aa00640 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5556d2272730 0x5556d2274b20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a67fc4cf82b040e server_cookie=6e7121ddd10a1db4 in_seq=0 out_seq=0 2024-02-26T06:33:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.931+0000 7f3354c00640 10 client.27362 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.948+0000 7f3354c00640 1 -- 172.21.15.49:0/3073098433 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5556d1e67600 con 0x5556d2272730 2024-02-26T06:33:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.948+0000 7f3354c00640 10 client.27362 handle_command_reply: tid=0 2024-02-26T06:33:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 client.27362 shutdown 2024-02-26T06:33:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 -- 172.21.15.49:0/3073098433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 msgr2=0x5556d220f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d220f110 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f335800cfa0 tx=0x7f33580ce3b0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 -- 172.21.15.49:0/3073098433 shutdown_connections 2024-02-26T06:33:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5556d2272730 0x5556d2274b20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5556d2201af0 0x5556d220f110 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556d220a300 0x5556d220f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f3354c00640 0 client.27362 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 --2- 172.21.15.49:0/3073098433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5556d2200d80 0x5556d2211130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 -- 172.21.15.49:0/3073098433 >> 172.21.15.49:0/3073098433 conn(0x5556d21c8480 msgr2=0x5556d2207330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 -- 172.21.15.49:0/3073098433 shutdown_connections 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.949+0000 7f33745f0740 1 -- 172.21.15.49:0/3073098433 wait complete. 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.950+0000 7f33745f0740 20 client.27362 trim_cache size 0 max 16384 2024-02-26T06:33:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:33:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:33:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:33:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:33:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 -- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f33500a2350 msgr2=0x7f33500a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f33500a2350 0x7f33500a4810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f335c009900 tx=0x7f335c005a20 comp rx=0 tx=0).stop 2024-02-26T06:33:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 -- 172.21.15.49:0/1856009034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c075fd0 msgr2=0x7f336c19e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c075fd0 0x7f336c19e340 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f335800e9f0 tx=0x7f335800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 -- 172.21.15.49:0/1856009034 shutdown_connections 2024-02-26T06:33:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f33500a2350 0x7f33500a4810 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f336c075fd0 0x7f336c19e340 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f336c111fd0 0x7f336c1a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 --2- 172.21.15.49:0/1856009034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c0756b0 0x7f336c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 -- 172.21.15.49:0/1856009034 >> 172.21.15.49:0/1856009034 conn(0x7f336c0fee00 msgr2=0x7f336c113a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 -- 172.21.15.49:0/1856009034 shutdown_connections 2024-02-26T06:33:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:27.951+0000 7f3371800640 1 -- 172.21.15.49:0/1856009034 wait complete. 2024-02-26T06:33:29.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:29 smithi080 ceph-mon[36206]: pgmap v313: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2024-02-26T06:33:29.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:29 smithi115 ceph-mon[36426]: pgmap v313: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2024-02-26T06:33:29.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:29 smithi049 ceph-mon[31696]: pgmap v313: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2024-02-26T06:33:31.306 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 580.0 s, 1704.4 tps, lat 58.133 ms stddev 69.544 2024-02-26T06:33:31.470 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:31 smithi049 ceph-mon[31696]: pgmap v314: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-02-26T06:33:31.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:31 smithi080 ceph-mon[36206]: pgmap v314: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-02-26T06:33:31.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:31 smithi115 ceph-mon[36426]: pgmap v314: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-02-26T06:33:33.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:33 smithi080 ceph-mon[36206]: pgmap v315: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2024-02-26T06:33:33.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:33 smithi115 ceph-mon[36426]: pgmap v315: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2024-02-26T06:33:33.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:33 smithi049 ceph-mon[31696]: pgmap v315: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2024-02-26T06:33:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:35 smithi080 ceph-mon[36206]: pgmap v316: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 19 MiB/s wr, 1.60k op/s 2024-02-26T06:33:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:35 smithi115 ceph-mon[36426]: pgmap v316: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 19 MiB/s wr, 1.60k op/s 2024-02-26T06:33:35.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:35 smithi049 ceph-mon[31696]: pgmap v316: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 19 MiB/s wr, 1.60k op/s 2024-02-26T06:33:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 585.0 s, 1581.6 tps, lat 63.430 ms stddev 71.136 2024-02-26T06:33:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:37 smithi080 ceph-mon[36206]: pgmap v317: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-02-26T06:33:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:37 smithi115 ceph-mon[36426]: pgmap v317: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-02-26T06:33:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:37 smithi049 ceph-mon[31696]: pgmap v317: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-02-26T06:33:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:39 smithi080 ceph-mon[36206]: pgmap v318: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-02-26T06:33:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:39 smithi115 ceph-mon[36426]: pgmap v318: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-02-26T06:33:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:39 smithi049 ceph-mon[31696]: pgmap v318: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-02-26T06:33:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.139+0000 7f2dbfc00640 1 -- 172.21.15.49:0/3809096908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2db80ff5d0 msgr2=0x7f2db80ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.139+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/3809096908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2db80ff5d0 0x7f2db80ff9b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009890 tx=0x7f2dac02bbc0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.140+0000 7f2dbfc00640 1 -- 172.21.15.49:0/3809096908 shutdown_connections 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.140+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/3809096908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2db8104500 0x7f2db81049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.140+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/3809096908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2db80ffef0 0x7f2db8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.140+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/3809096908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2db80ff5d0 0x7f2db80ff9b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.140+0000 7f2dbfc00640 1 -- 172.21.15.49:0/3809096908 >> 172.21.15.49:0/3809096908 conn(0x7f2db8076210 msgr2=0x7f2db80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.140+0000 7f2dbfc00640 1 -- 172.21.15.49:0/3809096908 shutdown_connections 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.140+0000 7f2dbfc00640 1 -- 172.21.15.49:0/3809096908 wait complete. 2024-02-26T06:33:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.141+0000 7f2dbfc00640 1 Processor -- start 2024-02-26T06:33:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.142+0000 7f2dbfc00640 1 -- start start 2024-02-26T06:33:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.142+0000 7f2dbfc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2db80ffef0 0x7f2db8107420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbfc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2db8104500 0x7f2db8107960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbfc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2db8107ea0 0x7f2db81aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbfc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db810b5b0 con 0x7f2db8104500 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbfc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db81aa600 con 0x7f2db80ffef0 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbfc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db81aa790 con 0x7f2db8107ea0 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbdc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2db8107ea0 0x7f2db81aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2db80ffef0 0x7f2db8107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbdc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2db8107ea0 0x7f2db81aa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33734/0 (socket says 172.21.15.49:33734) 2024-02-26T06:33:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.143+0000 7f2dbdc00640 1 -- 172.21.15.49:0/278713958 learned_addr learned my addr 172.21.15.49:0/278713958 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.144+0000 7f2dbdc00640 1 -- 172.21.15.49:0/278713958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2db80ffef0 msgr2=0x7f2db8107420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.144+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2db80ffef0 0x7f2db8107420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.144+0000 7f2db7e00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2db8104500 0x7f2db8107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.144+0000 7f2dbdc00640 1 -- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2db8104500 msgr2=0x7f2db8107960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.144+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2db8104500 0x7f2db8107960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.144+0000 7f2dbdc00640 1 -- 172.21.15.49:0/278713958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac009490 con 0x7f2db8107ea0 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.144+0000 7f2db7e00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2db8104500 0x7f2db8107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.145+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2db8107ea0 0x7f2db81aa0c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2db000cd10 tx=0x7f2db000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.145+0000 7f2db5600640 1 -- 172.21.15.49:0/278713958 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000f450 con 0x7f2db8107ea0 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.145+0000 7f2db5600640 1 -- 172.21.15.49:0/278713958 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2db000fa90 con 0x7f2db8107ea0 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.145+0000 7f2db5600640 1 -- 172.21.15.49:0/278713958 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0018b70 con 0x7f2db8107ea0 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.145+0000 7f2dbfc00640 1 -- 172.21.15.49:0/278713958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db81aaa40 con 0x7f2db8107ea0 2024-02-26T06:33:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.146+0000 7f2dbfc00640 1 -- 172.21.15.49:0/278713958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db81aaf00 con 0x7f2db8107ea0 2024-02-26T06:33:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.151+0000 7f2dc2a41740 1 Processor -- start 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.152+0000 7f2dc2a41740 1 -- start start 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.153+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbcd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.153+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.153+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.153+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcb895f0 con 0x5653dcbceaf0 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.153+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcb3b890 con 0x5653dcbcdd80 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.153+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcb38dc0 con 0x5653dcbe8310 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.153+0000 7f2db5600640 1 -- 172.21.15.49:0/278713958 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2db000fc00 con 0x7f2db8107ea0 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.155+0000 7f2dbd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.155+0000 7f2dbd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34884/0 (socket says 172.21.15.49:34884) 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.155+0000 7f2dbd200640 1 -- 172.21.15.49:0/195510575 learned_addr learned my addr 172.21.15.49:0/195510575 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.155+0000 7f2dbd200640 1 -- 172.21.15.49:0/195510575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 msgr2=0x5653dcbea700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.155+0000 7f2dbd200640 1 --2- 172.21.15.49:0/195510575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbea700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.155+0000 7f2dbd200640 1 -- 172.21.15.49:0/195510575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 msgr2=0x5653dcbcd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.155+0000 7f2dbd200640 1 --2- 172.21.15.49:0/195510575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbcd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.155+0000 7f2dbd200640 1 -- 172.21.15.49:0/195510575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653dcab5ad0 con 0x5653dcbcdd80 2024-02-26T06:33:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.156+0000 7f2dbd200640 1 --2- 172.21.15.49:0/195510575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbce200 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009890 tx=0x7f2dac02cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27339cb8be49b8f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.156+0000 7f2daa000640 1 -- 172.21.15.49:0/195510575 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac004240 con 0x5653dcbcdd80 2024-02-26T06:33:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.156+0000 7f2daa000640 1 -- 172.21.15.49:0/195510575 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2dac02ce50 con 0x5653dcbcdd80 2024-02-26T06:33:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.156+0000 7f2daa000640 1 -- 172.21.15.49:0/195510575 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac02db70 con 0x5653dcbcdd80 2024-02-26T06:33:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.157+0000 7f2dc2a41740 1 -- 172.21.15.49:0/195510575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 msgr2=0x5653dcbce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.157+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/195510575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbce200 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009890 tx=0x7f2dac02cc60 comp rx=0 tx=0).stop 2024-02-26T06:33:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.158+0000 7f2dc2a41740 1 -- 172.21.15.49:0/195510575 shutdown_connections 2024-02-26T06:33:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.158+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/195510575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbce200 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.158+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/195510575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.158+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/195510575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbcd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.158+0000 7f2dc2a41740 1 -- 172.21.15.49:0/195510575 >> 172.21.15.49:0/195510575 conn(0x5653dcb95480 msgr2=0x5653dcbc8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.158+0000 7f2dc2a41740 1 -- 172.21.15.49:0/195510575 shutdown_connections 2024-02-26T06:33:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.159+0000 7f2dc2a41740 1 -- 172.21.15.49:0/195510575 wait complete. 2024-02-26T06:33:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.160+0000 7f2dc2a41740 1 -- start start 2024-02-26T06:33:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.160+0000 7f2dc2a41740 1 Processor -- start 2024-02-26T06:33:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.161+0000 7f2db5600640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2da00a23c0 0x7f2da00a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.161+0000 7f2db5600640 1 -- 172.21.15.49:0/278713958 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f2db00ce460 con 0x7f2db8107ea0 2024-02-26T06:33:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.164+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbe53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.164+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbceaf0 0x5653dcbe58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.164+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbe8310 0x5653dcbe5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.165+0000 7f2db7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbceaf0 0x5653dcbe58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.165+0000 7f2db7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbceaf0 0x5653dcbe58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34898/0 (socket says 172.21.15.49:34898) 2024-02-26T06:33:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.165+0000 7f2dbd200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbe53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.165+0000 7f2dbd200640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2da00a23c0 0x7f2da00a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.165+0000 7f2dbd200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbe53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33748/0 (socket says 172.21.15.49:33748) 2024-02-26T06:33:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.166+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcb345a0 con 0x5653dcbe8310 2024-02-26T06:33:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.166+0000 7f2db7e00640 1 -- 172.21.15.49:0/1975049065 learned_addr learned my addr 172.21.15.49:0/1975049065 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.166+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcad9b20 con 0x5653dcbceaf0 2024-02-26T06:33:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.166+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcad97c0 con 0x5653dcbcdd80 2024-02-26T06:33:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.166+0000 7f2dc2a41740 10 client.0 _ll_register_callbacks cb 0x5653dcaed520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.169+0000 7f2db7e00640 1 -- 172.21.15.49:0/1975049065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 msgr2=0x5653dcbe53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.169+0000 7f2db7e00640 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbe53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.169+0000 7f2db7e00640 1 -- 172.21.15.49:0/1975049065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbe8310 msgr2=0x5653dcbe5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.169+0000 7f2db7e00640 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbe8310 0x5653dcbe5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.169+0000 7f2db7e00640 1 -- 172.21.15.49:0/1975049065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db8106240 con 0x5653dcbceaf0 2024-02-26T06:33:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.169+0000 7f2dbd200640 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbe53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-26T06:33:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.170+0000 7f2db7e00640 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbceaf0 0x5653dcbe58e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2da4004700 tx=0x7f2da4010a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20ae7bbcbbf9849 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.170+0000 7f2d9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:33:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.170+0000 7f2dbd200640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2da00a23c0 0x7f2da00a4880 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2dac037040 tx=0x7f2dac04d040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.171+0000 7f2d9ea00640 1 -- 172.21.15.49:0/1975049065 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4012920 con 0x5653dcbceaf0 2024-02-26T06:33:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.171+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653dcb06ca0 con 0x5653dcbceaf0 2024-02-26T06:33:41.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.171+0000 7f2d9ea00640 1 -- 172.21.15.49:0/1975049065 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f2da4016070 con 0x5653dcbceaf0 2024-02-26T06:33:41.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.173+0000 7f2d9ea00640 1 -- 172.21.15.49:0/1975049065 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2da4012920 con 0x5653dcbceaf0 2024-02-26T06:33:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.173+0000 7f2dc2a41740 10 client.27374 fetch_fsmap learned FSMap version 42 2024-02-26T06:33:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.173+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5653dcb32c20 con 0x5653dcbceaf0 2024-02-26T06:33:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.173+0000 7f2d9ea00640 1 -- 172.21.15.49:0/1975049065 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2da4027020 con 0x5653dcbceaf0 2024-02-26T06:33:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.174+0000 7f2d9ea00640 1 -- 172.21.15.49:0/1975049065 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2da401a420 con 0x5653dcbceaf0 2024-02-26T06:33:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.174+0000 7f2dc2a41740 10 client.27374 fetch_fsmap finished waiting for FSMap version 42 2024-02-26T06:33:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.174+0000 7f2dc2a41740 10 client.27374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.174+0000 7f2dc2a41740 20 client.27374 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.174+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653dcbf84d0 0x5653dcbfa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.174+0000 7f2dc2a41740 4 client.27374 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:33:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.174+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653dcb2acb0 con 0x5653dcbf84d0 2024-02-26T06:33:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.174+0000 7f2dbd200640 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653dcbf84d0 0x5653dcbfa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.175+0000 7f2dbd200640 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653dcbf84d0 0x5653dcbfa8f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa0b683cfdc6640 server_cookie=1b97619670b296f5 in_seq=0 out_seq=0 2024-02-26T06:33:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.177+0000 7f2d9ea00640 10 client.27374 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.177+0000 7f2d9ea00640 1 -- 172.21.15.49:0/1975049065 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653dcb2acb0 con 0x5653dcbf84d0 2024-02-26T06:33:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.177+0000 7f2d9ea00640 10 client.27374 handle_command_reply: tid=0 2024-02-26T06:33:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.177+0000 7f2dc2a41740 1 client.27374 shutdown 2024-02-26T06:33:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbceaf0 msgr2=0x5653dcbe58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbceaf0 0x5653dcbe58e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2da4004700 tx=0x7f2da4010a30 comp rx=0 tx=0).stop 2024-02-26T06:33:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 shutdown_connections 2024-02-26T06:33:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653dcbf84d0 0x5653dcbfa8f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbceaf0 0x5653dcbe58e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbe8310 0x5653dcbe5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1975049065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbe53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2d9ea00640 0 client.27374 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.180+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 >> 172.21.15.49:0/1975049065 conn(0x5653dcb95480 msgr2=0x5653dcbca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.181+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 shutdown_connections 2024-02-26T06:33:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.181+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1975049065 wait complete. 2024-02-26T06:33:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.181+0000 7f2dc2a41740 20 client.27374 trim_cache size 0 max 16384 2024-02-26T06:33:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.235+0000 7f2dc2a41740 1 Processor -- start 2024-02-26T06:33:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.236+0000 7f2dc2a41740 1 -- start start 2024-02-26T06:33:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbe7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbe5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dc87ce60 con 0x5653dcbceaf0 2024-02-26T06:33:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcbc9a70 con 0x5653dcbcdd80 2024-02-26T06:33:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcbffb00 con 0x5653dcbe8310 2024-02-26T06:33:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dbdc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dbdc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbef0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33754/0 (socket says 172.21.15.49:33754) 2024-02-26T06:33:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dbdc00640 1 -- 172.21.15.49:0/1286586404 learned_addr learned my addr 172.21.15.49:0/1286586404 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.237+0000 7f2dbd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbe7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.238+0000 7f2dbdc00640 1 -- 172.21.15.49:0/1286586404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 msgr2=0x5653dcbe7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.238+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/1286586404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbe7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.238+0000 7f2dbdc00640 1 -- 172.21.15.49:0/1286586404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 msgr2=0x5653dcbe5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.238+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/1286586404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbe5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.238+0000 7f2dbdc00640 1 -- 172.21.15.49:0/1286586404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653dcacf500 con 0x5653dcbe8310 2024-02-26T06:33:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.239+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/1286586404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbef0f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2db00a3040 tx=0x7f2db009a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b335a028be780bde server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.241+0000 7f2da9600640 1 -- 172.21.15.49:0/1286586404 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db009add0 con 0x5653dcbe8310 2024-02-26T06:33:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.241+0000 7f2da9600640 1 -- 172.21.15.49:0/1286586404 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2db009ef20 con 0x5653dcbe8310 2024-02-26T06:33:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.241+0000 7f2da9600640 1 -- 172.21.15.49:0/1286586404 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db009a480 con 0x5653dcbe8310 2024-02-26T06:33:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.243+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1286586404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 msgr2=0x5653dcbef0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.243+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1286586404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbef0f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2db00a3040 tx=0x7f2db009a910 comp rx=0 tx=0).stop 2024-02-26T06:33:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.243+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1286586404 shutdown_connections 2024-02-26T06:33:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.243+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1286586404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbe8310 0x5653dcbef0f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.243+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1286586404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbe5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.243+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/1286586404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbcdd80 0x5653dcbe7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.243+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1286586404 >> 172.21.15.49:0/1286586404 conn(0x5653dcb95480 msgr2=0x5653dcbc85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.245+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1286586404 shutdown_connections 2024-02-26T06:33:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.245+0000 7f2dc2a41740 1 -- 172.21.15.49:0/1286586404 wait complete. 2024-02-26T06:33:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.246+0000 7f2dc2a41740 1 -- start start 2024-02-26T06:33:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.246+0000 7f2dc2a41740 1 Processor -- start 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbe1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dc2a41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbe8310 0x5653dcbe2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcbe26a0 con 0x5653dcbceaf0 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcbe2810 con 0x5653dcbe8310 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dc2a41740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653dcbe29d0 con 0x5653dcbcdd80 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dc2a41740 10 client.0 _ll_register_callbacks cb 0x7f2da4009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dbdc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbe8310 0x5653dcbe2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dbdc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbe8310 0x5653dcbe2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:34922/0 (socket says 172.21.15.49:34922) 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dbdc00640 1 -- 172.21.15.49:0/3849444441 learned_addr learned my addr 172.21.15.49:0/3849444441 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dbdc00640 1 -- 172.21.15.49:0/3849444441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 msgr2=0x5653dcbef360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbef360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dbdc00640 1 -- 172.21.15.49:0/3849444441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 msgr2=0x5653dcbe1bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbe1bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.247+0000 7f2dbdc00640 1 -- 172.21.15.49:0/3849444441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db81aaf00 con 0x5653dcbe8310 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.248+0000 7f2dbdc00640 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbe8310 0x5653dcbe2130 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2db000ab20 tx=0x7f2db009a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4096c902d473ab12 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.248+0000 7f2d9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.248+0000 7f2d9ea00640 1 -- 172.21.15.49:0/3849444441 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db00cbdb0 con 0x5653dcbe8310 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.248+0000 7f2d9ea00640 1 -- 172.21.15.49:0/3849444441 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f2db009add0 con 0x5653dcbe8310 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.248+0000 7f2dc2a41740 1 -- 172.21.15.49:0/3849444441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653dcb59d00 con 0x5653dcbe8310 2024-02-26T06:33:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.249+0000 7f2d9ea00640 1 -- 172.21.15.49:0/3849444441 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db00a3570 con 0x5653dcbe8310 2024-02-26T06:33:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.249+0000 7f2dc2a41740 10 client.27380 fetch_fsmap learned FSMap version 42 2024-02-26T06:33:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.249+0000 7f2dc2a41740 1 -- 172.21.15.49:0/3849444441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5653dcaf5290 con 0x5653dcbe8310 2024-02-26T06:33:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.250+0000 7f2d9ea00640 1 -- 172.21.15.49:0/3849444441 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5653dcb59d00 con 0x5653dcbe8310 2024-02-26T06:33:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.250+0000 7f2d9ea00640 1 -- 172.21.15.49:0/3849444441 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2db0016020 con 0x5653dcbe8310 2024-02-26T06:33:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.251+0000 7f2dc2a41740 10 client.27380 fetch_fsmap finished waiting for FSMap version 42 2024-02-26T06:33:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.251+0000 7f2dc2a41740 10 client.27380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.251+0000 7f2dc2a41740 20 client.27380 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.251+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653dcc4fbd0 0x5653dcc51ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.251+0000 7f2dc2a41740 4 client.27380 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:33:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.251+0000 7f2dc2a41740 1 -- 172.21.15.49:0/3849444441 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5653dc834600 con 0x5653dcc4fbd0 2024-02-26T06:33:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.251+0000 7f2dbd200640 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653dcc4fbd0 0x5653dcc51ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.253+0000 7f2dbd200640 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653dcc4fbd0 0x5653dcc51ff0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928223dbf1fa536b server_cookie=50ad4ddd80f233e1 in_seq=0 out_seq=0 2024-02-26T06:33:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.253+0000 7f2d9ea00640 10 client.27380 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.273+0000 7f2d9ea00640 1 -- 172.21.15.49:0/3849444441 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5653dc834600 con 0x5653dcc4fbd0 2024-02-26T06:33:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.273+0000 7f2d9ea00640 10 client.27380 handle_command_reply: tid=0 2024-02-26T06:33:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.273+0000 7f2dc2a41740 1 client.27380 shutdown 2024-02-26T06:33:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.274+0000 7f2dc2a41740 1 -- 172.21.15.49:0/3849444441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbe8310 msgr2=0x5653dcbe2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.274+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbe8310 0x5653dcbe2130 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2db000ab20 tx=0x7f2db009a4d0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.274+0000 7f2dc2a41740 1 -- 172.21.15.49:0/3849444441 shutdown_connections 2024-02-26T06:33:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.274+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653dcc4fbd0 0x5653dcc51ff0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.274+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653dcbe8310 0x5653dcbe2130 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.274+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653dcbceaf0 0x5653dcbe1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.274+0000 7f2dc2a41740 1 --2- 172.21.15.49:0/3849444441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653dcbcdd80 0x5653dcbef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.274+0000 7f2dc2a41740 1 -- 172.21.15.49:0/3849444441 >> 172.21.15.49:0/3849444441 conn(0x5653dcb95480 msgr2=0x5653dcbe3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.275+0000 7f2d9ea00640 0 client.27380 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.275+0000 7f2dc2a41740 1 -- 172.21.15.49:0/3849444441 shutdown_connections 2024-02-26T06:33:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.275+0000 7f2dc2a41740 1 -- 172.21.15.49:0/3849444441 wait complete. 2024-02-26T06:33:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.275+0000 7f2dc2a41740 20 client.27380 trim_cache size 0 max 16384 2024-02-26T06:33:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:33:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:33:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:33:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 -- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2da00a23c0 msgr2=0x7f2da00a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2da00a23c0 0x7f2da00a4880 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2dac037040 tx=0x7f2dac04d040 comp rx=0 tx=0).stop 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 -- 172.21.15.49:0/278713958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2db8107ea0 msgr2=0x7f2db81aa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2db8107ea0 0x7f2db81aa0c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2db000cd10 tx=0x7f2db000a620 comp rx=0 tx=0).stop 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 -- 172.21.15.49:0/278713958 shutdown_connections 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f2da00a23c0 0x7f2da00a4880 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2db8107ea0 0x7f2db81aa0c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2db8104500 0x7f2db8107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 --2- 172.21.15.49:0/278713958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2db80ffef0 0x7f2db8107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 -- 172.21.15.49:0/278713958 >> 172.21.15.49:0/278713958 conn(0x7f2db8076210 msgr2=0x7f2db810b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 -- 172.21.15.49:0/278713958 shutdown_connections 2024-02-26T06:33:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:41.277+0000 7f2dbfc00640 1 -- 172.21.15.49:0/278713958 wait complete. 2024-02-26T06:33:41.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 590.0 s, 1371.6 tps, lat 72.559 ms stddev 113.280 2024-02-26T06:33:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:41 smithi080 ceph-mon[36206]: pgmap v319: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-02-26T06:33:41.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:41 smithi115 ceph-mon[36426]: pgmap v319: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-02-26T06:33:41.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:41 smithi049 ceph-mon[31696]: pgmap v319: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-02-26T06:33:43.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:43 smithi080 ceph-mon[36206]: pgmap v320: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2024-02-26T06:33:43.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:43 smithi115 ceph-mon[36426]: pgmap v320: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2024-02-26T06:33:43.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:43 smithi049 ceph-mon[31696]: pgmap v320: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2024-02-26T06:33:44.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:44 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:44.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:44 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:44.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:44 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:45 smithi080 ceph-mon[36206]: pgmap v321: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-02-26T06:33:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:45 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:45.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:45 smithi115 ceph-mon[36426]: pgmap v321: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-02-26T06:33:45.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:45 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:45 smithi049 ceph-mon[31696]: pgmap v321: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-02-26T06:33:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:45 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 595.0 s, 1500.2 tps, lat 67.006 ms stddev 81.771 2024-02-26T06:33:46.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:46 smithi080 ceph-mon[36206]: pgmap v322: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-02-26T06:33:46.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:46 smithi115 ceph-mon[36426]: pgmap v322: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-02-26T06:33:46.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:46 smithi049 ceph-mon[31696]: pgmap v322: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-02-26T06:33:48.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:48 smithi049 ceph-mon[31696]: pgmap v323: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2024-02-26T06:33:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:48 smithi080 ceph-mon[36206]: pgmap v323: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2024-02-26T06:33:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:48 smithi115 ceph-mon[36426]: pgmap v323: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2024-02-26T06:33:49.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:49 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:50.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:49 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:50.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:49 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:50 smithi080 ceph-mon[36206]: pgmap v324: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-02-26T06:33:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:50 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:50 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:33:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:50 smithi115 ceph-mon[36426]: pgmap v324: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-02-26T06:33:51.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:50 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:51.092 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:50 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:33:51.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:50 smithi049 ceph-mon[31696]: pgmap v324: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-02-26T06:33:51.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:50 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:33:51.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:50 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:33:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 600.0 s, 1715.2 tps, lat 57.827 ms stddev 75.499 2024-02-26T06:33:52.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:52 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:52 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:52 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:52 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:52 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:52 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:52 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:52 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:52 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:52 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:52 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:52 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:52 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:52 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:52 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:52 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:52 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:52.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:52 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:53.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:53 smithi080 ceph-mon[36206]: pgmap v325: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-02-26T06:33:53.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:53 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:53.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:53 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:53.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:53 smithi115 ceph-mon[36426]: pgmap v325: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-02-26T06:33:53.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:53 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:53.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:53 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:53.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:53 smithi049 ceph-mon[31696]: pgmap v325: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-02-26T06:33:53.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:53 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:53.647 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:53 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.478+0000 7fb154800640 1 -- 172.21.15.49:0/2409957289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c1142d0 msgr2=0x7fb14c1166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.478+0000 7fb154800640 1 --2- 172.21.15.49:0/2409957289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c1142d0 0x7fb14c1166c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb140009930 tx=0x7fb14002bc40 comp rx=0 tx=0).stop 2024-02-26T06:33:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.479+0000 7fb154800640 1 -- 172.21.15.49:0/2409957289 shutdown_connections 2024-02-26T06:33:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.479+0000 7fb154800640 1 --2- 172.21.15.49:0/2409957289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c1142d0 0x7fb14c1166c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.479+0000 7fb154800640 1 --2- 172.21.15.49:0/2409957289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb14c108cf0 0x7fb14c113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.479+0000 7fb154800640 1 --2- 172.21.15.49:0/2409957289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb14c1083d0 0x7fb14c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.479+0000 7fb154800640 1 -- 172.21.15.49:0/2409957289 >> 172.21.15.49:0/2409957289 conn(0x7fb14c06c220 msgr2=0x7fb14c06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.479+0000 7fb154800640 1 -- 172.21.15.49:0/2409957289 shutdown_connections 2024-02-26T06:33:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.480+0000 7fb154800640 1 -- 172.21.15.49:0/2409957289 wait complete. 2024-02-26T06:33:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.481+0000 7fb154800640 1 Processor -- start 2024-02-26T06:33:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.481+0000 7fb154800640 1 -- start start 2024-02-26T06:33:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb154800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb14c1083d0 0x7fb14c1a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb154800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c108cf0 0x7fb14c1a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb154800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb14c1142d0 0x7fb14c1a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb154800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1a7080 con 0x7fb14c1142d0 2024-02-26T06:33:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb154800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1a71f0 con 0x7fb14c108cf0 2024-02-26T06:33:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb154800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1a73b0 con 0x7fb14c1083d0 2024-02-26T06:33:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb151400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c108cf0 0x7fb14c1a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb151400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c108cf0 0x7fb14c1a2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43612/0 (socket says 172.21.15.49:43612) 2024-02-26T06:33:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.482+0000 7fb151400640 1 -- 172.21.15.49:0/2738171759 learned_addr learned my addr 172.21.15.49:0/2738171759 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.483+0000 7fb151e00640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb14c1083d0 0x7fb14c1a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.483+0000 7fb151400640 1 -- 172.21.15.49:0/2738171759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb14c1083d0 msgr2=0x7fb14c1a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.483+0000 7fb151400640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb14c1083d0 0x7fb14c1a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.483+0000 7fb151400640 1 -- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb14c1142d0 msgr2=0x7fb14c1a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.483+0000 7fb151400640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb14c1142d0 0x7fb14c1a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.483+0000 7fb151400640 1 -- 172.21.15.49:0/2738171759 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb140009560 con 0x7fb14c108cf0 2024-02-26T06:33:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.484+0000 7fb151400640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c108cf0 0x7fb14c1a2700 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b4a0 tx=0x7fb13c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.484+0000 7fb146a00640 1 -- 172.21.15.49:0/2738171759 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c010610 con 0x7fb14c108cf0 2024-02-26T06:33:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.485+0000 7fb146a00640 1 -- 172.21.15.49:0/2738171759 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb13c010c50 con 0x7fb14c108cf0 2024-02-26T06:33:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.485+0000 7fb146a00640 1 -- 172.21.15.49:0/2738171759 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c017e20 con 0x7fb14c108cf0 2024-02-26T06:33:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.486+0000 7fb154800640 1 -- 172.21.15.49:0/2738171759 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c1a7690 con 0x7fb14c108cf0 2024-02-26T06:33:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.486+0000 7fb154800640 1 -- 172.21.15.49:0/2738171759 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c10bf90 con 0x7fb14c108cf0 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.493+0000 7fb146a00640 1 -- 172.21.15.49:0/2738171759 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb13c017480 con 0x7fb14c108cf0 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.493+0000 7fb15779c740 1 Processor -- start 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.494+0000 7fb15779c740 1 -- start start 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.494+0000 7fb15779c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab45840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.494+0000 7fb15779c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab46200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.494+0000 7fb15779c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab4f300 0x557f4ab516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.494+0000 7fb15779c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4ab015f0 con 0x557f4ab4f300 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.494+0000 7fb15779c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4aab3890 con 0x557f4ab45d80 2024-02-26T06:33:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.494+0000 7fb15779c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4aab0dc0 con 0x557f4ab46af0 2024-02-26T06:33:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.495+0000 7fb151e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab46200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.495+0000 7fb151e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab46200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43620/0 (socket says 172.21.15.49:43620) 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.495+0000 7fb151e00640 1 -- 172.21.15.49:0/2339325872 learned_addr learned my addr 172.21.15.49:0/2339325872 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.496+0000 7fb152800640 1 --2- 172.21.15.49:0/2339325872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab45840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.496+0000 7fb151e00640 1 -- 172.21.15.49:0/2339325872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 msgr2=0x557f4ab45840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.496+0000 7fb151e00640 1 --2- 172.21.15.49:0/2339325872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab45840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.496+0000 7fb151e00640 1 -- 172.21.15.49:0/2339325872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab4f300 msgr2=0x557f4ab516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.496+0000 7fb151e00640 1 --2- 172.21.15.49:0/2339325872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab4f300 0x557f4ab516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.496+0000 7fb151e00640 1 -- 172.21.15.49:0/2339325872 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4aa2dad0 con 0x557f4ab45d80 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.497+0000 7fb146a00640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb1340a24b0 0x7fb1340a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.497+0000 7fb151e00640 1 --2- 172.21.15.49:0/2339325872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab46200 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb14800a5b0 tx=0x7fb1480096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fdbc434aaf045d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.497+0000 7fb146a00640 1 -- 172.21.15.49:0/2738171759 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fb13c0cd5b0 con 0x7fb14c108cf0 2024-02-26T06:33:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.498+0000 7fb152800640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb1340a24b0 0x7fb1340a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.498+0000 7fb13b000640 1 -- 172.21.15.49:0/2339325872 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148010040 con 0x557f4ab45d80 2024-02-26T06:33:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.498+0000 7fb13b000640 1 -- 172.21.15.49:0/2339325872 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb148014cf0 con 0x557f4ab45d80 2024-02-26T06:33:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.498+0000 7fb13b000640 1 -- 172.21.15.49:0/2339325872 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148013d80 con 0x557f4ab45d80 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.499+0000 7fb15779c740 1 -- 172.21.15.49:0/2339325872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 msgr2=0x557f4ab46200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.499+0000 7fb15779c740 1 --2- 172.21.15.49:0/2339325872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab46200 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb14800a5b0 tx=0x7fb1480096d0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.499+0000 7fb152800640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb1340a24b0 0x7fb1340a4970 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb140009900 tx=0x7fb140005e80 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.499+0000 7fb15779c740 1 -- 172.21.15.49:0/2339325872 shutdown_connections 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.499+0000 7fb15779c740 1 --2- 172.21.15.49:0/2339325872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab46200 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.499+0000 7fb15779c740 1 --2- 172.21.15.49:0/2339325872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab4f300 0x557f4ab516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.499+0000 7fb15779c740 1 --2- 172.21.15.49:0/2339325872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab45840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.499+0000 7fb15779c740 1 -- 172.21.15.49:0/2339325872 >> 172.21.15.49:0/2339325872 conn(0x557f4ab0d480 msgr2=0x557f4ab40a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.500+0000 7fb15779c740 1 -- 172.21.15.49:0/2339325872 shutdown_connections 2024-02-26T06:33:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.501+0000 7fb15779c740 1 -- 172.21.15.49:0/2339325872 wait complete. 2024-02-26T06:33:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.501+0000 7fb15779c740 1 -- start start 2024-02-26T06:33:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.501+0000 7fb15779c740 1 Processor -- start 2024-02-26T06:33:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.502+0000 7fb15779c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab4c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.502+0000 7fb15779c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab46af0 0x557f4ab4c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.502+0000 7fb15779c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab4f300 0x557f4ab4ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.502+0000 7fb15779c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4aaac5a0 con 0x557f4ab46af0 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.502+0000 7fb15779c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4aa51b20 con 0x557f4ab45d80 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.502+0000 7fb15779c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4aa517c0 con 0x557f4ab4f300 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.502+0000 7fb15779c740 10 client.0 _ll_register_callbacks cb 0x557f4aa65520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.503+0000 7fb152800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab4f300 0x557f4ab4ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.503+0000 7fb152800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab4f300 0x557f4ab4ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:52766/0 (socket says 172.21.15.49:52766) 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.503+0000 7fb152800640 1 -- 172.21.15.49:0/2224042225 learned_addr learned my addr 172.21.15.49:0/2224042225 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.503+0000 7fb151400640 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab46af0 0x557f4ab4c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.503+0000 7fb151e00640 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab4c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.503+0000 7fb152800640 1 -- 172.21.15.49:0/2224042225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 msgr2=0x557f4ab4c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb152800640 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab4c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb152800640 1 -- 172.21.15.49:0/2224042225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab46af0 msgr2=0x557f4ab4c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb152800640 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab46af0 0x557f4ab4c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb152800640 1 -- 172.21.15.49:0/2224042225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb148009d80 con 0x557f4ab4f300 2024-02-26T06:33:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb151400640 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab46af0 0x557f4ab4c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:33:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb152800640 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab4f300 0x557f4ab4ce80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb140035000 tx=0x7fb140005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98ef361f1dda2c06 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb12f400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:33:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb12f400640 1 -- 172.21.15.49:0/2224042225 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14002cc80 con 0x557f4ab4f300 2024-02-26T06:33:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.504+0000 7fb12f400640 1 -- 172.21.15.49:0/2224042225 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fb140047040 con 0x557f4ab4f300 2024-02-26T06:33:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.505+0000 7fb15779c740 1 -- 172.21.15.49:0/2224042225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f4aa7eca0 con 0x557f4ab4f300 2024-02-26T06:33:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.506+0000 7fb12f400640 1 -- 172.21.15.49:0/2224042225 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb140005000 con 0x557f4ab4f300 2024-02-26T06:33:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.506+0000 7fb15779c740 10 client.27346 fetch_fsmap learned FSMap version 46 2024-02-26T06:33:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.506+0000 7fb15779c740 1 -- 172.21.15.49:0/2224042225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x557f4aaaac20 con 0x557f4ab4f300 2024-02-26T06:33:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.506+0000 7fb12f400640 1 -- 172.21.15.49:0/2224042225 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557f4aa7eca0 con 0x557f4ab4f300 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.507+0000 7fb12f400640 1 -- 172.21.15.49:0/2224042225 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb140003c40 con 0x557f4ab4f300 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.507+0000 7fb15779c740 10 client.27346 fetch_fsmap finished waiting for FSMap version 46 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.507+0000 7fb15779c740 10 client.27346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.507+0000 7fb15779c740 20 client.27346 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.507+0000 7fb15779c740 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557f4ab5f3b0 0x557f4ab617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.507+0000 7fb15779c740 4 client.27346 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.507+0000 7fb15779c740 1 -- 172.21.15.49:0/2224042225 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f4aaa2cb0 con 0x557f4ab5f3b0 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.508+0000 7fb151e00640 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557f4ab5f3b0 0x557f4ab617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.509+0000 7fb12f400640 10 client.27346 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.509+0000 7fb151e00640 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557f4ab5f3b0 0x557f4ab617d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31d0b92a3551b5b server_cookie=4af36b903a691b1a in_seq=0 out_seq=0 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.510+0000 7fb12f400640 1 -- 172.21.15.49:0/2224042225 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f4aaa2cb0 con 0x557f4ab5f3b0 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.510+0000 7fb12f400640 10 client.27346 handle_command_reply: tid=0 2024-02-26T06:33:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.510+0000 7fb15779c740 1 client.27346 shutdown 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb15779c740 1 -- 172.21.15.49:0/2224042225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab4f300 msgr2=0x557f4ab4ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb15779c740 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab4f300 0x557f4ab4ce80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb140035000 tx=0x7fb140005fd0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb15779c740 1 -- 172.21.15.49:0/2224042225 shutdown_connections 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb15779c740 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557f4ab5f3b0 0x557f4ab617d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb15779c740 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab4f300 0x557f4ab4ce80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb15779c740 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab46af0 0x557f4ab4c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb15779c740 1 --2- 172.21.15.49:0/2224042225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab4c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb15779c740 1 -- 172.21.15.49:0/2224042225 >> 172.21.15.49:0/2224042225 conn(0x557f4ab0d480 msgr2=0x557f4ab427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.511+0000 7fb12f400640 0 client.27346 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.512+0000 7fb15779c740 1 -- 172.21.15.49:0/2224042225 shutdown_connections 2024-02-26T06:33:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.513+0000 7fb15779c740 1 -- 172.21.15.49:0/2224042225 wait complete. 2024-02-26T06:33:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.514+0000 7fb15779c740 20 client.27346 trim_cache size 0 max 16384 2024-02-26T06:33:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.545+0000 7fb15779c740 1 Processor -- start 2024-02-26T06:33:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.545+0000 7fb15779c740 1 -- start start 2024-02-26T06:33:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.545+0000 7fb15779c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 0x557f4ab4eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.545+0000 7fb15779c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab56120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.545+0000 7fb15779c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab56660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.545+0000 7fb15779c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4a7f4e60 con 0x557f4ab0d480 2024-02-26T06:33:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.545+0000 7fb15779c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4ab51f10 con 0x557f4ab45d80 2024-02-26T06:33:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.545+0000 7fb15779c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4ab667e0 con 0x557f4ab46af0 2024-02-26T06:33:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.546+0000 7fb151400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab56120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.546+0000 7fb151400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab56120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43642/0 (socket says 172.21.15.49:43642) 2024-02-26T06:33:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.546+0000 7fb151400640 1 -- 172.21.15.49:0/3618134233 learned_addr learned my addr 172.21.15.49:0/3618134233 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.546+0000 7fb152800640 1 --2- 172.21.15.49:0/3618134233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab56660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.547+0000 7fb151400640 1 -- 172.21.15.49:0/3618134233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 msgr2=0x557f4ab56660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.547+0000 7fb151400640 1 --2- 172.21.15.49:0/3618134233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab56660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.547+0000 7fb151400640 1 -- 172.21.15.49:0/3618134233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 msgr2=0x557f4ab4eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.547+0000 7fb151400640 1 --2- 172.21.15.49:0/3618134233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 0x557f4ab4eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.547+0000 7fb151400640 1 -- 172.21.15.49:0/3618134233 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4aa47500 con 0x557f4ab45d80 2024-02-26T06:33:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.547+0000 7fb151400640 1 --2- 172.21.15.49:0/3618134233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab56120 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb13c096490 tx=0x7fb13c0047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd09b9c11ee08588 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.547+0000 7fb13a600640 1 -- 172.21.15.49:0/3618134233 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c09e180 con 0x557f4ab45d80 2024-02-26T06:33:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.547+0000 7fb13a600640 1 -- 172.21.15.49:0/3618134233 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb13c0a27d0 con 0x557f4ab45d80 2024-02-26T06:33:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.548+0000 7fb13a600640 1 -- 172.21.15.49:0/3618134233 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c094ba0 con 0x557f4ab45d80 2024-02-26T06:33:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.548+0000 7fb15779c740 1 -- 172.21.15.49:0/3618134233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 msgr2=0x557f4ab56120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 --2- 172.21.15.49:0/3618134233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab56120 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb13c096490 tx=0x7fb13c0047d0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 -- 172.21.15.49:0/3618134233 shutdown_connections 2024-02-26T06:33:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 --2- 172.21.15.49:0/3618134233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab56120 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 --2- 172.21.15.49:0/3618134233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab56660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 --2- 172.21.15.49:0/3618134233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 0x557f4ab4eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 -- 172.21.15.49:0/3618134233 >> 172.21.15.49:0/3618134233 conn(0x557f4ab518d0 msgr2=0x557f4ab405a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 -- 172.21.15.49:0/3618134233 shutdown_connections 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 -- 172.21.15.49:0/3618134233 wait complete. 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 -- start start 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.549+0000 7fb15779c740 1 Processor -- start 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb15779c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 0x557f4ab49030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb15779c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab49570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb15779c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb15779c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4abb58f0 con 0x557f4ab0d480 2024-02-26T06:33:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb15779c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4abb5a30 con 0x557f4ab45d80 2024-02-26T06:33:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb15779c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4abb5b70 con 0x557f4ab46af0 2024-02-26T06:33:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb15779c740 10 client.0 _ll_register_callbacks cb 0x7fb1400047e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:33:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb151400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab49570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb151e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 0x557f4ab49030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb151400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab49570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43646/0 (socket says 172.21.15.49:43646) 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb151400640 1 -- 172.21.15.49:0/4102336361 learned_addr learned my addr 172.21.15.49:0/4102336361 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.550+0000 7fb152800640 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.551+0000 7fb151400640 1 -- 172.21.15.49:0/4102336361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 msgr2=0x557f4ab49ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.551+0000 7fb151400640 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab49ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.551+0000 7fb151400640 1 -- 172.21.15.49:0/4102336361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 msgr2=0x557f4ab49030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.551+0000 7fb151400640 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 0x557f4ab49030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.551+0000 7fb151400640 1 -- 172.21.15.49:0/4102336361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c10bf90 con 0x557f4ab45d80 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.551+0000 7fb151400640 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab49570 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0a22a0 tx=0x7fb13c017880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d0917b88bea56a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.551+0000 7fb12f400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.551+0000 7fb12f400640 1 -- 172.21.15.49:0/4102336361 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c09e180 con 0x557f4ab45d80 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.552+0000 7fb12f400640 1 -- 172.21.15.49:0/4102336361 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fb13c094ba0 con 0x557f4ab45d80 2024-02-26T06:33:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.552+0000 7fb15779c740 1 -- 172.21.15.49:0/4102336361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f4aad1d00 con 0x557f4ab45d80 2024-02-26T06:33:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.552+0000 7fb12f400640 1 -- 172.21.15.49:0/4102336361 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb13c094da0 con 0x557f4ab45d80 2024-02-26T06:33:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.552+0000 7fb15779c740 10 client.27398 fetch_fsmap learned FSMap version 46 2024-02-26T06:33:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb15779c740 1 -- 172.21.15.49:0/4102336361 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x557f4aa6d290 con 0x557f4ab45d80 2024-02-26T06:33:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb12f400640 1 -- 172.21.15.49:0/4102336361 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557f4aad1d00 con 0x557f4ab45d80 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb12f400640 1 -- 172.21.15.49:0/4102336361 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb13c020020 con 0x557f4ab45d80 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb15779c740 10 client.27398 fetch_fsmap finished waiting for FSMap version 46 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb15779c740 10 client.27398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb15779c740 20 client.27398 populate_metadata read hostname 'smithi049' 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb15779c740 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557f4ab4f300 0x557f4abb8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb15779c740 4 client.27398 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.553+0000 7fb15779c740 1 -- 172.21.15.49:0/4102336361 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f4a7ac600 con 0x557f4ab4f300 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.554+0000 7fb151e00640 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557f4ab4f300 0x557f4abb8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.554+0000 7fb151e00640 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557f4ab4f300 0x557f4abb8ec0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455bde2d0f408dc0 server_cookie=329c64a8a8b8096b in_seq=0 out_seq=0 2024-02-26T06:33:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.554+0000 7fb12f400640 10 client.27398 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:54 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.589+0000 7fb12f400640 1 -- 172.21.15.49:0/4102336361 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f4a7ac600 con 0x557f4ab4f300 2024-02-26T06:33:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.589+0000 7fb12f400640 10 client.27398 handle_command_reply: tid=0 2024-02-26T06:33:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.589+0000 7fb15779c740 1 client.27398 shutdown 2024-02-26T06:33:54.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:54.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:54.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:54 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.589+0000 7fb15779c740 1 -- 172.21.15.49:0/4102336361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 msgr2=0x557f4ab49570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.589+0000 7fb15779c740 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab49570 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0a22a0 tx=0x7fb13c017880 comp rx=0 tx=0).stop 2024-02-26T06:33:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 1 -- 172.21.15.49:0/4102336361 shutdown_connections 2024-02-26T06:33:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557f4ab4f300 0x557f4abb8ec0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557f4ab45d80 0x557f4ab49570 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f4ab46af0 0x557f4ab49ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 1 --2- 172.21.15.49:0/4102336361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f4ab0d480 0x557f4ab49030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb12f400640 0 client.27398 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 1 -- 172.21.15.49:0/4102336361 >> 172.21.15.49:0/4102336361 conn(0x557f4ab518d0 msgr2=0x557f4ab4ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 1 -- 172.21.15.49:0/4102336361 shutdown_connections 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 1 -- 172.21.15.49:0/4102336361 wait complete. 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.590+0000 7fb15779c740 20 client.27398 trim_cache size 0 max 16384 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:33:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:33:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:33:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:33:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 -- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb1340a24b0 msgr2=0x7fb1340a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb1340a24b0 0x7fb1340a4970 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb140009900 tx=0x7fb140005e80 comp rx=0 tx=0).stop 2024-02-26T06:33:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 -- 172.21.15.49:0/2738171759 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c108cf0 msgr2=0x7fb14c1a2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:33:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c108cf0 0x7fb14c1a2700 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b4a0 tx=0x7fb13c00b960 comp rx=0 tx=0).stop 2024-02-26T06:33:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 -- 172.21.15.49:0/2738171759 shutdown_connections 2024-02-26T06:33:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb1340a24b0 0x7fb1340a4970 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb14c108cf0 0x7fb14c1a2700 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb14c1142d0 0x7fb14c1a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 --2- 172.21.15.49:0/2738171759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb14c1083d0 0x7fb14c1a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:33:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 -- 172.21.15.49:0/2738171759 >> 172.21.15.49:0/2738171759 conn(0x7fb14c06c220 msgr2=0x7fb14c109b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:33:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.591+0000 7fb154800640 1 -- 172.21.15.49:0/2738171759 shutdown_connections 2024-02-26T06:33:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:33:54.592+0000 7fb154800640 1 -- 172.21.15.49:0/2738171759 wait complete. 2024-02-26T06:33:54.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:54.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:54.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:54.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:54 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:55.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:55 smithi080 ceph-mon[36206]: pgmap v326: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.1 MiB/s wr, 535 op/s 2024-02-26T06:33:55.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:55 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-02-26T06:33:55.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:55 smithi080 ceph-mon[36206]: Unable to set osd_memory_target on smithi115 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-02-26T06:33:55.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:55 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi080 to 174.1M 2024-02-26T06:33:55.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:55 smithi080 ceph-mon[36206]: Unable to set osd_memory_target on smithi080 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-26T06:33:55.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:55 smithi115 ceph-mon[36426]: pgmap v326: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.1 MiB/s wr, 535 op/s 2024-02-26T06:33:55.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:55 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-02-26T06:33:55.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:55 smithi115 ceph-mon[36426]: Unable to set osd_memory_target on smithi115 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-02-26T06:33:55.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:55 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi080 to 174.1M 2024-02-26T06:33:55.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:55 smithi115 ceph-mon[36426]: Unable to set osd_memory_target on smithi080 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-26T06:33:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:55 smithi049 ceph-mon[31696]: pgmap v326: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.1 MiB/s wr, 535 op/s 2024-02-26T06:33:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:55 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-02-26T06:33:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:55 smithi049 ceph-mon[31696]: Unable to set osd_memory_target on smithi115 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-02-26T06:33:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:55 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi080 to 174.1M 2024-02-26T06:33:55.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:55 smithi049 ceph-mon[31696]: Unable to set osd_memory_target on smithi080 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-26T06:33:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 605.0 s, 987.0 tps, lat 100.517 ms stddev 111.111 2024-02-26T06:33:56.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:56 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:56.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:56 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:56.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:56 smithi049 ceph-mon[31696]: pgmap v327: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.4 MiB/s wr, 680 op/s 2024-02-26T06:33:56.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:56 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:56.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:56 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:56.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:56 smithi080 ceph-mon[36206]: pgmap v327: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.4 MiB/s wr, 680 op/s 2024-02-26T06:33:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:56 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:56 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:56 smithi115 ceph-mon[36426]: pgmap v327: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.4 MiB/s wr, 680 op/s 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: Adjusting osd_memory_target on smithi049 to 176.7M 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: Unable to set osd_memory_target on smithi049 to 185363763: error parsing value: Value '185363763' is below minimum 939524096 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:33:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:57 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:57.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: Adjusting osd_memory_target on smithi049 to 176.7M 2024-02-26T06:33:57.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: Unable to set osd_memory_target on smithi049 to 185363763: error parsing value: Value '185363763' is below minimum 939524096 2024-02-26T06:33:57.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:33:57.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:33:57.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:57 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:57.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:33:57.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: Adjusting osd_memory_target on smithi049 to 176.7M 2024-02-26T06:33:57.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: Unable to set osd_memory_target on smithi049 to 185363763: error parsing value: Value '185363763' is below minimum 939524096 2024-02-26T06:33:57.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:33:57.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:33:57.841 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:57 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:33:58.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:33:58 smithi049 ceph-mon[31696]: pgmap v328: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.0 MiB/s wr, 625 op/s 2024-02-26T06:33:58.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:33:58 smithi080 ceph-mon[36206]: pgmap v328: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.0 MiB/s wr, 625 op/s 2024-02-26T06:33:58.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:33:58 smithi115 ceph-mon[36426]: pgmap v328: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.0 MiB/s wr, 625 op/s 2024-02-26T06:34:00.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:00 smithi049 ceph-mon[31696]: pgmap v329: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.4 MiB/s wr, 611 op/s 2024-02-26T06:34:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:00 smithi080 ceph-mon[36206]: pgmap v329: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.4 MiB/s wr, 611 op/s 2024-02-26T06:34:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:00 smithi115 ceph-mon[36426]: pgmap v329: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.4 MiB/s wr, 611 op/s 2024-02-26T06:34:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 610.0 s, 1054.4 tps, lat 88.656 ms stddev 99.137 2024-02-26T06:34:02.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:02 smithi049 ceph-mon[31696]: pgmap v330: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.4 MiB/s wr, 704 op/s 2024-02-26T06:34:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:02 smithi080 ceph-mon[36206]: pgmap v330: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.4 MiB/s wr, 704 op/s 2024-02-26T06:34:03.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:02 smithi115 ceph-mon[36426]: pgmap v330: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.4 MiB/s wr, 704 op/s 2024-02-26T06:34:04.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:04 smithi049 ceph-mon[31696]: pgmap v331: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-02-26T06:34:05.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:04 smithi080 ceph-mon[36206]: pgmap v331: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-02-26T06:34:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:04 smithi115 ceph-mon[36426]: pgmap v331: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-02-26T06:34:06.322 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 615.0 s, 132.6 tps, lat 748.560 ms stddev 424.052 2024-02-26T06:34:06.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:06 smithi049 ceph-mon[31696]: pgmap v332: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-02-26T06:34:06.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:06 smithi049 ceph-mon[31696]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T06:34:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:06 smithi080 ceph-mon[36206]: pgmap v332: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-02-26T06:34:07.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:06 smithi080 ceph-mon[36206]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T06:34:07.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:06 smithi115 ceph-mon[36426]: pgmap v332: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-02-26T06:34:07.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:06 smithi115 ceph-mon[36426]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T06:34:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 -- 172.21.15.49:0/1880766002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc408103f50 msgr2=0x7fc408107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 --2- 172.21.15.49:0/1880766002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc408103f50 0x7fc408107f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009900 tx=0x7fc3f002bc40 comp rx=0 tx=0).stop 2024-02-26T06:34:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 -- 172.21.15.49:0/1880766002 shutdown_connections 2024-02-26T06:34:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 --2- 172.21.15.49:0/1880766002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc408108670 0x7fc408110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 --2- 172.21.15.49:0/1880766002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc408103f50 0x7fc408107f90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 --2- 172.21.15.49:0/1880766002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4081035a0 0x7fc408103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 -- 172.21.15.49:0/1880766002 >> 172.21.15.49:0/1880766002 conn(0x7fc4080fee00 msgr2=0x7fc408101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 -- 172.21.15.49:0/1880766002 shutdown_connections 2024-02-26T06:34:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.781+0000 7fc40e600640 1 -- 172.21.15.49:0/1880766002 wait complete. 2024-02-26T06:34:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.783+0000 7fc40e600640 1 Processor -- start 2024-02-26T06:34:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.783+0000 7fc40e600640 1 -- start start 2024-02-26T06:34:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc40e600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4081035a0 0x7fc40819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc40e600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc408108670 0x7fc40819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc40e600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4081997d0 0x7fc408199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc40e600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40819a280 con 0x7fc4081997d0 2024-02-26T06:34:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc40e600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40819a3f0 con 0x7fc408108670 2024-02-26T06:34:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc40e600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40819a560 con 0x7fc4081035a0 2024-02-26T06:34:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc406a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc408108670 0x7fc40819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc406a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc408108670 0x7fc40819fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58644/0 (socket says 172.21.15.49:58644) 2024-02-26T06:34:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc406a00640 1 -- 172.21.15.49:0/3013915697 learned_addr learned my addr 172.21.15.49:0/3013915697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc407400640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4081035a0 0x7fc40819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc407e00640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4081997d0 0x7fc408199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc406a00640 1 -- 172.21.15.49:0/3013915697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4081035a0 msgr2=0x7fc40819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc406a00640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4081035a0 0x7fc40819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc406a00640 1 -- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4081997d0 msgr2=0x7fc408199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc406a00640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4081997d0 0x7fc408199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc406a00640 1 -- 172.21.15.49:0/3013915697 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f0009560 con 0x7fc408108670 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.784+0000 7fc407e00640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4081997d0 0x7fc408199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.785+0000 7fc406a00640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc408108670 0x7fc40819fb50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0036040 tx=0x7fc3f002c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.785+0000 7fc3ffe00640 1 -- 172.21.15.49:0/3013915697 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00043d0 con 0x7fc408108670 2024-02-26T06:34:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.785+0000 7fc3ffe00640 1 -- 172.21.15.49:0/3013915697 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3f00052d0 con 0x7fc408108670 2024-02-26T06:34:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.785+0000 7fc3ffe00640 1 -- 172.21.15.49:0/3013915697 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0003cd0 con 0x7fc408108670 2024-02-26T06:34:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.785+0000 7fc40e600640 1 -- 172.21.15.49:0/3013915697 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4081a6460 con 0x7fc408108670 2024-02-26T06:34:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.785+0000 7fc40e600640 1 -- 172.21.15.49:0/3013915697 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4081a68f0 con 0x7fc408108670 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.789+0000 7fc411587740 1 Processor -- start 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc411587740 1 -- start start 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc3ffe00640 1 -- 172.21.15.49:0/3013915697 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc3f0005440 con 0x7fc408108670 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc411587740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc411587740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 0x55cb4bde2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc411587740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 0x55cb4bded6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc411587740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bd9d5f0 con 0x55cb4bde1d80 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc411587740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bd4f890 con 0x55cb4bde2af0 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc411587740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bd4cdc0 con 0x55cb4bdeb300 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc407e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc407e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58660/0 (socket says 172.21.15.49:58660) 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc407e00640 1 -- 172.21.15.49:0/4188320824 learned_addr learned my addr 172.21.15.49:0/4188320824 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc406a00640 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 0x55cb4bded6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.790+0000 7fc407400640 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 0x55cb4bde2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.791+0000 7fc407e00640 1 -- 172.21.15.49:0/4188320824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 msgr2=0x55cb4bded6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.791+0000 7fc407e00640 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 0x55cb4bded6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.791+0000 7fc407e00640 1 -- 172.21.15.49:0/4188320824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 msgr2=0x55cb4bde2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.791+0000 7fc407e00640 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 0x55cb4bde2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.791+0000 7fc407e00640 1 -- 172.21.15.49:0/4188320824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb4bcc9ad0 con 0x55cb4bde2af0 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.791+0000 7fc407e00640 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde1840 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009c00 tx=0x7fc3f8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e6d519b3c9b5f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.792+0000 7fc3fcc00640 1 -- 172.21.15.49:0/4188320824 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8004030 con 0x55cb4bde2af0 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.792+0000 7fc3fcc00640 1 -- 172.21.15.49:0/4188320824 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3f80035f0 con 0x55cb4bde2af0 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.792+0000 7fc3fcc00640 1 -- 172.21.15.49:0/4188320824 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f80126b0 con 0x55cb4bde2af0 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.792+0000 7fc3ffe00640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc3ec0a2350 0x7fc3ec0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.792+0000 7fc3ffe00640 1 -- 172.21.15.49:0/3013915697 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fc3f00f0210 con 0x7fc408108670 2024-02-26T06:34:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.792+0000 7fc407400640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc3ec0a2350 0x7fc3ec0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 -- 172.21.15.49:0/4188320824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 msgr2=0x55cb4bde1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde1840 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009c00 tx=0x7fc3f8010740 comp rx=0 tx=0).stop 2024-02-26T06:34:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 -- 172.21.15.49:0/4188320824 shutdown_connections 2024-02-26T06:34:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 0x55cb4bded6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 0x55cb4bde2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 --2- 172.21.15.49:0/4188320824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde1840 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 -- 172.21.15.49:0/4188320824 >> 172.21.15.49:0/4188320824 conn(0x55cb4bda9480 msgr2=0x55cb4bddca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 -- 172.21.15.49:0/4188320824 shutdown_connections 2024-02-26T06:34:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc411587740 1 -- 172.21.15.49:0/4188320824 wait complete. 2024-02-26T06:34:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.793+0000 7fc407400640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc3ec0a2350 0x7fc3ec0a4810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55cb4bde3250 tx=0x7fc4000097c0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.794+0000 7fc411587740 1 -- start start 2024-02-26T06:34:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.794+0000 7fc411587740 1 Processor -- start 2024-02-26T06:34:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc411587740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde1d80 0x55cb4bde8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc411587740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc411587740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bdeb300 0x55cb4bde8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc411587740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bd485a0 con 0x55cb4bdeb300 2024-02-26T06:34:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc411587740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bcedb20 con 0x55cb4bde2af0 2024-02-26T06:34:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc411587740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bced7c0 con 0x55cb4bde1d80 2024-02-26T06:34:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc411587740 10 client.0 _ll_register_callbacks cb 0x55cb4bd01520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc406a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc407400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde1d80 0x55cb4bde8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc406a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58664/0 (socket says 172.21.15.49:58664) 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc406a00640 1 -- 172.21.15.49:0/3451915017 learned_addr learned my addr 172.21.15.49:0/3451915017 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc407e00640 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bdeb300 0x55cb4bde8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc406a00640 1 -- 172.21.15.49:0/3451915017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde1d80 msgr2=0x55cb4bde8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc406a00640 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde1d80 0x55cb4bde8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc406a00640 1 -- 172.21.15.49:0/3451915017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bdeb300 msgr2=0x55cb4bde8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc406a00640 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bdeb300 0x55cb4bde8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc406a00640 1 -- 172.21.15.49:0/3451915017 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f80098b0 con 0x55cb4bde2af0 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc407e00640 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bdeb300 0x55cb4bde8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.795+0000 7fc407400640 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde1d80 0x55cb4bde8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc406a00640 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde8890 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0000c00 tx=0x7fc3f00045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919b47160c452f42 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc3f5600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc3f5600640 1 -- 172.21.15.49:0/3451915017 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f003d5b0 con 0x55cb4bde2af0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc3f5600640 1 -- 172.21.15.49:0/3451915017 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fc3f00035f0 con 0x55cb4bde2af0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc411587740 1 -- 172.21.15.49:0/3451915017 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb4bd1aca0 con 0x55cb4bde2af0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc3f5600640 1 -- 172.21.15.49:0/3451915017 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3f00bc620 con 0x55cb4bde2af0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc411587740 10 client.27410 fetch_fsmap learned FSMap version 46 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc411587740 1 -- 172.21.15.49:0/3451915017 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55cb4bd46c20 con 0x55cb4bde2af0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.796+0000 7fc3f5600640 1 -- 172.21.15.49:0/3451915017 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cb4bd1aca0 con 0x55cb4bde2af0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.797+0000 7fc3f5600640 1 -- 172.21.15.49:0/3451915017 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc3f003c070 con 0x55cb4bde2af0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.797+0000 7fc411587740 10 client.27410 fetch_fsmap finished waiting for FSMap version 46 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.797+0000 7fc411587740 10 client.27410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.797+0000 7fc411587740 20 client.27410 populate_metadata read hostname 'smithi049' 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.797+0000 7fc411587740 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cb4bdfb0c0 0x55cb4bdfd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.797+0000 7fc411587740 4 client.27410 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.797+0000 7fc411587740 1 -- 172.21.15.49:0/3451915017 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb4bd3ecb0 con 0x55cb4bdfb0c0 2024-02-26T06:34:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.797+0000 7fc407400640 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cb4bdfb0c0 0x55cb4bdfd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.798+0000 7fc407400640 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cb4bdfb0c0 0x55cb4bdfd4e0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282e515cd10686c0 server_cookie=659ba119826e4e57 in_seq=0 out_seq=0 2024-02-26T06:34:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.798+0000 7fc3f5600640 10 client.27410 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.798+0000 7fc3f5600640 1 -- 172.21.15.49:0/3451915017 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb4bd3ecb0 con 0x55cb4bdfb0c0 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.798+0000 7fc3f5600640 10 client.27410 handle_command_reply: tid=0 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 client.27410 shutdown 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 -- 172.21.15.49:0/3451915017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 msgr2=0x55cb4bde8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde8890 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0000c00 tx=0x7fc3f00045b0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 -- 172.21.15.49:0/3451915017 shutdown_connections 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cb4bdfb0c0 0x55cb4bdfd4e0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde2af0 0x55cb4bde8890 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bdeb300 0x55cb4bde8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 --2- 172.21.15.49:0/3451915017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde1d80 0x55cb4bde8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc3f5600640 0 client.27410 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 -- 172.21.15.49:0/3451915017 >> 172.21.15.49:0/3451915017 conn(0x55cb4bda9480 msgr2=0x55cb4bdde7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 -- 172.21.15.49:0/3451915017 shutdown_connections 2024-02-26T06:34:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 1 -- 172.21.15.49:0/3451915017 wait complete. 2024-02-26T06:34:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.799+0000 7fc411587740 20 client.27410 trim_cache size 0 max 16384 2024-02-26T06:34:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.815+0000 7fc411587740 1 Processor -- start 2024-02-26T06:34:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.815+0000 7fc411587740 1 -- start start 2024-02-26T06:34:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.815+0000 7fc411587740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde1d80 0x55cb4bddc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.815+0000 7fc411587740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde2af0 0x55cb4bddcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.815+0000 7fc411587740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 0x55cb4bde6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.815+0000 7fc411587740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4ba90e60 con 0x55cb4bde2af0 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.815+0000 7fc411587740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bdddca0 con 0x55cb4bde1d80 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.815+0000 7fc411587740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bde8160 con 0x55cb4bdeb300 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde1d80 0x55cb4bddc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc406a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde2af0 0x55cb4bddcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde1d80 0x55cb4bddc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58674/0 (socket says 172.21.15.49:58674) 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 -- 172.21.15.49:0/2311823427 learned_addr learned my addr 172.21.15.49:0/2311823427 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407e00640 1 --2- 172.21.15.49:0/2311823427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 0x55cb4bde6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 -- 172.21.15.49:0/2311823427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 msgr2=0x55cb4bde6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 --2- 172.21.15.49:0/2311823427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 0x55cb4bde6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 -- 172.21.15.49:0/2311823427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde2af0 msgr2=0x55cb4bddcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 --2- 172.21.15.49:0/2311823427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde2af0 0x55cb4bddcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 -- 172.21.15.49:0/2311823427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb4bce3500 con 0x55cb4bde1d80 2024-02-26T06:34:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.816+0000 7fc407400640 1 --2- 172.21.15.49:0/2311823427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde1d80 0x55cb4bddc5a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc40001c4e0 tx=0x7fc400015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbb88ebdb48b318 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.817+0000 7fc3f7e00640 1 -- 172.21.15.49:0/2311823427 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400022040 con 0x55cb4bde1d80 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.817+0000 7fc3f7e00640 1 -- 172.21.15.49:0/2311823427 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc400027900 con 0x55cb4bde1d80 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.817+0000 7fc3f7e00640 1 -- 172.21.15.49:0/2311823427 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40001ce50 con 0x55cb4bde1d80 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 -- 172.21.15.49:0/2311823427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde1d80 msgr2=0x55cb4bddc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 --2- 172.21.15.49:0/2311823427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde1d80 0x55cb4bddc5a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc40001c4e0 tx=0x7fc400015000 comp rx=0 tx=0).stop 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 -- 172.21.15.49:0/2311823427 shutdown_connections 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 --2- 172.21.15.49:0/2311823427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bdeb300 0x55cb4bde6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 --2- 172.21.15.49:0/2311823427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde2af0 0x55cb4bddcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 --2- 172.21.15.49:0/2311823427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bde1d80 0x55cb4bddc5a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 -- 172.21.15.49:0/2311823427 >> 172.21.15.49:0/2311823427 conn(0x55cb4bda9480 msgr2=0x55cb4bde12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 -- 172.21.15.49:0/2311823427 shutdown_connections 2024-02-26T06:34:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.818+0000 7fc411587740 1 -- 172.21.15.49:0/2311823427 wait complete. 2024-02-26T06:34:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.819+0000 7fc411587740 1 -- start start 2024-02-26T06:34:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.819+0000 7fc411587740 1 Processor -- start 2024-02-26T06:34:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.819+0000 7fc411587740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 0x55cb4bdf07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.819+0000 7fc411587740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde2af0 0x55cb4bdf0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.819+0000 7fc411587740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bdeb300 0x55cb4bdf1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.819+0000 7fc411587740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bdf2170 con 0x55cb4bde1d80 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc411587740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bdf22e0 con 0x55cb4bdeb300 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc411587740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bdf24a0 con 0x55cb4bde2af0 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc411587740 10 client.0 _ll_register_callbacks cb 0x7fc3f00f3830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bdeb300 0x55cb4bdf1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc406a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde2af0 0x55cb4bdf0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bdeb300 0x55cb4bdf1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58682/0 (socket says 172.21.15.49:58682) 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 0x55cb4bdf07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407e00640 1 -- 172.21.15.49:0/176697204 learned_addr learned my addr 172.21.15.49:0/176697204 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc406a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde2af0 0x55cb4bdf0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:55102/0 (socket says 172.21.15.49:55102) 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407e00640 1 -- 172.21.15.49:0/176697204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde2af0 msgr2=0x55cb4bdf0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407e00640 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde2af0 0x55cb4bdf0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407e00640 1 -- 172.21.15.49:0/176697204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 msgr2=0x55cb4bdf07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407e00640 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 0x55cb4bdf07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc407e00640 1 -- 172.21.15.49:0/176697204 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc400004750 con 0x55cb4bdeb300 2024-02-26T06:34:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.820+0000 7fc406a00640 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde2af0 0x55cb4bdf0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.821+0000 7fc407e00640 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bdeb300 0x55cb4bdf1bb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8010c40 tx=0x7fc3f80050b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c61c04e59ab799c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.821+0000 7fc3f5600640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:34:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.821+0000 7fc3f5600640 1 -- 172.21.15.49:0/176697204 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f80055b0 con 0x55cb4bdeb300 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.821+0000 7fc3f5600640 1 -- 172.21.15.49:0/176697204 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fc3f8016030 con 0x55cb4bdeb300 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.821+0000 7fc411587740 1 -- 172.21.15.49:0/176697204 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb4bd6dd00 con 0x55cb4bdeb300 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.821+0000 7fc3f5600640 1 -- 172.21.15.49:0/176697204 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3f80035f0 con 0x55cb4bdeb300 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.821+0000 7fc411587740 10 client.27416 fetch_fsmap learned FSMap version 46 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.821+0000 7fc411587740 1 -- 172.21.15.49:0/176697204 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55cb4bd09290 con 0x55cb4bdeb300 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc3f5600640 1 -- 172.21.15.49:0/176697204 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3f8033080 con 0x55cb4bdeb300 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc3f5600640 1 -- 172.21.15.49:0/176697204 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 46) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc3f8012940 con 0x55cb4bdeb300 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc411587740 10 client.27416 fetch_fsmap finished waiting for FSMap version 46 2024-02-26T06:34:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc411587740 10 client.27416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:34:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc411587740 20 client.27416 populate_metadata read hostname 'smithi049' 2024-02-26T06:34:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc411587740 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cb4be52860 0x55cb4be54c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc411587740 4 client.27416 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:34:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc411587740 1 -- 172.21.15.49:0/176697204 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb4ba48600 con 0x55cb4be52860 2024-02-26T06:34:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.822+0000 7fc407400640 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cb4be52860 0x55cb4be54c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.823+0000 7fc407400640 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cb4be52860 0x55cb4be54c80 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3546fabeb8fbecc server_cookie=a8f4e74229245993 in_seq=0 out_seq=0 2024-02-26T06:34:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.823+0000 7fc3f5600640 10 client.27416 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.841+0000 7fc3f5600640 1 -- 172.21.15.49:0/176697204 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb4ba48600 con 0x55cb4be52860 2024-02-26T06:34:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.841+0000 7fc3f5600640 10 client.27416 handle_command_reply: tid=0 2024-02-26T06:34:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.841+0000 7fc411587740 1 client.27416 shutdown 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 -- 172.21.15.49:0/176697204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bdeb300 msgr2=0x55cb4bdf1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bdeb300 0x55cb4bdf1bb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8010c40 tx=0x7fc3f80050b0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 -- 172.21.15.49:0/176697204 shutdown_connections 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cb4be52860 0x55cb4be54c80 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb4bdeb300 0x55cb4bdf1bb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cb4bde2af0 0x55cb4bdf0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc3f5600640 0 client.27416 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 --2- 172.21.15.49:0/176697204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb4bde1d80 0x55cb4bdf07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 -- 172.21.15.49:0/176697204 >> 172.21.15.49:0/176697204 conn(0x55cb4bda9480 msgr2=0x55cb4bddc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 -- 172.21.15.49:0/176697204 shutdown_connections 2024-02-26T06:34:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 1 -- 172.21.15.49:0/176697204 wait complete. 2024-02-26T06:34:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.842+0000 7fc411587740 20 client.27416 trim_cache size 0 max 16384 2024-02-26T06:34:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:34:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:34:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:34:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 -- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc3ec0a2350 msgr2=0x7fc3ec0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc3ec0a2350 0x7fc3ec0a4810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55cb4bde3250 tx=0x7fc4000097c0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 -- 172.21.15.49:0/3013915697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc408108670 msgr2=0x7fc40819fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc408108670 0x7fc40819fb50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0036040 tx=0x7fc3f002c760 comp rx=0 tx=0).stop 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 -- 172.21.15.49:0/3013915697 shutdown_connections 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc3ec0a2350 0x7fc3ec0a4810 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc408108670 0x7fc40819fb50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4081997d0 0x7fc408199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 --2- 172.21.15.49:0/3013915697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4081035a0 0x7fc40819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 -- 172.21.15.49:0/3013915697 >> 172.21.15.49:0/3013915697 conn(0x7fc4080fee00 msgr2=0x7fc4081010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 -- 172.21.15.49:0/3013915697 shutdown_connections 2024-02-26T06:34:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:07.843+0000 7fc40e600640 1 -- 172.21.15.49:0/3013915697 wait complete. 2024-02-26T06:34:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:08 smithi049 ceph-mon[31696]: pgmap v334: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-02-26T06:34:09.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:08 smithi080 ceph-mon[36206]: pgmap v334: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-02-26T06:34:09.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:08 smithi115 ceph-mon[36426]: pgmap v334: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-02-26T06:34:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:10 smithi080 ceph-mon[36206]: pgmap v335: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.5 MiB/s wr, 399 op/s 2024-02-26T06:34:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:10 smithi115 ceph-mon[36426]: pgmap v335: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.5 MiB/s wr, 399 op/s 2024-02-26T06:34:11.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:10 smithi049 ceph-mon[31696]: pgmap v335: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.5 MiB/s wr, 399 op/s 2024-02-26T06:34:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 620.0 s, 136.8 tps, lat 695.152 ms stddev 412.963 2024-02-26T06:34:13.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:12 smithi080 ceph-mon[36206]: pgmap v336: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-02-26T06:34:13.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:12 smithi115 ceph-mon[36426]: pgmap v336: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-02-26T06:34:13.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:12 smithi049 ceph-mon[31696]: pgmap v336: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-02-26T06:34:15.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:14 smithi080 ceph-mon[36206]: pgmap v337: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-02-26T06:34:15.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:14 smithi115 ceph-mon[36426]: pgmap v337: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-02-26T06:34:15.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:14 smithi049 ceph-mon[31696]: pgmap v337: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-02-26T06:34:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 625.0 s, 448.6 tps, lat 250.749 ms stddev 426.581 2024-02-26T06:34:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:16 smithi080 ceph-mon[36206]: osdmap e99: 12 total, 12 up, 12 in 2024-02-26T06:34:16.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:16 smithi115 ceph-mon[36426]: osdmap e99: 12 total, 12 up, 12 in 2024-02-26T06:34:16.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:16 smithi049 ceph-mon[31696]: osdmap e99: 12 total, 12 up, 12 in 2024-02-26T06:34:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:17 smithi080 ceph-mon[36206]: pgmap v339: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-02-26T06:34:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:17 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:34:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:17 smithi115 ceph-mon[36426]: pgmap v339: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-02-26T06:34:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:17 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:34:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:17 smithi049 ceph-mon[31696]: pgmap v339: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-02-26T06:34:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:17 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:34:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:19 smithi080 ceph-mon[36206]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-02-26T06:34:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:19 smithi115 ceph-mon[36426]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-02-26T06:34:19.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:19 smithi049 ceph-mon[31696]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-02-26T06:34:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 -- 172.21.15.49:0/2019254737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa428068d40 msgr2=0x7fa428069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 --2- 172.21.15.49:0/2019254737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa428068d40 0x7fa428069180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa414009930 tx=0x7fa41402bcd0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 -- 172.21.15.49:0/2019254737 shutdown_connections 2024-02-26T06:34:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 --2- 172.21.15.49:0/2019254737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa428103b90 0x7fa428103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 --2- 172.21.15.49:0/2019254737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa428068d40 0x7fa428069180 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 --2- 172.21.15.49:0/2019254737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa428106140 0x7fa428068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 -- 172.21.15.49:0/2019254737 >> 172.21.15.49:0/2019254737 conn(0x7fa428076ed0 msgr2=0x7fa4280772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 -- 172.21.15.49:0/2019254737 shutdown_connections 2024-02-26T06:34:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.042+0000 7fa42d400640 1 -- 172.21.15.49:0/2019254737 wait complete. 2024-02-26T06:34:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.043+0000 7fa42d400640 1 Processor -- start 2024-02-26T06:34:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.043+0000 7fa42d400640 1 -- start start 2024-02-26T06:34:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.043+0000 7fa42d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa428068d40 0x7fa4281a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.043+0000 7fa42d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa428103b90 0x7fa4281a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.043+0000 7fa42d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa428106140 0x7fa42819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.043+0000 7fa42d400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42819e300 con 0x7fa428106140 2024-02-26T06:34:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.043+0000 7fa42d400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42819e470 con 0x7fa428068d40 2024-02-26T06:34:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.043+0000 7fa42d400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42819e630 con 0x7fa428103b90 2024-02-26T06:34:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.044+0000 7fa427200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa428106140 0x7fa42819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.044+0000 7fa426800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa428068d40 0x7fa4281a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.044+0000 7fa425e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa428103b90 0x7fa4281a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.044+0000 7fa427200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa428106140 0x7fa42819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46342/0 (socket says 172.21.15.49:46342) 2024-02-26T06:34:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.044+0000 7fa427200640 1 -- 172.21.15.49:0/3459811231 learned_addr learned my addr 172.21.15.49:0/3459811231 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.044+0000 7fa426800640 1 -- 172.21.15.49:0/3459811231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa428103b90 msgr2=0x7fa4281a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.045+0000 7fa426800640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa428103b90 0x7fa4281a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.045+0000 7fa426800640 1 -- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa428106140 msgr2=0x7fa42819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.045+0000 7fa426800640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa428106140 0x7fa42819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.045+0000 7fa426800640 1 -- 172.21.15.49:0/3459811231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa414009560 con 0x7fa428068d40 2024-02-26T06:34:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.045+0000 7fa426800640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa428068d40 0x7fa4281a3a60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa41800e9f0 tx=0x7fa41800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.046+0000 7fa41f400640 1 -- 172.21.15.49:0/3459811231 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41800ccc0 con 0x7fa428068d40 2024-02-26T06:34:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.046+0000 7fa41f400640 1 -- 172.21.15.49:0/3459811231 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa41800ce20 con 0x7fa428068d40 2024-02-26T06:34:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.046+0000 7fa41f400640 1 -- 172.21.15.49:0/3459811231 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418018920 con 0x7fa428068d40 2024-02-26T06:34:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.046+0000 7fa42d400640 1 -- 172.21.15.49:0/3459811231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42819e910 con 0x7fa428068d40 2024-02-26T06:34:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.047+0000 7fa42d400640 1 -- 172.21.15.49:0/3459811231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4281aa9e0 con 0x7fa428068d40 2024-02-26T06:34:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.051+0000 7fa41f400640 1 -- 172.21.15.49:0/3459811231 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa418018a80 con 0x7fa428068d40 2024-02-26T06:34:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.052+0000 7fa430436740 1 Processor -- start 2024-02-26T06:34:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.052+0000 7fa430436740 1 -- start start 2024-02-26T06:34:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.052+0000 7fa430436740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.052+0000 7fa430436740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464983c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.052+0000 7fa430436740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x5646498476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.052+0000 7fa430436740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646497f75f0 con 0x56464983caf0 2024-02-26T06:34:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.052+0000 7fa430436740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646497a9890 con 0x564649845300 2024-02-26T06:34:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.052+0000 7fa430436740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646497a6dc0 con 0x56464983bd80 2024-02-26T06:34:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.053+0000 7fa426800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464983c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.053+0000 7fa427200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.053+0000 7fa426800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464983c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46662/0 (socket says 172.21.15.49:46662) 2024-02-26T06:34:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.053+0000 7fa426800640 1 -- 172.21.15.49:0/2417782294 learned_addr learned my addr 172.21.15.49:0/2417782294 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.053+0000 7fa425e00640 1 --2- 172.21.15.49:0/2417782294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x5646498476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa427200640 1 -- 172.21.15.49:0/2417782294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 msgr2=0x56464983c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa427200640 1 --2- 172.21.15.49:0/2417782294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464983c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa427200640 1 -- 172.21.15.49:0/2417782294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 msgr2=0x5646498476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa427200640 1 --2- 172.21.15.49:0/2417782294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x5646498476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa427200640 1 -- 172.21.15.49:0/2417782294 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564649723ad0 con 0x56464983caf0 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa41f400640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa40c0a24b0 0x7fa40c0a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa41f400640 1 -- 172.21.15.49:0/3459811231 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa418014070 con 0x7fa428068d40 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa427200640 1 --2- 172.21.15.49:0/2417782294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983b840 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa420007e50 tx=0x7fa42000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b5563d6f8fd50e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.054+0000 7fa426800640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa40c0a24b0 0x7fa40c0a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.055+0000 7fa413400640 1 -- 172.21.15.49:0/2417782294 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420007920 con 0x56464983caf0 2024-02-26T06:34:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.055+0000 7fa413400640 1 -- 172.21.15.49:0/2417782294 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa420010d20 con 0x56464983caf0 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.055+0000 7fa413400640 1 -- 172.21.15.49:0/2417782294 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420018e50 con 0x56464983caf0 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 -- 172.21.15.49:0/2417782294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 msgr2=0x56464983b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 --2- 172.21.15.49:0/2417782294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983b840 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa420007e50 tx=0x7fa42000f790 comp rx=0 tx=0).stop 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa426800640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa40c0a24b0 0x7fa40c0a4970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa41800e9c0 tx=0x7fa4180d1b00 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 -- 172.21.15.49:0/2417782294 shutdown_connections 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 --2- 172.21.15.49:0/2417782294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x5646498476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 --2- 172.21.15.49:0/2417782294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464983c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 --2- 172.21.15.49:0/2417782294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983b840 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 -- 172.21.15.49:0/2417782294 >> 172.21.15.49:0/2417782294 conn(0x564649803480 msgr2=0x564649836a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 -- 172.21.15.49:0/2417782294 shutdown_connections 2024-02-26T06:34:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.056+0000 7fa430436740 1 -- 172.21.15.49:0/2417782294 wait complete. 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 1 -- start start 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 1 Processor -- start 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649842370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x5646498428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x564649842df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646497a25a0 con 0x56464983bd80 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564649747b20 con 0x564649845300 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646497477c0 con 0x56464983caf0 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.057+0000 7fa430436740 10 client.0 _ll_register_callbacks cb 0x56464975b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa427200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649842370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa427200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649842370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46362/0 (socket says 172.21.15.49:46362) 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa427200640 1 -- 172.21.15.49:0/2026490154 learned_addr learned my addr 172.21.15.49:0/2026490154 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa425e00640 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x5646498428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa427200640 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x564649842df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa425e00640 1 -- 172.21.15.49:0/2026490154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 msgr2=0x564649842df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa425e00640 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x564649842df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa425e00640 1 -- 172.21.15.49:0/2026490154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 msgr2=0x564649842370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa425e00640 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649842370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa425e00640 1 -- 172.21.15.49:0/2026490154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa420007a80 con 0x56464983caf0 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa427200640 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649842370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.058+0000 7fa425e00640 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x5646498428b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa414009230 tx=0x7fa414004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b930f97597d2079d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.059+0000 7fa407e00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:34:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.059+0000 7fa407e00640 1 -- 172.21.15.49:0/2026490154 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414004520 con 0x56464983caf0 2024-02-26T06:34:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.059+0000 7fa430436740 1 -- 172.21.15.49:0/2026490154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564649774ca0 con 0x56464983caf0 2024-02-26T06:34:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.059+0000 7fa407e00640 1 -- 172.21.15.49:0/2026490154 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa41402ca90 con 0x56464983caf0 2024-02-26T06:34:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.060+0000 7fa407e00640 1 -- 172.21.15.49:0/2026490154 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa41402cc50 con 0x56464983caf0 2024-02-26T06:34:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.060+0000 7fa430436740 10 client.27382 fetch_fsmap learned FSMap version 47 2024-02-26T06:34:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.060+0000 7fa430436740 1 -- 172.21.15.49:0/2026490154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5646497a0c20 con 0x56464983caf0 2024-02-26T06:34:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.061+0000 7fa407e00640 1 -- 172.21.15.49:0/2026490154 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564649774ca0 con 0x56464983caf0 2024-02-26T06:34:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.061+0000 7fa407e00640 1 -- 172.21.15.49:0/2026490154 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa414046020 con 0x56464983caf0 2024-02-26T06:34:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.062+0000 7fa430436740 10 client.27382 fetch_fsmap finished waiting for FSMap version 47 2024-02-26T06:34:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.062+0000 7fa430436740 10 client.27382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:34:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.062+0000 7fa430436740 20 client.27382 populate_metadata read hostname 'smithi049' 2024-02-26T06:34:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.062+0000 7fa430436740 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5646498552c0 0x5646498576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.062+0000 7fa430436740 4 client.27382 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:34:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.062+0000 7fa430436740 1 -- 172.21.15.49:0/2026490154 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564649798cb0 con 0x5646498552c0 2024-02-26T06:34:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.063+0000 7fa427200640 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5646498552c0 0x5646498576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.065+0000 7fa427200640 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5646498552c0 0x5646498576e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da12b00e1981bfdf server_cookie=55af3212eb4590fb in_seq=0 out_seq=0 2024-02-26T06:34:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.065+0000 7fa407e00640 10 client.27382 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.066+0000 7fa407e00640 1 -- 172.21.15.49:0/2026490154 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564649798cb0 con 0x5646498552c0 2024-02-26T06:34:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.066+0000 7fa407e00640 10 client.27382 handle_command_reply: tid=0 2024-02-26T06:34:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.066+0000 7fa430436740 1 client.27382 shutdown 2024-02-26T06:34:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 -- 172.21.15.49:0/2026490154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 msgr2=0x5646498428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x5646498428b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa414009230 tx=0x7fa414004270 comp rx=0 tx=0).stop 2024-02-26T06:34:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 -- 172.21.15.49:0/2026490154 shutdown_connections 2024-02-26T06:34:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5646498552c0 0x5646498576e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa407e00640 0 client.27382 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x564649842df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x5646498428b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 --2- 172.21.15.49:0/2026490154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649842370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 -- 172.21.15.49:0/2026490154 >> 172.21.15.49:0/2026490154 conn(0x564649803480 msgr2=0x5646498387c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.067+0000 7fa430436740 1 -- 172.21.15.49:0/2026490154 shutdown_connections 2024-02-26T06:34:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.068+0000 7fa430436740 1 -- 172.21.15.49:0/2026490154 wait complete. 2024-02-26T06:34:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.068+0000 7fa430436740 20 client.27382 trim_cache size 0 max 16384 2024-02-26T06:34:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.098+0000 7fa430436740 1 Processor -- start 2024-02-26T06:34:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.098+0000 7fa430436740 1 -- start start 2024-02-26T06:34:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa430436740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649844aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa430436740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x56464984c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa430436740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464984c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa430436740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646494eae60 con 0x56464983bd80 2024-02-26T06:34:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa430436740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564649837870 con 0x564649845300 2024-02-26T06:34:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa430436740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564649798cb0 con 0x56464983caf0 2024-02-26T06:34:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa425e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x56464984c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa425e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x56464984c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46684/0 (socket says 172.21.15.49:46684) 2024-02-26T06:34:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa425e00640 1 -- 172.21.15.49:0/3015394912 learned_addr learned my addr 172.21.15.49:0/3015394912 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.099+0000 7fa427200640 1 --2- 172.21.15.49:0/3015394912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464984c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.101+0000 7fa425e00640 1 -- 172.21.15.49:0/3015394912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 msgr2=0x56464984c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.101+0000 7fa425e00640 1 --2- 172.21.15.49:0/3015394912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464984c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.101+0000 7fa425e00640 1 -- 172.21.15.49:0/3015394912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 msgr2=0x564649844aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.101+0000 7fa425e00640 1 --2- 172.21.15.49:0/3015394912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649844aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.101+0000 7fa425e00640 1 -- 172.21.15.49:0/3015394912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464973d500 con 0x56464983caf0 2024-02-26T06:34:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.101+0000 7fa425e00640 1 --2- 172.21.15.49:0/3015394912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x56464984c120 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa414009850 tx=0x7fa41402bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efc493b7666d619f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.102+0000 7fa412a00640 1 -- 172.21.15.49:0/3015394912 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41402cc50 con 0x56464983caf0 2024-02-26T06:34:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.102+0000 7fa412a00640 1 -- 172.21.15.49:0/3015394912 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa414005d50 con 0x56464983caf0 2024-02-26T06:34:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.102+0000 7fa412a00640 1 -- 172.21.15.49:0/3015394912 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140346a0 con 0x56464983caf0 2024-02-26T06:34:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.103+0000 7fa430436740 1 -- 172.21.15.49:0/3015394912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 msgr2=0x56464984c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.103+0000 7fa430436740 1 --2- 172.21.15.49:0/3015394912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x56464984c120 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa414009850 tx=0x7fa41402bcd0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.106+0000 7fa430436740 1 -- 172.21.15.49:0/3015394912 shutdown_connections 2024-02-26T06:34:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.106+0000 7fa430436740 1 --2- 172.21.15.49:0/3015394912 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464984c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.106+0000 7fa430436740 1 --2- 172.21.15.49:0/3015394912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983caf0 0x56464984c120 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.106+0000 7fa430436740 1 --2- 172.21.15.49:0/3015394912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983bd80 0x564649844aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.106+0000 7fa430436740 1 -- 172.21.15.49:0/3015394912 >> 172.21.15.49:0/3015394912 conn(0x564649803480 msgr2=0x5646498365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.108+0000 7fa430436740 1 -- 172.21.15.49:0/3015394912 shutdown_connections 2024-02-26T06:34:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.108+0000 7fa430436740 1 -- 172.21.15.49:0/3015394912 wait complete. 2024-02-26T06:34:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.108+0000 7fa430436740 1 -- start start 2024-02-26T06:34:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.109+0000 7fa430436740 1 Processor -- start 2024-02-26T06:34:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.110+0000 7fa430436740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464984c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.110+0000 7fa430436740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.110+0000 7fa430436740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464983ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.110+0000 7fa430436740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464984c800 con 0x56464983caf0 2024-02-26T06:34:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.110+0000 7fa430436740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464984c970 con 0x564649845300 2024-02-26T06:34:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.110+0000 7fa430436740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464983f260 con 0x56464983bd80 2024-02-26T06:34:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.110+0000 7fa430436740 10 client.0 _ll_register_callbacks cb 0x7fa41402c750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:34:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.111+0000 7fa427200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464983ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.111+0000 7fa427200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464983ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:55520/0 (socket says 172.21.15.49:55520) 2024-02-26T06:34:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.111+0000 7fa427200640 1 -- 172.21.15.49:0/963522905 learned_addr learned my addr 172.21.15.49:0/963522905 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.111+0000 7fa427200640 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464984c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.111+0000 7fa425e00640 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.111+0000 7fa427200640 1 -- 172.21.15.49:0/963522905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 msgr2=0x56464984c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.111+0000 7fa427200640 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464984c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.111+0000 7fa427200640 1 -- 172.21.15.49:0/963522905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 msgr2=0x56464983e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.112+0000 7fa427200640 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.112+0000 7fa427200640 1 -- 172.21.15.49:0/963522905 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646497a0c20 con 0x564649845300 2024-02-26T06:34:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.112+0000 7fa425e00640 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:34:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.112+0000 7fa427200640 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464983ed20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa4200074f0 tx=0x7fa420005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4dd07e182f0061b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.112+0000 7fa407e00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:34:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.116+0000 7fa407e00640 1 -- 172.21.15.49:0/963522905 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42002bea0 con 0x564649845300 2024-02-26T06:34:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.116+0000 7fa407e00640 1 -- 172.21.15.49:0/963522905 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fa420004c10 con 0x564649845300 2024-02-26T06:34:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.117+0000 7fa430436740 1 -- 172.21.15.49:0/963522905 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646497c7d00 con 0x564649845300 2024-02-26T06:34:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.117+0000 7fa407e00640 1 -- 172.21.15.49:0/963522905 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa420004dd0 con 0x564649845300 2024-02-26T06:34:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.117+0000 7fa430436740 10 client.27434 fetch_fsmap learned FSMap version 47 2024-02-26T06:34:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.117+0000 7fa430436740 1 -- 172.21.15.49:0/963522905 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x564649763290 con 0x564649845300 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa407e00640 1 -- 172.21.15.49:0/963522905 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5646497c7d00 con 0x564649845300 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa407e00640 1 -- 172.21.15.49:0/963522905 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 47) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa420010970 con 0x564649845300 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa430436740 10 client.27434 fetch_fsmap finished waiting for FSMap version 47 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa430436740 10 client.27434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa430436740 20 client.27434 populate_metadata read hostname 'smithi049' 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa430436740 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5646498552c0 0x5646498ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa430436740 4 client.27434 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa430436740 1 -- 172.21.15.49:0/963522905 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5646494a2600 con 0x5646498552c0 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.118+0000 7fa425e00640 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5646498552c0 0x5646498ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.119+0000 7fa407e00640 10 client.27434 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.120+0000 7fa425e00640 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5646498552c0 0x5646498ae950 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22ab4cf7ae1ca23 server_cookie=44e8e9891ad5e9e2 in_seq=0 out_seq=0 2024-02-26T06:34:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.146+0000 7fa407e00640 1 -- 172.21.15.49:0/963522905 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5646494a2600 con 0x5646498552c0 2024-02-26T06:34:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa407e00640 10 client.27434 handle_command_reply: tid=0 2024-02-26T06:34:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 client.27434 shutdown 2024-02-26T06:34:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 -- 172.21.15.49:0/963522905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 msgr2=0x56464983ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464983ed20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa4200074f0 tx=0x7fa420005db0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 -- 172.21.15.49:0/963522905 shutdown_connections 2024-02-26T06:34:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5646498552c0 0x5646498ae950 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564649845300 0x56464983ed20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464983caf0 0x56464983e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 --2- 172.21.15.49:0/963522905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56464983bd80 0x56464984c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 -- 172.21.15.49:0/963522905 >> 172.21.15.49:0/963522905 conn(0x564649803480 msgr2=0x56464983b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa407e00640 0 client.27434 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.147+0000 7fa430436740 1 -- 172.21.15.49:0/963522905 shutdown_connections 2024-02-26T06:34:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.148+0000 7fa430436740 1 -- 172.21.15.49:0/963522905 wait complete. 2024-02-26T06:34:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.148+0000 7fa430436740 20 client.27434 trim_cache size 0 max 16384 2024-02-26T06:34:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:34:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:34:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:34:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:34:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 -- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa40c0a24b0 msgr2=0x7fa40c0a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa40c0a24b0 0x7fa40c0a4970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa41800e9c0 tx=0x7fa4180d1b00 comp rx=0 tx=0).stop 2024-02-26T06:34:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 -- 172.21.15.49:0/3459811231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa428068d40 msgr2=0x7fa4281a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa428068d40 0x7fa4281a3a60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa41800e9f0 tx=0x7fa41800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 -- 172.21.15.49:0/3459811231 shutdown_connections 2024-02-26T06:34:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa40c0a24b0 0x7fa40c0a4970 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa428106140 0x7fa42819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa428103b90 0x7fa4281a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 --2- 172.21.15.49:0/3459811231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa428068d40 0x7fa4281a3a60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 -- 172.21.15.49:0/3459811231 >> 172.21.15.49:0/3459811231 conn(0x7fa428076ed0 msgr2=0x7fa4280ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 -- 172.21.15.49:0/3459811231 shutdown_connections 2024-02-26T06:34:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:21.149+0000 7fa42d400640 1 -- 172.21.15.49:0/3459811231 wait complete. 2024-02-26T06:34:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 630.0 s, 1314.4 tps, lat 76.816 ms stddev 87.110 2024-02-26T06:34:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:21 smithi080 ceph-mon[36206]: pgmap v341: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 MiB/s wr, 1.61k op/s 2024-02-26T06:34:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:21 smithi115 ceph-mon[36426]: pgmap v341: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 MiB/s wr, 1.61k op/s 2024-02-26T06:34:21.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:21 smithi049 ceph-mon[31696]: pgmap v341: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 MiB/s wr, 1.61k op/s 2024-02-26T06:34:22.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:22 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:34:22.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:22 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:34:22.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:22 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:34:23.490 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:23 smithi115 ceph-mon[36426]: pgmap v342: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-02-26T06:34:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:23 smithi080 ceph-mon[36206]: pgmap v342: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-02-26T06:34:23.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:23 smithi049 ceph-mon[31696]: pgmap v342: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-02-26T06:34:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:25 smithi080 ceph-mon[36206]: pgmap v343: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-02-26T06:34:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:25 smithi115 ceph-mon[36426]: pgmap v343: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-02-26T06:34:25.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:25 smithi049 ceph-mon[31696]: pgmap v343: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-02-26T06:34:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 635.0 s, 1842.4 tps, lat 53.984 ms stddev 62.113 2024-02-26T06:34:27.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:27 smithi080 ceph-mon[36206]: pgmap v344: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 974 op/s 2024-02-26T06:34:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:27 smithi115 ceph-mon[36426]: pgmap v344: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 974 op/s 2024-02-26T06:34:27.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:27 smithi049 ceph-mon[31696]: pgmap v344: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 974 op/s 2024-02-26T06:34:29.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:29 smithi080 ceph-mon[36206]: pgmap v345: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 954 op/s 2024-02-26T06:34:29.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:29 smithi115 ceph-mon[36426]: pgmap v345: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 954 op/s 2024-02-26T06:34:29.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:29 smithi049 ceph-mon[31696]: pgmap v345: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 954 op/s 2024-02-26T06:34:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 640.0 s, 1973.4 tps, lat 50.694 ms stddev 69.369 2024-02-26T06:34:31.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:31 smithi080 ceph-mon[36206]: pgmap v346: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 643 op/s 2024-02-26T06:34:31.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:31 smithi115 ceph-mon[36426]: pgmap v346: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 643 op/s 2024-02-26T06:34:31.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:31 smithi049 ceph-mon[31696]: pgmap v346: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 643 op/s 2024-02-26T06:34:33.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:33 smithi080 ceph-mon[36206]: pgmap v347: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.3 MiB/s wr, 767 op/s 2024-02-26T06:34:33.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:33 smithi115 ceph-mon[36426]: pgmap v347: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.3 MiB/s wr, 767 op/s 2024-02-26T06:34:33.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:33 smithi049 ceph-mon[31696]: pgmap v347: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.3 MiB/s wr, 767 op/s 2024-02-26T06:34:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.350+0000 7f9a64600640 1 -- 172.21.15.49:0/3719286060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c106460 msgr2=0x7f9a5c10e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.350+0000 7f9a64600640 1 --2- 172.21.15.49:0/3719286060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c106460 0x7f9a5c10e030 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009cd0 tx=0x7f9a5402c0e0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.350+0000 7f9a64600640 1 -- 172.21.15.49:0/3719286060 shutdown_connections 2024-02-26T06:34:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.350+0000 7f9a64600640 1 --2- 172.21.15.49:0/3719286060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c106460 0x7f9a5c10e030 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.350+0000 7f9a64600640 1 --2- 172.21.15.49:0/3719286060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a5c101d40 0x7f9a5c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.350+0000 7f9a64600640 1 --2- 172.21.15.49:0/3719286060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5c101390 0x7f9a5c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.350+0000 7f9a64600640 1 -- 172.21.15.49:0/3719286060 >> 172.21.15.49:0/3719286060 conn(0x7f9a5c0fcc10 msgr2=0x7f9a5c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.350+0000 7f9a64600640 1 -- 172.21.15.49:0/3719286060 shutdown_connections 2024-02-26T06:34:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.351+0000 7f9a64600640 1 -- 172.21.15.49:0/3719286060 wait complete. 2024-02-26T06:34:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.352+0000 7f9a64600640 1 Processor -- start 2024-02-26T06:34:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.352+0000 7f9a64600640 1 -- start start 2024-02-26T06:34:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.353+0000 7f9a64600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5c101390 0x7f9a5c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.353+0000 7f9a64600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a5c106460 0x7f9a5c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.353+0000 7f9a64600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c1996b0 0x7f9a5c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.353+0000 7f9a64600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c19a180 con 0x7f9a5c101390 2024-02-26T06:34:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.353+0000 7f9a64600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c19a2f0 con 0x7f9a5c1996b0 2024-02-26T06:34:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.353+0000 7f9a64600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c19a4b0 con 0x7f9a5c106460 2024-02-26T06:34:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.354+0000 7f9a61c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5c101390 0x7f9a5c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.354+0000 7f9a61200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a5c106460 0x7f9a5c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.354+0000 7f9a61200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a5c106460 0x7f9a5c19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33662/0 (socket says 172.21.15.49:33662) 2024-02-26T06:34:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.354+0000 7f9a61200640 1 -- 172.21.15.49:0/1575157232 learned_addr learned my addr 172.21.15.49:0/1575157232 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.354+0000 7f9a62600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c1996b0 0x7f9a5c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a61200640 1 -- 172.21.15.49:0/1575157232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c1996b0 msgr2=0x7f9a5c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a61200640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c1996b0 0x7f9a5c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a61200640 1 -- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5c101390 msgr2=0x7f9a5c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a61200640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5c101390 0x7f9a5c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a61200640 1 -- 172.21.15.49:0/1575157232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54009900 con 0x7f9a5c106460 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a61c00640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5c101390 0x7f9a5c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a62600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c1996b0 0x7f9a5c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a61200640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a5c106460 0x7f9a5c19fb30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000d770 tx=0x7f9a5000db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a5a000640 1 -- 172.21.15.49:0/1575157232 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50009a20 con 0x7f9a5c106460 2024-02-26T06:34:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.355+0000 7f9a5a000640 1 -- 172.21.15.49:0/1575157232 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a50009b80 con 0x7f9a5c106460 2024-02-26T06:34:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.356+0000 7f9a5a000640 1 -- 172.21.15.49:0/1575157232 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000f6b0 con 0x7f9a5c106460 2024-02-26T06:34:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.356+0000 7f9a64600640 1 -- 172.21.15.49:0/1575157232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c1a6450 con 0x7f9a5c106460 2024-02-26T06:34:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.356+0000 7f9a64600640 1 -- 172.21.15.49:0/1575157232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c1a68e0 con 0x7f9a5c106460 2024-02-26T06:34:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.364+0000 7f9a675ae740 1 Processor -- start 2024-02-26T06:34:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.365+0000 7f9a675ae740 1 -- start start 2024-02-26T06:34:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.365+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c64050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.365+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 0x5626f5c64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.365+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c6ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.365+0000 7f9a675ae740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5c175f0 con 0x5626f5c6db10 2024-02-26T06:34:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.365+0000 7f9a675ae740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5bc9890 con 0x5626f5c64590 2024-02-26T06:34:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.365+0000 7f9a675ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5bc6dc0 con 0x5626f5c65300 2024-02-26T06:34:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.366+0000 7f9a5a000640 1 -- 172.21.15.49:0/1575157232 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9a50010460 con 0x7f9a5c106460 2024-02-26T06:34:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.368+0000 7f9a5a000640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a440a23d0 0x7f9a440a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.368+0000 7f9a5a000640 1 -- 172.21.15.49:0/1575157232 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f9a500cdcd0 con 0x7f9a5c106460 2024-02-26T06:34:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.370+0000 7f9a61200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c6ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.370+0000 7f9a61200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c6ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36798/0 (socket says 172.21.15.49:36798) 2024-02-26T06:34:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.370+0000 7f9a61200640 1 -- 172.21.15.49:0/3021011925 learned_addr learned my addr 172.21.15.49:0/3021011925 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.370+0000 7f9a62600640 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c64050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.371+0000 7f9a62600640 1 -- 172.21.15.49:0/3021011925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 msgr2=0x5626f5c64a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.371+0000 7f9a62600640 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 0x5626f5c64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.372+0000 7f9a62600640 1 -- 172.21.15.49:0/3021011925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 msgr2=0x5626f5c6ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.372+0000 7f9a62600640 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c6ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.372+0000 7f9a62600640 1 -- 172.21.15.49:0/3021011925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f5b43ad0 con 0x5626f5c65300 2024-02-26T06:34:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.372+0000 7f9a62600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a440a23d0 0x7f9a440a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.372+0000 7f9a61200640 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c6ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.372+0000 7f9a62600640 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c64050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9a54037040 tx=0x7f9a54032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca17c32093bb68ae server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.374+0000 7f9a4ea00640 1 -- 172.21.15.49:0/3021011925 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54034d30 con 0x5626f5c65300 2024-02-26T06:34:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.374+0000 7f9a4ea00640 1 -- 172.21.15.49:0/3021011925 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a54003970 con 0x5626f5c65300 2024-02-26T06:34:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.374+0000 7f9a4ea00640 1 -- 172.21.15.49:0/3021011925 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540045d0 con 0x5626f5c65300 2024-02-26T06:34:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.375+0000 7f9a675ae740 1 -- 172.21.15.49:0/3021011925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 msgr2=0x5626f5c64050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.375+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c64050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9a54037040 tx=0x7f9a54032e90 comp rx=0 tx=0).stop 2024-02-26T06:34:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.376+0000 7f9a675ae740 1 -- 172.21.15.49:0/3021011925 shutdown_connections 2024-02-26T06:34:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.376+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 0x5626f5c64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.376+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c6ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.376+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3021011925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c64050 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.376+0000 7f9a675ae740 1 -- 172.21.15.49:0/3021011925 >> 172.21.15.49:0/3021011925 conn(0x5626f5c23480 msgr2=0x5626f5c5f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.378+0000 7f9a675ae740 1 -- 172.21.15.49:0/3021011925 shutdown_connections 2024-02-26T06:34:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.381+0000 7f9a62600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a440a23d0 0x7f9a440a4890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9a54037040 tx=0x7f9a54032e90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.381+0000 7f9a675ae740 1 -- 172.21.15.49:0/3021011925 wait complete. 2024-02-26T06:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.382+0000 7f9a675ae740 1 -- start start 2024-02-26T06:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.382+0000 7f9a675ae740 1 Processor -- start 2024-02-26T06:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.383+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c6aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.383+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c65300 0x5626f5c6b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.384+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c6db10 0x5626f5c6b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.385+0000 7f9a61200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c65300 0x5626f5c6b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.385+0000 7f9a61200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c65300 0x5626f5c6b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:46718/0 (socket says 172.21.15.49:46718) 2024-02-26T06:34:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.386+0000 7f9a61200640 1 -- 172.21.15.49:0/1836049988 learned_addr learned my addr 172.21.15.49:0/1836049988 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.386+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5bc25a0 con 0x5626f5c64590 2024-02-26T06:34:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.386+0000 7f9a61c00640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c6aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.386+0000 7f9a62600640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c6db10 0x5626f5c6b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.386+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5b67b20 con 0x5626f5c65300 2024-02-26T06:34:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.386+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5b677c0 con 0x5626f5c6db10 2024-02-26T06:34:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.386+0000 7f9a675ae740 10 client.0 _ll_register_callbacks cb 0x5626f5b7b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:34:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.388+0000 7f9a61200640 1 -- 172.21.15.49:0/1836049988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c6db10 msgr2=0x5626f5c6b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.388+0000 7f9a61200640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c6db10 0x5626f5c6b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.388+0000 7f9a61200640 1 -- 172.21.15.49:0/1836049988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 msgr2=0x5626f5c6aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.389+0000 7f9a61200640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c6aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.389+0000 7f9a61200640 1 -- 172.21.15.49:0/1836049988 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c10e840 con 0x5626f5c65300 2024-02-26T06:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.389+0000 7f9a62600640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c6db10 0x5626f5c6b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.389+0000 7f9a61c00640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c6aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.389+0000 7f9a61200640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c65300 0x5626f5c6b0e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009610 tx=0x7f9a5009a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62eb2f9e715b4b61 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.389+0000 7f9a43400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.393+0000 7f9a43400640 1 -- 172.21.15.49:0/1836049988 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50009cf0 con 0x5626f5c65300 2024-02-26T06:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.393+0000 7f9a43400640 1 -- 172.21.15.49:0/1836049988 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f9a5009a640 con 0x5626f5c65300 2024-02-26T06:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.394+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626f5b94ca0 con 0x5626f5c65300 2024-02-26T06:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.394+0000 7f9a43400640 1 -- 172.21.15.49:0/1836049988 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a500cb830 con 0x5626f5c65300 2024-02-26T06:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.394+0000 7f9a675ae740 10 client.27443 fetch_fsmap learned FSMap version 48 2024-02-26T06:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.394+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5626f5bc0c20 con 0x5626f5c65300 2024-02-26T06:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.395+0000 7f9a43400640 1 -- 172.21.15.49:0/1836049988 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5626f5b94ca0 con 0x5626f5c65300 2024-02-26T06:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.395+0000 7f9a43400640 1 -- 172.21.15.49:0/1836049988 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9a50083b80 con 0x5626f5c65300 2024-02-26T06:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.395+0000 7f9a675ae740 10 client.27443 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.395+0000 7f9a675ae740 10 client.27443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.395+0000 7f9a675ae740 20 client.27443 populate_metadata read hostname 'smithi049' 2024-02-26T06:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.396+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5626f5c8eb90 0x5626f5c90fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.396+0000 7f9a675ae740 4 client.27443 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.396+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626f5bb8cb0 con 0x5626f5c8eb90 2024-02-26T06:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.396+0000 7f9a61c00640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5626f5c8eb90 0x5626f5c90fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.398+0000 7f9a61c00640 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5626f5c8eb90 0x5626f5c90fb0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e72c10685f386db server_cookie=6cc5ec0438fdfcf9 in_seq=0 out_seq=0 2024-02-26T06:34:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.398+0000 7f9a43400640 10 client.27443 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.400+0000 7f9a43400640 1 -- 172.21.15.49:0/1836049988 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626f5bb8cb0 con 0x5626f5c8eb90 2024-02-26T06:34:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.400+0000 7f9a43400640 10 client.27443 handle_command_reply: tid=0 2024-02-26T06:34:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.400+0000 7f9a675ae740 1 client.27443 shutdown 2024-02-26T06:34:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c65300 msgr2=0x5626f5c6b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c65300 0x5626f5c6b0e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009610 tx=0x7f9a5009a060 comp rx=0 tx=0).stop 2024-02-26T06:34:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 shutdown_connections 2024-02-26T06:34:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5626f5c8eb90 0x5626f5c90fb0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c65300 0x5626f5c6b0e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c6db10 0x5626f5c6b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1836049988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c6aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 >> 172.21.15.49:0/1836049988 conn(0x5626f5c23480 msgr2=0x5626f5c60740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.401+0000 7f9a43400640 0 client.27443 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.402+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 shutdown_connections 2024-02-26T06:34:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.402+0000 7f9a675ae740 1 -- 172.21.15.49:0/1836049988 wait complete. 2024-02-26T06:34:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.402+0000 7f9a675ae740 20 client.27443 trim_cache size 0 max 16384 2024-02-26T06:34:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.440+0000 7f9a675ae740 1 Processor -- start 2024-02-26T06:34:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.440+0000 7f9a675ae740 1 -- start start 2024-02-26T06:34:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.440+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 0x5626f5c6d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.440+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c85950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.440+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c85e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.440+0000 7f9a675ae740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f590ae60 con 0x5626f5c6db10 2024-02-26T06:34:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.440+0000 7f9a675ae740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5bb8cb0 con 0x5626f5c64590 2024-02-26T06:34:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.440+0000 7f9a675ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5b94ca0 con 0x5626f5c65300 2024-02-26T06:34:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c85950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c85950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33688/0 (socket says 172.21.15.49:33688) 2024-02-26T06:34:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61200640 1 -- 172.21.15.49:0/1035325339 learned_addr learned my addr 172.21.15.49:0/1035325339 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61c00640 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 0x5626f5c6d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a62600640 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c85e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61200640 1 -- 172.21.15.49:0/1035325339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 msgr2=0x5626f5c6d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61200640 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 0x5626f5c6d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61200640 1 -- 172.21.15.49:0/1035325339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 msgr2=0x5626f5c85e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61200640 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c85e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.441+0000 7f9a61200640 1 -- 172.21.15.49:0/1035325339 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f5b5d500 con 0x5626f5c65300 2024-02-26T06:34:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.442+0000 7f9a61200640 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c85950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9a50010cf0 tx=0x7f9a5009a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb7a0e6026b8cda server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.442+0000 7f9a4e000640 1 -- 172.21.15.49:0/1035325339 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a500cb830 con 0x5626f5c65300 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.442+0000 7f9a4e000640 1 -- 172.21.15.49:0/1035325339 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a5009a180 con 0x5626f5c65300 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.442+0000 7f9a4e000640 1 -- 172.21.15.49:0/1035325339 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5009e790 con 0x5626f5c65300 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 -- 172.21.15.49:0/1035325339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 msgr2=0x5626f5c85950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c85950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9a50010cf0 tx=0x7f9a5009a0d0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 -- 172.21.15.49:0/1035325339 shutdown_connections 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c6db10 0x5626f5c85e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c85950 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 --2- 172.21.15.49:0/1035325339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c64590 0x5626f5c6d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 -- 172.21.15.49:0/1035325339 >> 172.21.15.49:0/1035325339 conn(0x5626f5c23480 msgr2=0x5626f5c69470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 -- 172.21.15.49:0/1035325339 shutdown_connections 2024-02-26T06:34:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.443+0000 7f9a675ae740 1 -- 172.21.15.49:0/1035325339 wait complete. 2024-02-26T06:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.444+0000 7f9a675ae740 1 -- start start 2024-02-26T06:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.444+0000 7f9a675ae740 1 Processor -- start 2024-02-26T06:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.444+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c842d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.444+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c84810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.444+0000 7f9a675ae740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c6db10 0x5626f5c84d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c84810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c84810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33696/0 (socket says 172.21.15.49:33696) 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a675ae740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5c86270 con 0x5626f5c64590 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a675ae740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5ce50a0 con 0x5626f5c6db10 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a675ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5ce5260 con 0x5626f5c65300 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c842d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61200640 1 -- 172.21.15.49:0/3808138105 learned_addr learned my addr 172.21.15.49:0/3808138105 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a62600640 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c6db10 0x5626f5c84d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a675ae740 10 client.0 _ll_register_callbacks cb 0x7f9a500cb340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61200640 1 -- 172.21.15.49:0/3808138105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c6db10 msgr2=0x5626f5c84d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61200640 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c6db10 0x5626f5c84d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61200640 1 -- 172.21.15.49:0/3808138105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 msgr2=0x5626f5c842d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61200640 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c842d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.445+0000 7f9a61200640 1 -- 172.21.15.49:0/3808138105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626f5bc0c20 con 0x5626f5c65300 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a61200640 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c84810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9a5009a5d0 tx=0x7f9a500b32a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d5fb54cf77235a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a43400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a43400640 1 -- 172.21.15.49:0/3808138105 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a500cb830 con 0x5626f5c65300 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a43400640 1 -- 172.21.15.49:0/3808138105 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f9a50009cf0 con 0x5626f5c65300 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a675ae740 1 -- 172.21.15.49:0/3808138105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626f5be7d00 con 0x5626f5c65300 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a43400640 1 -- 172.21.15.49:0/3808138105 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a50099dd0 con 0x5626f5c65300 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a675ae740 10 client.27406 fetch_fsmap learned FSMap version 48 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a675ae740 1 -- 172.21.15.49:0/3808138105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5626f5b83290 con 0x5626f5c65300 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.446+0000 7f9a43400640 1 -- 172.21.15.49:0/3808138105 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5626f5be7d00 con 0x5626f5c65300 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.447+0000 7f9a43400640 1 -- 172.21.15.49:0/3808138105 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9a50096390 con 0x5626f5c65300 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.447+0000 7f9a675ae740 10 client.27406 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:34:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.447+0000 7f9a675ae740 10 client.27406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:34:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.447+0000 7f9a675ae740 20 client.27406 populate_metadata read hostname 'smithi049' 2024-02-26T06:34:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.447+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5626f5ce6590 0x5626f5ce89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.447+0000 7f9a675ae740 4 client.27406 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:34:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.447+0000 7f9a675ae740 1 -- 172.21.15.49:0/3808138105 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5626f58c2600 con 0x5626f5ce6590 2024-02-26T06:34:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.447+0000 7f9a61c00640 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5626f5ce6590 0x5626f5ce89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.449+0000 7f9a43400640 10 client.27406 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.449+0000 7f9a61c00640 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5626f5ce6590 0x5626f5ce89b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb31f24d2b751dd server_cookie=292f7660b14c46f0 in_seq=0 out_seq=0 2024-02-26T06:34:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.470+0000 7f9a43400640 1 -- 172.21.15.49:0/3808138105 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5626f58c2600 con 0x5626f5ce6590 2024-02-26T06:34:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.470+0000 7f9a43400640 10 client.27406 handle_command_reply: tid=0 2024-02-26T06:34:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.470+0000 7f9a675ae740 1 client.27406 shutdown 2024-02-26T06:34:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 -- 172.21.15.49:0/3808138105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 msgr2=0x5626f5c84810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c84810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9a5009a5d0 tx=0x7f9a500b32a0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 -- 172.21.15.49:0/3808138105 shutdown_connections 2024-02-26T06:34:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5626f5ce6590 0x5626f5ce89b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5626f5c6db10 0x5626f5c84d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a43400640 0 client.27406 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626f5c65300 0x5626f5c84810 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 --2- 172.21.15.49:0/3808138105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5626f5c64590 0x5626f5c842d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 -- 172.21.15.49:0/3808138105 >> 172.21.15.49:0/3808138105 conn(0x5626f5c23480 msgr2=0x5626f5c55480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.471+0000 7f9a675ae740 1 -- 172.21.15.49:0/3808138105 shutdown_connections 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.472+0000 7f9a675ae740 1 -- 172.21.15.49:0/3808138105 wait complete. 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.472+0000 7f9a675ae740 20 client.27406 trim_cache size 0 max 16384 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:34:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 -- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a440a23d0 msgr2=0x7f9a440a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a440a23d0 0x7f9a440a4890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9a54037040 tx=0x7f9a54032e90 comp rx=0 tx=0).stop 2024-02-26T06:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 -- 172.21.15.49:0/1575157232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a5c106460 msgr2=0x7f9a5c19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a5c106460 0x7f9a5c19fb30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000d770 tx=0x7f9a5000db70 comp rx=0 tx=0).stop 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 -- 172.21.15.49:0/1575157232 shutdown_connections 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a440a23d0 0x7f9a440a4890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a5c1996b0 0x7f9a5c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a5c106460 0x7f9a5c19fb30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 --2- 172.21.15.49:0/1575157232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5c101390 0x7f9a5c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 -- 172.21.15.49:0/1575157232 >> 172.21.15.49:0/1575157232 conn(0x7f9a5c0fcc10 msgr2=0x7f9a5c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 -- 172.21.15.49:0/1575157232 shutdown_connections 2024-02-26T06:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:34.473+0000 7f9a64600640 1 -- 172.21.15.49:0/1575157232 wait complete. 2024-02-26T06:34:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:35 smithi080 ceph-mon[36206]: pgmap v348: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-02-26T06:34:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:35 smithi115 ceph-mon[36426]: pgmap v348: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-02-26T06:34:35.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:35 smithi049 ceph-mon[31696]: pgmap v348: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-02-26T06:34:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 645.0 s, 2026.6 tps, lat 49.752 ms stddev 57.187 2024-02-26T06:34:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:37 smithi080 ceph-mon[36206]: pgmap v349: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.0 MiB/s wr, 768 op/s 2024-02-26T06:34:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:37 smithi115 ceph-mon[36426]: pgmap v349: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.0 MiB/s wr, 768 op/s 2024-02-26T06:34:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:37 smithi049 ceph-mon[31696]: pgmap v349: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.0 MiB/s wr, 768 op/s 2024-02-26T06:34:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:39 smithi080 ceph-mon[36206]: pgmap v350: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.5 MiB/s wr, 706 op/s 2024-02-26T06:34:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:39 smithi115 ceph-mon[36426]: pgmap v350: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.5 MiB/s wr, 706 op/s 2024-02-26T06:34:39.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:39 smithi049 ceph-mon[31696]: pgmap v350: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.5 MiB/s wr, 706 op/s 2024-02-26T06:34:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 650.0 s, 1781.8 tps, lat 56.188 ms stddev 124.347 2024-02-26T06:34:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:41 smithi080 ceph-mon[36206]: pgmap v351: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 725 op/s 2024-02-26T06:34:41.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:41 smithi115 ceph-mon[36426]: pgmap v351: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 725 op/s 2024-02-26T06:34:41.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:41 smithi049 ceph-mon[31696]: pgmap v351: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 725 op/s 2024-02-26T06:34:43.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:43 smithi080 ceph-mon[36206]: pgmap v352: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-02-26T06:34:43.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:43 smithi115 ceph-mon[36426]: pgmap v352: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-02-26T06:34:43.672 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:43 smithi049 ceph-mon[31696]: pgmap v352: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-02-26T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:45 smithi080 ceph-mon[36206]: pgmap v353: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-02-26T06:34:45.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:45 smithi115 ceph-mon[36426]: pgmap v353: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-02-26T06:34:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:45 smithi049 ceph-mon[31696]: pgmap v353: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-02-26T06:34:46.304 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 655.0 s, 1821.9 tps, lat 54.539 ms stddev 69.053 2024-02-26T06:34:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:47 smithi080 ceph-mon[36206]: pgmap v354: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-26T06:34:47.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:47 smithi115 ceph-mon[36426]: pgmap v354: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-26T06:34:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.666+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3932273619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc06a620 msgr2=0x7ff4bc105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.666+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3932273619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc06a620 0x7ff4bc105e10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009860 tx=0x7ff4a802bb70 comp rx=0 tx=0).stop 2024-02-26T06:34:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.666+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3932273619 shutdown_connections 2024-02-26T06:34:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.666+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3932273619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc106540 0x7ff4bc10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.666+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3932273619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc06a620 0x7ff4bc105e10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.666+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3932273619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4bc069c70 0x7ff4bc06a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.666+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3932273619 >> 172.21.15.49:0/3932273619 conn(0x7ff4bc076ec0 msgr2=0x7ff4bc0772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.666+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3932273619 shutdown_connections 2024-02-26T06:34:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.667+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3932273619 wait complete. 2024-02-26T06:34:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.668+0000 7ff4c3e00640 1 Processor -- start 2024-02-26T06:34:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.668+0000 7ff4c3e00640 1 -- start start 2024-02-26T06:34:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.668+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc069c70 0x7ff4bc19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.668+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4bc106540 0x7ff4bc19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.668+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc199670 0x7ff4bc199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.668+0000 7ff4c3e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc19a140 con 0x7ff4bc106540 2024-02-26T06:34:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.668+0000 7ff4c3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc19a2b0 con 0x7ff4bc199670 2024-02-26T06:34:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.668+0000 7ff4c3e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc19a470 con 0x7ff4bc069c70 2024-02-26T06:34:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc199670 0x7ff4bc199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc199670 0x7ff4bc199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58374/0 (socket says 172.21.15.49:58374) 2024-02-26T06:34:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1e00640 1 -- 172.21.15.49:0/3289656736 learned_addr learned my addr 172.21.15.49:0/3289656736 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1400640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc069c70 0x7ff4bc19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4bc106540 0x7ff4bc19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:47 smithi049 ceph-mon[31696]: pgmap v354: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-26T06:34:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1400640 1 -- 172.21.15.49:0/3289656736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc199670 msgr2=0x7ff4bc199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1400640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc199670 0x7ff4bc199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1400640 1 -- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4bc106540 msgr2=0x7ff4bc19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1400640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4bc106540 0x7ff4bc19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.669+0000 7ff4c1400640 1 -- 172.21.15.49:0/3289656736 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a8009490 con 0x7ff4bc069c70 2024-02-26T06:34:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.670+0000 7ff4c1e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc199670 0x7ff4bc199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.670+0000 7ff4c1400640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc069c70 0x7ff4bc19f5b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400ef60 tx=0x7ff4b400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.670+0000 7ff4ba000640 1 -- 172.21.15.49:0/3289656736 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400b570 con 0x7ff4bc069c70 2024-02-26T06:34:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.671+0000 7ff4ba000640 1 -- 172.21.15.49:0/3289656736 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4b400bbb0 con 0x7ff4bc069c70 2024-02-26T06:34:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.671+0000 7ff4ba000640 1 -- 172.21.15.49:0/3289656736 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4016cb0 con 0x7ff4bc069c70 2024-02-26T06:34:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.671+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3289656736 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4bc1a6410 con 0x7ff4bc069c70 2024-02-26T06:34:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.671+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3289656736 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc1a68a0 con 0x7ff4bc069c70 2024-02-26T06:34:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.675+0000 7ff4ba000640 1 -- 172.21.15.49:0/3289656736 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff4b400a040 con 0x7ff4bc069c70 2024-02-26T06:34:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.677+0000 7ff4c6ddc740 1 Processor -- start 2024-02-26T06:34:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.678+0000 7ff4c6ddc740 1 -- start start 2024-02-26T06:34:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.678+0000 7ff4ba000640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff4a00a23a0 0x7ff4a00a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.678+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4c5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.678+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.678+0000 7ff4ba000640 1 -- 172.21.15.49:0/3289656736 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7ff4b40ce070 con 0x7ff4bc069c70 2024-02-26T06:34:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.678+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.678+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf4785f0 con 0x55a5cf4c6300 2024-02-26T06:34:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.678+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf42a890 con 0x55a5cf4ceb10 2024-02-26T06:34:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.679+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf427dc0 con 0x55a5cf4c5590 2024-02-26T06:34:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.679+0000 7ff4c1e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff4a00a23a0 0x7ff4a00a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.679+0000 7ff4c1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.679+0000 7ff4c1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55840/0 (socket says 172.21.15.49:55840) 2024-02-26T06:34:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.679+0000 7ff4c1400640 1 -- 172.21.15.49:0/1620105012 learned_addr learned my addr 172.21.15.49:0/1620105012 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.679+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.680+0000 7ff4c0a00640 1 -- 172.21.15.49:0/1620105012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 msgr2=0x55a5cf4d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.680+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.680+0000 7ff4c0a00640 1 -- 172.21.15.49:0/1620105012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 msgr2=0x55a5cf4c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.680+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.680+0000 7ff4c0a00640 1 -- 172.21.15.49:0/1620105012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5cf3a4ad0 con 0x55a5cf4c5590 2024-02-26T06:34:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.680+0000 7ff4c1400640 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.680+0000 7ff4c1e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff4a00a23a0 0x7ff4a00a4860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009fd0 tx=0x7ff4ac009d20 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.681+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4c5a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc069820 tx=0x7ff4a8032410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac53bca429bf315c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.681+0000 7ff4b2600640 1 -- 172.21.15.49:0/1620105012 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8035d20 con 0x55a5cf4c5590 2024-02-26T06:34:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.681+0000 7ff4b2600640 1 -- 172.21.15.49:0/1620105012 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8005020 con 0x55a5cf4c5590 2024-02-26T06:34:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.681+0000 7ff4b2600640 1 -- 172.21.15.49:0/1620105012 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80347e0 con 0x55a5cf4c5590 2024-02-26T06:34:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.682+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/1620105012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 msgr2=0x55a5cf4c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.682+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4c5a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc069820 tx=0x7ff4a8032410 comp rx=0 tx=0).stop 2024-02-26T06:34:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.682+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/1620105012 shutdown_connections 2024-02-26T06:34:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.682+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.682+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4c5a10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.682+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/1620105012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.682+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/1620105012 >> 172.21.15.49:0/1620105012 conn(0x55a5cf484480 msgr2=0x55a5cf4c0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.682+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/1620105012 shutdown_connections 2024-02-26T06:34:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.683+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/1620105012 wait complete. 2024-02-26T06:34:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.684+0000 7ff4c6ddc740 1 -- start start 2024-02-26T06:34:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.684+0000 7ff4c6ddc740 1 Processor -- start 2024-02-26T06:34:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf4235a0 con 0x55a5cf4c6300 2024-02-26T06:34:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf3c8b20 con 0x55a5cf4ceb10 2024-02-26T06:34:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf3c87c0 con 0x55a5cf4c5590 2024-02-26T06:34:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40582/0 (socket says 172.21.15.49:40582) 2024-02-26T06:34:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.686+0000 7ff4c1400640 1 -- 172.21.15.49:0/2203307394 learned_addr learned my addr 172.21.15.49:0/2203307394 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.687+0000 7ff4c6ddc740 10 client.0 _ll_register_callbacks cb 0x55a5cf3dc520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:34:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.687+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.687+0000 7ff4c1e00640 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.688+0000 7ff4c1400640 1 -- 172.21.15.49:0/2203307394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 msgr2=0x55a5cf4cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.688+0000 7ff4c1400640 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.688+0000 7ff4c1400640 1 -- 172.21.15.49:0/2203307394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 msgr2=0x55a5cf4cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.688+0000 7ff4c1400640 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.688+0000 7ff4c1400640 1 -- 172.21.15.49:0/2203307394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc10e890 con 0x55a5cf4c5590 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.688+0000 7ff4c1400640 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4cbb70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4000f80 tx=0x7ff4b409ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa22714aedf1f364 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.688+0000 7ff4a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2203307394 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40b3200 con 0x55a5cf4c5590 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2203307394 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7ff4b40b3360 con 0x55a5cf4c5590 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2203307394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5cf3f5ca0 con 0x55a5cf4c5590 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2203307394 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4b40b3520 con 0x55a5cf4c5590 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 10 client.27418 fetch_fsmap learned FSMap version 48 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2203307394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a5cf421c20 con 0x55a5cf4c5590 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2203307394 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4b407d020 con 0x55a5cf4c5590 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2203307394 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff4b40969e0 con 0x55a5cf4c5590 2024-02-26T06:34:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 10 client.27418 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:34:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 10 client.27418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:34:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 20 client.27418 populate_metadata read hostname 'smithi049' 2024-02-26T06:34:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a5cf4dead0 0x55a5cf4e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 4 client.27418 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:34:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.689+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2203307394 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5cf419cb0 con 0x55a5cf4dead0 2024-02-26T06:34:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.690+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a5cf4dead0 0x55a5cf4e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.691+0000 7ff4a6a00640 10 client.27418 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.692+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a5cf4dead0 0x55a5cf4e0ef0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d508f9f13057a4 server_cookie=5b960ce4c1fa4dbd in_seq=0 out_seq=0 2024-02-26T06:34:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.693+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2203307394 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5cf419cb0 con 0x55a5cf4dead0 2024-02-26T06:34:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.693+0000 7ff4a6a00640 10 client.27418 handle_command_reply: tid=0 2024-02-26T06:34:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.693+0000 7ff4c6ddc740 1 client.27418 shutdown 2024-02-26T06:34:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2203307394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 msgr2=0x55a5cf4cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4cbb70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4000f80 tx=0x7ff4b409ec70 comp rx=0 tx=0).stop 2024-02-26T06:34:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2203307394 shutdown_connections 2024-02-26T06:34:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a5cf4dead0 0x55a5cf4e0ef0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2203307394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4cbb70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4a6a00640 0 client.27418 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.694+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2203307394 >> 172.21.15.49:0/2203307394 conn(0x55a5cf484480 msgr2=0x55a5cf4c1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.695+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2203307394 shutdown_connections 2024-02-26T06:34:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.695+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2203307394 wait complete. 2024-02-26T06:34:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.695+0000 7ff4c6ddc740 20 client.27418 trim_cache size 0 max 16384 2024-02-26T06:34:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.719+0000 7ff4c6ddc740 1 Processor -- start 2024-02-26T06:34:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.719+0000 7ff4c6ddc740 1 -- start start 2024-02-26T06:34:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.720+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.720+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.720+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4d5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.720+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf16be60 con 0x55a5cf4ceb10 2024-02-26T06:34:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.720+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf4e6130 con 0x55a5cf4c6300 2024-02-26T06:34:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.720+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf4c4890 con 0x55a5cf4c5590 2024-02-26T06:34:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.720+0000 7ff4c0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.721+0000 7ff4c1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.721+0000 7ff4c0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58394/0 (socket says 172.21.15.49:58394) 2024-02-26T06:34:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.721+0000 7ff4c0a00640 1 -- 172.21.15.49:0/2156050375 learned_addr learned my addr 172.21.15.49:0/2156050375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.721+0000 7ff4c1e00640 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4d5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.721+0000 7ff4c0a00640 1 -- 172.21.15.49:0/2156050375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 msgr2=0x55a5cf4ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.722+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.722+0000 7ff4c0a00640 1 -- 172.21.15.49:0/2156050375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4ceb10 msgr2=0x55a5cf4d5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.722+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4d5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.722+0000 7ff4c0a00640 1 -- 172.21.15.49:0/2156050375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5cf3be500 con 0x55a5cf4c6300 2024-02-26T06:34:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.722+0000 7ff4c1e00640 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4d5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:34:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.722+0000 7ff4c1400640 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.723+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d58e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005780 tx=0x7ff4a803f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e9187deee991687 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.723+0000 7ff4b1c00640 1 -- 172.21.15.49:0/2156050375 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8054070 con 0x55a5cf4c6300 2024-02-26T06:34:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.723+0000 7ff4b1c00640 1 -- 172.21.15.49:0/2156050375 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4a804eb70 con 0x55a5cf4c6300 2024-02-26T06:34:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.723+0000 7ff4b1c00640 1 -- 172.21.15.49:0/2156050375 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a804e5d0 con 0x55a5cf4c6300 2024-02-26T06:34:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.724+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2156050375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4c6300 msgr2=0x55a5cf4d58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.724+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d58e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005780 tx=0x7ff4a803f040 comp rx=0 tx=0).stop 2024-02-26T06:34:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.724+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2156050375 shutdown_connections 2024-02-26T06:34:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.724+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d58e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.724+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4ceb10 0x55a5cf4d5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.724+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2156050375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.724+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2156050375 >> 172.21.15.49:0/2156050375 conn(0x55a5cf484480 msgr2=0x55a5cf4bfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.725+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2156050375 shutdown_connections 2024-02-26T06:34:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.725+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2156050375 wait complete. 2024-02-26T06:34:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.725+0000 7ff4c6ddc740 1 -- start start 2024-02-26T06:34:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.726+0000 7ff4c6ddc740 1 Processor -- start 2024-02-26T06:34:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.726+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.726+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.726+0000 7ff4c6ddc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf534fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.726+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf4d54e0 con 0x55a5cf4c6300 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.726+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf535520 con 0x55a5cf4ceb10 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.726+0000 7ff4c6ddc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5cf535660 con 0x55a5cf4c5590 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.726+0000 7ff4c6ddc740 10 client.0 _ll_register_callbacks cb 0x7ff4b40cb6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.727+0000 7ff4c0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.727+0000 7ff4c1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.727+0000 7ff4c0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55858/0 (socket says 172.21.15.49:55858) 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.727+0000 7ff4c0a00640 1 -- 172.21.15.49:0/2011155764 learned_addr learned my addr 172.21.15.49:0/2011155764 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.727+0000 7ff4c1e00640 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf534fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.727+0000 7ff4c1400640 1 -- 172.21.15.49:0/2011155764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 msgr2=0x55a5cf534fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.727+0000 7ff4c1400640 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf534fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.727+0000 7ff4c1400640 1 -- 172.21.15.49:0/2011155764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 msgr2=0x55a5cf4d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:34:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.728+0000 7ff4c1400640 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.728+0000 7ff4c1400640 1 -- 172.21.15.49:0/2011155764 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5cf421c20 con 0x55a5cf4c5590 2024-02-26T06:34:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.728+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:34:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.729+0000 7ff4c1400640 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4d46f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40a2c50 tx=0x7ff4b40b3430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd136b43178461be server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:34:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.729+0000 7ff4a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:34:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.729+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2011155764 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4099d30 con 0x55a5cf4c5590 2024-02-26T06:34:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.730+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2011155764 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7ff4b407e080 con 0x55a5cf4c5590 2024-02-26T06:34:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.730+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2011155764 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5cf448d00 con 0x55a5cf4c5590 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2011155764 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4b4096d30 con 0x55a5cf4c5590 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4c6ddc740 10 client.27424 fetch_fsmap learned FSMap version 48 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2011155764 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a5cf3a4ad0 con 0x55a5cf4c5590 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2011155764 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4b407d070 con 0x55a5cf4c5590 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2011155764 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff4b4099ee0 con 0x55a5cf4c5590 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4c6ddc740 10 client.27424 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4c6ddc740 10 client.27424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4c6ddc740 20 client.27424 populate_metadata read hostname 'smithi049' 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a5cf536ad0 0x55a5cf538ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4c6ddc740 4 client.27424 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:34:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.731+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2011155764 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a5cf448d00 con 0x55a5cf536ad0 2024-02-26T06:34:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.732+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a5cf536ad0 0x55a5cf538ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:34:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.732+0000 7ff4c0a00640 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a5cf536ad0 0x55a5cf538ef0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d93b23b80e8d7d server_cookie=ca6df7e149a52a2b in_seq=0 out_seq=0 2024-02-26T06:34:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.733+0000 7ff4a6a00640 10 client.27424 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.815+0000 7ff4a6a00640 1 -- 172.21.15.49:0/2011155764 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a5cf448d00 con 0x55a5cf536ad0 2024-02-26T06:34:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.815+0000 7ff4a6a00640 10 client.27424 handle_command_reply: tid=0 2024-02-26T06:34:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.815+0000 7ff4c6ddc740 1 client.27424 shutdown 2024-02-26T06:34:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2011155764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 msgr2=0x55a5cf4d46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4d46f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40a2c50 tx=0x7ff4b40b3430 comp rx=0 tx=0).stop 2024-02-26T06:34:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2011155764 shutdown_connections 2024-02-26T06:34:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55a5cf536ad0 0x55a5cf538ef0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a5cf4ceb10 0x55a5cf534fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5cf4c6300 0x55a5cf4d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4c6ddc740 1 --2- 172.21.15.49:0/2011155764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5cf4c5590 0x55a5cf4d46f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2011155764 >> 172.21.15.49:0/2011155764 conn(0x55a5cf484480 msgr2=0x55a5cf4ca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.816+0000 7ff4a6a00640 0 client.27424 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.817+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2011155764 shutdown_connections 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.817+0000 7ff4c6ddc740 1 -- 172.21.15.49:0/2011155764 wait complete. 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.817+0000 7ff4c6ddc740 20 client.27424 trim_cache size 0 max 16384 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:34:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:34:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff4a00a23a0 msgr2=0x7ff4a00a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff4a00a23a0 0x7ff4a00a4860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009fd0 tx=0x7ff4ac009d20 comp rx=0 tx=0).stop 2024-02-26T06:34:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3289656736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc069c70 msgr2=0x7ff4bc19f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:34:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc069c70 0x7ff4bc19f5b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400ef60 tx=0x7ff4b400ef90 comp rx=0 tx=0).stop 2024-02-26T06:34:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3289656736 shutdown_connections 2024-02-26T06:34:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff4a00a23a0 0x7ff4a00a4860 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc199670 0x7ff4bc199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4bc106540 0x7ff4bc19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 --2- 172.21.15.49:0/3289656736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4bc069c70 0x7ff4bc19f5b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:34:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3289656736 >> 172.21.15.49:0/3289656736 conn(0x7ff4bc076ec0 msgr2=0x7ff4bc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:34:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3289656736 shutdown_connections 2024-02-26T06:34:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:34:47.818+0000 7ff4c3e00640 1 -- 172.21.15.49:0/3289656736 wait complete. 2024-02-26T06:34:49.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:49 smithi080 ceph-mon[36206]: pgmap v355: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-02-26T06:34:49.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:49 smithi115 ceph-mon[36426]: pgmap v355: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-02-26T06:34:49.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:49 smithi049 ceph-mon[31696]: pgmap v355: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-02-26T06:34:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 660.0 s, 1957.4 tps, lat 50.648 ms stddev 62.246 2024-02-26T06:34:51.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:51 smithi080 ceph-mon[36206]: pgmap v356: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-02-26T06:34:51.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:51 smithi115 ceph-mon[36426]: pgmap v356: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-02-26T06:34:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:51 smithi049 ceph-mon[31696]: pgmap v356: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-02-26T06:34:52.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:52 smithi049 ceph-mon[31696]: pgmap v357: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.73k op/s 2024-02-26T06:34:52.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:52 smithi080 ceph-mon[36206]: pgmap v357: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.73k op/s 2024-02-26T06:34:52.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:52 smithi115 ceph-mon[36426]: pgmap v357: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.73k op/s 2024-02-26T06:34:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:54 smithi080 ceph-mon[36206]: pgmap v358: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 555 op/s 2024-02-26T06:34:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:54 smithi115 ceph-mon[36426]: pgmap v358: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 555 op/s 2024-02-26T06:34:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:54 smithi049 ceph-mon[31696]: pgmap v358: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 555 op/s 2024-02-26T06:34:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 665.0 s, 1339.8 tps, lat 75.254 ms stddev 90.808 2024-02-26T06:34:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:56 smithi080 ceph-mon[36206]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.3 MiB/s wr, 717 op/s 2024-02-26T06:34:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:56 smithi115 ceph-mon[36426]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.3 MiB/s wr, 717 op/s 2024-02-26T06:34:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:56 smithi049 ceph-mon[31696]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.3 MiB/s wr, 717 op/s 2024-02-26T06:34:58.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:57 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:34:58.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:57 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:34:58.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:57 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:34:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:58 smithi080 ceph-mon[36206]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-02-26T06:34:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:58 smithi115 ceph-mon[36426]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-02-26T06:34:59.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:58 smithi049 ceph-mon[31696]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-02-26T06:35:00.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:59 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:35:00.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:59 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:35:00.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:34:59 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:35:00.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:59 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:35:00.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:59 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:35:00.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:34:59 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:35:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:59 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:35:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:59 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:35:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:34:59 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:35:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 -- 172.21.15.49:0/2482628338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 msgr2=0x7f5444102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 --2- 172.21.15.49:0/2482628338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 0x7f5444102510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5434009900 tx=0x7f543402bcd0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 -- 172.21.15.49:0/2482628338 shutdown_connections 2024-02-26T06:35:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 --2- 172.21.15.49:0/2482628338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f544410b550 0x7f544410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 --2- 172.21.15.49:0/2482628338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5444102a50 0x7f544410af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 --2- 172.21.15.49:0/2482628338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 0x7f5444102510 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 -- 172.21.15.49:0/2482628338 >> 172.21.15.49:0/2482628338 conn(0x7f54440fb990 msgr2=0x7f54440fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 -- 172.21.15.49:0/2482628338 shutdown_connections 2024-02-26T06:35:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.959+0000 7f5449800640 1 -- 172.21.15.49:0/2482628338 wait complete. 2024-02-26T06:35:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.960+0000 7f5449800640 1 Processor -- start 2024-02-26T06:35:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5449800640 1 -- start start 2024-02-26T06:35:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5449800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 0x7f544419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5449800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5444102a50 0x7f544419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5449800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f544410b550 0x7f5444199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5449800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444199c60 con 0x7f5444102a50 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5449800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444199dd0 con 0x7f544410b550 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5449800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444199f90 con 0x7f5444102130 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5443e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 0x7f544419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.961+0000 7f5443e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 0x7f544419f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48402/0 (socket says 172.21.15.49:48402) 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5443e00640 1 -- 172.21.15.49:0/4108890661 learned_addr learned my addr 172.21.15.49:0/4108890661 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5448e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f544410b550 0x7f5444199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5443400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5444102a50 0x7f544419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5443e00640 1 -- 172.21.15.49:0/4108890661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f544410b550 msgr2=0x7f5444199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5443e00640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f544410b550 0x7f5444199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5443e00640 1 -- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5444102a50 msgr2=0x7f544419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5443e00640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5444102a50 0x7f544419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5443e00640 1 -- 172.21.15.49:0/4108890661 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5434009560 con 0x7f5444102130 2024-02-26T06:35:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5448e00640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f544410b550 0x7f5444199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:35:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.962+0000 7f5443e00640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 0x7f544419f460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5434000c00 tx=0x7f543402c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.963+0000 7f5440c00640 1 -- 172.21.15.49:0/4108890661 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543402dbe0 con 0x7f5444102130 2024-02-26T06:35:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.963+0000 7f5440c00640 1 -- 172.21.15.49:0/4108890661 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f543402dd40 con 0x7f5444102130 2024-02-26T06:35:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.963+0000 7f5449800640 1 -- 172.21.15.49:0/4108890661 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544419a210 con 0x7f5444102130 2024-02-26T06:35:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.963+0000 7f5440c00640 1 -- 172.21.15.49:0/4108890661 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434003b20 con 0x7f5444102130 2024-02-26T06:35:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.963+0000 7f5449800640 1 -- 172.21.15.49:0/4108890661 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54441a6500 con 0x7f5444102130 2024-02-26T06:35:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.966+0000 7f5440c00640 1 -- 172.21.15.49:0/4108890661 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5434003c80 con 0x7f5444102130 2024-02-26T06:35:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.967+0000 7f544dc1e740 1 Processor -- start 2024-02-26T06:35:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.967+0000 7f544dc1e740 1 -- start start 2024-02-26T06:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x564464999a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x56446499a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5644649a3540 0x5644649a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f544dc1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644649555f0 con 0x5644649a3540 2024-02-26T06:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f544dc1e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464907890 con 0x564464999fc0 2024-02-26T06:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f544dc1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464904dc0 con 0x56446499ad30 2024-02-26T06:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5448e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x564464999a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5448e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x564464999a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48412/0 (socket says 172.21.15.49:48412) 2024-02-26T06:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5448e00640 1 -- 172.21.15.49:0/2543548115 learned_addr learned my addr 172.21.15.49:0/2543548115 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5443400640 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5644649a3540 0x5644649a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5443e00640 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x56446499a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5440c00640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54280a22a0 0x7f54280a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5448e00640 1 -- 172.21.15.49:0/2543548115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 msgr2=0x56446499a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5448e00640 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x56446499a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5448e00640 1 -- 172.21.15.49:0/2543548115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5644649a3540 msgr2=0x5644649a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.968+0000 7f5448e00640 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5644649a3540 0x5644649a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.969+0000 7f5448e00640 1 -- 172.21.15.49:0/2543548115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564464881ad0 con 0x56446499ad30 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.969+0000 7f5440c00640 1 -- 172.21.15.49:0/4108890661 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f54340f04b0 con 0x7f5444102130 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.969+0000 7f5448e00640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54280a22a0 0x7f54280a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.969+0000 7f5448e00640 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x564464999a80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f543c011790 tx=0x7f543c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57abaf3a324cf6f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.970+0000 7f5439600640 1 -- 172.21.15.49:0/2543548115 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c021070 con 0x56446499ad30 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.970+0000 7f5439600640 1 -- 172.21.15.49:0/2543548115 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f543c003db0 con 0x56446499ad30 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.970+0000 7f5439600640 1 -- 172.21.15.49:0/2543548115 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c011ea0 con 0x56446499ad30 2024-02-26T06:35:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.970+0000 7f5448e00640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54280a22a0 0x7f54280a4760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f543c020000 tx=0x7f543c021000 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 -- 172.21.15.49:0/2543548115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 msgr2=0x564464999a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x564464999a80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f543c011790 tx=0x7f543c011c50 comp rx=0 tx=0).stop 2024-02-26T06:35:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 -- 172.21.15.49:0/2543548115 shutdown_connections 2024-02-26T06:35:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x56446499a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5644649a3540 0x5644649a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 --2- 172.21.15.49:0/2543548115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x564464999a80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 -- 172.21.15.49:0/2543548115 >> 172.21.15.49:0/2543548115 conn(0x564464961480 msgr2=0x564464994cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 -- 172.21.15.49:0/2543548115 shutdown_connections 2024-02-26T06:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 -- 172.21.15.49:0/2543548115 wait complete. 2024-02-26T06:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.971+0000 7f544dc1e740 1 -- start start 2024-02-26T06:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.972+0000 7f544dc1e740 1 Processor -- start 2024-02-26T06:35:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.972+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x5644649a0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.972+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644649a3540 0x5644649a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f544dc1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644649005a0 con 0x56446499ad30 2024-02-26T06:35:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f544dc1e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644648a5b20 con 0x564464999fc0 2024-02-26T06:35:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f544dc1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644648a57c0 con 0x5644649a3540 2024-02-26T06:35:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f544dc1e740 10 client.0 _ll_register_callbacks cb 0x5644648b9520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x5644649a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55842/0 (socket says 172.21.15.49:55842) 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443400640 1 -- 172.21.15.49:0/1377580454 learned_addr learned my addr 172.21.15.49:0/1377580454 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5448e00640 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644649a3540 0x5644649a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443e00640 1 -- 172.21.15.49:0/1377580454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644649a3540 msgr2=0x5644649a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443e00640 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644649a3540 0x5644649a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443e00640 1 -- 172.21.15.49:0/1377580454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 msgr2=0x5644649a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443e00640 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.973+0000 7f5443e00640 1 -- 172.21.15.49:0/1377580454 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c011440 con 0x564464999fc0 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f5443e00640 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x5644649a0550 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f54340edc60 tx=0x7f5434009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=441848259bf157c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f542e000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f542e000640 1 -- 172.21.15.49:0/1377580454 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434005640 con 0x564464999fc0 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f542e000640 1 -- 172.21.15.49:0/1377580454 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f54340bd6c0 con 0x564464999fc0 2024-02-26T06:35:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f544dc1e740 1 -- 172.21.15.49:0/1377580454 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644648d2ca0 con 0x564464999fc0 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f542e000640 1 -- 172.21.15.49:0/1377580454 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54340bc920 con 0x564464999fc0 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f544dc1e740 10 client.27476 fetch_fsmap learned FSMap version 48 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f544dc1e740 1 -- 172.21.15.49:0/1377580454 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5644648fec20 con 0x564464999fc0 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.974+0000 7f542e000640 1 -- 172.21.15.49:0/1377580454 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5644648d2ca0 con 0x564464999fc0 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f542e000640 1 -- 172.21.15.49:0/1377580454 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f543403c070 con 0x564464999fc0 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f544dc1e740 10 client.27476 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f544dc1e740 10 client.27476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f544dc1e740 20 client.27476 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f544dc1e740 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5644649b33d0 0x5644649b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f544dc1e740 4 client.27476 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:35:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f544dc1e740 1 -- 172.21.15.49:0/1377580454 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644648f6cb0 con 0x5644649b33d0 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f5443400640 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5644649b33d0 0x5644649b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f5443400640 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5644649b33d0 0x5644649b5810 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99cfb75440dda53e server_cookie=c8249e7097acbbce in_seq=0 out_seq=0 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.975+0000 7f542e000640 10 client.27476 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f542e000640 1 -- 172.21.15.49:0/1377580454 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644648f6cb0 con 0x5644649b33d0 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f542e000640 10 client.27476 handle_command_reply: tid=0 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 client.27476 shutdown 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 -- 172.21.15.49:0/1377580454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 msgr2=0x5644649a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x5644649a0550 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f54340edc60 tx=0x7f5434009260 comp rx=0 tx=0).stop 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 -- 172.21.15.49:0/1377580454 shutdown_connections 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5644649b33d0 0x5644649b5810 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644649a3540 0x5644649a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 --2- 172.21.15.49:0/1377580454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464999fc0 0x5644649a0550 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f544dc1e740 1 -- 172.21.15.49:0/1377580454 >> 172.21.15.49:0/1377580454 conn(0x564464961480 msgr2=0x564464996a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.976+0000 7f542e000640 0 client.27476 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.977+0000 7f544dc1e740 1 -- 172.21.15.49:0/1377580454 shutdown_connections 2024-02-26T06:35:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.977+0000 7f544dc1e740 1 -- 172.21.15.49:0/1377580454 wait complete. 2024-02-26T06:35:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.977+0000 7f544dc1e740 20 client.27476 trim_cache size 0 max 16384 2024-02-26T06:35:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f544dc1e740 1 Processor -- start 2024-02-26T06:35:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f544dc1e740 1 -- start start 2024-02-26T06:35:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564464999fc0 0x56446499d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x56446499dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f544dc1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464648e60 con 0x564464999fc0 2024-02-26T06:35:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f544dc1e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644649a6150 con 0x564464961480 2024-02-26T06:35:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f544dc1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464994540 con 0x56446499ad30 2024-02-26T06:35:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f5443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f5443400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564464999fc0 0x56446499d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f5443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53314/0 (socket says 172.21.15.49:53314) 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f5443e00640 1 -- 172.21.15.49:0/994090553 learned_addr learned my addr 172.21.15.49:0/994090553 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.992+0000 7f5448e00640 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x56446499dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5443e00640 1 -- 172.21.15.49:0/994090553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 msgr2=0x56446499dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5443e00640 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x56446499dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5443e00640 1 -- 172.21.15.49:0/994090553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564464999fc0 msgr2=0x56446499d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5443e00640 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564464999fc0 0x56446499d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5443e00640 1 -- 172.21.15.49:0/994090553 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56446489b500 con 0x564464961480 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5448e00640 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x56446499dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:35:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5443e00640 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a2c80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f54340c51a0 tx=0x7f54340f03a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23ff72641f8e8a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5438c00640 1 -- 172.21.15.49:0/994090553 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434005640 con 0x564464961480 2024-02-26T06:35:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5438c00640 1 -- 172.21.15.49:0/994090553 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54340bcd90 con 0x564464961480 2024-02-26T06:35:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.993+0000 7f5438c00640 1 -- 172.21.15.49:0/994090553 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54340ee020 con 0x564464961480 2024-02-26T06:35:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.994+0000 7f544dc1e740 1 -- 172.21.15.49:0/994090553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 msgr2=0x5644649a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.994+0000 7f544dc1e740 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a2c80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f54340c51a0 tx=0x7f54340f03a0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 -- 172.21.15.49:0/994090553 shutdown_connections 2024-02-26T06:35:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56446499ad30 0x56446499dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564464999fc0 0x56446499d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 --2- 172.21.15.49:0/994090553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a2c80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 -- 172.21.15.49:0/994090553 >> 172.21.15.49:0/994090553 conn(0x564464990140 msgr2=0x56446499ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 -- 172.21.15.49:0/994090553 shutdown_connections 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 -- 172.21.15.49:0/994090553 wait complete. 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 -- start start 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.995+0000 7f544dc1e740 1 Processor -- start 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564464999fc0 0x5644649a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f544dc1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f544dc1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464a09900 con 0x56446499ad30 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f544dc1e740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464a09a70 con 0x564464961480 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f544dc1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464a09c30 con 0x564464999fc0 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f544dc1e740 10 client.0 _ll_register_callbacks cb 0x7f54340f3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f5443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f5443e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:53318/0 (socket says 172.21.15.49:53318) 2024-02-26T06:35:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f5443e00640 1 -- 172.21.15.49:0/170495420 learned_addr learned my addr 172.21.15.49:0/170495420 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f5443400640 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564464999fc0 0x5644649a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.996+0000 7f5448e00640 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f5443400640 1 -- 172.21.15.49:0/170495420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 msgr2=0x5644649a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f5443400640 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f5443400640 1 -- 172.21.15.49:0/170495420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 msgr2=0x5644649a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f5443400640 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f5443400640 1 -- 172.21.15.49:0/170495420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564464881ad0 con 0x564464999fc0 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f5443e00640 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f5443400640 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564464999fc0 0x5644649a93c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54340bb270 tx=0x7f54300109b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcbdc7792bf2ab02 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f542e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f542e000640 1 -- 172.21.15.49:0/170495420 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430017040 con 0x564464999fc0 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f542e000640 1 -- 172.21.15.49:0/170495420 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f5430005020 con 0x564464999fc0 2024-02-26T06:35:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.997+0000 7f544dc1e740 1 -- 172.21.15.49:0/170495420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564464925d00 con 0x564464999fc0 2024-02-26T06:35:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f542e000640 1 -- 172.21.15.49:0/170495420 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54300051e0 con 0x564464999fc0 2024-02-26T06:35:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f544dc1e740 10 client.27442 fetch_fsmap learned FSMap version 48 2024-02-26T06:35:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f544dc1e740 1 -- 172.21.15.49:0/170495420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5644648fec20 con 0x564464999fc0 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f542e000640 1 -- 172.21.15.49:0/170495420 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564464925d00 con 0x564464999fc0 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f542e000640 1 -- 172.21.15.49:0/170495420 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f54300263b0 con 0x564464999fc0 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f544dc1e740 10 client.27442 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f544dc1e740 10 client.27442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f544dc1e740 20 client.27442 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f544dc1e740 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5644649a3540 0x564464a0d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f544dc1e740 4 client.27442 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.998+0000 7f544dc1e740 1 -- 172.21.15.49:0/170495420 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564464600600 con 0x5644649a3540 2024-02-26T06:35:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.999+0000 7f5443e00640 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5644649a3540 0x564464a0d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.999+0000 7f5443e00640 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5644649a3540 0x564464a0d4b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6fe18478be5c9f server_cookie=aeba6d0c67b3ab88 in_seq=0 out_seq=0 2024-02-26T06:35:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:00.999+0000 7f542e000640 10 client.27442 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f542e000640 1 -- 172.21.15.49:0/170495420 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564464600600 con 0x5644649a3540 2024-02-26T06:35:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f542e000640 10 client.27442 handle_command_reply: tid=0 2024-02-26T06:35:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 client.27442 shutdown 2024-02-26T06:35:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 -- 172.21.15.49:0/170495420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564464999fc0 msgr2=0x5644649a93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564464999fc0 0x5644649a93c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54340bb270 tx=0x7f54300109b0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 -- 172.21.15.49:0/170495420 shutdown_connections 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5644649a3540 0x564464a0d4b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56446499ad30 0x5644649a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564464999fc0 0x5644649a93c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f542e000640 0 client.27442 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 --2- 172.21.15.49:0/170495420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564464961480 0x5644649a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.015+0000 7f544dc1e740 1 -- 172.21.15.49:0/170495420 >> 172.21.15.49:0/170495420 conn(0x564464990140 msgr2=0x564464993480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.016+0000 7f544dc1e740 1 -- 172.21.15.49:0/170495420 shutdown_connections 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.016+0000 7f544dc1e740 1 -- 172.21.15.49:0/170495420 wait complete. 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.016+0000 7f544dc1e740 20 client.27442 trim_cache size 0 max 16384 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:35:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:35:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:35:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:35:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.016+0000 7f5449800640 1 -- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54280a22a0 msgr2=0x7f54280a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.016+0000 7f5449800640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54280a22a0 0x7f54280a4760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f543c020000 tx=0x7f543c021000 comp rx=0 tx=0).stop 2024-02-26T06:35:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.016+0000 7f5449800640 1 -- 172.21.15.49:0/4108890661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 msgr2=0x7f544419f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 0x7f544419f460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5434000c00 tx=0x7f543402c6d0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 -- 172.21.15.49:0/4108890661 shutdown_connections 2024-02-26T06:35:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f54280a22a0 0x7f54280a4760 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f544410b550 0x7f5444199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5444102a50 0x7f544419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 --2- 172.21.15.49:0/4108890661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5444102130 0x7f544419f460 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 -- 172.21.15.49:0/4108890661 >> 172.21.15.49:0/4108890661 conn(0x7f54440fb990 msgr2=0x7f54440ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 -- 172.21.15.49:0/4108890661 shutdown_connections 2024-02-26T06:35:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:01.017+0000 7f5449800640 1 -- 172.21.15.49:0/4108890661 wait complete. 2024-02-26T06:35:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 670.0 s, 1200.2 tps, lat 79.405 ms stddev 97.894 2024-02-26T06:35:01.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:01 smithi049 ceph-mon[31696]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 570 op/s 2024-02-26T06:35:01.585 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:01 smithi080 ceph-mon[36206]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 570 op/s 2024-02-26T06:35:01.592 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:01 smithi115 ceph-mon[36426]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 570 op/s 2024-02-26T06:35:03.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:03 smithi049 ceph-mon[31696]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.3 MiB/s wr, 755 op/s 2024-02-26T06:35:03.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:03 smithi080 ceph-mon[36206]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.3 MiB/s wr, 755 op/s 2024-02-26T06:35:03.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:03 smithi115 ceph-mon[36426]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.3 MiB/s wr, 755 op/s 2024-02-26T06:35:05.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:05 smithi049 ceph-mon[31696]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 538 op/s 2024-02-26T06:35:05.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:05 smithi080 ceph-mon[36206]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 538 op/s 2024-02-26T06:35:05.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:05 smithi115 ceph-mon[36426]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 538 op/s 2024-02-26T06:35:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 675.0 s, 512.0 tps, lat 198.254 ms stddev 267.692 2024-02-26T06:35:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:07 smithi049 ceph-mon[31696]: pgmap v364: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-02-26T06:35:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:07 smithi049 ceph-mon[31696]: osdmap e100: 12 total, 12 up, 12 in 2024-02-26T06:35:07.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:07 smithi080 ceph-mon[36206]: pgmap v364: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-02-26T06:35:07.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:07 smithi080 ceph-mon[36206]: osdmap e100: 12 total, 12 up, 12 in 2024-02-26T06:35:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:07 smithi115 ceph-mon[36426]: pgmap v364: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-02-26T06:35:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:07 smithi115 ceph-mon[36426]: osdmap e100: 12 total, 12 up, 12 in 2024-02-26T06:35:09.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:09 smithi049 ceph-mon[31696]: pgmap v366: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.65k op/s 2024-02-26T06:35:09.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:09 smithi080 ceph-mon[36206]: pgmap v366: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.65k op/s 2024-02-26T06:35:09.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:09 smithi115 ceph-mon[36426]: pgmap v366: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.65k op/s 2024-02-26T06:35:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 680.0 s, 1443.0 tps, lat 71.136 ms stddev 97.677 2024-02-26T06:35:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:11 smithi080 ceph-mon[36206]: pgmap v367: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.63k op/s 2024-02-26T06:35:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:11 smithi115 ceph-mon[36426]: pgmap v367: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.63k op/s 2024-02-26T06:35:11.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:11 smithi049 ceph-mon[31696]: pgmap v367: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.63k op/s 2024-02-26T06:35:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:13 smithi080 ceph-mon[36206]: pgmap v368: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.65k op/s 2024-02-26T06:35:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:13 smithi115 ceph-mon[36426]: pgmap v368: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.65k op/s 2024-02-26T06:35:13.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:13 smithi049 ceph-mon[31696]: pgmap v368: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.65k op/s 2024-02-26T06:35:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.216+0000 7f9aa1c00640 1 -- 172.21.15.49:0/1002540397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c10c8f0 msgr2=0x7f9a9c10ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.216+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/1002540397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c10c8f0 0x7f9a9c10ccd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009900 tx=0x7f9a8402bc40 comp rx=0 tx=0).stop 2024-02-26T06:35:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.216+0000 7f9aa1c00640 1 -- 172.21.15.49:0/1002540397 shutdown_connections 2024-02-26T06:35:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.216+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/1002540397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a9c10d210 0x7f9a9c108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.216+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/1002540397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a9c1086c0 0x7f9a9c108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.216+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/1002540397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c10c8f0 0x7f9a9c10ccd0 secure :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009900 tx=0x7f9a8402bc40 comp rx=0 tx=0).stop 2024-02-26T06:35:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.216+0000 7f9aa1c00640 1 -- 172.21.15.49:0/1002540397 >> 172.21.15.49:0/1002540397 conn(0x7f9a9c06c6b0 msgr2=0x7f9a9c06cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.218+0000 7f9aa1c00640 1 -- 172.21.15.49:0/1002540397 shutdown_connections 2024-02-26T06:35:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.219+0000 7f9aa1c00640 1 -- 172.21.15.49:0/1002540397 wait complete. 2024-02-26T06:35:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.220+0000 7f9aa1c00640 1 Processor -- start 2024-02-26T06:35:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.222+0000 7f9aa1c00640 1 -- start start 2024-02-26T06:35:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.223+0000 7f9aa1c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c1086c0 0x7f9a9c1151e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.223+0000 7f9aa1c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a9c10d210 0x7f9a9c115720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.223+0000 7f9aa1c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a9c1a8230 0x7f9a9c1a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.223+0000 7f9a92000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a9c10d210 0x7f9a9c115720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.223+0000 7f9a92000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a9c10d210 0x7f9a9c115720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49414/0 (socket says 172.21.15.49:49414) 2024-02-26T06:35:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.224+0000 7f9aa1c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1a8e70 con 0x7f9a9c1086c0 2024-02-26T06:35:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.224+0000 7f9aa1c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1a8fe0 con 0x7f9a9c1a8230 2024-02-26T06:35:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.224+0000 7f9aa1c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c11aef0 con 0x7f9a9c10d210 2024-02-26T06:35:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.224+0000 7f9a9aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c1086c0 0x7f9a9c1151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.224+0000 7f9a9aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c1086c0 0x7f9a9c1151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36358/0 (socket says 172.21.15.49:36358) 2024-02-26T06:35:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.224+0000 7f9a9aa00640 1 -- 172.21.15.49:0/2432386656 learned_addr learned my addr 172.21.15.49:0/2432386656 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.226+0000 7f9a92000640 1 -- 172.21.15.49:0/2432386656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a9c1a8230 msgr2=0x7f9a9c1a86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.226+0000 7f9a92000640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a9c1a8230 0x7f9a9c1a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.226+0000 7f9a92000640 1 -- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c1086c0 msgr2=0x7f9a9c1151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.226+0000 7f9a92000640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c1086c0 0x7f9a9c1151e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.226+0000 7f9a92000640 1 -- 172.21.15.49:0/2432386656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009560 con 0x7f9a9c10d210 2024-02-26T06:35:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.227+0000 7f9a92000640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a9c10d210 0x7f9a9c115720 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00e9f0 tx=0x7f9a8c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.228+0000 7f9a93e00640 1 -- 172.21.15.49:0/2432386656 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00cd80 con 0x7f9a9c10d210 2024-02-26T06:35:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.228+0000 7f9a93e00640 1 -- 172.21.15.49:0/2432386656 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c011950 con 0x7f9a9c10d210 2024-02-26T06:35:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.228+0000 7f9a93e00640 1 -- 172.21.15.49:0/2432386656 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c01aa60 con 0x7f9a9c10d210 2024-02-26T06:35:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.228+0000 7f9aa1c00640 1 -- 172.21.15.49:0/2432386656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c11b1d0 con 0x7f9a9c10d210 2024-02-26T06:35:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.228+0000 7f9aa1c00640 1 -- 172.21.15.49:0/2432386656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c11b690 con 0x7f9a9c10d210 2024-02-26T06:35:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.237+0000 7f9aa4bc5740 1 Processor -- start 2024-02-26T06:35:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.238+0000 7f9a93e00640 1 -- 172.21.15.49:0/2432386656 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9a8c011ac0 con 0x7f9a9c10d210 2024-02-26T06:35:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.240+0000 7f9a93e00640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a800a2350 0x7f9a800a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.241+0000 7f9a93e00640 1 -- 172.21.15.49:0/2432386656 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f9a8c0ce510 con 0x7f9a9c10d210 2024-02-26T06:35:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.241+0000 7f9aa4bc5740 1 -- start start 2024-02-26T06:35:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.241+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb1af0 0x5653cfdb0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.241+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb0d80 0x5653cfdb1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.242+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfdbc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.242+0000 7f9a9b400640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a800a2350 0x7f9a800a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.242+0000 7f9a9aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb1af0 0x5653cfdb0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.242+0000 7f9a9aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb1af0 0x5653cfdb0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36370/0 (socket says 172.21.15.49:36370) 2024-02-26T06:35:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.242+0000 7f9a9aa00640 1 -- 172.21.15.49:0/694953114 learned_addr learned my addr 172.21.15.49:0/694953114 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.242+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/694953114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfd6c5f0 con 0x5653cfdb1af0 2024-02-26T06:35:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.242+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/694953114 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfd1e890 con 0x5653cfdba300 2024-02-26T06:35:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.243+0000 7f9a9b400640 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfdbc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.243+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/694953114 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfd1bdc0 con 0x5653cfdb0d80 2024-02-26T06:35:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.245+0000 7f9a92000640 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb0d80 0x5653cfdb1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.245+0000 7f9a9b400640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a800a2350 0x7f9a800a4810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400ad90 tx=0x7f9a940137d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.245+0000 7f9a9aa00640 1 -- 172.21.15.49:0/694953114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb0d80 msgr2=0x5653cfdb1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.245+0000 7f9a9aa00640 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb0d80 0x5653cfdb1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.245+0000 7f9a9aa00640 1 -- 172.21.15.49:0/694953114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 msgr2=0x5653cfdbc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.245+0000 7f9a9aa00640 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfdbc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.245+0000 7f9a9aa00640 1 -- 172.21.15.49:0/694953114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653cfc98ad0 con 0x5653cfdb1af0 2024-02-26T06:35:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.245+0000 7f9a92000640 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb0d80 0x5653cfdb1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:35:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.246+0000 7f9a9aa00640 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb1af0 0x5653cfdb0840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a84004740 tx=0x7f9a84004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aeb152353eecca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.246+0000 7f9a8be00640 1 -- 172.21.15.49:0/694953114 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8402c7f0 con 0x5653cfdb1af0 2024-02-26T06:35:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.247+0000 7f9a8be00640 1 -- 172.21.15.49:0/694953114 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a8402c950 con 0x5653cfdb1af0 2024-02-26T06:35:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.247+0000 7f9a8be00640 1 -- 172.21.15.49:0/694953114 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84005430 con 0x5653cfdb1af0 2024-02-26T06:35:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.250+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/694953114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb1af0 msgr2=0x5653cfdb0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.250+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb1af0 0x5653cfdb0840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a84004740 tx=0x7f9a84004770 comp rx=0 tx=0).stop 2024-02-26T06:35:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.251+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/694953114 shutdown_connections 2024-02-26T06:35:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.251+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfdbc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.251+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb0d80 0x5653cfdb1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.251+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/694953114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb1af0 0x5653cfdb0840 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a84004740 tx=0x7f9a84004770 comp rx=0 tx=0).stop 2024-02-26T06:35:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.251+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/694953114 >> 172.21.15.49:0/694953114 conn(0x5653cfd78480 msgr2=0x5653cfdaba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.251+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/694953114 shutdown_connections 2024-02-26T06:35:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.252+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/694953114 wait complete. 2024-02-26T06:35:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.252+0000 7f9aa4bc5740 1 -- start start 2024-02-26T06:35:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.252+0000 7f9aa4bc5740 1 Processor -- start 2024-02-26T06:35:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdb7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdba300 0x5653cfdb7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfd175a0 con 0x5653cfdba300 2024-02-26T06:35:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfcbcb20 con 0x5653cfdb0d80 2024-02-26T06:35:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfcbc7c0 con 0x5653cfdb7e90 2024-02-26T06:35:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9aa4bc5740 10 client.0 _ll_register_callbacks cb 0x5653cfcd0520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9a9b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9a9b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49432/0 (socket says 172.21.15.49:49432) 2024-02-26T06:35:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.255+0000 7f9a9b400640 1 -- 172.21.15.49:0/698306712 learned_addr learned my addr 172.21.15.49:0/698306712 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.256+0000 7f9a9aa00640 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdb7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.256+0000 7f9a9b400640 1 -- 172.21.15.49:0/698306712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 msgr2=0x5653cfdb7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.256+0000 7f9a9b400640 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdb7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.256+0000 7f9a9b400640 1 -- 172.21.15.49:0/698306712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdba300 msgr2=0x5653cfdb7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.256+0000 7f9a9b400640 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdba300 0x5653cfdb7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.256+0000 7f9a9b400640 1 -- 172.21.15.49:0/698306712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c10ef50 con 0x5653cfdb7e90 2024-02-26T06:35:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.256+0000 7f9a9b400640 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc6060 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a94010ab0 tx=0x7f9a94015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4afdae777d9ce890 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.257+0000 7f9a88c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:35:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.259+0000 7f9a88c00640 1 -- 172.21.15.49:0/698306712 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9401cd20 con 0x5653cfdb7e90 2024-02-26T06:35:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.259+0000 7f9a88c00640 1 -- 172.21.15.49:0/698306712 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f9a9401ce80 con 0x5653cfdb7e90 2024-02-26T06:35:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.261+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/698306712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653cfce9ca0 con 0x5653cfdb7e90 2024-02-26T06:35:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.261+0000 7f9a88c00640 1 -- 172.21.15.49:0/698306712 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a9401cd20 con 0x5653cfdb7e90 2024-02-26T06:35:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.262+0000 7f9aa4bc5740 10 client.27448 fetch_fsmap learned FSMap version 48 2024-02-26T06:35:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.262+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/698306712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5653cfd15c20 con 0x5653cfdb7e90 2024-02-26T06:35:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.262+0000 7f9a88c00640 1 -- 172.21.15.49:0/698306712 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a940035f0 con 0x5653cfdb7e90 2024-02-26T06:35:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.262+0000 7f9a88c00640 1 -- 172.21.15.49:0/698306712 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9a94025440 con 0x5653cfdb7e90 2024-02-26T06:35:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.263+0000 7f9aa4bc5740 10 client.27448 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:35:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.263+0000 7f9aa4bc5740 10 client.27448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.263+0000 7f9aa4bc5740 20 client.27448 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.263+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653cfdca6f0 0x5653cfdccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.263+0000 7f9aa4bc5740 4 client.27448 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:35:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.263+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/698306712 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653cfce9ca0 con 0x5653cfdca6f0 2024-02-26T06:35:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.264+0000 7f9a9aa00640 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653cfdca6f0 0x5653cfdccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.265+0000 7f9a9aa00640 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653cfdca6f0 0x5653cfdccb30 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9758c1ebaae63902 server_cookie=81f954402833ba4 in_seq=0 out_seq=0 2024-02-26T06:35:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.265+0000 7f9a88c00640 10 client.27448 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.265+0000 7f9a88c00640 1 -- 172.21.15.49:0/698306712 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653cfce9ca0 con 0x5653cfdca6f0 2024-02-26T06:35:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.265+0000 7f9a88c00640 10 client.27448 handle_command_reply: tid=0 2024-02-26T06:35:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.266+0000 7f9aa4bc5740 1 client.27448 shutdown 2024-02-26T06:35:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.266+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/698306712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 msgr2=0x5653cfdc6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.266+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc6060 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a94010ab0 tx=0x7f9a94015040 comp rx=0 tx=0).stop 2024-02-26T06:35:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.267+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/698306712 shutdown_connections 2024-02-26T06:35:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.267+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653cfdca6f0 0x5653cfdccb30 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.267+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc6060 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.267+0000 7f9a88c00640 0 client.27448 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.267+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdba300 0x5653cfdb7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.267+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/698306712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdb7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.267+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/698306712 >> 172.21.15.49:0/698306712 conn(0x5653cfd78480 msgr2=0x5653cfdad7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.268+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/698306712 shutdown_connections 2024-02-26T06:35:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.268+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/698306712 wait complete. 2024-02-26T06:35:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.268+0000 7f9aa4bc5740 20 client.27448 trim_cache size 0 max 16384 2024-02-26T06:35:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.302+0000 7f9aa4bc5740 1 Processor -- start 2024-02-26T06:35:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.302+0000 7f9aa4bc5740 1 -- start start 2024-02-26T06:35:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.303+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdc0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.303+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb7e90 0x5653cfdc14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.303+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdba300 0x5653cfdb5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.303+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfa5fe60 con 0x5653cfdb7e90 2024-02-26T06:35:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.303+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfd9d6a0 con 0x5653cfdb0d80 2024-02-26T06:35:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.303+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfdb7250 con 0x5653cfdba300 2024-02-26T06:35:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.304+0000 7f9a9aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdc0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.304+0000 7f9a9aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdc0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:37398/0 (socket says 172.21.15.49:37398) 2024-02-26T06:35:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.304+0000 7f9a9aa00640 1 -- 172.21.15.49:0/1814405309 learned_addr learned my addr 172.21.15.49:0/1814405309 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.304+0000 7f9a9b400640 1 --2- 172.21.15.49:0/1814405309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdba300 0x5653cfdb5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.305+0000 7f9a9b400640 1 -- 172.21.15.49:0/1814405309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 msgr2=0x5653cfdc0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.305+0000 7f9a9b400640 1 --2- 172.21.15.49:0/1814405309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdc0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.305+0000 7f9a9b400640 1 -- 172.21.15.49:0/1814405309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb7e90 msgr2=0x5653cfdc14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.305+0000 7f9a9b400640 1 --2- 172.21.15.49:0/1814405309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb7e90 0x5653cfdc14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.306+0000 7f9a9b400640 1 -- 172.21.15.49:0/1814405309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653cfcb2500 con 0x5653cfdba300 2024-02-26T06:35:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.307+0000 7f9a9b400640 1 --2- 172.21.15.49:0/1814405309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdba300 0x5653cfdb5d00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9a94025870 tx=0x7f9a94010f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296d82fa83aeb9b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.308+0000 7f9a8b400640 1 -- 172.21.15.49:0/1814405309 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400aa20 con 0x5653cfdba300 2024-02-26T06:35:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.308+0000 7f9a8b400640 1 -- 172.21.15.49:0/1814405309 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a9401cd20 con 0x5653cfdba300 2024-02-26T06:35:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.308+0000 7f9a8b400640 1 -- 172.21.15.49:0/1814405309 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940255c0 con 0x5653cfdba300 2024-02-26T06:35:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.310+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/1814405309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdba300 msgr2=0x5653cfdb5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.310+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/1814405309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdba300 0x5653cfdb5d00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9a94025870 tx=0x7f9a94010f70 comp rx=0 tx=0).stop 2024-02-26T06:35:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.311+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/1814405309 shutdown_connections 2024-02-26T06:35:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.311+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/1814405309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdba300 0x5653cfdb5d00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.311+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/1814405309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb7e90 0x5653cfdc14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.311+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/1814405309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdb0d80 0x5653cfdc0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.311+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/1814405309 >> 172.21.15.49:0/1814405309 conn(0x5653cfd78480 msgr2=0x5653cfdab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.311+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/1814405309 shutdown_connections 2024-02-26T06:35:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.312+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/1814405309 wait complete. 2024-02-26T06:35:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.313+0000 7f9aa4bc5740 1 -- start start 2024-02-26T06:35:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.313+0000 7f9aa4bc5740 1 Processor -- start 2024-02-26T06:35:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.316+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb0d80 0x5653cfdc0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9aa4bc5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfe20c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfe211a0 con 0x5653cfdb0d80 2024-02-26T06:35:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfe212e0 con 0x5653cfdba300 2024-02-26T06:35:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9aa4bc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cfe21420 con 0x5653cfdb7e90 2024-02-26T06:35:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9aa4bc5740 10 client.0 _ll_register_callbacks cb 0x7f9a9400af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9a9b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfe20c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9a9b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfe20c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:37406/0 (socket says 172.21.15.49:37406) 2024-02-26T06:35:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.317+0000 7f9a9b400640 1 -- 172.21.15.49:0/3887131310 learned_addr learned my addr 172.21.15.49:0/3887131310 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.318+0000 7f9a92000640 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.318+0000 7f9a9b400640 1 -- 172.21.15.49:0/3887131310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 msgr2=0x5653cfdc06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.318+0000 7f9a9b400640 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.318+0000 7f9a9b400640 1 -- 172.21.15.49:0/3887131310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb0d80 msgr2=0x5653cfdc0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.318+0000 7f9a9b400640 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb0d80 0x5653cfdc0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.318+0000 7f9a9b400640 1 -- 172.21.15.49:0/3887131310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653cfd15c20 con 0x5653cfdba300 2024-02-26T06:35:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.318+0000 7f9a9b400640 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfe20c60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400a3a0 tx=0x7f9a940256f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c0fac12e1963766 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.318+0000 7f9a88c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:35:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.319+0000 7f9a88c00640 1 -- 172.21.15.49:0/3887131310 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400aac0 con 0x5653cfdba300 2024-02-26T06:35:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.319+0000 7f9a88c00640 1 -- 172.21.15.49:0/3887131310 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f9a9401cb20 con 0x5653cfdba300 2024-02-26T06:35:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.319+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/3887131310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653cfd3cd00 con 0x5653cfdba300 2024-02-26T06:35:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.319+0000 7f9a88c00640 1 -- 172.21.15.49:0/3887131310 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a94004020 con 0x5653cfdba300 2024-02-26T06:35:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.320+0000 7f9aa4bc5740 10 client.27497 fetch_fsmap learned FSMap version 48 2024-02-26T06:35:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.320+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/3887131310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5653cfcd8290 con 0x5653cfdba300 2024-02-26T06:35:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.320+0000 7f9a88c00640 1 -- 172.21.15.49:0/3887131310 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5653cfd3cd00 con 0x5653cfdba300 2024-02-26T06:35:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.321+0000 7f9a88c00640 1 -- 172.21.15.49:0/3887131310 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9a94040070 con 0x5653cfdba300 2024-02-26T06:35:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.321+0000 7f9aa4bc5740 10 client.27497 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:35:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.321+0000 7f9aa4bc5740 10 client.27497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.321+0000 7f9aa4bc5740 20 client.27497 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.321+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653cfe22950 0x5653cfe24d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.321+0000 7f9aa4bc5740 4 client.27497 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:35:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.321+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/3887131310 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5653cfd0dcb0 con 0x5653cfe22950 2024-02-26T06:35:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.322+0000 7f9a9aa00640 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653cfe22950 0x5653cfe24d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.324+0000 7f9a9aa00640 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653cfe22950 0x5653cfe24d70 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b13dfb5bf38d7f97 server_cookie=852a651f30026c19 in_seq=0 out_seq=0 2024-02-26T06:35:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.324+0000 7f9a88c00640 10 client.27497 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9a88c00640 1 -- 172.21.15.49:0/3887131310 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5653cfd0dcb0 con 0x5653cfe22950 2024-02-26T06:35:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9a88c00640 10 client.27497 handle_command_reply: tid=0 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 client.27497 shutdown 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/3887131310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 msgr2=0x5653cfe20c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfe20c60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400a3a0 tx=0x7f9a940256f0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/3887131310 shutdown_connections 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5653cfe22950 0x5653cfe24d70 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653cfdba300 0x5653cfe20c60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5653cfdb7e90 0x5653cfdc06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 --2- 172.21.15.49:0/3887131310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5653cfdb0d80 0x5653cfdc0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/3887131310 >> 172.21.15.49:0/3887131310 conn(0x5653cfd78480 msgr2=0x5653cfdc5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.349+0000 7f9a88c00640 0 client.27497 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.350+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/3887131310 shutdown_connections 2024-02-26T06:35:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.350+0000 7f9aa4bc5740 1 -- 172.21.15.49:0/3887131310 wait complete. 2024-02-26T06:35:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.350+0000 7f9aa4bc5740 20 client.27497 trim_cache size 0 max 16384 2024-02-26T06:35:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:35:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:35:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:35:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:35:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 -- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a800a2350 msgr2=0x7f9a800a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a800a2350 0x7f9a800a4810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400ad90 tx=0x7f9a940137d0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 -- 172.21.15.49:0/2432386656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a9c10d210 msgr2=0x7f9a9c115720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a9c10d210 0x7f9a9c115720 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00e9f0 tx=0x7f9a8c00eeb0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 -- 172.21.15.49:0/2432386656 shutdown_connections 2024-02-26T06:35:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f9a800a2350 0x7f9a800a4810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a9c1a8230 0x7f9a9c1a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a9c10d210 0x7f9a9c115720 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 --2- 172.21.15.49:0/2432386656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a9c1086c0 0x7f9a9c1151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.351+0000 7f9aa1c00640 1 -- 172.21.15.49:0/2432386656 >> 172.21.15.49:0/2432386656 conn(0x7f9a9c06c6b0 msgr2=0x7f9a9c0701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.352+0000 7f9aa1c00640 1 -- 172.21.15.49:0/2432386656 shutdown_connections 2024-02-26T06:35:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:14.352+0000 7f9aa1c00640 1 -- 172.21.15.49:0/2432386656 wait complete. 2024-02-26T06:35:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:15 smithi080 ceph-mon[36206]: pgmap v369: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-02-26T06:35:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:15 smithi080 ceph-mon[36206]: osdmap e101: 12 total, 12 up, 12 in 2024-02-26T06:35:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:15 smithi115 ceph-mon[36426]: pgmap v369: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-02-26T06:35:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:15 smithi115 ceph-mon[36426]: osdmap e101: 12 total, 12 up, 12 in 2024-02-26T06:35:15.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:15 smithi049 ceph-mon[31696]: pgmap v369: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-02-26T06:35:15.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:15 smithi049 ceph-mon[31696]: osdmap e101: 12 total, 12 up, 12 in 2024-02-26T06:35:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 685.0 s, 1686.4 tps, lat 59.324 ms stddev 67.607 2024-02-26T06:35:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:17 smithi080 ceph-mon[36206]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.6 MiB/s wr, 948 op/s 2024-02-26T06:35:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:17 smithi115 ceph-mon[36426]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.6 MiB/s wr, 948 op/s 2024-02-26T06:35:17.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:17 smithi049 ceph-mon[31696]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.6 MiB/s wr, 948 op/s 2024-02-26T06:35:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:19 smithi080 ceph-mon[36206]: pgmap v372: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-02-26T06:35:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:19 smithi115 ceph-mon[36426]: pgmap v372: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-02-26T06:35:19.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:19 smithi049 ceph-mon[31696]: pgmap v372: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-02-26T06:35:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 690.0 s, 1191.8 tps, lat 83.302 ms stddev 92.716 2024-02-26T06:35:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:21 smithi080 ceph-mon[36206]: pgmap v373: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-02-26T06:35:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:21 smithi115 ceph-mon[36426]: pgmap v373: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-02-26T06:35:21.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:21 smithi049 ceph-mon[31696]: pgmap v373: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-02-26T06:35:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:23 smithi080 ceph-mon[36206]: pgmap v374: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 659 op/s 2024-02-26T06:35:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:23 smithi115 ceph-mon[36426]: pgmap v374: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 659 op/s 2024-02-26T06:35:23.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:23 smithi049 ceph-mon[31696]: pgmap v374: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 659 op/s 2024-02-26T06:35:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:25 smithi080 ceph-mon[36206]: pgmap v375: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 633 op/s 2024-02-26T06:35:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:25 smithi115 ceph-mon[36426]: pgmap v375: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 633 op/s 2024-02-26T06:35:25.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:25 smithi049 ceph-mon[31696]: pgmap v375: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 633 op/s 2024-02-26T06:35:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 695.0 s, 1392.4 tps, lat 72.849 ms stddev 88.161 2024-02-26T06:35:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.540+0000 7f6528c00640 1 -- 172.21.15.49:0/437733837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073520 msgr2=0x7f6524073900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.540+0000 7f6528c00640 1 --2- 172.21.15.49:0/437733837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073520 0x7f6524073900 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6514009900 tx=0x7f651402bcc0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.540+0000 7f6528c00640 1 -- 172.21.15.49:0/437733837 shutdown_connections 2024-02-26T06:35:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.540+0000 7f6528c00640 1 --2- 172.21.15.49:0/437733837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6524073e40 0x7f6524111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.540+0000 7f6528c00640 1 --2- 172.21.15.49:0/437733837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6524112140 0x7f6524114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.540+0000 7f6528c00640 1 --2- 172.21.15.49:0/437733837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073520 0x7f6524073900 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.540+0000 7f6528c00640 1 -- 172.21.15.49:0/437733837 >> 172.21.15.49:0/437733837 conn(0x7f65240fcc40 msgr2=0x7f65240ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.541+0000 7f6528c00640 1 -- 172.21.15.49:0/437733837 shutdown_connections 2024-02-26T06:35:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.541+0000 7f6528c00640 1 -- 172.21.15.49:0/437733837 wait complete. 2024-02-26T06:35:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.542+0000 7f6528c00640 1 Processor -- start 2024-02-26T06:35:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.543+0000 7f6528c00640 1 -- start start 2024-02-26T06:35:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.543+0000 7f6528c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6524073520 0x7f652419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.543+0000 7f6528c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073e40 0x7f652419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.543+0000 7f6528c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6524112140 0x7f65241a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.543+0000 7f6528c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65241a2c20 con 0x7f6524073520 2024-02-26T06:35:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6528c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65241a2d90 con 0x7f6524112140 2024-02-26T06:35:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6528c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65241a2f50 con 0x7f6524073e40 2024-02-26T06:35:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6521400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073e40 0x7f652419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6521e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6524073520 0x7f652419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6521400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073e40 0x7f652419e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:52842/0 (socket says 172.21.15.49:52842) 2024-02-26T06:35:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6522800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6524112140 0x7f65241a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6521400640 1 -- 172.21.15.49:0/2547877334 learned_addr learned my addr 172.21.15.49:0/2547877334 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6522800640 1 -- 172.21.15.49:0/2547877334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073e40 msgr2=0x7f652419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6522800640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073e40 0x7f652419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.544+0000 7f6522800640 1 -- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6524073520 msgr2=0x7f652419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.545+0000 7f6522800640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6524073520 0x7f652419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.545+0000 7f6522800640 1 -- 172.21.15.49:0/2547877334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514009560 con 0x7f6524112140 2024-02-26T06:35:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.545+0000 7f6521e00640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6524073520 0x7f652419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:35:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.545+0000 7f6522800640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6524112140 0x7f65241a2620 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f651c005fd0 tx=0x7f651c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.546+0000 7f651aa00640 1 -- 172.21.15.49:0/2547877334 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00cdf0 con 0x7f6524112140 2024-02-26T06:35:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.546+0000 7f651aa00640 1 -- 172.21.15.49:0/2547877334 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f651c005020 con 0x7f6524112140 2024-02-26T06:35:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.546+0000 7f6528c00640 1 -- 172.21.15.49:0/2547877334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65241a3230 con 0x7f6524112140 2024-02-26T06:35:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.546+0000 7f651aa00640 1 -- 172.21.15.49:0/2547877334 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c010690 con 0x7f6524112140 2024-02-26T06:35:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.546+0000 7f6528c00640 1 -- 172.21.15.49:0/2547877334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65241aaab0 con 0x7f6524112140 2024-02-26T06:35:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.550+0000 7f652bba0740 1 Processor -- start 2024-02-26T06:35:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.550+0000 7f651aa00640 1 -- 172.21.15.49:0/2547877334 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f651c005190 con 0x7f6524112140 2024-02-26T06:35:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.550+0000 7f652bba0740 1 -- start start 2024-02-26T06:35:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.551+0000 7f652bba0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db35840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.551+0000 7f652bba0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db35d80 0x56114db36200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.551+0000 7f652bba0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db3f300 0x56114db416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.551+0000 7f652bba0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114daf15f0 con 0x56114db35d80 2024-02-26T06:35:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.551+0000 7f652bba0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114daa3890 con 0x56114db36af0 2024-02-26T06:35:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.551+0000 7f652bba0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114daa0dc0 con 0x56114db3f300 2024-02-26T06:35:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.551+0000 7f6521400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db35840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.551+0000 7f6521e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db35d80 0x56114db36200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.552+0000 7f6521e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db35d80 0x56114db36200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52086/0 (socket says 172.21.15.49:52086) 2024-02-26T06:35:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.552+0000 7f6521e00640 1 -- 172.21.15.49:0/2994139880 learned_addr learned my addr 172.21.15.49:0/2994139880 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.552+0000 7f651aa00640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65080a2350 0x7f65080a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.552+0000 7f6521400640 1 --2- 172.21.15.49:0/2994139880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db3f300 0x56114db416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.552+0000 7f651aa00640 1 -- 172.21.15.49:0/2547877334 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f651c014070 con 0x7f6524112140 2024-02-26T06:35:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.553+0000 7f6521e00640 1 -- 172.21.15.49:0/2994139880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db3f300 msgr2=0x56114db416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.553+0000 7f6521e00640 1 --2- 172.21.15.49:0/2994139880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db3f300 0x56114db416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.553+0000 7f6521e00640 1 -- 172.21.15.49:0/2994139880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 msgr2=0x56114db35840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.553+0000 7f6521e00640 1 --2- 172.21.15.49:0/2994139880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db35840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.553+0000 7f6521e00640 1 -- 172.21.15.49:0/2994139880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114da1dad0 con 0x56114db35d80 2024-02-26T06:35:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.554+0000 7f6522800640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65080a2350 0x7f65080a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.555+0000 7f6521e00640 1 --2- 172.21.15.49:0/2994139880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db35d80 0x56114db36200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6514009900 tx=0x7f651402c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2660b6dc95e6fd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.555+0000 7f6522800640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65080a2350 0x7f65080a4810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f651c0cbed0 tx=0x7f651c005900 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.556+0000 7f650f000640 1 -- 172.21.15.49:0/2994139880 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514004040 con 0x56114db35d80 2024-02-26T06:35:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.556+0000 7f650f000640 1 -- 172.21.15.49:0/2994139880 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f651402c870 con 0x56114db35d80 2024-02-26T06:35:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.556+0000 7f650f000640 1 -- 172.21.15.49:0/2994139880 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514005230 con 0x56114db35d80 2024-02-26T06:35:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.557+0000 7f652bba0740 1 -- 172.21.15.49:0/2994139880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db35d80 msgr2=0x56114db36200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.557+0000 7f652bba0740 1 --2- 172.21.15.49:0/2994139880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db35d80 0x56114db36200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6514009900 tx=0x7f651402c700 comp rx=0 tx=0).stop 2024-02-26T06:35:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.558+0000 7f652bba0740 1 -- 172.21.15.49:0/2994139880 shutdown_connections 2024-02-26T06:35:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.558+0000 7f652bba0740 1 --2- 172.21.15.49:0/2994139880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db3f300 0x56114db416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.558+0000 7f652bba0740 1 --2- 172.21.15.49:0/2994139880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db35d80 0x56114db36200 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.558+0000 7f652bba0740 1 --2- 172.21.15.49:0/2994139880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db35840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.558+0000 7f652bba0740 1 -- 172.21.15.49:0/2994139880 >> 172.21.15.49:0/2994139880 conn(0x56114dafd480 msgr2=0x56114db30a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.561+0000 7f652bba0740 1 -- 172.21.15.49:0/2994139880 shutdown_connections 2024-02-26T06:35:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.561+0000 7f652bba0740 1 -- 172.21.15.49:0/2994139880 wait complete. 2024-02-26T06:35:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.562+0000 7f652bba0740 1 -- start start 2024-02-26T06:35:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.562+0000 7f652bba0740 1 Processor -- start 2024-02-26T06:35:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.564+0000 7f652bba0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db35d80 0x56114db3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.564+0000 7f652bba0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db36af0 0x56114db3c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.565+0000 7f652bba0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114db3ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.565+0000 7f652bba0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114da9c5a0 con 0x56114db3f300 2024-02-26T06:35:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.565+0000 7f652bba0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114da41b20 con 0x56114db35d80 2024-02-26T06:35:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.565+0000 7f652bba0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114da417c0 con 0x56114db36af0 2024-02-26T06:35:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.565+0000 7f652bba0740 10 client.0 _ll_register_callbacks cb 0x56114da55520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db35d80 0x56114db3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db35d80 0x56114db3c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39204/0 (socket says 172.21.15.49:39204) 2024-02-26T06:35:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521400640 1 -- 172.21.15.49:0/1791994667 learned_addr learned my addr 172.21.15.49:0/1791994667 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521e00640 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db36af0 0x56114db3c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521400640 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114db3ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521e00640 1 -- 172.21.15.49:0/1791994667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db35d80 msgr2=0x56114db3c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521e00640 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db35d80 0x56114db3c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521e00640 1 -- 172.21.15.49:0/1791994667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 msgr2=0x56114db3ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521e00640 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114db3ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.566+0000 7f6521e00640 1 -- 172.21.15.49:0/1791994667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6524114e60 con 0x56114db36af0 2024-02-26T06:35:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.567+0000 7f6521e00640 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db36af0 0x56114db3c940 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6514036040 tx=0x7f651402dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd15d059d1ef1729 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.567+0000 7f6503400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:35:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.571+0000 7f6503400640 1 -- 172.21.15.49:0/1791994667 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514005380 con 0x56114db36af0 2024-02-26T06:35:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.572+0000 7f6503400640 1 -- 172.21.15.49:0/1791994667 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f65140054e0 con 0x56114db36af0 2024-02-26T06:35:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.572+0000 7f652bba0740 1 -- 172.21.15.49:0/1791994667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56114da6eca0 con 0x56114db36af0 2024-02-26T06:35:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.572+0000 7f6503400640 1 -- 172.21.15.49:0/1791994667 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65140035f0 con 0x56114db36af0 2024-02-26T06:35:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.575+0000 7f652bba0740 10 client.27463 fetch_fsmap learned FSMap version 48 2024-02-26T06:35:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.575+0000 7f652bba0740 1 -- 172.21.15.49:0/1791994667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56114da9ac20 con 0x56114db36af0 2024-02-26T06:35:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.575+0000 7f6503400640 1 -- 172.21.15.49:0/1791994667 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56114da6eca0 con 0x56114db36af0 2024-02-26T06:35:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.576+0000 7f6503400640 1 -- 172.21.15.49:0/1791994667 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f651404e030 con 0x56114db36af0 2024-02-26T06:35:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.579+0000 7f652bba0740 10 client.27463 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:35:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.579+0000 7f652bba0740 10 client.27463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.579+0000 7f652bba0740 20 client.27463 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.579+0000 7f652bba0740 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56114db4f130 0x56114db51550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.579+0000 7f652bba0740 4 client.27463 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:35:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.579+0000 7f652bba0740 1 -- 172.21.15.49:0/1791994667 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56114da92cb0 con 0x56114db4f130 2024-02-26T06:35:27.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:27 smithi080 ceph-mon[36206]: pgmap v376: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-02-26T06:35:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.581+0000 7f6521400640 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56114db4f130 0x56114db51550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.586+0000 7f6503400640 10 client.27463 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.586+0000 7f6521400640 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56114db4f130 0x56114db51550 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c80acf655b4e41d server_cookie=ba5fc3dee0ec87d3 in_seq=0 out_seq=0 2024-02-26T06:35:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.587+0000 7f6503400640 1 -- 172.21.15.49:0/1791994667 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56114da92cb0 con 0x56114db4f130 2024-02-26T06:35:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.587+0000 7f6503400640 10 client.27463 handle_command_reply: tid=0 2024-02-26T06:35:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.588+0000 7f652bba0740 1 client.27463 shutdown 2024-02-26T06:35:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.589+0000 7f652bba0740 1 -- 172.21.15.49:0/1791994667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db36af0 msgr2=0x56114db3c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.589+0000 7f652bba0740 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db36af0 0x56114db3c940 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6514036040 tx=0x7f651402dab0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f652bba0740 1 -- 172.21.15.49:0/1791994667 shutdown_connections 2024-02-26T06:35:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f652bba0740 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56114db4f130 0x56114db51550 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f652bba0740 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114db3ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f652bba0740 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db36af0 0x56114db3c940 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f652bba0740 1 --2- 172.21.15.49:0/1791994667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db35d80 0x56114db3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f652bba0740 1 -- 172.21.15.49:0/1791994667 >> 172.21.15.49:0/1791994667 conn(0x56114dafd480 msgr2=0x56114db327c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f6503400640 0 client.27463 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f652bba0740 1 -- 172.21.15.49:0/1791994667 shutdown_connections 2024-02-26T06:35:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:27 smithi115 ceph-mon[36426]: pgmap v376: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-02-26T06:35:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.590+0000 7f652bba0740 1 -- 172.21.15.49:0/1791994667 wait complete. 2024-02-26T06:35:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.591+0000 7f652bba0740 20 client.27463 trim_cache size 0 max 16384 2024-02-26T06:35:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.623+0000 7f652bba0740 1 Processor -- start 2024-02-26T06:35:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.624+0000 7f652bba0740 1 -- start start 2024-02-26T06:35:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f652bba0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db3eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f652bba0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db3cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f652bba0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114db460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f652bba0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114d7e4e60 con 0x56114db3f300 2024-02-26T06:35:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f652bba0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114da92ce0 con 0x56114db36af0 2024-02-26T06:35:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f652bba0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114db37250 con 0x56114db35d80 2024-02-26T06:35:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f6521400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db3eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f6521e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db3cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f6521e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db3cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39206/0 (socket says 172.21.15.49:39206) 2024-02-26T06:35:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f6521e00640 1 -- 172.21.15.49:0/2584328137 learned_addr learned my addr 172.21.15.49:0/2584328137 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.626+0000 7f6521e00640 1 -- 172.21.15.49:0/2584328137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 msgr2=0x56114db3eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.625+0000 7f6521400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db3eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:52862/0 (socket says 172.21.15.49:52862) 2024-02-26T06:35:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.626+0000 7f6521e00640 1 --2- 172.21.15.49:0/2584328137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db3eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.626+0000 7f6521e00640 1 -- 172.21.15.49:0/2584328137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 msgr2=0x56114db460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.626+0000 7f6521e00640 1 --2- 172.21.15.49:0/2584328137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114db460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.626+0000 7f6521e00640 1 -- 172.21.15.49:0/2584328137 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114da37500 con 0x56114db36af0 2024-02-26T06:35:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.626+0000 7f6521400640 1 --2- 172.21.15.49:0/2584328137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db3eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-26T06:35:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.627+0000 7f6521e00640 1 --2- 172.21.15.49:0/2584328137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db3cca0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6514004be0 tx=0x7f651402bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a015bb3665177101 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.627+0000 7f650e600640 1 -- 172.21.15.49:0/2584328137 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514004df0 con 0x56114db36af0 2024-02-26T06:35:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.628+0000 7f650e600640 1 -- 172.21.15.49:0/2584328137 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6514005d50 con 0x56114db36af0 2024-02-26T06:35:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.628+0000 7f650e600640 1 -- 172.21.15.49:0/2584328137 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514004df0 con 0x56114db36af0 2024-02-26T06:35:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 -- 172.21.15.49:0/2584328137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 msgr2=0x56114db3cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 --2- 172.21.15.49:0/2584328137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db3cca0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6514004be0 tx=0x7f651402bcc0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 -- 172.21.15.49:0/2584328137 shutdown_connections 2024-02-26T06:35:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 --2- 172.21.15.49:0/2584328137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db3cca0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 --2- 172.21.15.49:0/2584328137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114db460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 --2- 172.21.15.49:0/2584328137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db3eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 -- 172.21.15.49:0/2584328137 >> 172.21.15.49:0/2584328137 conn(0x56114dafd480 msgr2=0x56114db305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 -- 172.21.15.49:0/2584328137 shutdown_connections 2024-02-26T06:35:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.629+0000 7f652bba0740 1 -- 172.21.15.49:0/2584328137 wait complete. 2024-02-26T06:35:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.630+0000 7f652bba0740 1 -- start start 2024-02-26T06:35:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.630+0000 7f652bba0740 1 Processor -- start 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.631+0000 7f652bba0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db42290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.631+0000 7f652bba0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.631+0000 7f652bba0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114dba4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.631+0000 7f652bba0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114db42f30 con 0x56114db3f300 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.631+0000 7f652bba0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114dba4b60 con 0x56114db36af0 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.631+0000 7f652bba0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114dbb5d30 con 0x56114db35d80 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.631+0000 7f652bba0740 10 client.0 _ll_register_callbacks cb 0x7f6510009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.632+0000 7f6521e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.632+0000 7f6521e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db427d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:39210/0 (socket says 172.21.15.49:39210) 2024-02-26T06:35:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.632+0000 7f6521e00640 1 -- 172.21.15.49:0/2275305643 learned_addr learned my addr 172.21.15.49:0/2275305643 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.632+0000 7f6521400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db42290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.632+0000 7f6521400640 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114dba4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6521e00640 1 -- 172.21.15.49:0/2275305643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 msgr2=0x56114db42290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6521e00640 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db42290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6521e00640 1 -- 172.21.15.49:0/2275305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 msgr2=0x56114dba4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6521e00640 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114dba4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6521e00640 1 -- 172.21.15.49:0/2275305643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56114da9ac20 con 0x56114db36af0 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6521400640 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114dba4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6521e00640 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db427d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6514000c00 tx=0x7f6514004e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0d2fce1946846d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6503400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6503400640 1 -- 172.21.15.49:0/2275305643 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514005590 con 0x56114db36af0 2024-02-26T06:35:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.633+0000 7f6503400640 1 -- 172.21.15.49:0/2275305643 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6514000e40 con 0x56114db36af0 2024-02-26T06:35:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 1 -- 172.21.15.49:0/2275305643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56114dac1d00 con 0x56114db36af0 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f6503400640 1 -- 172.21.15.49:0/2275305643 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65140056f0 con 0x56114db36af0 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 10 client.27515 fetch_fsmap learned FSMap version 48 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 1 -- 172.21.15.49:0/2275305643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56114da5d290 con 0x56114db36af0 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f6503400640 1 -- 172.21.15.49:0/2275305643 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56114dac1d00 con 0x56114db36af0 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f6503400640 1 -- 172.21.15.49:0/2275305643 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 48) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6514062070 con 0x56114db36af0 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 10 client.27515 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 10 client.27515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 20 client.27515 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56114dbb70b0 0x56114dbb94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 4 client.27515 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:35:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.634+0000 7f652bba0740 1 -- 172.21.15.49:0/2275305643 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56114d79c600 con 0x56114dbb70b0 2024-02-26T06:35:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.635+0000 7f6521400640 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56114dbb70b0 0x56114dbb94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.635+0000 7f6521400640 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56114dbb70b0 0x56114dbb94a0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfecbb295e37ccad server_cookie=c6e1d2e68c36469f in_seq=0 out_seq=0 2024-02-26T06:35:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.635+0000 7f6503400640 10 client.27515 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f6503400640 1 -- 172.21.15.49:0/2275305643 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56114d79c600 con 0x56114dbb70b0 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f6503400640 10 client.27515 handle_command_reply: tid=0 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 client.27515 shutdown 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 -- 172.21.15.49:0/2275305643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 msgr2=0x56114db427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db427d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6514000c00 tx=0x7f6514004e70 comp rx=0 tx=0).stop 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 -- 172.21.15.49:0/2275305643 shutdown_connections 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56114dbb70b0 0x56114dbb94a0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56114db36af0 0x56114db427d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56114db3f300 0x56114dba4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 --2- 172.21.15.49:0/2275305643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56114db35d80 0x56114db42290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 -- 172.21.15.49:0/2275305643 >> 172.21.15.49:0/2275305643 conn(0x56114dafd480 msgr2=0x56114db3ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f6503400640 0 client.27515 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 -- 172.21.15.49:0/2275305643 shutdown_connections 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.657+0000 7f652bba0740 1 -- 172.21.15.49:0/2275305643 wait complete. 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f652bba0740 20 client.27515 trim_cache size 0 max 16384 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:35:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:35:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 -- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65080a2350 msgr2=0x7f65080a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65080a2350 0x7f65080a4810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f651c0cbed0 tx=0x7f651c005900 comp rx=0 tx=0).stop 2024-02-26T06:35:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 -- 172.21.15.49:0/2547877334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6524112140 msgr2=0x7f65241a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6524112140 0x7f65241a2620 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f651c005fd0 tx=0x7f651c00ef10 comp rx=0 tx=0).stop 2024-02-26T06:35:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 -- 172.21.15.49:0/2547877334 shutdown_connections 2024-02-26T06:35:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65080a2350 0x7f65080a4810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6524112140 0x7f65241a2620 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6524073e40 0x7f652419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 --2- 172.21.15.49:0/2547877334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6524073520 0x7f652419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 -- 172.21.15.49:0/2547877334 >> 172.21.15.49:0/2547877334 conn(0x7f65240fcc40 msgr2=0x7f65240fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.658+0000 7f6528c00640 1 -- 172.21.15.49:0/2547877334 shutdown_connections 2024-02-26T06:35:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:27.659+0000 7f6528c00640 1 -- 172.21.15.49:0/2547877334 wait complete. 2024-02-26T06:35:27.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:27 smithi049 ceph-mon[31696]: pgmap v376: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-02-26T06:35:29.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:29 smithi080 ceph-mon[36206]: pgmap v377: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-02-26T06:35:29.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:29 smithi115 ceph-mon[36426]: pgmap v377: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-02-26T06:35:29.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:29 smithi049 ceph-mon[31696]: pgmap v377: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-02-26T06:35:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 700.0 s, 1620.0 tps, lat 61.147 ms stddev 68.335 2024-02-26T06:35:31.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:31 smithi080 ceph-mon[36206]: pgmap v378: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-02-26T06:35:31.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:31 smithi115 ceph-mon[36426]: pgmap v378: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-02-26T06:35:31.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:31 smithi049 ceph-mon[31696]: pgmap v378: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-02-26T06:35:33.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:33 smithi080 ceph-mon[36206]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.8 MiB/s wr, 812 op/s 2024-02-26T06:35:33.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:33 smithi115 ceph-mon[36426]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.8 MiB/s wr, 812 op/s 2024-02-26T06:35:33.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:33 smithi049 ceph-mon[31696]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.8 MiB/s wr, 812 op/s 2024-02-26T06:35:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:35 smithi080 ceph-mon[36206]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-02-26T06:35:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:35 smithi115 ceph-mon[36426]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-02-26T06:35:35.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:35 smithi049 ceph-mon[31696]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-02-26T06:35:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 705.0 s, 1851.0 tps, lat 54.209 ms stddev 65.994 2024-02-26T06:35:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:37 smithi080 ceph-mon[36206]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 724 op/s 2024-02-26T06:35:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:37 smithi115 ceph-mon[36426]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 724 op/s 2024-02-26T06:35:37.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:37 smithi049 ceph-mon[31696]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 724 op/s 2024-02-26T06:35:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:39 smithi080 ceph-mon[36206]: pgmap v382: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 646 op/s 2024-02-26T06:35:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:39 smithi115 ceph-mon[36426]: pgmap v382: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 646 op/s 2024-02-26T06:35:39.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:39 smithi049 ceph-mon[31696]: pgmap v382: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 646 op/s 2024-02-26T06:35:40.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:40 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:35:40.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:40 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:35:40.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:40 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:35:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.844+0000 7feaeae00640 1 -- 172.21.15.49:0/77873893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feae4103f70 msgr2=0x7feae4107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.844+0000 7feaeae00640 1 --2- 172.21.15.49:0/77873893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feae4103f70 0x7feae4107fb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fead0009930 tx=0x7fead002bc30 comp rx=0 tx=0).stop 2024-02-26T06:35:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.844+0000 7feaeae00640 1 -- 172.21.15.49:0/77873893 shutdown_connections 2024-02-26T06:35:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.844+0000 7feaeae00640 1 --2- 172.21.15.49:0/77873893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feae4108690 0x7feae4110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.844+0000 7feaeae00640 1 --2- 172.21.15.49:0/77873893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feae4103f70 0x7feae4107fb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.844+0000 7feaeae00640 1 --2- 172.21.15.49:0/77873893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae41035c0 0x7feae41039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.844+0000 7feaeae00640 1 -- 172.21.15.49:0/77873893 >> 172.21.15.49:0/77873893 conn(0x7feae40fee40 msgr2=0x7feae4101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.844+0000 7feaeae00640 1 -- 172.21.15.49:0/77873893 shutdown_connections 2024-02-26T06:35:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.845+0000 7feaeae00640 1 -- 172.21.15.49:0/77873893 wait complete. 2024-02-26T06:35:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.845+0000 7feaeae00640 1 Processor -- start 2024-02-26T06:35:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.845+0000 7feaeae00640 1 -- start start 2024-02-26T06:35:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feaeae00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae41035c0 0x7feae419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feaeae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feae4108690 0x7feae419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feaeae00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feae4199770 0x7feae4199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feaeae00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae419a240 con 0x7feae4199770 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feaeae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae419a3b0 con 0x7feae4108690 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feaeae00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae419a570 con 0x7feae41035c0 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feae3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feae4108690 0x7feae419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feae3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae41035c0 0x7feae419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feae3e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae41035c0 0x7feae419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34824/0 (socket says 172.21.15.49:34824) 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feae3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feae4108690 0x7feae419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43838/0 (socket says 172.21.15.49:43838) 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feae3e00640 1 -- 172.21.15.49:0/2551161943 learned_addr learned my addr 172.21.15.49:0/2551161943 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.846+0000 7feae8e00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feae4199770 0x7feae4199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae3400640 1 -- 172.21.15.49:0/2551161943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae41035c0 msgr2=0x7feae419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae3400640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae41035c0 0x7feae419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae3400640 1 -- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feae4199770 msgr2=0x7feae4199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae3400640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feae4199770 0x7feae4199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae3400640 1 -- 172.21.15.49:0/2551161943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0009560 con 0x7feae4108690 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae3400640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feae4108690 0x7feae419fbf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fead002c170 tx=0x7fead002c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae0c00640 1 -- 172.21.15.49:0/2551161943 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead00042d0 con 0x7feae4108690 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae0c00640 1 -- 172.21.15.49:0/2551161943 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead0004430 con 0x7feae4108690 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feaeae00640 1 -- 172.21.15.49:0/2551161943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae41a64b0 con 0x7feae4108690 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feae0c00640 1 -- 172.21.15.49:0/2551161943 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0003c90 con 0x7feae4108690 2024-02-26T06:35:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.847+0000 7feaeae00640 1 -- 172.21.15.49:0/2551161943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae41a69e0 con 0x7feae4108690 2024-02-26T06:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.851+0000 7feaedc83740 1 Processor -- start 2024-02-26T06:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.852+0000 7feaedc83740 1 -- start start 2024-02-26T06:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.852+0000 7feaedc83740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe99af0 0x55ec2fe98840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feaedc83740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe99200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feaedc83740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feaedc83740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fe545f0 con 0x55ec2fe99af0 2024-02-26T06:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feaedc83740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fe06890 con 0x55ec2fe98d80 2024-02-26T06:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feaedc83740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fe03dc0 con 0x55ec2fea2300 2024-02-26T06:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feae8e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe99af0 0x55ec2fe98840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feae8e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe99af0 0x55ec2fe98840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49262/0 (socket says 172.21.15.49:49262) 2024-02-26T06:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feae8e00640 1 -- 172.21.15.49:0/1945378338 learned_addr learned my addr 172.21.15.49:0/1945378338 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.853+0000 7feae3e00640 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe99200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7feae3e00640 1 -- 172.21.15.49:0/1945378338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 msgr2=0x55ec2fea46f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7feae3e00640 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea46f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7feae3e00640 1 -- 172.21.15.49:0/1945378338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe99af0 msgr2=0x55ec2fe98840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7feae3e00640 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe99af0 0x55ec2fe98840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7feae3e00640 1 -- 172.21.15.49:0/1945378338 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2fd80ad0 con 0x55ec2fe98d80 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7feae8e00640 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe99af0 0x55ec2fe98840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7feae3e00640 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe99200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fead4009cf0 tx=0x7fead4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=866fca5a10dee175 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7fead9600640 1 -- 172.21.15.49:0/1945378338 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4005220 con 0x55ec2fe98d80 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7fead9600640 1 -- 172.21.15.49:0/1945378338 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead4004c80 con 0x55ec2fe98d80 2024-02-26T06:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.854+0000 7fead9600640 1 -- 172.21.15.49:0/1945378338 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4019890 con 0x55ec2fe98d80 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feae0c00640 1 -- 172.21.15.49:0/2551161943 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fead0004df0 con 0x7feae4108690 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feaedc83740 1 -- 172.21.15.49:0/1945378338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 msgr2=0x55ec2fe99200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feaedc83740 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe99200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fead4009cf0 tx=0x7fead4011810 comp rx=0 tx=0).stop 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feaedc83740 1 -- 172.21.15.49:0/1945378338 shutdown_connections 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feaedc83740 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe99200 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feaedc83740 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feaedc83740 1 --2- 172.21.15.49:0/1945378338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe99af0 0x55ec2fe98840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feaedc83740 1 -- 172.21.15.49:0/1945378338 >> 172.21.15.49:0/1945378338 conn(0x55ec2fe60480 msgr2=0x55ec2fe93a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.855+0000 7feaedc83740 1 -- 172.21.15.49:0/1945378338 shutdown_connections 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.856+0000 7feaedc83740 1 -- 172.21.15.49:0/1945378338 wait complete. 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.856+0000 7feaedc83740 1 -- start start 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.856+0000 7feae0c00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feac80a2260 0x7feac80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.856+0000 7feaedc83740 1 Processor -- start 2024-02-26T06:35:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.856+0000 7feae0c00640 1 -- 172.21.15.49:0/2551161943 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fead00efe90 con 0x7feae4108690 2024-02-26T06:35:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.856+0000 7feae8e00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feac80a2260 0x7feac80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feaedc83740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feaedc83740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fe9f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feaedc83740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fe9fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feaedc83740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fdff5a0 con 0x55ec2fe98d80 2024-02-26T06:35:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feaedc83740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fda4b20 con 0x55ec2fe99af0 2024-02-26T06:35:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feaedc83740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fda47c0 con 0x55ec2fea2300 2024-02-26T06:35:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feaedc83740 10 client.0 _ll_register_callbacks cb 0x55ec2fdb8520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feae3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fe9f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feae3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fe9f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43868/0 (socket says 172.21.15.49:43868) 2024-02-26T06:35:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feae3400640 1 -- 172.21.15.49:0/2882129903 learned_addr learned my addr 172.21.15.49:0/2882129903 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feae3e00640 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.857+0000 7feae8e00640 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fe9fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feae3400640 1 -- 172.21.15.49:0/2882129903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 msgr2=0x55ec2fe9fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feae3400640 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fe9fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feae3400640 1 -- 172.21.15.49:0/2882129903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 msgr2=0x55ec2fe9f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feae3400640 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feae3400640 1 -- 172.21.15.49:0/2882129903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead4009920 con 0x55ec2fe99af0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feae3400640 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fe9f880 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fead00ed730 tx=0x7fead00bc570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e363a19dba4ae server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feace000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feae8e00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feac80a2260 0x7feac80a4720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feadc005fd0 tx=0x7feadc011670 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feace000640 1 -- 172.21.15.49:0/2882129903 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead00c0ac0 con 0x55ec2fe99af0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feace000640 1 -- 172.21.15.49:0/2882129903 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fead00c5110 con 0x55ec2fe99af0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.858+0000 7feaedc83740 1 -- 172.21.15.49:0/2882129903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec2fdd1ca0 con 0x55ec2fe99af0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.859+0000 7feace000640 1 -- 172.21.15.49:0/2882129903 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fead00b8fe0 con 0x55ec2fe99af0 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.859+0000 7feaedc83740 10 client.27527 fetch_fsmap learned FSMap version 49 2024-02-26T06:35:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.859+0000 7feaedc83740 1 -- 172.21.15.49:0/2882129903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ec2fdfdc20 con 0x55ec2fe99af0 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.859+0000 7feace000640 1 -- 172.21.15.49:0/2882129903 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ec2fdd1ca0 con 0x55ec2fe99af0 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.860+0000 7feace000640 1 -- 172.21.15.49:0/2882129903 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fead003c070 con 0x55ec2fe99af0 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.860+0000 7feaedc83740 10 client.27527 fetch_fsmap finished waiting for FSMap version 49 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.860+0000 7feaedc83740 10 client.27527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.860+0000 7feaedc83740 20 client.27527 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.860+0000 7feaedc83740 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2fec3030 0x55ec2fec5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.860+0000 7feaedc83740 4 client.27527 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.860+0000 7feaedc83740 1 -- 172.21.15.49:0/2882129903 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec2fdf5cb0 con 0x55ec2fec3030 2024-02-26T06:35:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.860+0000 7feae3e00640 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2fec3030 0x55ec2fec5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.861+0000 7feae3e00640 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2fec3030 0x55ec2fec5450 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2b5047ec9ea9b server_cookie=3bc1453bac4c77f0 in_seq=0 out_seq=0 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.861+0000 7feace000640 10 client.27527 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.861+0000 7feace000640 1 -- 172.21.15.49:0/2882129903 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec2fdf5cb0 con 0x55ec2fec3030 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.861+0000 7feace000640 10 client.27527 handle_command_reply: tid=0 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.861+0000 7feaedc83740 1 client.27527 shutdown 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.861+0000 7feaedc83740 1 -- 172.21.15.49:0/2882129903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 msgr2=0x55ec2fe9f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.861+0000 7feaedc83740 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fe9f880 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fead00ed730 tx=0x7fead00bc570 comp rx=0 tx=0).stop 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 1 -- 172.21.15.49:0/2882129903 shutdown_connections 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2fec3030 0x55ec2fec5450 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feace000640 0 client.27527 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fe9f880 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fe9fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 1 --2- 172.21.15.49:0/2882129903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 1 -- 172.21.15.49:0/2882129903 >> 172.21.15.49:0/2882129903 conn(0x55ec2fe60480 msgr2=0x55ec2fe957c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 1 -- 172.21.15.49:0/2882129903 shutdown_connections 2024-02-26T06:35:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 1 -- 172.21.15.49:0/2882129903 wait complete. 2024-02-26T06:35:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.862+0000 7feaedc83740 20 client.27527 trim_cache size 0 max 16384 2024-02-26T06:35:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.882+0000 7feaedc83740 1 Processor -- start 2024-02-26T06:35:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.882+0000 7feaedc83740 1 -- start start 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feaedc83740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feaedc83740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fe99af0 0x55ec2fe92b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feaedc83740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fea2300 0x55ec2fea9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feaedc83740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fb47e60 con 0x55ec2fea2300 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feaedc83740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fe9f160 con 0x55ec2fe98d80 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feaedc83740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fe85670 con 0x55ec2fe99af0 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feae3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feae3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fe99af0 0x55ec2fe92b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feae8e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fea2300 0x55ec2fea9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feae3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fe99af0 0x55ec2fe92b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34844/0 (socket says 172.21.15.49:34844) 2024-02-26T06:35:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feae3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43882/0 (socket says 172.21.15.49:43882) 2024-02-26T06:35:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.883+0000 7feae3400640 1 -- 172.21.15.49:0/1449589282 learned_addr learned my addr 172.21.15.49:0/1449589282 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.884+0000 7feae3400640 1 -- 172.21.15.49:0/1449589282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 msgr2=0x55ec2fe9ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.884+0000 7feae3400640 1 --2- 172.21.15.49:0/1449589282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.884+0000 7feae3400640 1 -- 172.21.15.49:0/1449589282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fea2300 msgr2=0x55ec2fea9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.884+0000 7feae3400640 1 --2- 172.21.15.49:0/1449589282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fea2300 0x55ec2fea9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.884+0000 7feae3400640 1 -- 172.21.15.49:0/1449589282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2fd9a500 con 0x55ec2fe99af0 2024-02-26T06:35:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.884+0000 7feae3e00640 1 --2- 172.21.15.49:0/1449589282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:35:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.885+0000 7feae3400640 1 --2- 172.21.15.49:0/1449589282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fe99af0 0x55ec2fe92b90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fead0006010 tx=0x7fead00bd0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26ab8f96f848cf2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.885+0000 7fead8c00640 1 -- 172.21.15.49:0/1449589282 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead00bc740 con 0x55ec2fe99af0 2024-02-26T06:35:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.885+0000 7fead8c00640 1 -- 172.21.15.49:0/1449589282 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead00c5110 con 0x55ec2fe99af0 2024-02-26T06:35:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.886+0000 7fead8c00640 1 -- 172.21.15.49:0/1449589282 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead00c4ec0 con 0x55ec2fe99af0 2024-02-26T06:35:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 -- 172.21.15.49:0/1449589282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fe99af0 msgr2=0x55ec2fe92b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 --2- 172.21.15.49:0/1449589282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fe99af0 0x55ec2fe92b90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fead0006010 tx=0x7fead00bd0f0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 -- 172.21.15.49:0/1449589282 shutdown_connections 2024-02-26T06:35:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 --2- 172.21.15.49:0/1449589282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fea2300 0x55ec2fea9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 --2- 172.21.15.49:0/1449589282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fe99af0 0x55ec2fe92b90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 --2- 172.21.15.49:0/1449589282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe98d80 0x55ec2fe9ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 -- 172.21.15.49:0/1449589282 >> 172.21.15.49:0/1449589282 conn(0x55ec2fe60480 msgr2=0x55ec2fe92480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 -- 172.21.15.49:0/1449589282 shutdown_connections 2024-02-26T06:35:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 -- 172.21.15.49:0/1449589282 wait complete. 2024-02-26T06:35:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.887+0000 7feaedc83740 1 -- start start 2024-02-26T06:35:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.888+0000 7feaedc83740 1 Processor -- start 2024-02-26T06:35:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feaedc83740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 0x55ec2fea9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feaedc83740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fea56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feaedc83740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feaedc83740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fea6160 con 0x55ec2fe98d80 2024-02-26T06:35:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feaedc83740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fea62d0 con 0x55ec2fe99af0 2024-02-26T06:35:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feaedc83740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2fea6440 con 0x55ec2fea2300 2024-02-26T06:35:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feaedc83740 10 client.0 _ll_register_callbacks cb 0x7fead00b8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feae8e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feae8e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34848/0 (socket says 172.21.15.49:34848) 2024-02-26T06:35:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feae3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 0x55ec2fea9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feae8e00640 1 -- 172.21.15.49:0/2082793584 learned_addr learned my addr 172.21.15.49:0/2082793584 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.889+0000 7feae3400640 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fea56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feae8e00640 1 -- 172.21.15.49:0/2082793584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 msgr2=0x55ec2fea56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feae8e00640 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fea56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feae8e00640 1 -- 172.21.15.49:0/2082793584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 msgr2=0x55ec2fea9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feae8e00640 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 0x55ec2fea9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feae8e00640 1 -- 172.21.15.49:0/2082793584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead00d4050 con 0x55ec2fea2300 2024-02-26T06:35:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feae8e00640 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea5c20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7feadc020e40 tx=0x7feadc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8a0c80704d927c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feace000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feace000640 1 -- 172.21.15.49:0/2082793584 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc0228a0 con 0x55ec2fea2300 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feaedc83740 1 -- 172.21.15.49:0/2082793584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec2fe24d00 con 0x55ec2fea2300 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.890+0000 7feace000640 1 -- 172.21.15.49:0/2082793584 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7feadc003e00 con 0x55ec2fea2300 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feace000640 1 -- 172.21.15.49:0/2082793584 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7feadc004030 con 0x55ec2fea2300 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feaedc83740 10 client.27481 fetch_fsmap learned FSMap version 49 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feaedc83740 1 -- 172.21.15.49:0/2082793584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ec2fdc0290 con 0x55ec2fea2300 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feace000640 1 -- 172.21.15.49:0/2082793584 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ec2fe24d00 con 0x55ec2fea2300 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feace000640 1 -- 172.21.15.49:0/2082793584 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7feadc02c7f0 con 0x55ec2fea2300 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feaedc83740 10 client.27481 fetch_fsmap finished waiting for FSMap version 49 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feaedc83740 10 client.27481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feaedc83740 20 client.27481 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feaedc83740 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2ff1aa80 0x55ec2ff1cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feaedc83740 4 client.27481 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:35:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.891+0000 7feaedc83740 1 -- 172.21.15.49:0/2082793584 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ec2faff600 con 0x55ec2ff1aa80 2024-02-26T06:35:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.892+0000 7feae3e00640 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2ff1aa80 0x55ec2ff1cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.892+0000 7feae3e00640 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2ff1aa80 0x55ec2ff1cea0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9f32763ad7ac41 server_cookie=d1a5fa4c8912224 in_seq=0 out_seq=0 2024-02-26T06:35:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.892+0000 7feace000640 10 client.27481 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.907+0000 7feace000640 1 -- 172.21.15.49:0/2082793584 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ec2faff600 con 0x55ec2ff1aa80 2024-02-26T06:35:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.907+0000 7feace000640 10 client.27481 handle_command_reply: tid=0 2024-02-26T06:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.907+0000 7feaedc83740 1 client.27481 shutdown 2024-02-26T06:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 -- 172.21.15.49:0/2082793584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 msgr2=0x55ec2fea5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea5c20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7feadc020e40 tx=0x7feadc004a40 comp rx=0 tx=0).stop 2024-02-26T06:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 -- 172.21.15.49:0/2082793584 shutdown_connections 2024-02-26T06:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55ec2ff1aa80 0x55ec2ff1cea0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ec2fe99af0 0x55ec2fea56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2fea2300 0x55ec2fea5c20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feace000640 0 client.27481 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 --2- 172.21.15.49:0/2082793584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec2fe98d80 0x55ec2fea9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 -- 172.21.15.49:0/2082793584 >> 172.21.15.49:0/2082793584 conn(0x55ec2fe60480 msgr2=0x55ec2fe92d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.908+0000 7feaedc83740 1 -- 172.21.15.49:0/2082793584 shutdown_connections 2024-02-26T06:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.909+0000 7feaedc83740 1 -- 172.21.15.49:0/2082793584 wait complete. 2024-02-26T06:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.909+0000 7feaedc83740 20 client.27481 trim_cache size 0 max 16384 2024-02-26T06:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:35:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 -- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feac80a2260 msgr2=0x7feac80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feac80a2260 0x7feac80a4720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feadc005fd0 tx=0x7feadc011670 comp rx=0 tx=0).stop 2024-02-26T06:35:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 -- 172.21.15.49:0/2551161943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feae4108690 msgr2=0x7feae419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feae4108690 0x7feae419fbf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fead002c170 tx=0x7fead002c5d0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 -- 172.21.15.49:0/2551161943 shutdown_connections 2024-02-26T06:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7feac80a2260 0x7feac80a4720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feae4108690 0x7feae419fbf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feae4199770 0x7feae4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 --2- 172.21.15.49:0/2551161943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae41035c0 0x7feae419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 -- 172.21.15.49:0/2551161943 >> 172.21.15.49:0/2551161943 conn(0x7feae40fee40 msgr2=0x7feae4101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 -- 172.21.15.49:0/2551161943 shutdown_connections 2024-02-26T06:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:40.910+0000 7feaeae00640 1 -- 172.21.15.49:0/2551161943 wait complete. 2024-02-26T06:35:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 710.0 s, 1266.6 tps, lat 79.062 ms stddev 96.611 2024-02-26T06:35:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:41 smithi080 ceph-mon[36206]: pgmap v383: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 557 op/s 2024-02-26T06:35:41.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:41 smithi115 ceph-mon[36426]: pgmap v383: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 557 op/s 2024-02-26T06:35:41.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:41 smithi049 ceph-mon[31696]: pgmap v383: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 557 op/s 2024-02-26T06:35:43.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:43 smithi080 ceph-mon[36206]: pgmap v384: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.7 MiB/s wr, 827 op/s 2024-02-26T06:35:43.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:43 smithi115 ceph-mon[36426]: pgmap v384: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.7 MiB/s wr, 827 op/s 2024-02-26T06:35:43.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:43 smithi049 ceph-mon[31696]: pgmap v384: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.7 MiB/s wr, 827 op/s 2024-02-26T06:35:45.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:45 smithi115 ceph-mon[36426]: pgmap v385: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.0 MiB/s wr, 564 op/s 2024-02-26T06:35:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:45 smithi049 ceph-mon[31696]: pgmap v385: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.0 MiB/s wr, 564 op/s 2024-02-26T06:35:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:45 smithi080 ceph-mon[36206]: pgmap v385: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.0 MiB/s wr, 564 op/s 2024-02-26T06:35:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 715.0 s, 1182.0 tps, lat 84.743 ms stddev 163.154 2024-02-26T06:35:47.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:47 smithi115 ceph-mon[36426]: pgmap v386: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.39k op/s 2024-02-26T06:35:47.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:47 smithi049 ceph-mon[31696]: pgmap v386: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.39k op/s 2024-02-26T06:35:47.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:47 smithi080 ceph-mon[36206]: pgmap v386: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.39k op/s 2024-02-26T06:35:48.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:48 smithi049 ceph-mon[31696]: pgmap v387: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 1.65k op/s 2024-02-26T06:35:48.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:48 smithi080 ceph-mon[36206]: pgmap v387: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 1.65k op/s 2024-02-26T06:35:48.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:48 smithi115 ceph-mon[36426]: pgmap v387: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 1.65k op/s 2024-02-26T06:35:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:50 smithi080 ceph-mon[36206]: pgmap v388: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-02-26T06:35:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:50 smithi115 ceph-mon[36426]: pgmap v388: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-02-26T06:35:51.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:50 smithi049 ceph-mon[31696]: pgmap v388: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-02-26T06:35:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 720.0 s, 2119.8 tps, lat 47.080 ms stddev 58.199 2024-02-26T06:35:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:52 smithi080 ceph-mon[36206]: pgmap v389: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 22 MiB/s wr, 1.84k op/s 2024-02-26T06:35:53.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:52 smithi115 ceph-mon[36426]: pgmap v389: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 22 MiB/s wr, 1.84k op/s 2024-02-26T06:35:53.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:52 smithi049 ceph-mon[31696]: pgmap v389: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 22 MiB/s wr, 1.84k op/s 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.098+0000 7ff6a5800640 1 -- 172.21.15.49:0/2434875192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a0103f70 msgr2=0x7ff6a0107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.098+0000 7ff6a5800640 1 --2- 172.21.15.49:0/2434875192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a0103f70 0x7ff6a0107fb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009900 tx=0x7ff68c02bc50 comp rx=0 tx=0).stop 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.098+0000 7ff6a5800640 1 -- 172.21.15.49:0/2434875192 shutdown_connections 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.098+0000 7ff6a5800640 1 --2- 172.21.15.49:0/2434875192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a0108690 0x7ff6a0110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.098+0000 7ff6a5800640 1 --2- 172.21.15.49:0/2434875192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a0103f70 0x7ff6a0107fb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.098+0000 7ff6a5800640 1 --2- 172.21.15.49:0/2434875192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6a01035c0 0x7ff6a01039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.098+0000 7ff6a5800640 1 -- 172.21.15.49:0/2434875192 >> 172.21.15.49:0/2434875192 conn(0x7ff6a00fee40 msgr2=0x7ff6a0101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.099+0000 7ff6a5800640 1 -- 172.21.15.49:0/2434875192 shutdown_connections 2024-02-26T06:35:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.099+0000 7ff6a5800640 1 -- 172.21.15.49:0/2434875192 wait complete. 2024-02-26T06:35:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.100+0000 7ff6a5800640 1 Processor -- start 2024-02-26T06:35:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.100+0000 7ff6a5800640 1 -- start start 2024-02-26T06:35:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff6a5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6a01035c0 0x7ff6a010fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff6a5800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a0108690 0x7ff6a010cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff6a5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a010d500 0x7ff6a010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff6a5800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a01aa3a0 con 0x7ff6a010d500 2024-02-26T06:35:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff6a5800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0198b30 con 0x7ff6a01035c0 2024-02-26T06:35:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff6a5800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0198cf0 con 0x7ff6a0108690 2024-02-26T06:35:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff69ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6a01035c0 0x7ff6a010fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff69ea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6a01035c0 0x7ff6a010fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38536/0 (socket says 172.21.15.49:38536) 2024-02-26T06:35:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.101+0000 7ff69ea00640 1 -- 172.21.15.49:0/1335248549 learned_addr learned my addr 172.21.15.49:0/1335248549 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.102+0000 7ff69f400640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a010d500 0x7ff6a010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.102+0000 7ff69ea00640 1 -- 172.21.15.49:0/1335248549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a0108690 msgr2=0x7ff6a010cfc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-26T06:35:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.102+0000 7ff69ea00640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a0108690 0x7ff6a010cfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.102+0000 7ff69ea00640 1 -- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a010d500 msgr2=0x7ff6a010d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.102+0000 7ff69ea00640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a010d500 0x7ff6a010d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.102+0000 7ff69ea00640 1 -- 172.21.15.49:0/1335248549 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c009560 con 0x7ff6a01035c0 2024-02-26T06:35:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.102+0000 7ff69f400640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a010d500 0x7ff6a010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:35:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.105+0000 7ff69ea00640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6a01035c0 0x7ff6a010fed0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff69000d980 tx=0x7ff69000de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.105+0000 7ff697400640 1 -- 172.21.15.49:0/1335248549 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690009c30 con 0x7ff6a01035c0 2024-02-26T06:35:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.105+0000 7ff697400640 1 -- 172.21.15.49:0/1335248549 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff69000b660 con 0x7ff6a01035c0 2024-02-26T06:35:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.105+0000 7ff697400640 1 -- 172.21.15.49:0/1335248549 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69000f6b0 con 0x7ff6a01035c0 2024-02-26T06:35:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.105+0000 7ff6a5800640 1 -- 172.21.15.49:0/1335248549 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a0198fd0 con 0x7ff6a01035c0 2024-02-26T06:35:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.106+0000 7ff6a5800640 1 -- 172.21.15.49:0/1335248549 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a0199490 con 0x7ff6a01035c0 2024-02-26T06:35:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.110+0000 7ff6a87a4740 1 Processor -- start 2024-02-26T06:35:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.113+0000 7ff6a87a4740 1 -- start start 2024-02-26T06:35:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.114+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb50840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.114+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb50d80 0x55cacfb51200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.114+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb5a300 0x55cacfb5c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.114+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfb0c5f0 con 0x55cacfb50d80 2024-02-26T06:35:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.115+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfabe890 con 0x55cacfb51af0 2024-02-26T06:35:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.115+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfabbdc0 con 0x55cacfb5a300 2024-02-26T06:35:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.115+0000 7ff69f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb50840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.115+0000 7ff69f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb50840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38552/0 (socket says 172.21.15.49:38552) 2024-02-26T06:35:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.115+0000 7ff69f400640 1 -- 172.21.15.49:0/2069397434 learned_addr learned my addr 172.21.15.49:0/2069397434 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.116+0000 7ff69e000640 1 --2- 172.21.15.49:0/2069397434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb5a300 0x55cacfb5c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.118+0000 7ff697400640 1 -- 172.21.15.49:0/1335248549 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff690010460 con 0x7ff6a01035c0 2024-02-26T06:35:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.119+0000 7ff69f400640 1 -- 172.21.15.49:0/2069397434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb5a300 msgr2=0x55cacfb5c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.119+0000 7ff69f400640 1 --2- 172.21.15.49:0/2069397434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb5a300 0x55cacfb5c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.119+0000 7ff69f400640 1 -- 172.21.15.49:0/2069397434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb50d80 msgr2=0x55cacfb51200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:35:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.120+0000 7ff69f400640 1 --2- 172.21.15.49:0/2069397434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb50d80 0x55cacfb51200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.120+0000 7ff69f400640 1 -- 172.21.15.49:0/2069397434 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cacfa38ad0 con 0x55cacfb51af0 2024-02-26T06:35:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.121+0000 7ff697400640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff6840a23d0 0x7ff6840a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.121+0000 7ff69f400640 1 --2- 172.21.15.49:0/2069397434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb50840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff6980107a0 tx=0x7ff698010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca589ed1d6ed69a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.121+0000 7ff697400640 1 -- 172.21.15.49:0/1335248549 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7ff6900cdd80 con 0x7ff6a01035c0 2024-02-26T06:35:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.121+0000 7ff68be00640 1 -- 172.21.15.49:0/2069397434 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698003b70 con 0x55cacfb51af0 2024-02-26T06:35:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.121+0000 7ff68be00640 1 -- 172.21.15.49:0/2069397434 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff698015070 con 0x55cacfb51af0 2024-02-26T06:35:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.122+0000 7ff68be00640 1 -- 172.21.15.49:0/2069397434 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698004560 con 0x55cacfb51af0 2024-02-26T06:35:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.122+0000 7ff69ea00640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff6840a23d0 0x7ff6840a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.124+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2069397434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 msgr2=0x55cacfb50840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.124+0000 7ff69ea00640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff6840a23d0 0x7ff6840a4890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55cacfb52250 tx=0x7ff6900cdf90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.124+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2069397434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb50840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff6980107a0 tx=0x7ff698010c60 comp rx=0 tx=0).stop 2024-02-26T06:35:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.124+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2069397434 shutdown_connections 2024-02-26T06:35:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.124+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2069397434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb5a300 0x55cacfb5c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.124+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2069397434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb50d80 0x55cacfb51200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.124+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2069397434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb50840 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.125+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2069397434 >> 172.21.15.49:0/2069397434 conn(0x55cacfb18480 msgr2=0x55cacfb4ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.125+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2069397434 shutdown_connections 2024-02-26T06:35:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.125+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2069397434 wait complete. 2024-02-26T06:35:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.126+0000 7ff6a87a4740 1 -- start start 2024-02-26T06:35:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.126+0000 7ff6a87a4740 1 Processor -- start 2024-02-26T06:35:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb57910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfab75a0 con 0x55cacfb5a300 2024-02-26T06:35:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfa5cb20 con 0x55cacfb51af0 2024-02-26T06:35:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfa5c7c0 con 0x55cacfb50d80 2024-02-26T06:35:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff6a87a4740 10 client.0 _ll_register_callbacks cb 0x55cacfa70520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff69f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff69f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48456/0 (socket says 172.21.15.49:48456) 2024-02-26T06:35:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.128+0000 7ff69f400640 1 -- 172.21.15.49:0/1607305770 learned_addr learned my addr 172.21.15.49:0/1607305770 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.129+0000 7ff69e000640 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb57910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.129+0000 7ff69f400640 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.129+0000 7ff69e000640 1 -- 172.21.15.49:0/1607305770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 msgr2=0x55cacfb573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.129+0000 7ff69e000640 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.129+0000 7ff69e000640 1 -- 172.21.15.49:0/1607305770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 msgr2=0x55cacfb57e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.129+0000 7ff69e000640 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.129+0000 7ff69e000640 1 -- 172.21.15.49:0/1607305770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6980103d0 con 0x55cacfb51af0 2024-02-26T06:35:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.129+0000 7ff69f400640 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:35:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.130+0000 7ff69e000640 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb57910 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff6a010e730 tx=0x7ff68c02c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd313e71a333d655 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.130+0000 7ff688c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:35:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.131+0000 7ff688c00640 1 -- 172.21.15.49:0/1607305770 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c004040 con 0x55cacfb51af0 2024-02-26T06:35:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.131+0000 7ff688c00640 1 -- 172.21.15.49:0/1607305770 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7ff68c0041a0 con 0x55cacfb51af0 2024-02-26T06:35:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.133+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1607305770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cacfa89ca0 con 0x55cacfb51af0 2024-02-26T06:35:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.133+0000 7ff688c00640 1 -- 172.21.15.49:0/1607305770 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff68c004380 con 0x55cacfb51af0 2024-02-26T06:35:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.134+0000 7ff6a87a4740 10 client.27542 fetch_fsmap learned FSMap version 49 2024-02-26T06:35:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.134+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1607305770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55cacfab5c20 con 0x55cacfb51af0 2024-02-26T06:35:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff688c00640 1 -- 172.21.15.49:0/1607305770 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff68c04c030 con 0x55cacfb51af0 2024-02-26T06:35:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff688c00640 1 -- 172.21.15.49:0/1607305770 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff68c046020 con 0x55cacfb51af0 2024-02-26T06:35:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff6a87a4740 10 client.27542 fetch_fsmap finished waiting for FSMap version 49 2024-02-26T06:35:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff6a87a4740 10 client.27542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff6a87a4740 20 client.27542 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cacfb6a360 0x55cacfb6c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff6a87a4740 4 client.27542 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:35:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1607305770 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cacfaadcb0 con 0x55cacfb6a360 2024-02-26T06:35:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.135+0000 7ff69f400640 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cacfb6a360 0x55cacfb6c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.136+0000 7ff69f400640 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cacfb6a360 0x55cacfb6c780 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124d54745c925b9a server_cookie=5c7783fc6317c1ef in_seq=0 out_seq=0 2024-02-26T06:35:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.136+0000 7ff688c00640 10 client.27542 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.137+0000 7ff688c00640 1 -- 172.21.15.49:0/1607305770 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cacfaadcb0 con 0x55cacfb6a360 2024-02-26T06:35:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.137+0000 7ff688c00640 10 client.27542 handle_command_reply: tid=0 2024-02-26T06:35:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.137+0000 7ff6a87a4740 1 client.27542 shutdown 2024-02-26T06:35:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 msgr2=0x55cacfb57910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb57910 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff6a010e730 tx=0x7ff68c02c830 comp rx=0 tx=0).stop 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1607305770 shutdown_connections 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cacfb6a360 0x55cacfb6c780 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb51af0 0x55cacfb57910 secure :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff6a010e730 tx=0x7ff68c02c830 comp rx=0 tx=0).stop 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1607305770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1607305770 >> 172.21.15.49:0/1607305770 conn(0x55cacfb18480 msgr2=0x55cacfb4d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff688c00640 0 client.27542 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1607305770 shutdown_connections 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.138+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1607305770 wait complete. 2024-02-26T06:35:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.139+0000 7ff6a87a4740 20 client.27542 trim_cache size 0 max 16384 2024-02-26T06:35:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.170+0000 7ff6a87a4740 1 Processor -- start 2024-02-26T06:35:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.170+0000 7ff6a87a4740 1 -- start start 2024-02-26T06:35:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.171+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb59b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.171+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.171+0000 7ff69f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb59b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.171+0000 7ff69f400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb59b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48462/0 (socket says 172.21.15.49:48462) 2024-02-26T06:35:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb50910 0x55cacfb61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacf7ffe60 con 0x55cacfb5a300 2024-02-26T06:35:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfb71990 con 0x55cacfb50910 2024-02-26T06:35:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfb50050 con 0x55cacfb50d80 2024-02-26T06:35:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff69f400640 1 -- 172.21.15.49:0/1922271518 learned_addr learned my addr 172.21.15.49:0/1922271518 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff69e000640 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff69f400640 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb50910 0x55cacfb61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff69f400640 1 -- 172.21.15.49:0/1922271518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb50910 msgr2=0x55cacfb61120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff69f400640 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb50910 0x55cacfb61120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff69f400640 1 -- 172.21.15.49:0/1922271518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 msgr2=0x55cacfb57c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff69f400640 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.172+0000 7ff69f400640 1 -- 172.21.15.49:0/1922271518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cacfa52500 con 0x55cacfb50d80 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.173+0000 7ff69e000640 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.173+0000 7ff69f400640 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb59b00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff698004190 tx=0x7ff6980096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11582c4b2251d22 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.173+0000 7ff68b400640 1 -- 172.21.15.49:0/1922271518 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69801e040 con 0x55cacfb50d80 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.173+0000 7ff68b400640 1 -- 172.21.15.49:0/1922271518 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff698015070 con 0x55cacfb50d80 2024-02-26T06:35:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.173+0000 7ff68b400640 1 -- 172.21.15.49:0/1922271518 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698005620 con 0x55cacfb50d80 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.174+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1922271518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 msgr2=0x55cacfb59b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.174+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb59b00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff698004190 tx=0x7ff6980096a0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.174+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1922271518 shutdown_connections 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.174+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb50910 0x55cacfb61120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.174+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb5a300 0x55cacfb57c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.174+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/1922271518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb59b00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.174+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1922271518 >> 172.21.15.49:0/1922271518 conn(0x55cacfb18480 msgr2=0x55cacfb55ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.175+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1922271518 shutdown_connections 2024-02-26T06:35:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.175+0000 7ff6a87a4740 1 -- 172.21.15.49:0/1922271518 wait complete. 2024-02-26T06:35:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.175+0000 7ff6a87a4740 1 -- start start 2024-02-26T06:35:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.176+0000 7ff6a87a4740 1 Processor -- start 2024-02-26T06:35:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.180+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb50910 0x55cacfb54760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.180+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb54ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.180+0000 7ff6a87a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb5a300 0x55cacfbc0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.180+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfb55400 con 0x55cacfb50910 2024-02-26T06:35:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.180+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfbc0da0 con 0x55cacfb5a300 2024-02-26T06:35:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.180+0000 7ff6a87a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacfbc0f60 con 0x55cacfb50d80 2024-02-26T06:35:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.180+0000 7ff6a87a4740 10 client.0 _ll_register_callbacks cb 0x7ff68c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:35:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.181+0000 7ff69e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb54ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.181+0000 7ff69e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb54ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48476/0 (socket says 172.21.15.49:48476) 2024-02-26T06:35:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.181+0000 7ff69e000640 1 -- 172.21.15.49:0/2156832717 learned_addr learned my addr 172.21.15.49:0/2156832717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:35:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.181+0000 7ff69e000640 1 -- 172.21.15.49:0/2156832717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb5a300 msgr2=0x55cacfbc0860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:35:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.181+0000 7ff69e000640 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb5a300 0x55cacfbc0860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.181+0000 7ff69e000640 1 -- 172.21.15.49:0/2156832717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb50910 msgr2=0x55cacfb54760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.181+0000 7ff69e000640 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb50910 0x55cacfb54760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.181+0000 7ff69e000640 1 -- 172.21.15.49:0/2156832717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff698021040 con 0x55cacfb50d80 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.182+0000 7ff69e000640 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb54ca0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff68c02c190 tx=0x7ff68c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e2c0ffda503c2f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.182+0000 7ff688c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.182+0000 7ff688c00640 1 -- 172.21.15.49:0/2156832717 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c004100 con 0x55cacfb50d80 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.182+0000 7ff688c00640 1 -- 172.21.15.49:0/2156832717 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7ff68c004260 con 0x55cacfb50d80 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.182+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2156832717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cacfadcd00 con 0x55cacfb50d80 2024-02-26T06:35:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff688c00640 1 -- 172.21.15.49:0/2156832717 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff68c004440 con 0x55cacfb50d80 2024-02-26T06:35:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff6a87a4740 10 client.27496 fetch_fsmap learned FSMap version 49 2024-02-26T06:35:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2156832717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55cacfa78290 con 0x55cacfb50d80 2024-02-26T06:35:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff688c00640 1 -- 172.21.15.49:0/2156832717 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cacfadcd00 con 0x55cacfb50d80 2024-02-26T06:35:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff688c00640 1 -- 172.21.15.49:0/2156832717 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff68c046040 con 0x55cacfb50d80 2024-02-26T06:35:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff6a87a4740 10 client.27496 fetch_fsmap finished waiting for FSMap version 49 2024-02-26T06:35:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff6a87a4740 10 client.27496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:35:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff6a87a4740 20 client.27496 populate_metadata read hostname 'smithi049' 2024-02-26T06:35:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cacfbc24c0 0x55cacfbc48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:35:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff6a87a4740 4 client.27496 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:35:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.183+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2156832717 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cacf7b7600 con 0x55cacfbc24c0 2024-02-26T06:35:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.184+0000 7ff69f400640 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cacfbc24c0 0x55cacfbc48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:35:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.184+0000 7ff69f400640 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cacfbc24c0 0x55cacfbc48e0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c4fed8255e719 server_cookie=fec9b36e5d2340ed in_seq=0 out_seq=0 2024-02-26T06:35:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.184+0000 7ff688c00640 10 client.27496 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.203+0000 7ff688c00640 1 -- 172.21.15.49:0/2156832717 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cacf7b7600 con 0x55cacfbc24c0 2024-02-26T06:35:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.203+0000 7ff688c00640 10 client.27496 handle_command_reply: tid=0 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.203+0000 7ff6a87a4740 1 client.27496 shutdown 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2156832717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 msgr2=0x55cacfb54ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb54ca0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff68c02c190 tx=0x7ff68c003820 comp rx=0 tx=0).stop 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2156832717 shutdown_connections 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55cacfbc24c0 0x55cacfbc48e0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cacfb5a300 0x55cacfbc0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cacfb50d80 0x55cacfb54ca0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 --2- 172.21.15.49:0/2156832717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cacfb50910 0x55cacfb54760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2156832717 >> 172.21.15.49:0/2156832717 conn(0x55cacfb18480 msgr2=0x55cacfb4a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff688c00640 0 client.27496 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2156832717 shutdown_connections 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 1 -- 172.21.15.49:0/2156832717 wait complete. 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.204+0000 7ff6a87a4740 20 client.27496 trim_cache size 0 max 16384 2024-02-26T06:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:35:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:35:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:35:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:35:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.205+0000 7ff6a5800640 1 -- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff6840a23d0 msgr2=0x7ff6840a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.205+0000 7ff6a5800640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff6840a23d0 0x7ff6840a4890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55cacfb52250 tx=0x7ff6900cdf90 comp rx=0 tx=0).stop 2024-02-26T06:35:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.206+0000 7ff6a5800640 1 -- 172.21.15.49:0/1335248549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6a01035c0 msgr2=0x7ff6a010fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:35:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.206+0000 7ff6a5800640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6a01035c0 0x7ff6a010fed0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff69000d980 tx=0x7ff69000de40 comp rx=0 tx=0).stop 2024-02-26T06:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.206+0000 7ff6a5800640 1 -- 172.21.15.49:0/1335248549 shutdown_connections 2024-02-26T06:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.206+0000 7ff6a5800640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff6840a23d0 0x7ff6840a4890 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.206+0000 7ff6a5800640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a010d500 0x7ff6a010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.206+0000 7ff6a5800640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a0108690 0x7ff6a010cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.206+0000 7ff6a5800640 1 --2- 172.21.15.49:0/1335248549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6a01035c0 0x7ff6a010fed0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.206+0000 7ff6a5800640 1 -- 172.21.15.49:0/1335248549 >> 172.21.15.49:0/1335248549 conn(0x7ff6a00fee40 msgr2=0x7ff6a00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.207+0000 7ff6a5800640 1 -- 172.21.15.49:0/1335248549 shutdown_connections 2024-02-26T06:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:35:54.207+0000 7ff6a5800640 1 -- 172.21.15.49:0/1335248549 wait complete. 2024-02-26T06:35:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:54 smithi080 ceph-mon[36206]: pgmap v390: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.59k op/s 2024-02-26T06:35:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:54 smithi115 ceph-mon[36426]: pgmap v390: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.59k op/s 2024-02-26T06:35:55.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:54 smithi049 ceph-mon[31696]: pgmap v390: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.59k op/s 2024-02-26T06:35:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 725.0 s, 1704.4 tps, lat 58.880 ms stddev 69.910 2024-02-26T06:35:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:56 smithi080 ceph-mon[36206]: pgmap v391: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 1.71k op/s 2024-02-26T06:35:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:56 smithi115 ceph-mon[36426]: pgmap v391: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 1.71k op/s 2024-02-26T06:35:57.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:56 smithi049 ceph-mon[31696]: pgmap v391: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 1.71k op/s 2024-02-26T06:35:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:58 smithi080 ceph-mon[36206]: pgmap v392: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 999 op/s 2024-02-26T06:35:59.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:58 smithi115 ceph-mon[36426]: pgmap v392: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 999 op/s 2024-02-26T06:35:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:58 smithi049 ceph-mon[31696]: pgmap v392: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 999 op/s 2024-02-26T06:36:00.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:35:59 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:36:00.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:35:59 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:36:00.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:35:59 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:36:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:00 smithi080 ceph-mon[36206]: pgmap v393: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-02-26T06:36:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:00 smithi115 ceph-mon[36426]: pgmap v393: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-02-26T06:36:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:00 smithi049 ceph-mon[31696]: pgmap v393: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-02-26T06:36:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 730.0 s, 1312.2 tps, lat 74.888 ms stddev 83.893 2024-02-26T06:36:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:01 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:36:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:01 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:36:02.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:01 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:36:02.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:01 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:36:02.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:01 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:36:02.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:01 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:36:02.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:01 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:36:02.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:01 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:36:02.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:01 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:36:03.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:03 smithi080 ceph-mon[36206]: pgmap v394: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 784 op/s 2024-02-26T06:36:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:03 smithi115 ceph-mon[36426]: pgmap v394: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 784 op/s 2024-02-26T06:36:03.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:03 smithi049 ceph-mon[31696]: pgmap v394: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 784 op/s 2024-02-26T06:36:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:05 smithi080 ceph-mon[36206]: pgmap v395: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 545 op/s 2024-02-26T06:36:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:05 smithi115 ceph-mon[36426]: pgmap v395: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 545 op/s 2024-02-26T06:36:05.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:05 smithi049 ceph-mon[31696]: pgmap v395: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 545 op/s 2024-02-26T06:36:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 735.0 s, 693.6 tps, lat 145.488 ms stddev 216.490 2024-02-26T06:36:06.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:06 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:06.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:06 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:06.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:06 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:07 smithi080 ceph-mon[36206]: pgmap v396: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-02-26T06:36:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:07 smithi115 ceph-mon[36426]: pgmap v396: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-02-26T06:36:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.412+0000 7f985e600640 1 -- 172.21.15.49:0/4204801471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f985810c510 msgr2=0x7f98581189d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.412+0000 7f985e600640 1 --2- 172.21.15.49:0/4204801471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f985810c510 0x7f98581189d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9844009860 tx=0x7f984402bb70 comp rx=0 tx=0).stop 2024-02-26T06:36:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.413+0000 7f985e600640 1 -- 172.21.15.49:0/4204801471 shutdown_connections 2024-02-26T06:36:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.413+0000 7f985e600640 1 --2- 172.21.15.49:0/4204801471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f985810c510 0x7f98581189d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.413+0000 7f985e600640 1 --2- 172.21.15.49:0/4204801471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f985810bb50 0x7f985810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.413+0000 7f985e600640 1 --2- 172.21.15.49:0/4204801471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9858111c10 0x7f9858111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.413+0000 7f985e600640 1 -- 172.21.15.49:0/4204801471 >> 172.21.15.49:0/4204801471 conn(0x7f985806c390 msgr2=0x7f985810a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.414+0000 7f985e600640 1 -- 172.21.15.49:0/4204801471 shutdown_connections 2024-02-26T06:36:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.414+0000 7f985e600640 1 -- 172.21.15.49:0/4204801471 wait complete. 2024-02-26T06:36:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:07 smithi049 ceph-mon[31696]: pgmap v396: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-02-26T06:36:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.417+0000 7f985e600640 1 Processor -- start 2024-02-26T06:36:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.417+0000 7f985e600640 1 -- start start 2024-02-26T06:36:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.417+0000 7f985e600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f985810bb50 0x7f9858118600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.417+0000 7f985e600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f985810c510 0x7f98581156a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.417+0000 7f985e600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858111c10 0x7f9858115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.417+0000 7f985e600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581161e0 con 0x7f9858111c10 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.417+0000 7f985e600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858116350 con 0x7f985810c510 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.417+0000 7f985e600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858116510 con 0x7f985810bb50 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.418+0000 7f9856a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f985810c510 0x7f98581156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.418+0000 7f9857400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f985810bb50 0x7f9858118600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.418+0000 7f9856a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f985810c510 0x7f98581156a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:44622/0 (socket says 172.21.15.49:44622) 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.418+0000 7f9856a00640 1 -- 172.21.15.49:0/1562235797 learned_addr learned my addr 172.21.15.49:0/1562235797 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.418+0000 7f9857e00640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858111c10 0x7f9858115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.419+0000 7f9857e00640 1 -- 172.21.15.49:0/1562235797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f985810bb50 msgr2=0x7f9858118600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.419+0000 7f9857e00640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f985810bb50 0x7f9858118600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.419+0000 7f9857e00640 1 -- 172.21.15.49:0/1562235797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f985810c510 msgr2=0x7f98581156a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.419+0000 7f9857e00640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f985810c510 0x7f98581156a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.419+0000 7f9857e00640 1 -- 172.21.15.49:0/1562235797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848009900 con 0x7f9858111c10 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.419+0000 7f9856a00640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f985810c510 0x7f98581156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:36:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.419+0000 7f9857e00640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858111c10 0x7f9858115be0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f984402c180 tx=0x7f9844004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.420+0000 7f984fe00640 1 -- 172.21.15.49:0/1562235797 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844034990 con 0x7f9858111c10 2024-02-26T06:36:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.420+0000 7f985e600640 1 -- 172.21.15.49:0/1562235797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9844009490 con 0x7f9858111c10 2024-02-26T06:36:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.421+0000 7f984fe00640 1 -- 172.21.15.49:0/1562235797 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f984402c9f0 con 0x7f9858111c10 2024-02-26T06:36:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.421+0000 7f984fe00640 1 -- 172.21.15.49:0/1562235797 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844032df0 con 0x7f9858111c10 2024-02-26T06:36:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.421+0000 7f985e600640 1 -- 172.21.15.49:0/1562235797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98581a8580 con 0x7f9858111c10 2024-02-26T06:36:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.428+0000 7f984fe00640 1 -- 172.21.15.49:0/1562235797 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f984402c510 con 0x7f9858111c10 2024-02-26T06:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.429+0000 7f986146a740 1 Processor -- start 2024-02-26T06:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.429+0000 7f986146a740 1 -- start start 2024-02-26T06:36:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.429+0000 7f986146a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307becaaf0 0x56307bec9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.429+0000 7f986146a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307beca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.429+0000 7f986146a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307bed3300 0x56307bed56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.429+0000 7f986146a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307be855f0 con 0x56307becaaf0 2024-02-26T06:36:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.429+0000 7f986146a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307be37890 con 0x56307bed3300 2024-02-26T06:36:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.429+0000 7f986146a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307be34dc0 con 0x56307bec9d80 2024-02-26T06:36:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.430+0000 7f9857400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307beca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.431+0000 7f9857400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307beca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43160/0 (socket says 172.21.15.49:43160) 2024-02-26T06:36:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.431+0000 7f9857400640 1 -- 172.21.15.49:0/4053801596 learned_addr learned my addr 172.21.15.49:0/4053801596 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.432+0000 7f9856a00640 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307bed3300 0x56307bed56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.433+0000 7f9856a00640 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307becaaf0 0x56307bec9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.433+0000 7f9857400640 1 -- 172.21.15.49:0/4053801596 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307bed3300 msgr2=0x56307bed56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.433+0000 7f9857400640 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307bed3300 0x56307bed56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.433+0000 7f9857400640 1 -- 172.21.15.49:0/4053801596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307becaaf0 msgr2=0x56307bec9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.434+0000 7f9857400640 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307becaaf0 0x56307bec9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.434+0000 7f9857400640 1 -- 172.21.15.49:0/4053801596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56307bdb1ad0 con 0x56307bec9d80 2024-02-26T06:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.435+0000 7f9856a00640 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307becaaf0 0x56307bec9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.435+0000 7f9857400640 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307beca200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9850009c80 tx=0x7f98500117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118fa82f96973fe5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.436+0000 7f984fe00640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f983c0a23a0 0x7f983c0a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.436+0000 7f984fe00640 1 -- 172.21.15.49:0/1562235797 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f98441114a0 con 0x7f9858111c10 2024-02-26T06:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.437+0000 7f984cc00640 1 -- 172.21.15.49:0/4053801596 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850009750 con 0x56307bec9d80 2024-02-26T06:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.437+0000 7f984cc00640 1 -- 172.21.15.49:0/4053801596 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9850012930 con 0x56307bec9d80 2024-02-26T06:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.437+0000 7f984cc00640 1 -- 172.21.15.49:0/4053801596 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000fb80 con 0x56307bec9d80 2024-02-26T06:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.437+0000 7f9856a00640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f983c0a23a0 0x7f983c0a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.438+0000 7f986146a740 1 -- 172.21.15.49:0/4053801596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 msgr2=0x56307beca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.438+0000 7f986146a740 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307beca200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9850009c80 tx=0x7f98500117b0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.441+0000 7f986146a740 1 -- 172.21.15.49:0/4053801596 shutdown_connections 2024-02-26T06:36:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.441+0000 7f986146a740 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307bed3300 0x56307bed56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.441+0000 7f986146a740 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307beca200 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.441+0000 7f986146a740 1 --2- 172.21.15.49:0/4053801596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307becaaf0 0x56307bec9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.441+0000 7f986146a740 1 -- 172.21.15.49:0/4053801596 >> 172.21.15.49:0/4053801596 conn(0x56307be91480 msgr2=0x56307bec4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.441+0000 7f986146a740 1 -- 172.21.15.49:0/4053801596 shutdown_connections 2024-02-26T06:36:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.441+0000 7f986146a740 1 -- 172.21.15.49:0/4053801596 wait complete. 2024-02-26T06:36:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.441+0000 7f9856a00640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f983c0a23a0 0x7f983c0a4860 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56307bec5c00 tx=0x7f98480095c0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.442+0000 7f986146a740 1 -- start start 2024-02-26T06:36:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.442+0000 7f986146a740 1 Processor -- start 2024-02-26T06:36:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f986146a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f986146a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f986146a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307bed0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f986146a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307be305a0 con 0x56307bed3300 2024-02-26T06:36:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f986146a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bdd5b20 con 0x56307becaaf0 2024-02-26T06:36:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f986146a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bdd57c0 con 0x56307bec9d80 2024-02-26T06:36:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f986146a740 10 client.0 _ll_register_callbacks cb 0x56307bde9520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f9857400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f9857400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43166/0 (socket says 172.21.15.49:43166) 2024-02-26T06:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.443+0000 7f9857400640 1 -- 172.21.15.49:0/2132571637 learned_addr learned my addr 172.21.15.49:0/2132571637 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.444+0000 7f9857e00640 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307bed0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.444+0000 7f9856a00640 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.444+0000 7f9857e00640 1 -- 172.21.15.49:0/2132571637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 msgr2=0x56307bed0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.444+0000 7f9857e00640 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.444+0000 7f9857e00640 1 -- 172.21.15.49:0/2132571637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 msgr2=0x56307bed08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.444+0000 7f9857e00640 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.444+0000 7f9857e00640 1 -- 172.21.15.49:0/2132571637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98500098b0 con 0x56307bed3300 2024-02-26T06:36:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.445+0000 7f9857e00640 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307bed0df0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f984410ece0 tx=0x7f98440dd970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a70e6026a582e2f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.445+0000 7f9841600640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:36:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.446+0000 7f9841600640 1 -- 172.21.15.49:0/2132571637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98440e2160 con 0x56307bed3300 2024-02-26T06:36:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.446+0000 7f9841600640 1 -- 172.21.15.49:0/2132571637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f98440e6720 con 0x56307bed3300 2024-02-26T06:36:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.446+0000 7f986146a740 1 -- 172.21.15.49:0/2132571637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56307be02ca0 con 0x56307bed3300 2024-02-26T06:36:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.448+0000 7f9841600640 1 -- 172.21.15.49:0/2132571637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984402cb60 con 0x56307bed3300 2024-02-26T06:36:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.449+0000 7f986146a740 10 client.16884 fetch_fsmap learned FSMap version 50 2024-02-26T06:36:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.449+0000 7f986146a740 1 -- 172.21.15.49:0/2132571637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56307be2ec20 con 0x56307bed3300 2024-02-26T06:36:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.449+0000 7f9841600640 1 -- 172.21.15.49:0/2132571637 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56307be02ca0 con 0x56307bed3300 2024-02-26T06:36:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.450+0000 7f9841600640 1 -- 172.21.15.49:0/2132571637 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f98440e63a0 con 0x56307bed3300 2024-02-26T06:36:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.451+0000 7f986146a740 10 client.16884 fetch_fsmap finished waiting for FSMap version 50 2024-02-26T06:36:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.451+0000 7f986146a740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:36:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.451+0000 7f986146a740 20 client.16884 populate_metadata read hostname 'smithi049' 2024-02-26T06:36:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.451+0000 7f986146a740 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56307bee3030 0x56307bee5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.451+0000 7f986146a740 4 client.16884 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:36:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.451+0000 7f986146a740 1 -- 172.21.15.49:0/2132571637 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56307be26cb0 con 0x56307bee3030 2024-02-26T06:36:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.452+0000 7f9857400640 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56307bee3030 0x56307bee5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.456+0000 7f9857400640 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56307bee3030 0x56307bee5450 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=426ca3084ad5830b server_cookie=c5fbf053a6d18e35 in_seq=0 out_seq=0 2024-02-26T06:36:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.456+0000 7f9841600640 10 client.16884 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.457+0000 7f9841600640 1 -- 172.21.15.49:0/2132571637 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56307be26cb0 con 0x56307bee3030 2024-02-26T06:36:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.457+0000 7f9841600640 10 client.16884 handle_command_reply: tid=0 2024-02-26T06:36:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.457+0000 7f986146a740 1 client.16884 shutdown 2024-02-26T06:36:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.457+0000 7f986146a740 1 -- 172.21.15.49:0/2132571637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 msgr2=0x56307bed0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.457+0000 7f986146a740 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307bed0df0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f984410ece0 tx=0x7f98440dd970 comp rx=0 tx=0).stop 2024-02-26T06:36:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.458+0000 7f986146a740 1 -- 172.21.15.49:0/2132571637 shutdown_connections 2024-02-26T06:36:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.458+0000 7f986146a740 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56307bee3030 0x56307bee5450 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.458+0000 7f986146a740 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.458+0000 7f986146a740 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307bed0df0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.458+0000 7f986146a740 1 --2- 172.21.15.49:0/2132571637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.458+0000 7f986146a740 1 -- 172.21.15.49:0/2132571637 >> 172.21.15.49:0/2132571637 conn(0x56307be91480 msgr2=0x56307bec67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.458+0000 7f9841600640 0 client.16884 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.458+0000 7f986146a740 1 -- 172.21.15.49:0/2132571637 shutdown_connections 2024-02-26T06:36:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.459+0000 7f986146a740 1 -- 172.21.15.49:0/2132571637 wait complete. 2024-02-26T06:36:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.459+0000 7f986146a740 20 client.16884 trim_cache size 0 max 16384 2024-02-26T06:36:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.491+0000 7f986146a740 1 Processor -- start 2024-02-26T06:36:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f986146a740 1 -- start start 2024-02-26T06:36:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f986146a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f986146a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f986146a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307becec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f986146a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bb78e60 con 0x56307bed3300 2024-02-26T06:36:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f986146a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bedfeb0 con 0x56307becaaf0 2024-02-26T06:36:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f986146a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bed5f40 con 0x56307bec9d80 2024-02-26T06:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f9857400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f9857400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43180/0 (socket says 172.21.15.49:43180) 2024-02-26T06:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f9857400640 1 -- 172.21.15.49:0/790625750 learned_addr learned my addr 172.21.15.49:0/790625750 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f9856a00640 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f9857400640 1 -- 172.21.15.49:0/790625750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 msgr2=0x56307bed0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f9857400640 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f9857e00640 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307becec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.492+0000 7f9857400640 1 -- 172.21.15.49:0/790625750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 msgr2=0x56307becec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.493+0000 7f9857400640 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307becec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.493+0000 7f9857400640 1 -- 172.21.15.49:0/790625750 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56307bdcb500 con 0x56307bec9d80 2024-02-26T06:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.493+0000 7f9857400640 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed2aa0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f9850005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07d3615f6e15486 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.494+0000 7f9843e00640 1 -- 172.21.15.49:0/790625750 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000f4b0 con 0x56307bec9d80 2024-02-26T06:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.494+0000 7f9843e00640 1 -- 172.21.15.49:0/790625750 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f985000f610 con 0x56307bec9d80 2024-02-26T06:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.494+0000 7f9843e00640 1 -- 172.21.15.49:0/790625750 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850003850 con 0x56307bec9d80 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 -- 172.21.15.49:0/790625750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 msgr2=0x56307bed2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed2aa0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f9850005f90 comp rx=0 tx=0).stop 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 -- 172.21.15.49:0/790625750 shutdown_connections 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bed3300 0x56307becec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 --2- 172.21.15.49:0/790625750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bec9d80 0x56307bed2aa0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 -- 172.21.15.49:0/790625750 >> 172.21.15.49:0/790625750 conn(0x56307be91480 msgr2=0x56307becb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 -- 172.21.15.49:0/790625750 shutdown_connections 2024-02-26T06:36:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.496+0000 7f986146a740 1 -- 172.21.15.49:0/790625750 wait complete. 2024-02-26T06:36:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.497+0000 7f986146a740 1 -- start start 2024-02-26T06:36:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.497+0000 7f986146a740 1 Processor -- start 2024-02-26T06:36:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f986146a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bec9d80 0x56307beda340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f986146a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f986146a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bed3300 0x56307bed8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f986146a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bed91f0 con 0x56307bec9d80 2024-02-26T06:36:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f986146a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bed9360 con 0x56307becaaf0 2024-02-26T06:36:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f986146a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bed9520 con 0x56307bed3300 2024-02-26T06:36:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f986146a740 10 client.0 _ll_register_callbacks cb 0x7f98440e4040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:36:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f9857e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bed3300 0x56307bed8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f9857e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bed3300 0x56307bed8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43188/0 (socket says 172.21.15.49:43188) 2024-02-26T06:36:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.498+0000 7f9857e00640 1 -- 172.21.15.49:0/4264827968 learned_addr learned my addr 172.21.15.49:0/4264827968 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.499+0000 7f9857400640 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bec9d80 0x56307beda340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.499+0000 7f9857400640 1 -- 172.21.15.49:0/4264827968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bed3300 msgr2=0x56307bed8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.499+0000 7f9857400640 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bed3300 0x56307bed8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.499+0000 7f9857400640 1 -- 172.21.15.49:0/4264827968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 msgr2=0x56307bed8770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:36:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.499+0000 7f9857400640 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed8770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.499+0000 7f9857400640 1 -- 172.21.15.49:0/4264827968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56307bdb1ad0 con 0x56307bec9d80 2024-02-26T06:36:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.500+0000 7f9857400640 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bec9d80 0x56307beda340 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f985000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e55fb094454238 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.500+0000 7f9841600640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:36:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.503+0000 7f9841600640 1 -- 172.21.15.49:0/4264827968 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000fa40 con 0x56307bec9d80 2024-02-26T06:36:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.503+0000 7f9841600640 1 -- 172.21.15.49:0/4264827968 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f98500050b0 con 0x56307bec9d80 2024-02-26T06:36:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.503+0000 7f986146a740 1 -- 172.21.15.49:0/4264827968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56307be55d00 con 0x56307bec9d80 2024-02-26T06:36:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.504+0000 7f9841600640 1 -- 172.21.15.49:0/4264827968 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9850005270 con 0x56307bec9d80 2024-02-26T06:36:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.504+0000 7f986146a740 10 client.16887 fetch_fsmap learned FSMap version 50 2024-02-26T06:36:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.504+0000 7f986146a740 1 -- 172.21.15.49:0/4264827968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56307bdf1290 con 0x56307bec9d80 2024-02-26T06:36:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.505+0000 7f9841600640 1 -- 172.21.15.49:0/4264827968 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56307be55d00 con 0x56307bec9d80 2024-02-26T06:36:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.509+0000 7f9841600640 1 -- 172.21.15.49:0/4264827968 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 50) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9850022040 con 0x56307bec9d80 2024-02-26T06:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.509+0000 7f986146a740 10 client.16887 fetch_fsmap finished waiting for FSMap version 50 2024-02-26T06:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.509+0000 7f986146a740 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.509+0000 7f986146a740 20 client.16887 populate_metadata read hostname 'smithi049' 2024-02-26T06:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.509+0000 7f986146a740 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56307bf3a730 0x56307bf3cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.509+0000 7f986146a740 4 client.16887 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.509+0000 7f986146a740 1 -- 172.21.15.49:0/4264827968 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56307bb30600 con 0x56307bf3a730 2024-02-26T06:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.509+0000 7f9856a00640 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56307bf3a730 0x56307bf3cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.512+0000 7f9856a00640 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56307bf3a730 0x56307bf3cb50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a566d8ad713219 server_cookie=2ce6bed28e5991b4 in_seq=0 out_seq=0 2024-02-26T06:36:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.512+0000 7f9841600640 10 client.16887 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.573+0000 7f9841600640 1 -- 172.21.15.49:0/4264827968 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56307bb30600 con 0x56307bf3a730 2024-02-26T06:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.573+0000 7f9841600640 10 client.16887 handle_command_reply: tid=0 2024-02-26T06:36:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.573+0000 7f986146a740 1 client.16887 shutdown 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 -- 172.21.15.49:0/4264827968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bec9d80 msgr2=0x56307beda340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bec9d80 0x56307beda340 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f985000f580 comp rx=0 tx=0).stop 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 -- 172.21.15.49:0/4264827968 shutdown_connections 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x56307bf3a730 0x56307bf3cb50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56307becaaf0 0x56307bed8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56307bed3300 0x56307bed8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 --2- 172.21.15.49:0/4264827968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56307bec9d80 0x56307beda340 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 -- 172.21.15.49:0/4264827968 >> 172.21.15.49:0/4264827968 conn(0x56307be91480 msgr2=0x56307beceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f9841600640 0 client.16887 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 -- 172.21.15.49:0/4264827968 shutdown_connections 2024-02-26T06:36:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.575+0000 7f986146a740 1 -- 172.21.15.49:0/4264827968 wait complete. 2024-02-26T06:36:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.576+0000 7f986146a740 20 client.16887 trim_cache size 0 max 16384 2024-02-26T06:36:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:36:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:36:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:36:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:36:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.577+0000 7f985e600640 1 -- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f983c0a23a0 msgr2=0x7f983c0a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.577+0000 7f985e600640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f983c0a23a0 0x7f983c0a4860 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56307bec5c00 tx=0x7f98480095c0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.577+0000 7f985e600640 1 -- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858111c10 msgr2=0x7f9858115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.577+0000 7f985e600640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858111c10 0x7f9858115be0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f984402c180 tx=0x7f9844004580 comp rx=0 tx=0).stop 2024-02-26T06:36:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.577+0000 7f985e600640 1 -- 172.21.15.49:0/1562235797 shutdown_connections 2024-02-26T06:36:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.578+0000 7f985e600640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f983c0a23a0 0x7f983c0a4860 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56307bec5c00 tx=0x7f98480095c0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.578+0000 7f985e600640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f985810c510 0x7f98581156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.578+0000 7f985e600640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9858111c10 0x7f9858115be0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.578+0000 7f985e600640 1 --2- 172.21.15.49:0/1562235797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f985810bb50 0x7f9858118600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.578+0000 7f985e600640 1 -- 172.21.15.49:0/1562235797 >> 172.21.15.49:0/1562235797 conn(0x7f985806c390 msgr2=0x7f985810a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.578+0000 7f985e600640 1 -- 172.21.15.49:0/1562235797 shutdown_connections 2024-02-26T06:36:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:07.578+0000 7f985e600640 1 -- 172.21.15.49:0/1562235797 wait complete. 2024-02-26T06:36:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:09 smithi080 ceph-mon[36206]: pgmap v397: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 1.44k op/s 2024-02-26T06:36:09.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:09 smithi115 ceph-mon[36426]: pgmap v397: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 1.44k op/s 2024-02-26T06:36:09.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:09 smithi049 ceph-mon[31696]: pgmap v397: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 1.44k op/s 2024-02-26T06:36:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 740.0 s, 1942.0 tps, lat 50.198 ms stddev 64.756 2024-02-26T06:36:11.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:11 smithi080 ceph-mon[36206]: pgmap v398: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-02-26T06:36:11.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:11 smithi115 ceph-mon[36426]: pgmap v398: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-02-26T06:36:11.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:11 smithi049 ceph-mon[31696]: pgmap v398: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-02-26T06:36:12.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:12 smithi049 ceph-mon[31696]: osdmap e102: 12 total, 12 up, 12 in 2024-02-26T06:36:12.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:12 smithi080 ceph-mon[36206]: osdmap e102: 12 total, 12 up, 12 in 2024-02-26T06:36:12.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:12 smithi115 ceph-mon[36426]: osdmap e102: 12 total, 12 up, 12 in 2024-02-26T06:36:13.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:13 smithi049 ceph-mon[31696]: pgmap v400: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.59k op/s 2024-02-26T06:36:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:13 smithi080 ceph-mon[36206]: pgmap v400: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.59k op/s 2024-02-26T06:36:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:13 smithi115 ceph-mon[36426]: pgmap v400: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.59k op/s 2024-02-26T06:36:15.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:15 smithi049 ceph-mon[31696]: pgmap v401: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-02-26T06:36:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:15 smithi080 ceph-mon[36206]: pgmap v401: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-02-26T06:36:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:15 smithi115 ceph-mon[36426]: pgmap v401: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-02-26T06:36:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 745.0 s, 1191.0 tps, lat 86.814 ms stddev 116.379 2024-02-26T06:36:17.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:17 smithi049 ceph-mon[31696]: pgmap v402: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.9 MiB/s wr, 1.01k op/s 2024-02-26T06:36:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:17 smithi080 ceph-mon[36206]: pgmap v402: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.9 MiB/s wr, 1.01k op/s 2024-02-26T06:36:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:17 smithi115 ceph-mon[36426]: pgmap v402: 97 pgs: 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 84 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.9 MiB/s wr, 1.01k op/s 2024-02-26T06:36:19.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:19 smithi049 ceph-mon[31696]: pgmap v403: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-02-26T06:36:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:19 smithi080 ceph-mon[36206]: pgmap v403: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-02-26T06:36:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:19 smithi115 ceph-mon[36426]: pgmap v403: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-02-26T06:36:20.418 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:20 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:20.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:20 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:20.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:20 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.772+0000 7f7ed4200640 1 -- 172.21.15.49:0/3646788803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 msgr2=0x7f7ecc107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.772+0000 7f7ed4200640 1 --2- 172.21.15.49:0/3646788803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 0x7f7ecc107bd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009930 tx=0x7f7eb802bc30 comp rx=0 tx=0).stop 2024-02-26T06:36:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.773+0000 7f7ed4200640 1 -- 172.21.15.49:0/3646788803 shutdown_connections 2024-02-26T06:36:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.773+0000 7f7ed4200640 1 --2- 172.21.15.49:0/3646788803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ecc108110 0x7f7ecc114620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.773+0000 7f7ed4200640 1 --2- 172.21.15.49:0/3646788803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 0x7f7ecc107bd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.773+0000 7f7ed4200640 1 --2- 172.21.15.49:0/3646788803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ecc10d760 0x7f7ecc10db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.773+0000 7f7ed4200640 1 -- 172.21.15.49:0/3646788803 >> 172.21.15.49:0/3646788803 conn(0x7f7ecc0756f0 msgr2=0x7f7ecc077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.773+0000 7f7ed4200640 1 -- 172.21.15.49:0/3646788803 shutdown_connections 2024-02-26T06:36:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.773+0000 7f7ed4200640 1 -- 172.21.15.49:0/3646788803 wait complete. 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.774+0000 7f7ed4200640 1 Processor -- start 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed4200640 1 -- start start 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed4200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 0x7f7ecc19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed4200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ecc108110 0x7f7ecc19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed4200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ecc10d760 0x7f7ecc1a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed4200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc1a2c00 con 0x7f7ecc107750 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed4200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc1a2d70 con 0x7f7ecc10d760 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed4200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc1a2f30 con 0x7f7ecc108110 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed1800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 0x7f7ecc19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ecc108110 0x7f7ecc19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ecc108110 0x7f7ecc19e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60372/0 (socket says 172.21.15.49:60372) 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed0e00640 1 -- 172.21.15.49:0/4105991600 learned_addr learned my addr 172.21.15.49:0/4105991600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.775+0000 7f7ed2200640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ecc10d760 0x7f7ecc1a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.776+0000 7f7ed0e00640 1 -- 172.21.15.49:0/4105991600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ecc10d760 msgr2=0x7f7ecc1a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.776+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ecc10d760 0x7f7ecc1a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.776+0000 7f7ed0e00640 1 -- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 msgr2=0x7f7ecc19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.776+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 0x7f7ecc19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.776+0000 7f7ed0e00640 1 -- 172.21.15.49:0/4105991600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb8009560 con 0x7f7ecc108110 2024-02-26T06:36:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.776+0000 7f7ed1800640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 0x7f7ecc19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:36:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.777+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ecc108110 0x7f7ecc19e280 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000c00 tx=0x7f7eb802c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.778+0000 7f7eca000640 1 -- 172.21.15.49:0/4105991600 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80042d0 con 0x7f7ecc108110 2024-02-26T06:36:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.778+0000 7f7eca000640 1 -- 172.21.15.49:0/4105991600 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb8004430 con 0x7f7ecc108110 2024-02-26T06:36:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.778+0000 7f7eca000640 1 -- 172.21.15.49:0/4105991600 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8003c30 con 0x7f7ecc108110 2024-02-26T06:36:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.778+0000 7f7ed4200640 1 -- 172.21.15.49:0/4105991600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc1a31b0 con 0x7f7ecc108110 2024-02-26T06:36:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.778+0000 7f7ed4200640 1 -- 172.21.15.49:0/4105991600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc1aaa90 con 0x7f7ecc108110 2024-02-26T06:36:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.785+0000 7f7eca000640 1 -- 172.21.15.49:0/4105991600 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7eb8004df0 con 0x7f7ecc108110 2024-02-26T06:36:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.786+0000 7f7ed702f740 1 Processor -- start 2024-02-26T06:36:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.786+0000 7f7ed702f740 1 -- start start 2024-02-26T06:36:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.786+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd4029840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.786+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4029d80 0x55afd402a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.786+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 0x55afd40356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.786+0000 7f7ed702f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd3fe55f0 con 0x55afd402aaf0 2024-02-26T06:36:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.786+0000 7f7ed702f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd3f97890 con 0x55afd4029d80 2024-02-26T06:36:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed702f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd3f94dc0 con 0x55afd4033300 2024-02-26T06:36:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed2200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd4029840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 0x55afd40356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed2200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd4029840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38688/0 (socket says 172.21.15.49:38688) 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed2200640 1 -- 172.21.15.49:0/1182905341 learned_addr learned my addr 172.21.15.49:0/1182905341 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 0x55afd40356f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60382/0 (socket says 172.21.15.49:60382) 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed1800640 1 --2- 172.21.15.49:0/1182905341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4029d80 0x55afd402a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed2200640 1 -- 172.21.15.49:0/1182905341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 msgr2=0x55afd40356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed2200640 1 --2- 172.21.15.49:0/1182905341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 0x55afd40356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.787+0000 7f7ed2200640 1 -- 172.21.15.49:0/1182905341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4029d80 msgr2=0x55afd402a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.788+0000 7f7ed2200640 1 --2- 172.21.15.49:0/1182905341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4029d80 0x55afd402a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.788+0000 7f7ed2200640 1 -- 172.21.15.49:0/1182905341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd3f11ad0 con 0x55afd402aaf0 2024-02-26T06:36:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.788+0000 7f7ed2200640 1 --2- 172.21.15.49:0/1182905341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd4029840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009fd0 tx=0x7f7ec40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59e929304da4e91 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.789+0000 7f7ebea00640 1 -- 172.21.15.49:0/1182905341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4004d20 con 0x55afd402aaf0 2024-02-26T06:36:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.789+0000 7f7ebea00640 1 -- 172.21.15.49:0/1182905341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ec4004e80 con 0x55afd402aaf0 2024-02-26T06:36:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.789+0000 7f7ebea00640 1 -- 172.21.15.49:0/1182905341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4005280 con 0x55afd402aaf0 2024-02-26T06:36:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7eca000640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7eb40a22d0 0x7f7eb40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7eca000640 1 -- 172.21.15.49:0/4105991600 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f7eb80f0600 con 0x7f7ecc108110 2024-02-26T06:36:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7ed702f740 1 -- 172.21.15.49:0/1182905341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 msgr2=0x55afd4029840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1182905341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd4029840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009fd0 tx=0x7f7ec40118d0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7ed702f740 1 -- 172.21.15.49:0/1182905341 shutdown_connections 2024-02-26T06:36:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1182905341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4029d80 0x55afd402a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1182905341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 0x55afd40356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1182905341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd4029840 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.790+0000 7f7ed702f740 1 -- 172.21.15.49:0/1182905341 >> 172.21.15.49:0/1182905341 conn(0x55afd3ff1480 msgr2=0x55afd4024a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.791+0000 7f7ed1800640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7eb40a22d0 0x7f7eb40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.792+0000 7f7ed702f740 1 -- 172.21.15.49:0/1182905341 shutdown_connections 2024-02-26T06:36:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.793+0000 7f7ed1800640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7eb40a22d0 0x7f7eb40a4790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55afd402b250 tx=0x7f7ec0005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.793+0000 7f7ed702f740 1 -- 172.21.15.49:0/1182905341 wait complete. 2024-02-26T06:36:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.794+0000 7f7ed702f740 1 -- start start 2024-02-26T06:36:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.794+0000 7f7ed702f740 1 Processor -- start 2024-02-26T06:36:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.795+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4029d80 0x55afd4030360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.795+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd40308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.795+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd4030de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.796+0000 7f7ed702f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd3f905a0 con 0x55afd402aaf0 2024-02-26T06:36:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.796+0000 7f7ed702f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd3f35b20 con 0x55afd4033300 2024-02-26T06:36:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.796+0000 7f7ed702f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd3f357c0 con 0x55afd4029d80 2024-02-26T06:36:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.796+0000 7f7ed702f740 10 client.0 _ll_register_callbacks cb 0x55afd3f49520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:36:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.796+0000 7f7ed1800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4029d80 0x55afd4030360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.796+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd40308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd40308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38698/0 (socket says 172.21.15.49:38698) 2024-02-26T06:36:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed0e00640 1 -- 172.21.15.49:0/3723857214 learned_addr learned my addr 172.21.15.49:0/3723857214 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed2200640 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd4030de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed0e00640 1 -- 172.21.15.49:0/3723857214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4029d80 msgr2=0x55afd4030360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4029d80 0x55afd4030360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed0e00640 1 -- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 msgr2=0x55afd4030de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd4030de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed0e00640 1 -- 172.21.15.49:0/3723857214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec4009c80 con 0x55afd402aaf0 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.797+0000 7f7ed2200640 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd4030de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.798+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd40308a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7eb802c780 tx=0x7f7eb8003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c686ae8250f12d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.798+0000 7f7eb3400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.798+0000 7f7eb3400640 1 -- 172.21.15.49:0/3723857214 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80bca40 con 0x55afd402aaf0 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.798+0000 7f7eb3400640 1 -- 172.21.15.49:0/3723857214 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f7eb80bd7e0 con 0x55afd402aaf0 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.798+0000 7f7ed702f740 1 -- 172.21.15.49:0/3723857214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afd3f62ca0 con 0x55afd402aaf0 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.798+0000 7f7eb3400640 1 -- 172.21.15.49:0/3723857214 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7eb80bd0e0 con 0x55afd402aaf0 2024-02-26T06:36:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.798+0000 7f7ed702f740 10 client.16899 fetch_fsmap learned FSMap version 51 2024-02-26T06:36:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.798+0000 7f7ed702f740 1 -- 172.21.15.49:0/3723857214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55afd3f8ec20 con 0x55afd402aaf0 2024-02-26T06:36:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.799+0000 7f7eb3400640 1 -- 172.21.15.49:0/3723857214 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55afd3f62ca0 con 0x55afd402aaf0 2024-02-26T06:36:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.799+0000 7f7eb3400640 1 -- 172.21.15.49:0/3723857214 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 51) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7eb80d4020 con 0x55afd402aaf0 2024-02-26T06:36:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.800+0000 7f7ed702f740 10 client.16899 fetch_fsmap finished waiting for FSMap version 51 2024-02-26T06:36:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.800+0000 7f7ed702f740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:36:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.800+0000 7f7ed702f740 20 client.16899 populate_metadata read hostname 'smithi049' 2024-02-26T06:36:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.800+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55afd4043160 0x55afd4045580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.800+0000 7f7ed702f740 4 client.16899 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:36:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.800+0000 7f7ed702f740 1 -- 172.21.15.49:0/3723857214 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afd3f86cb0 con 0x55afd4043160 2024-02-26T06:36:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.800+0000 7f7ed1800640 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55afd4043160 0x55afd4045580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.801+0000 7f7ed1800640 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55afd4043160 0x55afd4045580 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cfac140f160e1a3 server_cookie=72ebd8e9e82e2fc4 in_seq=0 out_seq=0 2024-02-26T06:36:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.801+0000 7f7eb3400640 10 client.16899 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.802+0000 7f7eb3400640 1 -- 172.21.15.49:0/3723857214 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55afd3f86cb0 con 0x55afd4043160 2024-02-26T06:36:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.802+0000 7f7eb3400640 10 client.16899 handle_command_reply: tid=0 2024-02-26T06:36:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.802+0000 7f7ed702f740 1 client.16899 shutdown 2024-02-26T06:36:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7ed702f740 1 -- 172.21.15.49:0/3723857214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 msgr2=0x55afd40308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd40308a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7eb802c780 tx=0x7f7eb8003670 comp rx=0 tx=0).stop 2024-02-26T06:36:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7ed702f740 1 -- 172.21.15.49:0/3723857214 shutdown_connections 2024-02-26T06:36:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55afd4043160 0x55afd4045580 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7eb3400640 0 client.16899 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd4030de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd402aaf0 0x55afd40308a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3723857214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4029d80 0x55afd4030360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.803+0000 7f7ed702f740 1 -- 172.21.15.49:0/3723857214 >> 172.21.15.49:0/3723857214 conn(0x55afd3ff1480 msgr2=0x55afd40267c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.804+0000 7f7ed702f740 1 -- 172.21.15.49:0/3723857214 shutdown_connections 2024-02-26T06:36:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.804+0000 7f7ed702f740 1 -- 172.21.15.49:0/3723857214 wait complete. 2024-02-26T06:36:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.804+0000 7f7ed702f740 20 client.16899 trim_cache size 0 max 16384 2024-02-26T06:36:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.828+0000 7f7ed702f740 1 Processor -- start 2024-02-26T06:36:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.828+0000 7f7ed702f740 1 -- start start 2024-02-26T06:36:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.828+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 0x55afd4032a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.828+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd402aaf0 0x55afd402d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.828+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd402db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.828+0000 7f7ed702f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd3cd8e60 con 0x55afd4029d80 2024-02-26T06:36:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.828+0000 7f7ed702f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4025a70 con 0x55afd4033300 2024-02-26T06:36:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.828+0000 7f7ed702f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd404a790 con 0x55afd402aaf0 2024-02-26T06:36:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd402aaf0 0x55afd402d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd402aaf0 0x55afd402d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:60400/0 (socket says 172.21.15.49:60400) 2024-02-26T06:36:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed1800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 0x55afd4032a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed0e00640 1 -- 172.21.15.49:0/3688220870 learned_addr learned my addr 172.21.15.49:0/3688220870 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed2200640 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd402db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed1800640 1 -- 172.21.15.49:0/3688220870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd402aaf0 msgr2=0x55afd402d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed1800640 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd402aaf0 0x55afd402d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed1800640 1 -- 172.21.15.49:0/3688220870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 msgr2=0x55afd402db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed1800640 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd402db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.829+0000 7f7ed1800640 1 -- 172.21.15.49:0/3688220870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd3f2b500 con 0x55afd4029d80 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.830+0000 7f7ed2200640 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd402db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.830+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd402aaf0 0x55afd402d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.830+0000 7f7ed1800640 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 0x55afd4032a90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80f62f0 tx=0x7f7ec001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95f4dcc5a5eedf3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.830+0000 7f7ebe000640 1 -- 172.21.15.49:0/3688220870 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0023040 con 0x55afd4029d80 2024-02-26T06:36:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.830+0000 7f7ebe000640 1 -- 172.21.15.49:0/3688220870 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ec0024040 con 0x55afd4029d80 2024-02-26T06:36:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.830+0000 7f7ebe000640 1 -- 172.21.15.49:0/3688220870 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0017070 con 0x55afd4029d80 2024-02-26T06:36:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.832+0000 7f7ed702f740 1 -- 172.21.15.49:0/3688220870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 msgr2=0x55afd4032a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.832+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 0x55afd4032a90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80f62f0 tx=0x7f7ec001b000 comp rx=0 tx=0).stop 2024-02-26T06:36:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.832+0000 7f7ed702f740 1 -- 172.21.15.49:0/3688220870 shutdown_connections 2024-02-26T06:36:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.832+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd4033300 0x55afd402db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.832+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd402aaf0 0x55afd402d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.832+0000 7f7ed702f740 1 --2- 172.21.15.49:0/3688220870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 0x55afd4032a90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.832+0000 7f7ed702f740 1 -- 172.21.15.49:0/3688220870 >> 172.21.15.49:0/3688220870 conn(0x55afd3ff1480 msgr2=0x55afd402ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.832+0000 7f7ed702f740 1 -- 172.21.15.49:0/3688220870 shutdown_connections 2024-02-26T06:36:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.833+0000 7f7ed702f740 1 -- 172.21.15.49:0/3688220870 wait complete. 2024-02-26T06:36:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.834+0000 7f7ed702f740 1 -- start start 2024-02-26T06:36:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.835+0000 7f7ed702f740 1 Processor -- start 2024-02-26T06:36:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.836+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 0x55afd4038f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.836+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd402aaf0 0x55afd40394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.836+0000 7f7ed702f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 0x55afd4099670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.836+0000 7f7ed702f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4039d70 con 0x55afd4029d80 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.836+0000 7f7ed702f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4099bb0 con 0x55afd402aaf0 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.836+0000 7f7ed702f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4099cf0 con 0x55afd4033300 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.836+0000 7f7ed702f740 10 client.0 _ll_register_callbacks cb 0x7f7eb8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.837+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd402aaf0 0x55afd40394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.837+0000 7f7ed0e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd402aaf0 0x55afd40394c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:48304/0 (socket says 172.21.15.49:48304) 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.837+0000 7f7ed0e00640 1 -- 172.21.15.49:0/1006559788 learned_addr learned my addr 172.21.15.49:0/1006559788 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.837+0000 7f7ed0e00640 1 -- 172.21.15.49:0/1006559788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 msgr2=0x55afd4099670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.838+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 0x55afd4099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.838+0000 7f7ed0e00640 1 -- 172.21.15.49:0/1006559788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 msgr2=0x55afd4038f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:36:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.838+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 0x55afd4038f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.838+0000 7f7ed0e00640 1 -- 172.21.15.49:0/1006559788 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afd3f8ec20 con 0x55afd402aaf0 2024-02-26T06:36:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.838+0000 7f7ed0e00640 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd402aaf0 0x55afd40394c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80b9ab0 tx=0x7f7eb802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98bbdcccc84d2278 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.840+0000 7f7eb3400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:36:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.840+0000 7f7eb3400640 1 -- 172.21.15.49:0/1006559788 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80d2070 con 0x55afd402aaf0 2024-02-26T06:36:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.840+0000 7f7eb3400640 1 -- 172.21.15.49:0/1006559788 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f7eb80bd7e0 con 0x55afd402aaf0 2024-02-26T06:36:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.840+0000 7f7ed702f740 1 -- 172.21.15.49:0/1006559788 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afd3fb5d00 con 0x55afd402aaf0 2024-02-26T06:36:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.843+0000 7f7eb3400640 1 -- 172.21.15.49:0/1006559788 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7eb80bca40 con 0x55afd402aaf0 2024-02-26T06:36:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.843+0000 7f7ed702f740 10 client.27578 fetch_fsmap learned FSMap version 51 2024-02-26T06:36:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.843+0000 7f7ed702f740 1 -- 172.21.15.49:0/1006559788 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55afd3f11ad0 con 0x55afd402aaf0 2024-02-26T06:36:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.844+0000 7f7eb3400640 1 -- 172.21.15.49:0/1006559788 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55afd3fb5d00 con 0x55afd402aaf0 2024-02-26T06:36:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.844+0000 7f7eb3400640 1 -- 172.21.15.49:0/1006559788 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 51) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7eb80f8070 con 0x55afd402aaf0 2024-02-26T06:36:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.844+0000 7f7ed702f740 10 client.27578 fetch_fsmap finished waiting for FSMap version 51 2024-02-26T06:36:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.844+0000 7f7ed702f740 10 client.27578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:36:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.844+0000 7f7ed702f740 20 client.27578 populate_metadata read hostname 'smithi049' 2024-02-26T06:36:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.844+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55afd4043160 0x55afd409cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.844+0000 7f7ed702f740 4 client.27578 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:36:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.844+0000 7f7ed702f740 1 -- 172.21.15.49:0/1006559788 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55afd3c90600 con 0x55afd4043160 2024-02-26T06:36:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.845+0000 7f7ed1800640 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55afd4043160 0x55afd409cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.845+0000 7f7ed1800640 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55afd4043160 0x55afd409cf80 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e06ad56b8bcff server_cookie=378f1188c554e8e4 in_seq=0 out_seq=0 2024-02-26T06:36:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.845+0000 7f7eb3400640 10 client.27578 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7eb3400640 1 -- 172.21.15.49:0/1006559788 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55afd3c90600 con 0x55afd4043160 2024-02-26T06:36:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7eb3400640 10 client.27578 handle_command_reply: tid=0 2024-02-26T06:36:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 client.27578 shutdown 2024-02-26T06:36:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 -- 172.21.15.49:0/1006559788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd402aaf0 msgr2=0x55afd40394c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd402aaf0 0x55afd40394c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80b9ab0 tx=0x7f7eb802bc30 comp rx=0 tx=0).stop 2024-02-26T06:36:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 -- 172.21.15.49:0/1006559788 shutdown_connections 2024-02-26T06:36:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55afd4043160 0x55afd409cf80 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55afd402aaf0 0x55afd40394c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55afd4033300 0x55afd4099670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 --2- 172.21.15.49:0/1006559788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55afd4029d80 0x55afd4038f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7ed702f740 1 -- 172.21.15.49:0/1006559788 >> 172.21.15.49:0/1006559788 conn(0x55afd3ff1480 msgr2=0x55afd4023480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:20.999+0000 7f7eb3400640 0 client.27578 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.000+0000 7f7ed702f740 1 -- 172.21.15.49:0/1006559788 shutdown_connections 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.000+0000 7f7ed702f740 1 -- 172.21.15.49:0/1006559788 wait complete. 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.000+0000 7f7ed702f740 20 client.27578 trim_cache size 0 max 16384 2024-02-26T06:36:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:36:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:36:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:36:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:36:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.001+0000 7f7ed4200640 1 -- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7eb40a22d0 msgr2=0x7f7eb40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.001+0000 7f7ed4200640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7eb40a22d0 0x7f7eb40a4790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55afd402b250 tx=0x7f7ec0005f90 comp rx=0 tx=0).stop 2024-02-26T06:36:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.002+0000 7f7ed4200640 1 -- 172.21.15.49:0/4105991600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ecc108110 msgr2=0x7f7ecc19e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.002+0000 7f7ed4200640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ecc108110 0x7f7ecc19e280 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000c00 tx=0x7f7eb802c5f0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.003+0000 7f7ed4200640 1 -- 172.21.15.49:0/4105991600 shutdown_connections 2024-02-26T06:36:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.003+0000 7f7ed4200640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7eb40a22d0 0x7f7eb40a4790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.003+0000 7f7ed4200640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ecc10d760 0x7f7ecc1a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.003+0000 7f7ed4200640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ecc108110 0x7f7ecc19e280 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.003+0000 7f7ed4200640 1 --2- 172.21.15.49:0/4105991600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ecc107750 0x7f7ecc19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.003+0000 7f7ed4200640 1 -- 172.21.15.49:0/4105991600 >> 172.21.15.49:0/4105991600 conn(0x7f7ecc0756f0 msgr2=0x7f7ecc075c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.004+0000 7f7ed4200640 1 -- 172.21.15.49:0/4105991600 shutdown_connections 2024-02-26T06:36:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:21.004+0000 7f7ed4200640 1 -- 172.21.15.49:0/4105991600 wait complete. 2024-02-26T06:36:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 750.0 s, 2139.8 tps, lat 46.304 ms stddev 53.994 2024-02-26T06:36:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:21 smithi080 ceph-mon[36206]: pgmap v404: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-02-26T06:36:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:21 smithi080 ceph-mon[36206]: osdmap e103: 12 total, 12 up, 12 in 2024-02-26T06:36:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:21 smithi115 ceph-mon[36426]: pgmap v404: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-02-26T06:36:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:21 smithi115 ceph-mon[36426]: osdmap e103: 12 total, 12 up, 12 in 2024-02-26T06:36:21.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:21 smithi049 ceph-mon[31696]: pgmap v404: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-02-26T06:36:21.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:21 smithi049 ceph-mon[31696]: osdmap e103: 12 total, 12 up, 12 in 2024-02-26T06:36:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:23 smithi080 ceph-mon[36206]: pgmap v406: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 630 op/s 2024-02-26T06:36:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:23 smithi115 ceph-mon[36426]: pgmap v406: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 630 op/s 2024-02-26T06:36:23.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:23 smithi049 ceph-mon[31696]: pgmap v406: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 630 op/s 2024-02-26T06:36:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:25 smithi080 ceph-mon[36206]: pgmap v407: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 633 op/s 2024-02-26T06:36:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:25 smithi115 ceph-mon[36426]: pgmap v407: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 633 op/s 2024-02-26T06:36:25.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:25 smithi049 ceph-mon[31696]: pgmap v407: 97 pgs: 97 active+clean; 4.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 633 op/s 2024-02-26T06:36:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 755.0 s, 1695.2 tps, lat 59.058 ms stddev 74.659 2024-02-26T06:36:27.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:27 smithi080 ceph-mon[36206]: pgmap v408: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 685 op/s 2024-02-26T06:36:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:27 smithi115 ceph-mon[36426]: pgmap v408: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 685 op/s 2024-02-26T06:36:27.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:27 smithi049 ceph-mon[31696]: pgmap v408: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 685 op/s 2024-02-26T06:36:29.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:29 smithi080 ceph-mon[36206]: pgmap v409: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-02-26T06:36:29.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:29 smithi115 ceph-mon[36426]: pgmap v409: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-02-26T06:36:29.680 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:29 smithi049 ceph-mon[31696]: pgmap v409: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-02-26T06:36:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 760.0 s, 1622.4 tps, lat 61.892 ms stddev 76.848 2024-02-26T06:36:31.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:31 smithi080 ceph-mon[36206]: pgmap v410: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-02-26T06:36:31.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:31 smithi115 ceph-mon[36426]: pgmap v410: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-02-26T06:36:31.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:31 smithi049 ceph-mon[31696]: pgmap v410: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-02-26T06:36:33.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:33 smithi080 ceph-mon[36206]: pgmap v411: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 624 op/s 2024-02-26T06:36:33.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:33 smithi115 ceph-mon[36426]: pgmap v411: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 624 op/s 2024-02-26T06:36:33.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:33 smithi049 ceph-mon[31696]: pgmap v411: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 624 op/s 2024-02-26T06:36:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.196+0000 7fba38600640 1 -- 172.21.15.49:0/2262745211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 msgr2=0x7fba3010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.196+0000 7fba38600640 1 --2- 172.21.15.49:0/2262745211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 0x7fba3010da20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fba2001a910 tx=0x7fba2003cca0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.197+0000 7fba38600640 1 -- 172.21.15.49:0/2262745211 shutdown_connections 2024-02-26T06:36:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.197+0000 7fba38600640 1 --2- 172.21.15.49:0/2262745211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 0x7fba3010da20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.197+0000 7fba38600640 1 --2- 172.21.15.49:0/2262745211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3011a960 0x7fba3011cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.197+0000 7fba38600640 1 --2- 172.21.15.49:0/2262745211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba3010f8c0 0x7fba3010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.197+0000 7fba38600640 1 -- 172.21.15.49:0/2262745211 >> 172.21.15.49:0/2262745211 conn(0x7fba3006c220 msgr2=0x7fba3006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.197+0000 7fba38600640 1 -- 172.21.15.49:0/2262745211 shutdown_connections 2024-02-26T06:36:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.197+0000 7fba38600640 1 -- 172.21.15.49:0/2262745211 wait complete. 2024-02-26T06:36:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.198+0000 7fba38600640 1 Processor -- start 2024-02-26T06:36:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.199+0000 7fba38600640 1 -- start start 2024-02-26T06:36:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.201+0000 7fba38600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3010f8c0 0x7fba30119e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.202+0000 7fba38600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 0x7fba3011a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.202+0000 7fba38600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba3011a960 0x7fba30116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.202+0000 7fba38600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30116890 con 0x7fba3010f8c0 2024-02-26T06:36:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.202+0000 7fba38600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30116a00 con 0x7fba3010fd30 2024-02-26T06:36:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.202+0000 7fba38600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30116bc0 con 0x7fba3011a960 2024-02-26T06:36:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.202+0000 7fba35c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3010f8c0 0x7fba30119e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 0x7fba3011a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 0x7fba3011a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49930/0 (socket says 172.21.15.49:49930) 2024-02-26T06:36:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35200640 1 -- 172.21.15.49:0/3275279036 learned_addr learned my addr 172.21.15.49:0/3275279036 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba36600640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba3011a960 0x7fba30116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35c00640 1 -- 172.21.15.49:0/3275279036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba3011a960 msgr2=0x7fba30116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35c00640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba3011a960 0x7fba30116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35c00640 1 -- 172.21.15.49:0/3275279036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 msgr2=0x7fba3011a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35c00640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 0x7fba3011a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35c00640 1 -- 172.21.15.49:0/3275279036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2001a570 con 0x7fba3010f8c0 2024-02-26T06:36:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.203+0000 7fba35200640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 0x7fba3011a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:36:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.204+0000 7fba35c00640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3010f8c0 0x7fba30119e90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c970 tx=0x7fba2400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.204+0000 7fba2e000640 1 -- 172.21.15.49:0/3275279036 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24007bb0 con 0x7fba3010f8c0 2024-02-26T06:36:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.205+0000 7fba38600640 1 -- 172.21.15.49:0/3275279036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba301aa5f0 con 0x7fba3010f8c0 2024-02-26T06:36:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.205+0000 7fba38600640 1 -- 172.21.15.49:0/3275279036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba301aaa10 con 0x7fba3010f8c0 2024-02-26T06:36:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.205+0000 7fba2e000640 1 -- 172.21.15.49:0/3275279036 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba24007d10 con 0x7fba3010f8c0 2024-02-26T06:36:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.205+0000 7fba2e000640 1 -- 172.21.15.49:0/3275279036 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24018760 con 0x7fba3010f8c0 2024-02-26T06:36:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.211+0000 7fba3b434740 1 Processor -- start 2024-02-26T06:36:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.213+0000 7fba3b434740 1 -- start start 2024-02-26T06:36:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.213+0000 7fba3b434740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d69aaf0 0x55d54d699840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.213+0000 7fba3b434740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d69a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.213+0000 7fba3b434740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 0x55d54d6a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.213+0000 7fba3b434740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d6555f0 con 0x55d54d6a3300 2024-02-26T06:36:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.213+0000 7fba3b434740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d607890 con 0x55d54d699d80 2024-02-26T06:36:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.213+0000 7fba3b434740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d604dc0 con 0x55d54d69aaf0 2024-02-26T06:36:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.214+0000 7fba36600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d69aaf0 0x55d54d699840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.214+0000 7fba36600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d69aaf0 0x55d54d699840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38834/0 (socket says 172.21.15.49:38834) 2024-02-26T06:36:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.214+0000 7fba36600640 1 -- 172.21.15.49:0/2052651678 learned_addr learned my addr 172.21.15.49:0/2052651678 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.214+0000 7fba35200640 1 --2- 172.21.15.49:0/2052651678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 0x55d54d6a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.215+0000 7fba36600640 1 -- 172.21.15.49:0/2052651678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 msgr2=0x55d54d69a200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:36:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.215+0000 7fba36600640 1 --2- 172.21.15.49:0/2052651678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d69a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.215+0000 7fba36600640 1 -- 172.21.15.49:0/2052651678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 msgr2=0x55d54d6a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.215+0000 7fba36600640 1 --2- 172.21.15.49:0/2052651678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 0x55d54d6a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.215+0000 7fba36600640 1 -- 172.21.15.49:0/2052651678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d54d581ad0 con 0x55d54d69aaf0 2024-02-26T06:36:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.215+0000 7fba2e000640 1 -- 172.21.15.49:0/3275279036 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fba240189a0 con 0x7fba3010f8c0 2024-02-26T06:36:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.216+0000 7fba36600640 1 --2- 172.21.15.49:0/2052651678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d69aaf0 0x55d54d699840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fba2800a400 tx=0x7fba28009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6332713d7c88e246 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.217+0000 7fba1ea00640 1 -- 172.21.15.49:0/2052651678 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28010040 con 0x55d54d69aaf0 2024-02-26T06:36:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.217+0000 7fba1ea00640 1 -- 172.21.15.49:0/2052651678 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba28013c70 con 0x55d54d69aaf0 2024-02-26T06:36:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.217+0000 7fba1ea00640 1 -- 172.21.15.49:0/2052651678 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28014da0 con 0x55d54d69aaf0 2024-02-26T06:36:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.218+0000 7fba2e000640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fba180a23d0 0x7fba180a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.218+0000 7fba2e000640 1 -- 172.21.15.49:0/3275279036 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fba240cdf70 con 0x7fba3010f8c0 2024-02-26T06:36:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.219+0000 7fba3b434740 1 -- 172.21.15.49:0/2052651678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d69aaf0 msgr2=0x55d54d699840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.219+0000 7fba3b434740 1 --2- 172.21.15.49:0/2052651678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d69aaf0 0x55d54d699840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fba2800a400 tx=0x7fba28009650 comp rx=0 tx=0).stop 2024-02-26T06:36:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.220+0000 7fba3b434740 1 -- 172.21.15.49:0/2052651678 shutdown_connections 2024-02-26T06:36:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.220+0000 7fba3b434740 1 --2- 172.21.15.49:0/2052651678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d69a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.220+0000 7fba3b434740 1 --2- 172.21.15.49:0/2052651678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 0x55d54d6a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.220+0000 7fba3b434740 1 --2- 172.21.15.49:0/2052651678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d69aaf0 0x55d54d699840 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.220+0000 7fba35c00640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fba180a23d0 0x7fba180a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.220+0000 7fba3b434740 1 -- 172.21.15.49:0/2052651678 >> 172.21.15.49:0/2052651678 conn(0x55d54d661480 msgr2=0x55d54d694a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.220+0000 7fba3b434740 1 -- 172.21.15.49:0/2052651678 shutdown_connections 2024-02-26T06:36:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.221+0000 7fba3b434740 1 -- 172.21.15.49:0/2052651678 wait complete. 2024-02-26T06:36:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.221+0000 7fba35c00640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fba180a23d0 0x7fba180a4890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55d54d69b250 tx=0x7fba24018930 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.221+0000 7fba3b434740 1 -- start start 2024-02-26T06:36:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.222+0000 7fba3b434740 1 Processor -- start 2024-02-26T06:36:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.222+0000 7fba3b434740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d699d80 0x55d54d6a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba3b434740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba3b434740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 0x55d54d6a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba3b434740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d6005a0 con 0x55d54d6a3300 2024-02-26T06:36:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba3b434740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d5a5b20 con 0x55d54d69aaf0 2024-02-26T06:36:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba3b434740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d5a57c0 con 0x55d54d699d80 2024-02-26T06:36:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba3b434740 10 client.0 _ll_register_callbacks cb 0x55d54d5b9520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:36:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba36600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d699d80 0x55d54d6a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba36600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d699d80 0x55d54d6a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38838/0 (socket says 172.21.15.49:38838) 2024-02-26T06:36:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba36600640 1 -- 172.21.15.49:0/4291005088 learned_addr learned my addr 172.21.15.49:0/4291005088 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.223+0000 7fba35200640 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.224+0000 7fba36600640 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 0x55d54d6a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.224+0000 7fba36600640 1 -- 172.21.15.49:0/4291005088 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 msgr2=0x55d54d6a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.224+0000 7fba36600640 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.224+0000 7fba36600640 1 -- 172.21.15.49:0/4291005088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 msgr2=0x55d54d6a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.224+0000 7fba36600640 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 0x55d54d6a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.224+0000 7fba36600640 1 -- 172.21.15.49:0/4291005088 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba28009c60 con 0x55d54d699d80 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.225+0000 7fba36600640 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d699d80 0x55d54d6a02c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fba280094c0 tx=0x7fba2800c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=711152ad0c52293b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.225+0000 7fba13400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.225+0000 7fba13400640 1 -- 172.21.15.49:0/4291005088 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28013590 con 0x55d54d699d80 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.225+0000 7fba13400640 1 -- 172.21.15.49:0/4291005088 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fba280136f0 con 0x55d54d699d80 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.225+0000 7fba3b434740 1 -- 172.21.15.49:0/4291005088 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d54d5d2ca0 con 0x55d54d699d80 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.226+0000 7fba13400640 1 -- 172.21.15.49:0/4291005088 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba280138b0 con 0x55d54d699d80 2024-02-26T06:36:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.226+0000 7fba3b434740 10 client.27538 fetch_fsmap learned FSMap version 51 2024-02-26T06:36:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.226+0000 7fba3b434740 1 -- 172.21.15.49:0/4291005088 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d54d5fec20 con 0x55d54d699d80 2024-02-26T06:36:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.227+0000 7fba13400640 1 -- 172.21.15.49:0/4291005088 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d54d5d2ca0 con 0x55d54d699d80 2024-02-26T06:36:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.227+0000 7fba13400640 1 -- 172.21.15.49:0/4291005088 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 51) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fba28008030 con 0x55d54d699d80 2024-02-26T06:36:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.228+0000 7fba3b434740 10 client.27538 fetch_fsmap finished waiting for FSMap version 51 2024-02-26T06:36:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.228+0000 7fba3b434740 10 client.27538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:36:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.229+0000 7fba3b434740 20 client.27538 populate_metadata read hostname 'smithi049' 2024-02-26T06:36:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.229+0000 7fba3b434740 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d54d6b32a0 0x55d54d6b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.229+0000 7fba3b434740 4 client.27538 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:36:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.229+0000 7fba3b434740 1 -- 172.21.15.49:0/4291005088 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d54d5f6cb0 con 0x55d54d6b32a0 2024-02-26T06:36:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.229+0000 7fba35200640 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d54d6b32a0 0x55d54d6b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.231+0000 7fba35200640 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d54d6b32a0 0x55d54d6b56c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd7109441a937ed server_cookie=2baf9cb1fdf31190 in_seq=0 out_seq=0 2024-02-26T06:36:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.232+0000 7fba13400640 10 client.27538 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.233+0000 7fba13400640 1 -- 172.21.15.49:0/4291005088 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d54d5f6cb0 con 0x55d54d6b32a0 2024-02-26T06:36:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.233+0000 7fba13400640 10 client.27538 handle_command_reply: tid=0 2024-02-26T06:36:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.233+0000 7fba3b434740 1 client.27538 shutdown 2024-02-26T06:36:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.233+0000 7fba3b434740 1 -- 172.21.15.49:0/4291005088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d699d80 msgr2=0x55d54d6a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.233+0000 7fba3b434740 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d699d80 0x55d54d6a02c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fba280094c0 tx=0x7fba2800c040 comp rx=0 tx=0).stop 2024-02-26T06:36:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.233+0000 7fba3b434740 1 -- 172.21.15.49:0/4291005088 shutdown_connections 2024-02-26T06:36:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.233+0000 7fba3b434740 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d54d6b32a0 0x55d54d6b56c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.233+0000 7fba3b434740 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.234+0000 7fba3b434740 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d6a3300 0x55d54d6a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.234+0000 7fba3b434740 1 --2- 172.21.15.49:0/4291005088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d699d80 0x55d54d6a02c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.234+0000 7fba3b434740 1 -- 172.21.15.49:0/4291005088 >> 172.21.15.49:0/4291005088 conn(0x55d54d661480 msgr2=0x55d54d6967c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.234+0000 7fba13400640 0 client.27538 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.234+0000 7fba3b434740 1 -- 172.21.15.49:0/4291005088 shutdown_connections 2024-02-26T06:36:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.234+0000 7fba3b434740 1 -- 172.21.15.49:0/4291005088 wait complete. 2024-02-26T06:36:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.234+0000 7fba3b434740 20 client.27538 trim_cache size 0 max 16384 2024-02-26T06:36:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.259+0000 7fba3b434740 1 Processor -- start 2024-02-26T06:36:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.259+0000 7fba3b434740 1 -- start start 2024-02-26T06:36:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.259+0000 7fba3b434740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d699d80 0x55d54d6a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.259+0000 7fba3b434740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.259+0000 7fba3b434740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d69b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.260+0000 7fba3b434740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d348e60 con 0x55d54d699d80 2024-02-26T06:36:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.260+0000 7fba3b434740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d69b0e0 con 0x55d54d69aaf0 2024-02-26T06:36:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.260+0000 7fba3b434740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d6a5f10 con 0x55d54d6a3300 2024-02-26T06:36:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.260+0000 7fba35200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.260+0000 7fba35200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49948/0 (socket says 172.21.15.49:49948) 2024-02-26T06:36:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.260+0000 7fba36600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d699d80 0x55d54d6a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.261+0000 7fba36600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d699d80 0x55d54d6a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33536/0 (socket says 172.21.15.49:33536) 2024-02-26T06:36:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.261+0000 7fba36600640 1 -- 172.21.15.49:0/1700701703 learned_addr learned my addr 172.21.15.49:0/1700701703 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.262+0000 7fba35200640 1 -- 172.21.15.49:0/1700701703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 msgr2=0x55d54d69b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.262+0000 7fba36600640 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d69b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.263+0000 7fba35200640 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d69b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.263+0000 7fba35200640 1 -- 172.21.15.49:0/1700701703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d699d80 msgr2=0x55d54d6a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.263+0000 7fba35200640 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d699d80 0x55d54d6a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.263+0000 7fba35200640 1 -- 172.21.15.49:0/1700701703 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d54d59b500 con 0x55d54d69aaf0 2024-02-26T06:36:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.263+0000 7fba36600640 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d69b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:36:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.264+0000 7fba35200640 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0b60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fba2003d2b0 tx=0x7fba2001a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab601a8fb76a4f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.264+0000 7fba1e000640 1 -- 172.21.15.49:0/1700701703 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20057040 con 0x55d54d69aaf0 2024-02-26T06:36:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.264+0000 7fba1e000640 1 -- 172.21.15.49:0/1700701703 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba20059c30 con 0x55d54d69aaf0 2024-02-26T06:36:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.264+0000 7fba1e000640 1 -- 172.21.15.49:0/1700701703 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba200052a0 con 0x55d54d69aaf0 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 -- 172.21.15.49:0/1700701703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 msgr2=0x55d54d6a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0b60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fba2003d2b0 tx=0x7fba2001a270 comp rx=0 tx=0).stop 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 -- 172.21.15.49:0/1700701703 shutdown_connections 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d69aaf0 0x55d54d6a0b60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d69b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 --2- 172.21.15.49:0/1700701703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d699d80 0x55d54d6a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 -- 172.21.15.49:0/1700701703 >> 172.21.15.49:0/1700701703 conn(0x55d54d661480 msgr2=0x55d54d6ba020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 -- 172.21.15.49:0/1700701703 shutdown_connections 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.265+0000 7fba3b434740 1 -- 172.21.15.49:0/1700701703 wait complete. 2024-02-26T06:36:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 1 -- start start 2024-02-26T06:36:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 1 Processor -- start 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d6a8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d69aaf0 0x55d54d6a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d7087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d6a8f70 con 0x55d54d69aaf0 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d708ce0 con 0x55d54d699d80 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54d708e50 con 0x55d54d6a3300 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.266+0000 7fba3b434740 10 client.0 _ll_register_callbacks cb 0x7fba28009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d6a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d6a8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:49962/0 (socket says 172.21.15.49:49962) 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 -- 172.21.15.49:0/2021422427 learned_addr learned my addr 172.21.15.49:0/2021422427 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d7087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba35200640 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d69aaf0 0x55d54d6a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 -- 172.21.15.49:0/2021422427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 msgr2=0x55d54d7087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d7087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 -- 172.21.15.49:0/2021422427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d69aaf0 msgr2=0x55d54d6a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d69aaf0 0x55d54d6a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 -- 172.21.15.49:0/2021422427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3011d640 con 0x55d54d699d80 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba36600640 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d6a8330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fba28004a80 tx=0x7fba2802e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc5e0bbbd33387a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.267+0000 7fba13400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.268+0000 7fba13400640 1 -- 172.21.15.49:0/2021422427 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2800ad80 con 0x55d54d699d80 2024-02-26T06:36:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.268+0000 7fba3b434740 1 -- 172.21.15.49:0/2021422427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d54d625d00 con 0x55d54d699d80 2024-02-26T06:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.268+0000 7fba13400640 1 -- 172.21.15.49:0/2021422427 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fba280146d0 con 0x55d54d699d80 2024-02-26T06:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.268+0000 7fba13400640 1 -- 172.21.15.49:0/2021422427 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba28014420 con 0x55d54d699d80 2024-02-26T06:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.268+0000 7fba3b434740 10 client.27593 fetch_fsmap learned FSMap version 52 2024-02-26T06:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.268+0000 7fba3b434740 1 -- 172.21.15.49:0/2021422427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55d54d5c1290 con 0x55d54d699d80 2024-02-26T06:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.268+0000 7fba13400640 1 -- 172.21.15.49:0/2021422427 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d54d625d00 con 0x55d54d699d80 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.269+0000 7fba13400640 1 -- 172.21.15.49:0/2021422427 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fba28008040 con 0x55d54d699d80 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.269+0000 7fba3b434740 10 client.27593 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.269+0000 7fba3b434740 10 client.27593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.269+0000 7fba3b434740 20 client.27593 populate_metadata read hostname 'smithi049' 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.269+0000 7fba3b434740 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d54d70a350 0x55d54d70c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.269+0000 7fba3b434740 4 client.27593 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.269+0000 7fba3b434740 1 -- 172.21.15.49:0/2021422427 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d54d300600 con 0x55d54d70a350 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.269+0000 7fba35200640 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d54d70a350 0x55d54d70c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.270+0000 7fba35200640 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d54d70a350 0x55d54d70c770 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be8a61b9e41bcf8 server_cookie=8a1b7d4cfd12f877 in_seq=0 out_seq=0 2024-02-26T06:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.270+0000 7fba13400640 10 client.27593 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba13400640 1 -- 172.21.15.49:0/2021422427 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d54d300600 con 0x55d54d70a350 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba13400640 10 client.27593 handle_command_reply: tid=0 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 client.27593 shutdown 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 -- 172.21.15.49:0/2021422427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 msgr2=0x55d54d6a8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d6a8330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fba28004a80 tx=0x7fba2802e040 comp rx=0 tx=0).stop 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 -- 172.21.15.49:0/2021422427 shutdown_connections 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d54d70a350 0x55d54d70c770 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d54d6a3300 0x55d54d7087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba13400640 0 client.27593 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d54d69aaf0 0x55d54d6a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 --2- 172.21.15.49:0/2021422427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d54d699d80 0x55d54d6a8330 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 -- 172.21.15.49:0/2021422427 >> 172.21.15.49:0/2021422427 conn(0x55d54d661480 msgr2=0x55d54d69e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 -- 172.21.15.49:0/2021422427 shutdown_connections 2024-02-26T06:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 1 -- 172.21.15.49:0/2021422427 wait complete. 2024-02-26T06:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.286+0000 7fba3b434740 20 client.27593 trim_cache size 0 max 16384 2024-02-26T06:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 -- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fba180a23d0 msgr2=0x7fba180a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fba180a23d0 0x7fba180a4890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55d54d69b250 tx=0x7fba24018930 comp rx=0 tx=0).stop 2024-02-26T06:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 -- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3010f8c0 msgr2=0x7fba30119e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3010f8c0 0x7fba30119e90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c970 tx=0x7fba2400ce30 comp rx=0 tx=0).stop 2024-02-26T06:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 -- 172.21.15.49:0/3275279036 shutdown_connections 2024-02-26T06:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fba180a23d0 0x7fba180a4890 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba3010fd30 0x7fba3011a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba3011a960 0x7fba30116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 --2- 172.21.15.49:0/3275279036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3010f8c0 0x7fba30119e90 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c970 tx=0x7fba2400ce30 comp rx=0 tx=0).stop 2024-02-26T06:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.287+0000 7fba38600640 1 -- 172.21.15.49:0/3275279036 >> 172.21.15.49:0/3275279036 conn(0x7fba3006c220 msgr2=0x7fba3011b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.288+0000 7fba38600640 1 -- 172.21.15.49:0/3275279036 shutdown_connections 2024-02-26T06:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:34.288+0000 7fba38600640 1 -- 172.21.15.49:0/3275279036 wait complete. 2024-02-26T06:36:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:35 smithi080 ceph-mon[36206]: pgmap v412: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.6 MiB/s wr, 591 op/s 2024-02-26T06:36:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:35 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:35 smithi115 ceph-mon[36426]: pgmap v412: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.6 MiB/s wr, 591 op/s 2024-02-26T06:36:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:35 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:35.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:35 smithi049 ceph-mon[31696]: pgmap v412: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.6 MiB/s wr, 591 op/s 2024-02-26T06:36:35.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:35 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:36:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 765.0 s, 1437.2 tps, lat 68.718 ms stddev 85.165 2024-02-26T06:36:37.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:37 smithi080 ceph-mon[36206]: pgmap v413: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 764 op/s 2024-02-26T06:36:37.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:37 smithi115 ceph-mon[36426]: pgmap v413: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 764 op/s 2024-02-26T06:36:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:37 smithi049 ceph-mon[31696]: pgmap v413: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 764 op/s 2024-02-26T06:36:39.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:39 smithi080 ceph-mon[36206]: pgmap v414: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 647 op/s 2024-02-26T06:36:39.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:39 smithi115 ceph-mon[36426]: pgmap v414: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 647 op/s 2024-02-26T06:36:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:39 smithi049 ceph-mon[31696]: pgmap v414: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 647 op/s 2024-02-26T06:36:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 770.0 s, 1297.4 tps, lat 77.689 ms stddev 92.000 2024-02-26T06:36:41.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:41 smithi080 ceph-mon[36206]: pgmap v415: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-02-26T06:36:41.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:41 smithi115 ceph-mon[36426]: pgmap v415: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-02-26T06:36:41.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:41 smithi049 ceph-mon[31696]: pgmap v415: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-02-26T06:36:43.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:43 smithi080 ceph-mon[36206]: pgmap v416: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 730 op/s 2024-02-26T06:36:43.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:43 smithi115 ceph-mon[36426]: pgmap v416: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 730 op/s 2024-02-26T06:36:43.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:43 smithi049 ceph-mon[31696]: pgmap v416: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 730 op/s 2024-02-26T06:36:45.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:45 smithi080 ceph-mon[36206]: pgmap v417: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-02-26T06:36:45.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:45 smithi115 ceph-mon[36426]: pgmap v417: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-02-26T06:36:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:45 smithi049 ceph-mon[31696]: pgmap v417: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-02-26T06:36:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 775.0 s, 965.8 tps, lat 103.838 ms stddev 195.239 2024-02-26T06:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.485+0000 7fd4c8600640 1 -- 172.21.15.49:0/2306606785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4c01035a0 msgr2=0x7fd4c0103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.485+0000 7fd4c8600640 1 --2- 172.21.15.49:0/2306606785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4c01035a0 0x7fd4c0103980 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009900 tx=0x7fd4b402bc30 comp rx=0 tx=0).stop 2024-02-26T06:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.485+0000 7fd4c8600640 1 -- 172.21.15.49:0/2306606785 shutdown_connections 2024-02-26T06:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.485+0000 7fd4c8600640 1 --2- 172.21.15.49:0/2306606785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4c0108670 0x7fd4c0110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.485+0000 7fd4c8600640 1 --2- 172.21.15.49:0/2306606785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4c0103f50 0x7fd4c0107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.485+0000 7fd4c8600640 1 --2- 172.21.15.49:0/2306606785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4c01035a0 0x7fd4c0103980 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.485+0000 7fd4c8600640 1 -- 172.21.15.49:0/2306606785 >> 172.21.15.49:0/2306606785 conn(0x7fd4c00fee00 msgr2=0x7fd4c0101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.485+0000 7fd4c8600640 1 -- 172.21.15.49:0/2306606785 shutdown_connections 2024-02-26T06:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.486+0000 7fd4c8600640 1 -- 172.21.15.49:0/2306606785 wait complete. 2024-02-26T06:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.487+0000 7fd4c8600640 1 Processor -- start 2024-02-26T06:36:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.488+0000 7fd4c8600640 1 -- start start 2024-02-26T06:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.488+0000 7fd4c8600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4c01035a0 0x7fd4c010ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.488+0000 7fd4c8600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4c0108670 0x7fd4c010cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.488+0000 7fd4c8600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4c010d4e0 0x7fd4c010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.489+0000 7fd4c8600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c01aa3a0 con 0x7fd4c0108670 2024-02-26T06:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.489+0000 7fd4c8600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c01aa4e0 con 0x7fd4c010d4e0 2024-02-26T06:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.489+0000 7fd4c8600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c01aa620 con 0x7fd4c01035a0 2024-02-26T06:36:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.489+0000 7fd4c6600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4c010d4e0 0x7fd4c010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.489+0000 7fd4c6600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4c010d4e0 0x7fd4c010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58334/0 (socket says 172.21.15.49:58334) 2024-02-26T06:36:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.489+0000 7fd4c6600640 1 -- 172.21.15.49:0/909551246 learned_addr learned my addr 172.21.15.49:0/909551246 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.489+0000 7fd4c5200640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4c0108670 0x7fd4c010cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.490+0000 7fd4c5c00640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4c01035a0 0x7fd4c010ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.490+0000 7fd4c6600640 1 -- 172.21.15.49:0/909551246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4c01035a0 msgr2=0x7fd4c010ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.490+0000 7fd4c6600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4c01035a0 0x7fd4c010ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.490+0000 7fd4c6600640 1 -- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4c0108670 msgr2=0x7fd4c010cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.490+0000 7fd4c6600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4c0108670 0x7fd4c010cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.490+0000 7fd4c6600640 1 -- 172.21.15.49:0/909551246 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b4009560 con 0x7fd4c010d4e0 2024-02-26T06:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.491+0000 7fd4c6600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4c010d4e0 0x7fd4c010d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800cb00 tx=0x7fd4b800a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.491+0000 7fd4be000640 1 -- 172.21.15.49:0/909551246 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800ae90 con 0x7fd4c010d4e0 2024-02-26T06:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.491+0000 7fd4be000640 1 -- 172.21.15.49:0/909551246 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4b8005020 con 0x7fd4c010d4e0 2024-02-26T06:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.491+0000 7fd4be000640 1 -- 172.21.15.49:0/909551246 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800f6b0 con 0x7fd4c010d4e0 2024-02-26T06:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.492+0000 7fd4c8600640 1 -- 172.21.15.49:0/909551246 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c01aa8b0 con 0x7fd4c010d4e0 2024-02-26T06:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.492+0000 7fd4c8600640 1 -- 172.21.15.49:0/909551246 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c01aad70 con 0x7fd4c010d4e0 2024-02-26T06:36:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.496+0000 7fd4cb42a740 1 Processor -- start 2024-02-26T06:36:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.497+0000 7fd4be000640 1 -- 172.21.15.49:0/909551246 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd4b800f810 con 0x7fd4c010d4e0 2024-02-26T06:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.497+0000 7fd4cb42a740 1 -- start start 2024-02-26T06:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x564772075840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772076200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x5647720816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647720315f0 con 0x56477207f300 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564771fe3890 con 0x564772075d80 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564771fe0dc0 con 0x564772076af0 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4c5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x564772075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4c5200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x564772075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:36030/0 (socket says 172.21.15.49:36030) 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.498+0000 7fd4c5200640 1 -- 172.21.15.49:0/1706760561 learned_addr learned my addr 172.21.15.49:0/1706760561 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.499+0000 7fd4c5200640 1 --2- 172.21.15.49:0/1706760561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x5647720816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.499+0000 7fd4c5200640 1 -- 172.21.15.49:0/1706760561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 msgr2=0x564772076200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.499+0000 7fd4c5200640 1 --2- 172.21.15.49:0/1706760561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772076200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.499+0000 7fd4c5200640 1 -- 172.21.15.49:0/1706760561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 msgr2=0x5647720816f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.499+0000 7fd4c5200640 1 --2- 172.21.15.49:0/1706760561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x5647720816f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.499+0000 7fd4c5200640 1 -- 172.21.15.49:0/1706760561 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564771f5dad0 con 0x564772076af0 2024-02-26T06:36:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.500+0000 7fd4c5200640 1 --2- 172.21.15.49:0/1706760561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x564772075840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac012540 tx=0x7fd4ac00e5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad2dd54e83a4cfd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.500+0000 7fd4be000640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4a80a2430 0x7fd4a80a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.500+0000 7fd4b2a00640 1 -- 172.21.15.49:0/1706760561 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac004020 con 0x564772076af0 2024-02-26T06:36:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.500+0000 7fd4b2a00640 1 -- 172.21.15.49:0/1706760561 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4ac005500 con 0x564772076af0 2024-02-26T06:36:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.500+0000 7fd4b2a00640 1 -- 172.21.15.49:0/1706760561 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac004cd0 con 0x564772076af0 2024-02-26T06:36:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.500+0000 7fd4be000640 1 -- 172.21.15.49:0/909551246 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fd4b80ce100 con 0x7fd4c010d4e0 2024-02-26T06:36:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4c5c00640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4a80a2430 0x7fd4a80a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4cb42a740 1 -- 172.21.15.49:0/1706760561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 msgr2=0x564772075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/1706760561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x564772075840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac012540 tx=0x7fd4ac00e5f0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4cb42a740 1 -- 172.21.15.49:0/1706760561 shutdown_connections 2024-02-26T06:36:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/1706760561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772076200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/1706760561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x5647720816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/1706760561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x564772075840 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4cb42a740 1 -- 172.21.15.49:0/1706760561 >> 172.21.15.49:0/1706760561 conn(0x56477203d480 msgr2=0x564772070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.501+0000 7fd4cb42a740 1 -- 172.21.15.49:0/1706760561 shutdown_connections 2024-02-26T06:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.502+0000 7fd4cb42a740 1 -- 172.21.15.49:0/1706760561 wait complete. 2024-02-26T06:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.502+0000 7fd4cb42a740 1 -- start start 2024-02-26T06:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.502+0000 7fd4c5c00640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4a80a2430 0x7fd4a80a48f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009230 tx=0x7fd4b4005a20 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.502+0000 7fd4cb42a740 1 Processor -- start 2024-02-26T06:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564772075d80 0x56477207c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x56477207c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56477207f300 0x56477207ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564771fdc5a0 con 0x564772075d80 2024-02-26T06:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564771f81b20 con 0x56477207f300 2024-02-26T06:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564771f817c0 con 0x564772076af0 2024-02-26T06:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4cb42a740 10 client.0 _ll_register_callbacks cb 0x564771f95520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4c5c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564772075d80 0x56477207c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4c5c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564772075d80 0x56477207c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57660/0 (socket says 172.21.15.49:57660) 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4c5c00640 1 -- 172.21.15.49:0/879850712 learned_addr learned my addr 172.21.15.49:0/879850712 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.503+0000 7fd4c6600640 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56477207f300 0x56477207ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.504+0000 7fd4c5200640 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x56477207c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.504+0000 7fd4c6600640 1 -- 172.21.15.49:0/879850712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 msgr2=0x56477207c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.504+0000 7fd4c6600640 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x56477207c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.504+0000 7fd4c6600640 1 -- 172.21.15.49:0/879850712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564772075d80 msgr2=0x56477207c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.504+0000 7fd4c6600640 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564772075d80 0x56477207c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.504+0000 7fd4c6600640 1 -- 172.21.15.49:0/879850712 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac00ed00 con 0x56477207f300 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.504+0000 7fd4c6600640 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56477207f300 0x56477207ce20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80cb7d0 tx=0x7fd4b809b2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f324a7de0a9bcba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.504+0000 7fd4a7400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4a7400640 1 -- 172.21.15.49:0/879850712 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80a3210 con 0x56477207f300 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4a7400640 1 -- 172.21.15.49:0/879850712 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fd4b80970e0 con 0x56477207f300 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4cb42a740 1 -- 172.21.15.49:0/879850712 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564771faeca0 con 0x56477207f300 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4a7400640 1 -- 172.21.15.49:0/879850712 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4b80972a0 con 0x56477207f300 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4cb42a740 10 client.27599 fetch_fsmap learned FSMap version 52 2024-02-26T06:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4cb42a740 1 -- 172.21.15.49:0/879850712 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x564771fdac20 con 0x56477207f300 2024-02-26T06:36:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4a7400640 1 -- 172.21.15.49:0/879850712 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564771faeca0 con 0x56477207f300 2024-02-26T06:36:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4a7400640 1 -- 172.21.15.49:0/879850712 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd4b8016030 con 0x56477207f300 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4cb42a740 10 client.27599 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.505+0000 7fd4cb42a740 10 client.27599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.506+0000 7fd4cb42a740 20 client.27599 populate_metadata read hostname 'smithi049' 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.506+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647720a03d0 0x5647720a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.506+0000 7fd4cb42a740 4 client.27599 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.506+0000 7fd4cb42a740 1 -- 172.21.15.49:0/879850712 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564771fd2cb0 con 0x5647720a03d0 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.506+0000 7fd4c5c00640 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647720a03d0 0x5647720a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.507+0000 7fd4c5c00640 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647720a03d0 0x5647720a27f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0e20d414e45790 server_cookie=b6659b0037b2d7ee in_seq=0 out_seq=0 2024-02-26T06:36:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.507+0000 7fd4a7400640 10 client.27599 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.508+0000 7fd4a7400640 1 -- 172.21.15.49:0/879850712 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564771fd2cb0 con 0x5647720a03d0 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.508+0000 7fd4a7400640 10 client.27599 handle_command_reply: tid=0 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.508+0000 7fd4cb42a740 1 client.27599 shutdown 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 -- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56477207f300 msgr2=0x56477207ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56477207f300 0x56477207ce20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80cb7d0 tx=0x7fd4b809b2b0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 -- 172.21.15.49:0/879850712 shutdown_connections 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647720a03d0 0x5647720a27f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56477207f300 0x56477207ce20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x56477207c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/879850712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564772075d80 0x56477207c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 -- 172.21.15.49:0/879850712 >> 172.21.15.49:0/879850712 conn(0x56477203d480 msgr2=0x5647720727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4a7400640 0 client.27599 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.509+0000 7fd4cb42a740 1 -- 172.21.15.49:0/879850712 shutdown_connections 2024-02-26T06:36:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.510+0000 7fd4cb42a740 1 -- 172.21.15.49:0/879850712 wait complete. 2024-02-26T06:36:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.510+0000 7fd4cb42a740 20 client.27599 trim_cache size 0 max 16384 2024-02-26T06:36:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.532+0000 7fd4cb42a740 1 Processor -- start 2024-02-26T06:36:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.532+0000 7fd4cb42a740 1 -- start start 2024-02-26T06:36:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.532+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772075d80 0x56477207ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.532+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772076af0 0x564772097130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.532+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772097670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.532+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564771d24e60 con 0x56477207f300 2024-02-26T06:36:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.532+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564771fd2cb0 con 0x564772076af0 2024-02-26T06:36:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.532+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564772071870 con 0x564772075d80 2024-02-26T06:36:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772097670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c5c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772075d80 0x56477207ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772097670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57672/0 (socket says 172.21.15.49:57672) 2024-02-26T06:36:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 -- 172.21.15.49:0/4227716853 learned_addr learned my addr 172.21.15.49:0/4227716853 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c5200640 1 --2- 172.21.15.49:0/4227716853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772076af0 0x564772097130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 -- 172.21.15.49:0/4227716853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772075d80 msgr2=0x56477207ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 --2- 172.21.15.49:0/4227716853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772075d80 0x56477207ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 -- 172.21.15.49:0/4227716853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772076af0 msgr2=0x564772097130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 --2- 172.21.15.49:0/4227716853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772076af0 0x564772097130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 -- 172.21.15.49:0/4227716853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564771f77500 con 0x56477207f300 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.533+0000 7fd4c6600640 1 --2- 172.21.15.49:0/4227716853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772097670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8096e80 tx=0x7fd4b8007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1817961a88a14d86 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.534+0000 7fd4b2000640 1 -- 172.21.15.49:0/4227716853 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80a3210 con 0x56477207f300 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.534+0000 7fd4b2000640 1 -- 172.21.15.49:0/4227716853 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4b80970e0 con 0x56477207f300 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.534+0000 7fd4b2000640 1 -- 172.21.15.49:0/4227716853 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b809a0c0 con 0x56477207f300 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 -- 172.21.15.49:0/4227716853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 msgr2=0x564772097670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/4227716853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772097670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8096e80 tx=0x7fd4b8007810 comp rx=0 tx=0).stop 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 -- 172.21.15.49:0/4227716853 shutdown_connections 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/4227716853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772076af0 0x564772097130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/4227716853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772097670 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/4227716853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772075d80 0x56477207ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 -- 172.21.15.49:0/4227716853 >> 172.21.15.49:0/4227716853 conn(0x56477203d480 msgr2=0x5647720a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 -- 172.21.15.49:0/4227716853 shutdown_connections 2024-02-26T06:36:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 -- 172.21.15.49:0/4227716853 wait complete. 2024-02-26T06:36:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.535+0000 7fd4cb42a740 1 -- start start 2024-02-26T06:36:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4cb42a740 1 Processor -- start 2024-02-26T06:36:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772097140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x5647720789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4cb42a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772078f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647720979f0 con 0x56477207f300 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564772079460 con 0x564772075d80 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4cb42a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647720795d0 con 0x564772076af0 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4cb42a740 10 client.0 _ll_register_callbacks cb 0x7fd4b8096db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4c5c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772097140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4c5c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772097140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:58360/0 (socket says 172.21.15.49:58360) 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4c5c00640 1 -- 172.21.15.49:0/3802660485 learned_addr learned my addr 172.21.15.49:0/3802660485 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4c5200640 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x5647720789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.536+0000 7fd4c6600640 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772078f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4c5c00640 1 -- 172.21.15.49:0/3802660485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 msgr2=0x5647720789e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4c5c00640 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x5647720789e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4c5c00640 1 -- 172.21.15.49:0/3802660485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 msgr2=0x564772078f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:36:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4c5c00640 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772078f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4c5c00640 1 -- 172.21.15.49:0/3802660485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564771fdac20 con 0x564772075d80 2024-02-26T06:36:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4c5200640 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x5647720789e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:36:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4c5c00640 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772097140 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4036040 tx=0x7fd4b4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b75f0990a379e1c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:36:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4a7400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:36:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4a7400640 1 -- 172.21.15.49:0/3802660485 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4005000 con 0x564772075d80 2024-02-26T06:36:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.537+0000 7fd4cb42a740 1 -- 172.21.15.49:0/3802660485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564772001d00 con 0x564772075d80 2024-02-26T06:36:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.538+0000 7fd4a7400640 1 -- 172.21.15.49:0/3802660485 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fd4b4049040 con 0x564772075d80 2024-02-26T06:36:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.538+0000 7fd4a7400640 1 -- 172.21.15.49:0/3802660485 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4b4005160 con 0x564772075d80 2024-02-26T06:36:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.538+0000 7fd4cb42a740 10 client.27605 fetch_fsmap learned FSMap version 52 2024-02-26T06:36:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.538+0000 7fd4cb42a740 1 -- 172.21.15.49:0/3802660485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x564771f9d290 con 0x564772075d80 2024-02-26T06:36:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.539+0000 7fd4a7400640 1 -- 172.21.15.49:0/3802660485 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564772001d00 con 0x564772075d80 2024-02-26T06:36:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.539+0000 7fd4a7400640 1 -- 172.21.15.49:0/3802660485 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd4b4050070 con 0x564772075d80 2024-02-26T06:36:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.539+0000 7fd4cb42a740 10 client.27605 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:36:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.539+0000 7fd4cb42a740 10 client.27605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:36:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.539+0000 7fd4cb42a740 20 client.27605 populate_metadata read hostname 'smithi049' 2024-02-26T06:36:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.539+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647720f7c30 0x5647720fa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:36:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.539+0000 7fd4cb42a740 4 client.27605 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:36:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.539+0000 7fd4cb42a740 1 -- 172.21.15.49:0/3802660485 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564771cdc600 con 0x5647720f7c30 2024-02-26T06:36:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.540+0000 7fd4c5200640 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647720f7c30 0x5647720fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:36:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.540+0000 7fd4a7400640 10 client.27605 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.542+0000 7fd4c5200640 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647720f7c30 0x5647720fa020 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322eeb5a03ffccca server_cookie=ea50dc9f1c48e572 in_seq=0 out_seq=0 2024-02-26T06:36:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.552+0000 7fd4a7400640 1 -- 172.21.15.49:0/3802660485 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564771cdc600 con 0x5647720f7c30 2024-02-26T06:36:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.552+0000 7fd4a7400640 10 client.27605 handle_command_reply: tid=0 2024-02-26T06:36:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.553+0000 7fd4cb42a740 1 client.27605 shutdown 2024-02-26T06:36:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.553+0000 7fd4cb42a740 1 -- 172.21.15.49:0/3802660485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 msgr2=0x564772097140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.553+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772097140 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4036040 tx=0x7fd4b4009260 comp rx=0 tx=0).stop 2024-02-26T06:36:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.553+0000 7fd4cb42a740 1 -- 172.21.15.49:0/3802660485 shutdown_connections 2024-02-26T06:36:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.553+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5647720f7c30 0x5647720fa020 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.553+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56477207f300 0x564772078f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.553+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564772076af0 0x5647720789e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.554+0000 7fd4cb42a740 1 --2- 172.21.15.49:0/3802660485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564772075d80 0x564772097140 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.554+0000 7fd4cb42a740 1 -- 172.21.15.49:0/3802660485 >> 172.21.15.49:0/3802660485 conn(0x56477203d480 msgr2=0x5647720705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.554+0000 7fd4a7400640 0 client.27605 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:36:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.554+0000 7fd4cb42a740 1 -- 172.21.15.49:0/3802660485 shutdown_connections 2024-02-26T06:36:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.554+0000 7fd4cb42a740 1 -- 172.21.15.49:0/3802660485 wait complete. 2024-02-26T06:36:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.554+0000 7fd4cb42a740 20 client.27605 trim_cache size 0 max 16384 2024-02-26T06:36:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:36:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:36:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:36:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:36:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 -- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4a80a2430 msgr2=0x7fd4a80a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4a80a2430 0x7fd4a80a48f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009230 tx=0x7fd4b4005a20 comp rx=0 tx=0).stop 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 -- 172.21.15.49:0/909551246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4c010d4e0 msgr2=0x7fd4c010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4c010d4e0 0x7fd4c010d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800cb00 tx=0x7fd4b800a250 comp rx=0 tx=0).stop 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 -- 172.21.15.49:0/909551246 shutdown_connections 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd4a80a2430 0x7fd4a80a48f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4c010d4e0 0x7fd4c010d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4c0108670 0x7fd4c010cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 --2- 172.21.15.49:0/909551246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4c01035a0 0x7fd4c010ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 -- 172.21.15.49:0/909551246 >> 172.21.15.49:0/909551246 conn(0x7fd4c00fee00 msgr2=0x7fd4c01010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 -- 172.21.15.49:0/909551246 shutdown_connections 2024-02-26T06:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:36:47.555+0000 7fd4c8600640 1 -- 172.21.15.49:0/909551246 wait complete. 2024-02-26T06:36:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:47 smithi080 ceph-mon[36206]: pgmap v418: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-02-26T06:36:47.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:47 smithi115 ceph-mon[36426]: pgmap v418: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-02-26T06:36:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:47 smithi049 ceph-mon[31696]: pgmap v418: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-02-26T06:36:49.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:49 smithi080 ceph-mon[36206]: pgmap v419: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-02-26T06:36:49.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:49 smithi115 ceph-mon[36426]: pgmap v419: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-02-26T06:36:49.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:49 smithi049 ceph-mon[31696]: pgmap v419: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-02-26T06:36:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 780.0 s, 1976.4 tps, lat 50.840 ms stddev 71.627 2024-02-26T06:36:51.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:51 smithi080 ceph-mon[36206]: pgmap v420: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.55k op/s 2024-02-26T06:36:51.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:51 smithi115 ceph-mon[36426]: pgmap v420: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.55k op/s 2024-02-26T06:36:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:51 smithi049 ceph-mon[31696]: pgmap v420: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.55k op/s 2024-02-26T06:36:53.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:53 smithi080 ceph-mon[36206]: pgmap v421: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2024-02-26T06:36:53.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:53 smithi115 ceph-mon[36426]: pgmap v421: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2024-02-26T06:36:53.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:53 smithi049 ceph-mon[31696]: pgmap v421: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2024-02-26T06:36:55.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:55 smithi080 ceph-mon[36206]: pgmap v422: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-02-26T06:36:55.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:55 smithi115 ceph-mon[36426]: pgmap v422: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-02-26T06:36:55.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:55 smithi049 ceph-mon[31696]: pgmap v422: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-02-26T06:36:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 785.0 s, 1960.6 tps, lat 50.314 ms stddev 59.289 2024-02-26T06:36:57.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:57 smithi115 ceph-mon[36426]: pgmap v423: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 1.77k op/s 2024-02-26T06:36:57.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:57 smithi049 ceph-mon[31696]: pgmap v423: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 1.77k op/s 2024-02-26T06:36:57.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:57 smithi080 ceph-mon[36206]: pgmap v423: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 1.77k op/s 2024-02-26T06:36:59.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:36:59 smithi049 ceph-mon[31696]: pgmap v424: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 979 op/s 2024-02-26T06:36:59.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:36:59 smithi080 ceph-mon[36206]: pgmap v424: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 979 op/s 2024-02-26T06:36:59.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:36:59 smithi115 ceph-mon[36426]: pgmap v424: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 979 op/s 2024-02-26T06:37:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.763+0000 7f4f48600640 1 -- 172.21.15.49:0/1591457491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 msgr2=0x7f4f401166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.763+0000 7f4f48600640 1 --2- 172.21.15.49:0/1591457491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 0x7f4f401166c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009be0 tx=0x7f4f3802bef0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.763+0000 7f4f48600640 1 -- 172.21.15.49:0/1591457491 shutdown_connections 2024-02-26T06:37:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.763+0000 7f4f48600640 1 --2- 172.21.15.49:0/1591457491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 0x7f4f401166c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.763+0000 7f4f48600640 1 --2- 172.21.15.49:0/1591457491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f40108cf0 0x7f4f40113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.763+0000 7f4f48600640 1 --2- 172.21.15.49:0/1591457491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f401083d0 0x7f4f401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.763+0000 7f4f48600640 1 -- 172.21.15.49:0/1591457491 >> 172.21.15.49:0/1591457491 conn(0x7f4f4006c220 msgr2=0x7f4f4006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.763+0000 7f4f48600640 1 -- 172.21.15.49:0/1591457491 shutdown_connections 2024-02-26T06:37:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.764+0000 7f4f48600640 1 -- 172.21.15.49:0/1591457491 wait complete. 2024-02-26T06:37:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.765+0000 7f4f48600640 1 Processor -- start 2024-02-26T06:37:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.765+0000 7f4f48600640 1 -- start start 2024-02-26T06:37:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.765+0000 7f4f48600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401083d0 0x7f4f40109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f48600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f40108cf0 0x7f4f4010a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f48600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 0x7f4f4010a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f48600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f401ae1f0 con 0x7f4f401083d0 2024-02-26T06:37:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f48600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f401ae360 con 0x7f4f40108cf0 2024-02-26T06:37:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f48600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f401ae520 con 0x7f4f401142d0 2024-02-26T06:37:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f45200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f40108cf0 0x7f4f4010a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f46600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 0x7f4f4010a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f45200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f40108cf0 0x7f4f4010a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:59696/0 (socket says 172.21.15.49:59696) 2024-02-26T06:37:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.767+0000 7f4f45200640 1 -- 172.21.15.49:0/2008102624 learned_addr learned my addr 172.21.15.49:0/2008102624 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.766+0000 7f4f46600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 0x7f4f4010a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47872/0 (socket says 172.21.15.49:47872) 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.767+0000 7f4f45200640 1 -- 172.21.15.49:0/2008102624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 msgr2=0x7f4f4010a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.767+0000 7f4f45200640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 0x7f4f4010a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.767+0000 7f4f45200640 1 -- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401083d0 msgr2=0x7f4f40109bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.767+0000 7f4f45200640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401083d0 0x7f4f40109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.767+0000 7f4f45200640 1 -- 172.21.15.49:0/2008102624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f30009490 con 0x7f4f40108cf0 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.767+0000 7f4f45200640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f40108cf0 0x7f4f4010a130 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000ea60 tx=0x7f4f3000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.768+0000 7f4f3e000640 1 -- 172.21.15.49:0/2008102624 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000cde0 con 0x7f4f40108cf0 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.768+0000 7f4f48600640 1 -- 172.21.15.49:0/2008102624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f38009840 con 0x7f4f40108cf0 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.768+0000 7f4f48600640 1 -- 172.21.15.49:0/2008102624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f401a0020 con 0x7f4f40108cf0 2024-02-26T06:37:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.768+0000 7f4f3e000640 1 -- 172.21.15.49:0/2008102624 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f300035f0 con 0x7f4f40108cf0 2024-02-26T06:37:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.768+0000 7f4f3e000640 1 -- 172.21.15.49:0/2008102624 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30010690 con 0x7f4f40108cf0 2024-02-26T06:37:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.772+0000 7f4f3e000640 1 -- 172.21.15.49:0/2008102624 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4f300107f0 con 0x7f4f40108cf0 2024-02-26T06:37:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.774+0000 7f4f4b5ca740 1 Processor -- start 2024-02-26T06:37:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.774+0000 7f4f4b5ca740 1 -- start start 2024-02-26T06:37:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.775+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29ba7af0 0x55db29ba6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.775+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29ba7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f46600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29ba7af0 0x55db29ba6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f46600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29ba7af0 0x55db29ba6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47886/0 (socket says 172.21.15.49:47886) 2024-02-26T06:37:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29bb0300 0x55db29bb26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b625f0 con 0x55db29ba6d80 2024-02-26T06:37:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b14890 con 0x55db29bb0300 2024-02-26T06:37:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b11dc0 con 0x55db29ba7af0 2024-02-26T06:37:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f3e000640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f280a2400 0x7f4f280a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f45c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29ba7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f45c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29ba7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41078/0 (socket says 172.21.15.49:41078) 2024-02-26T06:37:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.776+0000 7f4f45c00640 1 -- 172.21.15.49:0/2250258459 learned_addr learned my addr 172.21.15.49:0/2250258459 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.777+0000 7f4f45200640 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29bb0300 0x55db29bb26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.777+0000 7f4f3e000640 1 -- 172.21.15.49:0/2008102624 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f4f30014070 con 0x7f4f40108cf0 2024-02-26T06:37:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.777+0000 7f4f45200640 1 -- 172.21.15.49:0/2250258459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29ba7af0 msgr2=0x55db29ba6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.777+0000 7f4f45200640 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29ba7af0 0x55db29ba6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.777+0000 7f4f45200640 1 -- 172.21.15.49:0/2250258459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 msgr2=0x55db29ba7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.777+0000 7f4f45200640 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29ba7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.777+0000 7f4f45200640 1 -- 172.21.15.49:0/2250258459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db29a8ead0 con 0x55db29bb0300 2024-02-26T06:37:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.777+0000 7f4f45c00640 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29ba7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:37:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.778+0000 7f4f46600640 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29ba7af0 0x55db29ba6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:37:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.778+0000 7f4f46600640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f280a2400 0x7f4f280a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.778+0000 7f4f45200640 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29bb0300 0x55db29bb26f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4f300cbbb0 tx=0x7f4f30097220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4fd81752a89a55f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.778+0000 7f4f2ea00640 1 -- 172.21.15.49:0/2250258459 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f300aeea0 con 0x55db29bb0300 2024-02-26T06:37:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.779+0000 7f4f2ea00640 1 -- 172.21.15.49:0/2250258459 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f3009ae50 con 0x55db29bb0300 2024-02-26T06:37:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.779+0000 7f4f2ea00640 1 -- 172.21.15.49:0/2250258459 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30097690 con 0x55db29bb0300 2024-02-26T06:37:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.780+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/2250258459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29bb0300 msgr2=0x55db29bb26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.780+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29bb0300 0x55db29bb26f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4f300cbbb0 tx=0x7f4f30097220 comp rx=0 tx=0).stop 2024-02-26T06:37:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.780+0000 7f4f46600640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f280a2400 0x7f4f280a48c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f38006010 tx=0x7f4f3802e040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.781+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/2250258459 shutdown_connections 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.781+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29bb0300 0x55db29bb26f0 secure :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4f300cbbb0 tx=0x7f4f30097220 comp rx=0 tx=0).stop 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.781+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29ba7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.781+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/2250258459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29ba7af0 0x55db29ba6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.781+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/2250258459 >> 172.21.15.49:0/2250258459 conn(0x55db29b6e480 msgr2=0x55db29ba1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.781+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/2250258459 shutdown_connections 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.782+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/2250258459 wait complete. 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.782+0000 7f4f4b5ca740 1 -- start start 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.782+0000 7f4f4b5ca740 1 Processor -- start 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.783+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29bad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.783+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba7af0 0x55db29bad940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.783+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bbc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.783+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b0d5a0 con 0x55db29ba6d80 2024-02-26T06:37:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.783+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29ab2b20 con 0x55db29ba7af0 2024-02-26T06:37:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.783+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29ab27c0 con 0x55db29bade80 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.784+0000 7f4f45c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29bad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.784+0000 7f4f4b5ca740 10 client.0 _ll_register_callbacks cb 0x55db29ac6520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.784+0000 7f4f45c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29bad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41088/0 (socket says 172.21.15.49:41088) 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.784+0000 7f4f45c00640 1 -- 172.21.15.49:0/861488910 learned_addr learned my addr 172.21.15.49:0/861488910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.784+0000 7f4f45200640 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba7af0 0x55db29bad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.784+0000 7f4f46600640 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bbc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.785+0000 7f4f45c00640 1 -- 172.21.15.49:0/861488910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 msgr2=0x55db29bbc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.785+0000 7f4f45c00640 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bbc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.785+0000 7f4f45c00640 1 -- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba7af0 msgr2=0x55db29bad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.785+0000 7f4f45c00640 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba7af0 0x55db29bad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.785+0000 7f4f45c00640 1 -- 172.21.15.49:0/861488910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f401a0020 con 0x55db29ba6d80 2024-02-26T06:37:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.785+0000 7f4f45200640 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba7af0 0x55db29bad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:37:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.786+0000 7f4f45c00640 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29bad400 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400b8f0 tx=0x7f4f3400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3495f1a96e0250bc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.786+0000 7f4f23400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-02-26T06:37:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.786+0000 7f4f23400640 1 -- 172.21.15.49:0/861488910 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34005020 con 0x55db29ba6d80 2024-02-26T06:37:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.786+0000 7f4f23400640 1 -- 172.21.15.49:0/861488910 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f4f34005180 con 0x55db29ba6d80 2024-02-26T06:37:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.787+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/861488910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db29adfca0 con 0x55db29ba6d80 2024-02-26T06:37:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.787+0000 7f4f23400640 1 -- 172.21.15.49:0/861488910 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f34005340 con 0x55db29ba6d80 2024-02-26T06:37:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.787+0000 7f4f4b5ca740 10 client.16941 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.787+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/861488910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55db29b0bc20 con 0x55db29ba6d80 2024-02-26T06:37:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.788+0000 7f4f23400640 1 -- 172.21.15.49:0/861488910 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55db29adfca0 con 0x55db29ba6d80 2024-02-26T06:37:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.789+0000 7f4f23400640 1 -- 172.21.15.49:0/861488910 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4f34011720 con 0x55db29ba6d80 2024-02-26T06:37:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.789+0000 7f4f4b5ca740 10 client.16941 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.789+0000 7f4f4b5ca740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.790+0000 7f4f4b5ca740 20 client.16941 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.790+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55db29bc08e0 0x55db29bc2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.790+0000 7f4f4b5ca740 4 client.16941 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:37:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.790+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/861488910 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db29b03cb0 con 0x55db29bc08e0 2024-02-26T06:37:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.791+0000 7f4f45200640 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55db29bc08e0 0x55db29bc2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.792+0000 7f4f45200640 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55db29bc08e0 0x55db29bc2d20 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ef85f536cad3e8 server_cookie=d6deef2230d3761 in_seq=0 out_seq=0 2024-02-26T06:37:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.792+0000 7f4f23400640 10 client.16941 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.793+0000 7f4f23400640 1 -- 172.21.15.49:0/861488910 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db29b03cb0 con 0x55db29bc08e0 2024-02-26T06:37:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.793+0000 7f4f23400640 10 client.16941 handle_command_reply: tid=0 2024-02-26T06:37:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 client.16941 shutdown 2024-02-26T06:37:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/861488910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 msgr2=0x55db29bad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29bad400 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400b8f0 tx=0x7f4f3400bdb0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/861488910 shutdown_connections 2024-02-26T06:37:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55db29bc08e0 0x55db29bc2d20 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba7af0 0x55db29bad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bbc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/861488910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba6d80 0x55db29bad400 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/861488910 >> 172.21.15.49:0/861488910 conn(0x55db29b6e480 msgr2=0x55db29ba37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f23400640 0 client.16941 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.794+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/861488910 shutdown_connections 2024-02-26T06:37:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.795+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/861488910 wait complete. 2024-02-26T06:37:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.795+0000 7f4f4b5ca740 20 client.16941 trim_cache size 0 max 16384 2024-02-26T06:37:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.823+0000 7f4f4b5ca740 1 Processor -- start 2024-02-26T06:37:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.823+0000 7f4f4b5ca740 1 -- start start 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.824+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 0x55db29bb7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.824+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.824+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29babcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f46600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29babcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f46600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29babcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47904/0 (socket says 172.21.15.49:47904) 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29855e60 con 0x55db29ba7af0 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29ba2aa0 con 0x55db29ba6d80 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29bbf080 con 0x55db29bade80 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f45200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f45c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 0x55db29bb7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f46600640 1 -- 172.21.15.49:0/1925836146 learned_addr learned my addr 172.21.15.49:0/1925836146 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f46600640 1 -- 172.21.15.49:0/1925836146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 msgr2=0x55db29bb7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f46600640 1 --2- 172.21.15.49:0/1925836146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 0x55db29bb7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f46600640 1 -- 172.21.15.49:0/1925836146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 msgr2=0x55db29bb7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f46600640 1 --2- 172.21.15.49:0/1925836146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.825+0000 7f4f46600640 1 -- 172.21.15.49:0/1925836146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db29aa8500 con 0x55db29bade80 2024-02-26T06:37:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.826+0000 7f4f46600640 1 --2- 172.21.15.49:0/1925836146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29babcd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009be0 tx=0x7f4f38049000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b4aa3ca1572b20 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.826+0000 7f4f2e000640 1 -- 172.21.15.49:0/1925836146 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38034c60 con 0x55db29bade80 2024-02-26T06:37:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.827+0000 7f4f2e000640 1 -- 172.21.15.49:0/1925836146 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f38054040 con 0x55db29bade80 2024-02-26T06:37:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.827+0000 7f4f2e000640 1 -- 172.21.15.49:0/1925836146 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38004860 con 0x55db29bade80 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.828+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/1925836146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 msgr2=0x55db29babcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.828+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/1925836146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29babcd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009be0 tx=0x7f4f38049000 comp rx=0 tx=0).stop 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.828+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/1925836146 shutdown_connections 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.828+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/1925836146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29babcd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.828+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/1925836146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.828+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/1925836146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 0x55db29bb7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.828+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/1925836146 >> 172.21.15.49:0/1925836146 conn(0x55db29b6e480 msgr2=0x55db29ba15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.828+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/1925836146 shutdown_connections 2024-02-26T06:37:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.829+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/1925836146 wait complete. 2024-02-26T06:37:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.829+0000 7f4f4b5ca740 1 -- start start 2024-02-26T06:37:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.829+0000 7f4f4b5ca740 1 Processor -- start 2024-02-26T06:37:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.830+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 0x55db29bb7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.830+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.830+0000 7f4f4b5ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bb3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.830+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29bb4140 con 0x55db29ba7af0 2024-02-26T06:37:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.830+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29bb42b0 con 0x55db29ba6d80 2024-02-26T06:37:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.830+0000 7f4f4b5ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29bb4470 con 0x55db29bade80 2024-02-26T06:37:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.830+0000 7f4f4b5ca740 10 client.0 _ll_register_callbacks cb 0x7f4f34004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.831+0000 7f4f46600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bb3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.831+0000 7f4f45c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 0x55db29bb7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.831+0000 7f4f46600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bb3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47912/0 (socket says 172.21.15.49:47912) 2024-02-26T06:37:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.831+0000 7f4f46600640 1 -- 172.21.15.49:0/700842216 learned_addr learned my addr 172.21.15.49:0/700842216 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.831+0000 7f4f45200640 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.832+0000 7f4f46600640 1 -- 172.21.15.49:0/700842216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 msgr2=0x55db29bb7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.832+0000 7f4f46600640 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 0x55db29bb7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.833+0000 7f4f46600640 1 -- 172.21.15.49:0/700842216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 msgr2=0x55db29bb36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.833+0000 7f4f46600640 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.833+0000 7f4f46600640 1 -- 172.21.15.49:0/700842216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f40117050 con 0x55db29bade80 2024-02-26T06:37:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.833+0000 7f4f45200640 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:37:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.833+0000 7f4f46600640 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bb3c00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4f3802c430 tx=0x7f4f38004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45a8145424fe2a54 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.833+0000 7f4f23400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:37:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.838+0000 7f4f23400640 1 -- 172.21.15.49:0/700842216 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38004180 con 0x55db29bade80 2024-02-26T06:37:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.838+0000 7f4f23400640 1 -- 172.21.15.49:0/700842216 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f4f38003b80 con 0x55db29bade80 2024-02-26T06:37:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.838+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/700842216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db29b32d00 con 0x55db29bade80 2024-02-26T06:37:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.840+0000 7f4f23400640 1 -- 172.21.15.49:0/700842216 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f38003d40 con 0x55db29bade80 2024-02-26T06:37:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.840+0000 7f4f4b5ca740 10 client.27577 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.840+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/700842216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55db29ace290 con 0x55db29bade80 2024-02-26T06:37:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.841+0000 7f4f23400640 1 -- 172.21.15.49:0/700842216 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f38069030 con 0x55db29bade80 2024-02-26T06:37:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.841+0000 7f4f23400640 1 -- 172.21.15.49:0/700842216 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4f38062450 con 0x55db29bade80 2024-02-26T06:37:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.841+0000 7f4f4b5ca740 10 client.27577 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.841+0000 7f4f4b5ca740 10 client.27577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.841+0000 7f4f4b5ca740 20 client.27577 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.841+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55db29c17fa0 0x55db29c1a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.841+0000 7f4f4b5ca740 4 client.27577 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:37:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.841+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/700842216 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db2980d600 con 0x55db29c17fa0 2024-02-26T06:37:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.842+0000 7f4f45c00640 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55db29c17fa0 0x55db29c1a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.843+0000 7f4f45c00640 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55db29c17fa0 0x55db29c1a3c0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20729bf906f2258f server_cookie=1534ceeba726a9fa in_seq=0 out_seq=0 2024-02-26T06:37:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.843+0000 7f4f23400640 10 client.27577 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.967+0000 7f4f23400640 1 -- 172.21.15.49:0/700842216 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db2980d600 con 0x55db29c17fa0 2024-02-26T06:37:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.967+0000 7f4f23400640 10 client.27577 handle_command_reply: tid=0 2024-02-26T06:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.968+0000 7f4f4b5ca740 1 client.27577 shutdown 2024-02-26T06:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.968+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/700842216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 msgr2=0x55db29bb3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.968+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bb3c00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4f3802c430 tx=0x7f4f38004690 comp rx=0 tx=0).stop 2024-02-26T06:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.969+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/700842216 shutdown_connections 2024-02-26T06:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.969+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55db29c17fa0 0x55db29c1a3c0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.969+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db29bade80 0x55db29bb3c00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.969+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db29ba7af0 0x55db29bb36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.969+0000 7f4f4b5ca740 1 --2- 172.21.15.49:0/700842216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55db29ba6d80 0x55db29bb7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.969+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/700842216 >> 172.21.15.49:0/700842216 conn(0x55db29b6e480 msgr2=0x55db29ba0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.969+0000 7f4f23400640 0 client.27577 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.970+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/700842216 shutdown_connections 2024-02-26T06:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.970+0000 7f4f4b5ca740 1 -- 172.21.15.49:0/700842216 wait complete. 2024-02-26T06:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.970+0000 7f4f4b5ca740 20 client.27577 trim_cache size 0 max 16384 2024-02-26T06:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.971+0000 7f4f48600640 1 -- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f280a2400 msgr2=0x7f4f280a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.971+0000 7f4f48600640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f280a2400 0x7f4f280a48c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f38006010 tx=0x7f4f3802e040 comp rx=0 tx=0).stop 2024-02-26T06:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.971+0000 7f4f48600640 1 -- 172.21.15.49:0/2008102624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f40108cf0 msgr2=0x7f4f4010a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.971+0000 7f4f48600640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f40108cf0 0x7f4f4010a130 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000ea60 tx=0x7f4f3000ef20 comp rx=0 tx=0).stop 2024-02-26T06:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.972+0000 7f4f48600640 1 -- 172.21.15.49:0/2008102624 shutdown_connections 2024-02-26T06:37:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.972+0000 7f4f48600640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4f280a2400 0x7f4f280a48c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.972+0000 7f4f48600640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f40108cf0 0x7f4f4010a130 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.972+0000 7f4f48600640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f401142d0 0x7f4f4010a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.972+0000 7f4f48600640 1 --2- 172.21.15.49:0/2008102624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f401083d0 0x7f4f40109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.972+0000 7f4f48600640 1 -- 172.21.15.49:0/2008102624 >> 172.21.15.49:0/2008102624 conn(0x7f4f4006c220 msgr2=0x7f4f4006fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.972+0000 7f4f48600640 1 -- 172.21.15.49:0/2008102624 shutdown_connections 2024-02-26T06:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:00.972+0000 7f4f48600640 1 -- 172.21.15.49:0/2008102624 wait complete. 2024-02-26T06:37:01.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 790.0 s, 1981.0 tps, lat 49.776 ms stddev 58.634 2024-02-26T06:37:01.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:01 smithi049 ceph-mon[31696]: pgmap v425: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-02-26T06:37:01.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:01 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:37:01.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:01 smithi080 ceph-mon[36206]: pgmap v425: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-02-26T06:37:01.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:01 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:37:01.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:01 smithi115 ceph-mon[36426]: pgmap v425: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-02-26T06:37:01.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:01 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:37:03.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:03 smithi049 ceph-mon[31696]: pgmap v426: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2024-02-26T06:37:03.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:03 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:37:03.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:03 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:37:03.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:03 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:37:03.833 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:03 smithi080 ceph-mon[36206]: pgmap v426: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2024-02-26T06:37:03.833 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:03 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:37:03.833 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:03 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:37:03.833 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:03 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:37:03.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:03 smithi115 ceph-mon[36426]: pgmap v426: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2024-02-26T06:37:03.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:03 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:37:03.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:03 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:37:03.842 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:03 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:37:05.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:05 smithi049 ceph-mon[31696]: pgmap v427: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-02-26T06:37:05.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:05 smithi080 ceph-mon[36206]: pgmap v427: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-02-26T06:37:05.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:05 smithi115 ceph-mon[36426]: pgmap v427: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-02-26T06:37:06.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 795.0 s, 275.4 tps, lat 312.441 ms stddev 269.848 2024-02-26T06:37:07.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:07 smithi049 ceph-mon[31696]: pgmap v428: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.6 MiB/s wr, 719 op/s 2024-02-26T06:37:07.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:07 smithi049 ceph-mon[31696]: osdmap e104: 12 total, 12 up, 12 in 2024-02-26T06:37:07.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:07 smithi080 ceph-mon[36206]: pgmap v428: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.6 MiB/s wr, 719 op/s 2024-02-26T06:37:07.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:07 smithi080 ceph-mon[36206]: osdmap e104: 12 total, 12 up, 12 in 2024-02-26T06:37:07.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:07 smithi115 ceph-mon[36426]: pgmap v428: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.6 MiB/s wr, 719 op/s 2024-02-26T06:37:07.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:07 smithi115 ceph-mon[36426]: osdmap e104: 12 total, 12 up, 12 in 2024-02-26T06:37:08.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:08 smithi080 ceph-mon[36206]: pgmap v430: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 972 op/s 2024-02-26T06:37:08.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:08 smithi115 ceph-mon[36426]: pgmap v430: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 972 op/s 2024-02-26T06:37:08.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:08 smithi049 ceph-mon[31696]: pgmap v430: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 972 op/s 2024-02-26T06:37:11.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:10 smithi080 ceph-mon[36206]: pgmap v431: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 972 op/s 2024-02-26T06:37:11.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:10 smithi115 ceph-mon[36426]: pgmap v431: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 972 op/s 2024-02-26T06:37:11.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:10 smithi049 ceph-mon[31696]: pgmap v431: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 972 op/s 2024-02-26T06:37:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 800.0 s, 1165.6 tps, lat 98.695 ms stddev 177.558 2024-02-26T06:37:13.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:12 smithi080 ceph-mon[36206]: pgmap v432: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-02-26T06:37:13.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:12 smithi115 ceph-mon[36426]: pgmap v432: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-02-26T06:37:13.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:12 smithi049 ceph-mon[31696]: pgmap v432: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-02-26T06:37:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.166+0000 7f37c4800640 1 -- 172.21.15.49:0/2810442259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 msgr2=0x7f37bc0ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.166+0000 7f37c4800640 1 --2- 172.21.15.49:0/2810442259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 0x7f37bc0ffae0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f37ac01a910 tx=0x7f37ac03cc70 comp rx=0 tx=0).stop 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.166+0000 7f37c4800640 1 -- 172.21.15.49:0/2810442259 shutdown_connections 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.166+0000 7f37c4800640 1 --2- 172.21.15.49:0/2810442259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37bc100020 0x7f37bc103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.166+0000 7f37c4800640 1 --2- 172.21.15.49:0/2810442259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 0x7f37bc0ffae0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.166+0000 7f37c4800640 1 --2- 172.21.15.49:0/2810442259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37bc106150 0x7f37bc106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.167+0000 7f37c4800640 1 -- 172.21.15.49:0/2810442259 >> 172.21.15.49:0/2810442259 conn(0x7f37bc076610 msgr2=0x7f37bc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.167+0000 7f37c4800640 1 -- 172.21.15.49:0/2810442259 shutdown_connections 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.168+0000 7f37c4800640 1 -- 172.21.15.49:0/2810442259 wait complete. 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.170+0000 7f37c4800640 1 Processor -- start 2024-02-26T06:37:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.170+0000 7f37c4800640 1 -- start start 2024-02-26T06:37:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c4800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 0x7f37bc071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c4800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37bc100020 0x7f37bc071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c4800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37bc106150 0x7f37bc10dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c4800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc10e0b0 con 0x7f37bc106150 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c4800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc10e220 con 0x7f37bc100020 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c4800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc10e3e0 con 0x7f37bc0ff680 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 0x7f37bc071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 0x7f37bc071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42522/0 (socket says 172.21.15.49:42522) 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c1e00640 1 -- 172.21.15.49:0/3884806058 learned_addr learned my addr 172.21.15.49:0/3884806058 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c2800640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37bc106150 0x7f37bc10dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.171+0000 7f37c1400640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37bc100020 0x7f37bc071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.172+0000 7f37c1e00640 1 -- 172.21.15.49:0/3884806058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37bc100020 msgr2=0x7f37bc071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.172+0000 7f37c1e00640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37bc100020 0x7f37bc071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.172+0000 7f37c1e00640 1 -- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37bc106150 msgr2=0x7f37bc10dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.172+0000 7f37c1e00640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37bc106150 0x7f37bc10dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.172+0000 7f37c1e00640 1 -- 172.21.15.49:0/3884806058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8009840 con 0x7f37bc0ff680 2024-02-26T06:37:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.172+0000 7f37c2800640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37bc106150 0x7f37bc10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:37:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.173+0000 7f37c1e00640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 0x7f37bc071970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b4a0 tx=0x7f37b000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.173+0000 7f37b6a00640 1 -- 172.21.15.49:0/3884806058 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0010610 con 0x7f37bc0ff680 2024-02-26T06:37:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.174+0000 7f37b6a00640 1 -- 172.21.15.49:0/3884806058 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37b0010c50 con 0x7f37bc0ff680 2024-02-26T06:37:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.174+0000 7f37b6a00640 1 -- 172.21.15.49:0/3884806058 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0017e20 con 0x7f37bc0ff680 2024-02-26T06:37:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.174+0000 7f37c4800640 1 -- 172.21.15.49:0/3884806058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac01a570 con 0x7f37bc0ff680 2024-02-26T06:37:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.174+0000 7f37c4800640 1 -- 172.21.15.49:0/3884806058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc114ba0 con 0x7f37bc0ff680 2024-02-26T06:37:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.178+0000 7f37b6a00640 1 -- 172.21.15.49:0/3884806058 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f37b0017480 con 0x7f37bc0ff680 2024-02-26T06:37:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.182+0000 7f37b6a00640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37a40a2480 0x7f37a40a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.182+0000 7f37b6a00640 1 -- 172.21.15.49:0/3884806058 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f37b00cd560 con 0x7f37bc0ff680 2024-02-26T06:37:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.182+0000 7f37c1400640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37a40a2480 0x7f37a40a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.183+0000 7f37c7322740 1 Processor -- start 2024-02-26T06:37:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.183+0000 7f37c7322740 1 -- start start 2024-02-26T06:37:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.184+0000 7f37c7322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 0x55f011710840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.184+0000 7f37c7322740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710d80 0x55f011711200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.184+0000 7f37c7322740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 0x55f01171c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.184+0000 7f37c7322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0116cc5f0 con 0x55f011711af0 2024-02-26T06:37:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.184+0000 7f37c7322740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01167e890 con 0x55f011710d80 2024-02-26T06:37:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.184+0000 7f37c7322740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01167bdc0 con 0x55f01171a300 2024-02-26T06:37:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.184+0000 7f37c1400640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37a40a2480 0x7f37a40a4940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac03f040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.184+0000 7f37c1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710d80 0x55f011711200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.185+0000 7f37c1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710d80 0x55f011711200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:55946/0 (socket says 172.21.15.49:55946) 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.185+0000 7f37c1400640 1 -- 172.21.15.49:0/4135075738 learned_addr learned my addr 172.21.15.49:0/4135075738 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.185+0000 7f37c1400640 1 -- 172.21.15.49:0/4135075738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 msgr2=0x55f01171c6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.185+0000 7f37c1400640 1 --2- 172.21.15.49:0/4135075738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 0x55f01171c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.185+0000 7f37c1400640 1 -- 172.21.15.49:0/4135075738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 msgr2=0x55f011710840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.185+0000 7f37c1400640 1 --2- 172.21.15.49:0/4135075738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 0x55f011710840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.185+0000 7f37c1400640 1 -- 172.21.15.49:0/4135075738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0115f8ad0 con 0x55f011710d80 2024-02-26T06:37:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.186+0000 7f37c1400640 1 --2- 172.21.15.49:0/4135075738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710d80 0x55f011711200 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f37ac046000 tx=0x7f37ac04f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e43b4d9451ea89a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.186+0000 7f37ab000640 1 -- 172.21.15.49:0/4135075738 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0051c0 con 0x55f011710d80 2024-02-26T06:37:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.186+0000 7f37ab000640 1 -- 172.21.15.49:0/4135075738 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37ac03dd00 con 0x55f011710d80 2024-02-26T06:37:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.186+0000 7f37ab000640 1 -- 172.21.15.49:0/4135075738 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac003c00 con 0x55f011710d80 2024-02-26T06:37:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.187+0000 7f37c7322740 1 -- 172.21.15.49:0/4135075738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710d80 msgr2=0x55f011711200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.187+0000 7f37c7322740 1 --2- 172.21.15.49:0/4135075738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710d80 0x55f011711200 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f37ac046000 tx=0x7f37ac04f000 comp rx=0 tx=0).stop 2024-02-26T06:37:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.187+0000 7f37c7322740 1 -- 172.21.15.49:0/4135075738 shutdown_connections 2024-02-26T06:37:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.187+0000 7f37c7322740 1 --2- 172.21.15.49:0/4135075738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710d80 0x55f011711200 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.187+0000 7f37c7322740 1 --2- 172.21.15.49:0/4135075738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 0x55f01171c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.187+0000 7f37c7322740 1 --2- 172.21.15.49:0/4135075738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 0x55f011710840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.187+0000 7f37c7322740 1 -- 172.21.15.49:0/4135075738 >> 172.21.15.49:0/4135075738 conn(0x55f0116d8480 msgr2=0x55f01170ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.188+0000 7f37c7322740 1 -- 172.21.15.49:0/4135075738 shutdown_connections 2024-02-26T06:37:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.188+0000 7f37c7322740 1 -- 172.21.15.49:0/4135075738 wait complete. 2024-02-26T06:37:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.188+0000 7f37c7322740 1 -- start start 2024-02-26T06:37:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.189+0000 7f37c7322740 1 Processor -- start 2024-02-26T06:37:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.189+0000 7f37c7322740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710d80 0x55f0117173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.189+0000 7f37c7322740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011717910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.189+0000 7f37c7322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 0x55f011717e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.189+0000 7f37c7322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0116775a0 con 0x55f01171a300 2024-02-26T06:37:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.189+0000 7f37c7322740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01161cb20 con 0x55f011711af0 2024-02-26T06:37:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.189+0000 7f37c7322740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01161c7c0 con 0x55f011710d80 2024-02-26T06:37:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.190+0000 7f37c7322740 10 client.0 _ll_register_callbacks cb 0x55f011630520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.190+0000 7f37c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710d80 0x55f0117173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.190+0000 7f37c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710d80 0x55f0117173d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42534/0 (socket says 172.21.15.49:42534) 2024-02-26T06:37:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.190+0000 7f37c1e00640 1 -- 172.21.15.49:0/526328778 learned_addr learned my addr 172.21.15.49:0/526328778 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.190+0000 7f37c1400640 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011717910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.191+0000 7f37c2800640 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 0x55f011717e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.191+0000 7f37c1e00640 1 -- 172.21.15.49:0/526328778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 msgr2=0x55f011717910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.191+0000 7f37c1e00640 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011717910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.191+0000 7f37c1e00640 1 -- 172.21.15.49:0/526328778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 msgr2=0x55f011717e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.191+0000 7f37c1e00640 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 0x55f011717e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.191+0000 7f37c1e00640 1 -- 172.21.15.49:0/526328778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc1044a0 con 0x55f011710d80 2024-02-26T06:37:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.192+0000 7f37c1e00640 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710d80 0x55f0117173d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f37b0099fa0 tx=0x7f37b0099a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6db3cf81392ab70f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.192+0000 7f379f400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:37:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.192+0000 7f379f400640 1 -- 172.21.15.49:0/526328778 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00a27b0 con 0x55f011710d80 2024-02-26T06:37:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.192+0000 7f379f400640 1 -- 172.21.15.49:0/526328778 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f37b00caee0 con 0x55f011710d80 2024-02-26T06:37:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.192+0000 7f37c7322740 1 -- 172.21.15.49:0/526328778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f011649ca0 con 0x55f011710d80 2024-02-26T06:37:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.193+0000 7f379f400640 1 -- 172.21.15.49:0/526328778 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b00cb0a0 con 0x55f011710d80 2024-02-26T06:37:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.193+0000 7f37c7322740 10 client.27586 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.193+0000 7f37c7322740 1 -- 172.21.15.49:0/526328778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55f011675c20 con 0x55f011710d80 2024-02-26T06:37:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.194+0000 7f379f400640 1 -- 172.21.15.49:0/526328778 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f011649ca0 con 0x55f011710d80 2024-02-26T06:37:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.194+0000 7f379f400640 1 -- 172.21.15.49:0/526328778 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37b0099460 con 0x55f011710d80 2024-02-26T06:37:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.195+0000 7f37c7322740 10 client.27586 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.195+0000 7f37c7322740 10 client.27586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.195+0000 7f37c7322740 20 client.27586 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.195+0000 7f37c7322740 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f01172a320 0x55f01172c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.195+0000 7f37c7322740 4 client.27586 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:37:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.195+0000 7f37c7322740 1 -- 172.21.15.49:0/526328778 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f01166dcb0 con 0x55f01172a320 2024-02-26T06:37:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.196+0000 7f37c1400640 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f01172a320 0x55f01172c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.197+0000 7f37c1400640 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f01172a320 0x55f01172c710 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2cc9974e1c241b server_cookie=ef83ff5847d5e0e7 in_seq=0 out_seq=0 2024-02-26T06:37:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.197+0000 7f379f400640 10 client.27586 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.198+0000 7f379f400640 1 -- 172.21.15.49:0/526328778 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f01166dcb0 con 0x55f01172a320 2024-02-26T06:37:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.198+0000 7f379f400640 10 client.27586 handle_command_reply: tid=0 2024-02-26T06:37:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.199+0000 7f37c7322740 1 client.27586 shutdown 2024-02-26T06:37:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f37c7322740 1 -- 172.21.15.49:0/526328778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710d80 msgr2=0x55f0117173d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f37c7322740 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710d80 0x55f0117173d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f37b0099fa0 tx=0x7f37b0099a60 comp rx=0 tx=0).stop 2024-02-26T06:37:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f37c7322740 1 -- 172.21.15.49:0/526328778 shutdown_connections 2024-02-26T06:37:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f37c7322740 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f01172a320 0x55f01172c710 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f37c7322740 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011717910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f37c7322740 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 0x55f011717e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f379f400640 0 client.27586 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f37c7322740 1 --2- 172.21.15.49:0/526328778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710d80 0x55f0117173d0 secure :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f37b0099fa0 tx=0x7f37b0099a60 comp rx=0 tx=0).stop 2024-02-26T06:37:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.200+0000 7f37c7322740 1 -- 172.21.15.49:0/526328778 >> 172.21.15.49:0/526328778 conn(0x55f0116d8480 msgr2=0x55f01170d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.201+0000 7f37c7322740 1 -- 172.21.15.49:0/526328778 shutdown_connections 2024-02-26T06:37:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.201+0000 7f37c7322740 1 -- 172.21.15.49:0/526328778 wait complete. 2024-02-26T06:37:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.201+0000 7f37c7322740 20 client.27586 trim_cache size 0 max 16384 2024-02-26T06:37:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.232+0000 7f37c7322740 1 Processor -- start 2024-02-26T06:37:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.233+0000 7f37c7322740 1 -- start start 2024-02-26T06:37:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.233+0000 7f37c7322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 0x55f011719b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.233+0000 7f37c7322740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 0x55f011717ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.233+0000 7f37c7322740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710910 0x55f0117210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.233+0000 7f37c7322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0113bfe60 con 0x55f011711af0 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.233+0000 7f37c7322740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f011707770 con 0x55f011710910 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.233+0000 7f37c7322740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f011726e30 con 0x55f01171a300 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.234+0000 7f37c2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710910 0x55f0117210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.234+0000 7f37c1400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 0x55f011717ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.234+0000 7f37c1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 0x55f011719b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.234+0000 7f37c2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710910 0x55f0117210c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:55968/0 (socket says 172.21.15.49:55968) 2024-02-26T06:37:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.234+0000 7f37c2800640 1 -- 172.21.15.49:0/2942047174 learned_addr learned my addr 172.21.15.49:0/2942047174 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.234+0000 7f37c2800640 1 -- 172.21.15.49:0/2942047174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 msgr2=0x55f011717ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.234+0000 7f37c2800640 1 --2- 172.21.15.49:0/2942047174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 0x55f011717ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.234+0000 7f37c2800640 1 -- 172.21.15.49:0/2942047174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 msgr2=0x55f011719b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.235+0000 7f37c2800640 1 --2- 172.21.15.49:0/2942047174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 0x55f011719b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.235+0000 7f37c2800640 1 -- 172.21.15.49:0/2942047174 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f011612500 con 0x55f011710910 2024-02-26T06:37:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.235+0000 7f37c1e00640 1 --2- 172.21.15.49:0/2942047174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 0x55f011719b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:37:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.235+0000 7f37c2800640 1 --2- 172.21.15.49:0/2942047174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710910 0x55f0117210c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f37b8012580 tx=0x7f37b8012a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96cc44dd9e4ba6c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.236+0000 7f37aa600640 1 -- 172.21.15.49:0/2942047174 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b80107d0 con 0x55f011710910 2024-02-26T06:37:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.236+0000 7f37aa600640 1 -- 172.21.15.49:0/2942047174 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37b8010e10 con 0x55f011710910 2024-02-26T06:37:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.236+0000 7f37aa600640 1 -- 172.21.15.49:0/2942047174 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800e3c0 con 0x55f011710910 2024-02-26T06:37:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 -- 172.21.15.49:0/2942047174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710910 msgr2=0x55f0117210c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 --2- 172.21.15.49:0/2942047174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710910 0x55f0117210c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f37b8012580 tx=0x7f37b8012a40 comp rx=0 tx=0).stop 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 -- 172.21.15.49:0/2942047174 shutdown_connections 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 --2- 172.21.15.49:0/2942047174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011710910 0x55f0117210c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 --2- 172.21.15.49:0/2942047174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f01171a300 0x55f011717ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 --2- 172.21.15.49:0/2942047174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f011711af0 0x55f011719b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 -- 172.21.15.49:0/2942047174 >> 172.21.15.49:0/2942047174 conn(0x55f0116d8480 msgr2=0x55f011715ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 -- 172.21.15.49:0/2942047174 shutdown_connections 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.237+0000 7f37c7322740 1 -- 172.21.15.49:0/2942047174 wait complete. 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.238+0000 7f37c7322740 1 -- start start 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.238+0000 7f37c7322740 1 Processor -- start 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.238+0000 7f37c7322740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710910 0x55f0117210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c7322740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011721610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c7322740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 0x55f01171f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c7322740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01171f990 con 0x55f01171a300 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c7322740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01171fb00 con 0x55f011711af0 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c7322740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01171fcc0 con 0x55f011710910 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c7322740 10 client.0 _ll_register_callbacks cb 0x7f37b0017360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710910 0x55f0117210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c1e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710910 0x55f0117210d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:42544/0 (socket says 172.21.15.49:42544) 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c1e00640 1 -- 172.21.15.49:0/1146798638 learned_addr learned my addr 172.21.15.49:0/1146798638 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.239+0000 7f37c1400640 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011721610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.240+0000 7f37c2800640 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 0x55f01171f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.240+0000 7f37c1400640 1 -- 172.21.15.49:0/1146798638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710910 msgr2=0x55f0117210d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.240+0000 7f37c1400640 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710910 0x55f0117210d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.240+0000 7f37c1400640 1 -- 172.21.15.49:0/1146798638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 msgr2=0x55f01171f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.240+0000 7f37c1400640 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 0x55f01171f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.240+0000 7f37c1400640 1 -- 172.21.15.49:0/1146798638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f011675c20 con 0x55f011711af0 2024-02-26T06:37:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.241+0000 7f37c1400640 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011721610 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f37ac047040 tx=0x7f37ac06ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d58f663a1cddbab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.241+0000 7f379f400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.242+0000 7f379f400640 1 -- 172.21.15.49:0/1146798638 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac06c040 con 0x55f011711af0 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.242+0000 7f379f400640 1 -- 172.21.15.49:0/1146798638 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f37ac057070 con 0x55f011711af0 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.242+0000 7f37c7322740 1 -- 172.21.15.49:0/1146798638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f01169cd00 con 0x55f011711af0 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.243+0000 7f379f400640 1 -- 172.21.15.49:0/1146798638 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37ac06c040 con 0x55f011711af0 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.243+0000 7f37c7322740 10 client.27641 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.244+0000 7f37c7322740 1 -- 172.21.15.49:0/1146798638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f37b8009dc0 con 0x55f011711af0 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.244+0000 7f379f400640 1 -- 172.21.15.49:0/1146798638 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37ac0053a0 con 0x55f011711af0 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.244+0000 7f379f400640 1 -- 172.21.15.49:0/1146798638 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37ac04d040 con 0x55f011711af0 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.245+0000 7f37c7322740 10 client.27641 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.245+0000 7f37c7322740 10 client.27641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.245+0000 7f37c7322740 20 client.27641 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.245+0000 7f37c7322740 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f011782850 0x55f011784c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.245+0000 7f37c7322740 4 client.27641 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:37:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.245+0000 7f37c7322740 1 -- 172.21.15.49:0/1146798638 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f01169cd00 con 0x55f011782850 2024-02-26T06:37:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.245+0000 7f37c1e00640 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f011782850 0x55f011784c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.246+0000 7f37c1e00640 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f011782850 0x55f011784c70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a802f336135ae48 server_cookie=b7a35056a75487b5 in_seq=0 out_seq=0 2024-02-26T06:37:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.246+0000 7f379f400640 10 client.27641 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.263+0000 7f379f400640 1 -- 172.21.15.49:0/1146798638 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f01169cd00 con 0x55f011782850 2024-02-26T06:37:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.263+0000 7f379f400640 10 client.27641 handle_command_reply: tid=0 2024-02-26T06:37:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.263+0000 7f37c7322740 1 client.27641 shutdown 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 -- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 msgr2=0x55f011721610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011721610 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f37ac047040 tx=0x7f37ac06ecd0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 -- 172.21.15.49:0/1146798638 shutdown_connections 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55f011782850 0x55f011784c70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f011711af0 0x55f011721610 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f379f400640 0 client.27641 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f01171a300 0x55f01171f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 --2- 172.21.15.49:0/1146798638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f011710910 0x55f0117210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 -- 172.21.15.49:0/1146798638 >> 172.21.15.49:0/1146798638 conn(0x55f0116d8480 msgr2=0x55f01170ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 -- 172.21.15.49:0/1146798638 shutdown_connections 2024-02-26T06:37:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 1 -- 172.21.15.49:0/1146798638 wait complete. 2024-02-26T06:37:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.264+0000 7f37c7322740 20 client.27641 trim_cache size 0 max 16384 2024-02-26T06:37:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:37:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:37:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:37:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 -- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37a40a2480 msgr2=0x7f37a40a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37a40a2480 0x7f37a40a4940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac03f040 comp rx=0 tx=0).stop 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 -- 172.21.15.49:0/3884806058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 msgr2=0x7f37bc071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 0x7f37bc071970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b4a0 tx=0x7f37b000b960 comp rx=0 tx=0).stop 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 -- 172.21.15.49:0/3884806058 shutdown_connections 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f37a40a2480 0x7f37a40a4940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37bc100020 0x7f37bc071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37bc106150 0x7f37bc10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 --2- 172.21.15.49:0/3884806058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37bc0ff680 0x7f37bc071970 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 -- 172.21.15.49:0/3884806058 >> 172.21.15.49:0/3884806058 conn(0x7f37bc076610 msgr2=0x7f37bc101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 -- 172.21.15.49:0/3884806058 shutdown_connections 2024-02-26T06:37:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:14.265+0000 7f37c4800640 1 -- 172.21.15.49:0/3884806058 wait complete. 2024-02-26T06:37:15.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:14 smithi080 ceph-mon[36206]: pgmap v433: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-02-26T06:37:15.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:14 smithi115 ceph-mon[36426]: pgmap v433: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-02-26T06:37:15.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:14 smithi049 ceph-mon[31696]: pgmap v433: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-02-26T06:37:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 805.0 s, 1043.2 tps, lat 93.617 ms stddev 119.679 2024-02-26T06:37:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:16 smithi080 ceph-mon[36206]: osdmap e105: 12 total, 12 up, 12 in 2024-02-26T06:37:16.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:16 smithi115 ceph-mon[36426]: osdmap e105: 12 total, 12 up, 12 in 2024-02-26T06:37:16.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:16 smithi049 ceph-mon[31696]: osdmap e105: 12 total, 12 up, 12 in 2024-02-26T06:37:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:17 smithi080 ceph-mon[36206]: pgmap v435: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-02-26T06:37:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:17 smithi115 ceph-mon[36426]: pgmap v435: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-02-26T06:37:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:17 smithi049 ceph-mon[31696]: pgmap v435: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-02-26T06:37:19.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:19 smithi080 ceph-mon[36206]: pgmap v436: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-02-26T06:37:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:19 smithi115 ceph-mon[36426]: pgmap v436: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-02-26T06:37:19.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:19 smithi049 ceph-mon[31696]: pgmap v436: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-02-26T06:37:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 810.0 s, 1124.8 tps, lat 91.207 ms stddev 109.365 2024-02-26T06:37:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:21 smithi080 ceph-mon[36206]: pgmap v437: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-02-26T06:37:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:21 smithi115 ceph-mon[36426]: pgmap v437: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-02-26T06:37:21.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:21 smithi049 ceph-mon[31696]: pgmap v437: 97 pgs: 97 active+clean; 4.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-02-26T06:37:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:23 smithi080 ceph-mon[36206]: pgmap v438: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.7 MiB/s wr, 565 op/s 2024-02-26T06:37:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:23 smithi115 ceph-mon[36426]: pgmap v438: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.7 MiB/s wr, 565 op/s 2024-02-26T06:37:23.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:23 smithi049 ceph-mon[31696]: pgmap v438: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.7 MiB/s wr, 565 op/s 2024-02-26T06:37:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:25 smithi080 ceph-mon[36206]: pgmap v439: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-02-26T06:37:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:25 smithi115 ceph-mon[36426]: pgmap v439: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-02-26T06:37:25.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:25 smithi049 ceph-mon[31696]: pgmap v439: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-02-26T06:37:26.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 815.0 s, 1224.8 tps, lat 80.231 ms stddev 90.649 2024-02-26T06:37:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 -- 172.21.15.49:0/3707457124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 msgr2=0x7fe224069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 --2- 172.21.15.49:0/3707457124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 0x7fe224069180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe210009930 tx=0x7fe21002bcb0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 -- 172.21.15.49:0/3707457124 shutdown_connections 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 --2- 172.21.15.49:0/3707457124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 0x7fe224069180 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 --2- 172.21.15.49:0/3707457124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe224103b90 0x7fe224103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 --2- 172.21.15.49:0/3707457124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe224106140 0x7fe224068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 -- 172.21.15.49:0/3707457124 >> 172.21.15.49:0/3707457124 conn(0x7fe224076ed0 msgr2=0x7fe2240772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 -- 172.21.15.49:0/3707457124 shutdown_connections 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.517+0000 7fe22ac00640 1 -- 172.21.15.49:0/3707457124 wait complete. 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.518+0000 7fe22ac00640 1 Processor -- start 2024-02-26T06:37:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.519+0000 7fe22ac00640 1 -- start start 2024-02-26T06:37:28.273 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:27 smithi115 ceph-mon[36426]: pgmap v440: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-02-26T06:37:28.273 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:27 smithi080 ceph-mon[36206]: pgmap v440: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-02-26T06:37:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe22ac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 0x7fe2241a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe22ac00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe224103b90 0x7fe2241a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe22ac00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe224106140 0x7fe22419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe22ac00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22419e140 con 0x7fe224103b90 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe22ac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22419e2b0 con 0x7fe224068d40 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe22ac00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22419e470 con 0x7fe224106140 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe223e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 0x7fe2241a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe228c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe224106140 0x7fe22419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe223e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 0x7fe2241a3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36608/0 (socket says 172.21.15.49:36608) 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.520+0000 7fe223e00640 1 -- 172.21.15.49:0/2990655984 learned_addr learned my addr 172.21.15.49:0/2990655984 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe228c00640 1 -- 172.21.15.49:0/2990655984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 msgr2=0x7fe2241a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe228c00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 0x7fe2241a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe228c00640 1 -- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe224103b90 msgr2=0x7fe2241a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe228c00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe224103b90 0x7fe2241a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe228c00640 1 -- 172.21.15.49:0/2990655984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210009560 con 0x7fe224106140 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe228c00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe224106140 0x7fe22419daf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00be20 tx=0x7fe21c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe220c00640 1 -- 172.21.15.49:0/2990655984 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00cdf0 con 0x7fe224106140 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe220c00640 1 -- 172.21.15.49:0/2990655984 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe21c014920 con 0x7fe224106140 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe220c00640 1 -- 172.21.15.49:0/2990655984 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c0129b0 con 0x7fe224106140 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe22ac00640 1 -- 172.21.15.49:0/2990655984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22419e750 con 0x7fe224106140 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.521+0000 7fe22ac00640 1 -- 172.21.15.49:0/2990655984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2241aa9e0 con 0x7fe224106140 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.526+0000 7fe22db93740 1 Processor -- start 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.526+0000 7fe22db93740 1 -- start start 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.527+0000 7fe22db93740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x557755987840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.527+0000 7fe22db93740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x557755988200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.527+0000 7fe22db93740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755991300 0x5577559936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.527+0000 7fe22db93740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577559435f0 con 0x557755988af0 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.527+0000 7fe22db93740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577558f5890 con 0x557755991300 2024-02-26T06:37:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.527+0000 7fe22db93740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577558f2dc0 con 0x557755987d80 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x557755988200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755991300 0x5577559936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755991300 0x5577559936f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36624/0 (socket says 172.21.15.49:36624) 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 -- 172.21.15.49:0/3658093831 learned_addr learned my addr 172.21.15.49:0/3658093831 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 --2- 172.21.15.49:0/3658093831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x557755987840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 -- 172.21.15.49:0/3658093831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 msgr2=0x557755988200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 --2- 172.21.15.49:0/3658093831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x557755988200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 -- 172.21.15.49:0/3658093831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 msgr2=0x557755987840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 --2- 172.21.15.49:0/3658093831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x557755987840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.528+0000 7fe223400640 1 -- 172.21.15.49:0/3658093831 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55775586fad0 con 0x557755991300 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.529+0000 7fe223400640 1 --2- 172.21.15.49:0/3658093831 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755991300 0x5577559936f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe210005ab0 tx=0x7fe21002c1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d37d82fa5c57de9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.529+0000 7fe219200640 1 -- 172.21.15.49:0/3658093831 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210004b20 con 0x557755991300 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.529+0000 7fe219200640 1 -- 172.21.15.49:0/3658093831 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe210004510 con 0x557755991300 2024-02-26T06:37:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.529+0000 7fe219200640 1 -- 172.21.15.49:0/3658093831 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210004b20 con 0x557755991300 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.530+0000 7fe220c00640 1 -- 172.21.15.49:0/2990655984 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe21c014a90 con 0x7fe224106140 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.530+0000 7fe22db93740 1 -- 172.21.15.49:0/3658093831 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755991300 msgr2=0x5577559936f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.530+0000 7fe22db93740 1 --2- 172.21.15.49:0/3658093831 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755991300 0x5577559936f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe210005ab0 tx=0x7fe21002c1d0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 -- 172.21.15.49:0/3658093831 shutdown_connections 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 --2- 172.21.15.49:0/3658093831 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755991300 0x5577559936f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 --2- 172.21.15.49:0/3658093831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x557755988200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 --2- 172.21.15.49:0/3658093831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x557755987840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 -- 172.21.15.49:0/3658093831 >> 172.21.15.49:0/3658093831 conn(0x55775594f480 msgr2=0x557755982a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 -- 172.21.15.49:0/3658093831 shutdown_connections 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 -- 172.21.15.49:0/3658093831 wait complete. 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 -- start start 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.531+0000 7fe22db93740 1 Processor -- start 2024-02-26T06:37:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe220c00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe2080a22b0 0x7fe2080a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe220c00640 1 -- 172.21.15.49:0/2990655984 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fe21c0cdc00 con 0x7fe224106140 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe22db93740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x55775598e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe22db93740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755988af0 0x55775598e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe22db93740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 0x55775598edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe22db93740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577558ee5a0 con 0x557755991300 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe22db93740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755893b20 con 0x557755988af0 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe22db93740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577558937c0 con 0x557755987d80 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.532+0000 7fe22db93740 10 client.0 _ll_register_callbacks cb 0x5577558a7520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.533+0000 7fe223400640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe2080a22b0 0x7fe2080a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.533+0000 7fe223e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x55775598e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.533+0000 7fe223e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x55775598e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:56646/0 (socket says 172.21.15.49:56646) 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.533+0000 7fe223400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755988af0 0x55775598e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.533+0000 7fe228c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 0x55775598edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.533+0000 7fe223e00640 1 -- 172.21.15.49:0/2703223417 learned_addr learned my addr 172.21.15.49:0/2703223417 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.534+0000 7fe223e00640 1 -- 172.21.15.49:0/2703223417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755988af0 msgr2=0x55775598e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.534+0000 7fe223e00640 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755988af0 0x55775598e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.534+0000 7fe223e00640 1 -- 172.21.15.49:0/2703223417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 msgr2=0x55775598edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.534+0000 7fe223e00640 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 0x55775598edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.534+0000 7fe223e00640 1 -- 172.21.15.49:0/2703223417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2241048b0 con 0x557755987d80 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.534+0000 7fe223400640 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755988af0 0x55775598e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.534+0000 7fe223e00640 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x55775598e370 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe21400bc00 tx=0x7fe2140039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2137ffc94217845 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.535+0000 7fe20d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.535+0000 7fe223400640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe2080a22b0 0x7fe2080a4770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe210036040 tx=0x7fe21002c7c0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.535+0000 7fe20d600640 1 -- 172.21.15.49:0/2703223417 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214016070 con 0x557755987d80 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.535+0000 7fe20d600640 1 -- 172.21.15.49:0/2703223417 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fe214005020 con 0x557755987d80 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.535+0000 7fe22db93740 1 -- 172.21.15.49:0/2703223417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577558c0ca0 con 0x557755987d80 2024-02-26T06:37:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.535+0000 7fe20d600640 1 -- 172.21.15.49:0/2703223417 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2140042c0 con 0x557755987d80 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.536+0000 7fe22db93740 10 client.27604 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.536+0000 7fe22db93740 1 -- 172.21.15.49:0/2703223417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5577558ecc20 con 0x557755987d80 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.536+0000 7fe20d600640 1 -- 172.21.15.49:0/2703223417 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5577558c0ca0 con 0x557755987d80 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.537+0000 7fe20d600640 1 -- 172.21.15.49:0/2703223417 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe214011650 con 0x557755987d80 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.537+0000 7fe22db93740 10 client.27604 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.537+0000 7fe22db93740 10 client.27604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.537+0000 7fe22db93740 20 client.27604 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.537+0000 7fe22db93740 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5577559a11b0 0x5577559a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.537+0000 7fe22db93740 4 client.27604 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.537+0000 7fe22db93740 1 -- 172.21.15.49:0/2703223417 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577558e4cb0 con 0x5577559a11b0 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.538+0000 7fe223400640 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5577559a11b0 0x5577559a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.539+0000 7fe223400640 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5577559a11b0 0x5577559a35d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1c64b145bcb3d8 server_cookie=7f3b40dc19535ffe in_seq=0 out_seq=0 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.539+0000 7fe20d600640 10 client.27604 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.540+0000 7fe20d600640 1 -- 172.21.15.49:0/2703223417 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577558e4cb0 con 0x5577559a11b0 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.540+0000 7fe20d600640 10 client.27604 handle_command_reply: tid=0 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.540+0000 7fe22db93740 1 client.27604 shutdown 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 -- 172.21.15.49:0/2703223417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 msgr2=0x55775598e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x55775598e370 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe21400bc00 tx=0x7fe2140039c0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 -- 172.21.15.49:0/2703223417 shutdown_connections 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5577559a11b0 0x5577559a35d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755988af0 0x55775598e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 0x55775598edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 --2- 172.21.15.49:0/2703223417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755987d80 0x55775598e370 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 -- 172.21.15.49:0/2703223417 >> 172.21.15.49:0/2703223417 conn(0x55775594f480 msgr2=0x5577559847c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe20d600640 0 client.27604 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 -- 172.21.15.49:0/2703223417 shutdown_connections 2024-02-26T06:37:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 1 -- 172.21.15.49:0/2703223417 wait complete. 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.541+0000 7fe22db93740 20 client.27604 trim_cache size 0 max 16384 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.571+0000 7fe22db93740 1 Processor -- start 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.571+0000 7fe22db93740 1 -- start start 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.571+0000 7fe22db93740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755990aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.571+0000 7fe22db93740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x55775598ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.571+0000 7fe22db93740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755991300 0x55775598cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.571+0000 7fe22db93740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755636e60 con 0x557755988af0 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.571+0000 7fe22db93740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577558c0ca0 con 0x557755987d80 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.571+0000 7fe22db93740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775599de20 con 0x557755991300 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755990aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755990aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:36646/0 (socket says 172.21.15.49:36646) 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 -- 172.21.15.49:0/3079087241 learned_addr learned my addr 172.21.15.49:0/3079087241 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223400640 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x55775598ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe228c00640 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755991300 0x55775598cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 -- 172.21.15.49:0/3079087241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755991300 msgr2=0x55775598cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755991300 0x55775598cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 -- 172.21.15.49:0/3079087241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 msgr2=0x55775598ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x55775598ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 -- 172.21.15.49:0/3079087241 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557755889500 con 0x557755987d80 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223400640 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x55775598ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.572+0000 7fe223e00640 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755990aa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe214004530 tx=0x7fe21400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e69bd016e48968af server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.573+0000 7fe20fe00640 1 -- 172.21.15.49:0/3079087241 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214016040 con 0x557755987d80 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.573+0000 7fe20fe00640 1 -- 172.21.15.49:0/3079087241 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe214005520 con 0x557755987d80 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.573+0000 7fe20fe00640 1 -- 172.21.15.49:0/3079087241 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214004850 con 0x557755987d80 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.574+0000 7fe22db93740 1 -- 172.21.15.49:0/3079087241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 msgr2=0x557755990aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.574+0000 7fe22db93740 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755990aa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe214004530 tx=0x7fe21400b860 comp rx=0 tx=0).stop 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.574+0000 7fe22db93740 1 -- 172.21.15.49:0/3079087241 shutdown_connections 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.574+0000 7fe22db93740 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755991300 0x55775598cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.574+0000 7fe22db93740 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755988af0 0x55775598ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.574+0000 7fe22db93740 1 --2- 172.21.15.49:0/3079087241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755990aa0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.574+0000 7fe22db93740 1 -- 172.21.15.49:0/3079087241 >> 172.21.15.49:0/3079087241 conn(0x55775594f480 msgr2=0x557755989660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.575+0000 7fe22db93740 1 -- 172.21.15.49:0/3079087241 shutdown_connections 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.575+0000 7fe22db93740 1 -- 172.21.15.49:0/3079087241 wait complete. 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.576+0000 7fe22db93740 1 -- start start 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.576+0000 7fe22db93740 1 Processor -- start 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe22db93740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755998160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe22db93740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755988af0 0x557755996500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe22db93740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 0x557755996a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe22db93740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755998a30 con 0x557755991300 2024-02-26T06:37:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe22db93740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755996fb0 con 0x557755987d80 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe22db93740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755997170 con 0x557755988af0 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe22db93740 10 client.0 _ll_register_callbacks cb 0x7fe214004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe223400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755988af0 0x557755996500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe223e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755998160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe223400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755988af0 0x557755996500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:56668/0 (socket says 172.21.15.49:56668) 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe223400640 1 -- 172.21.15.49:0/1150202900 learned_addr learned my addr 172.21.15.49:0/1150202900 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.577+0000 7fe228c00640 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 0x557755996a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.578+0000 7fe223400640 1 -- 172.21.15.49:0/1150202900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 msgr2=0x557755998160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.578+0000 7fe223400640 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755998160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.578+0000 7fe223400640 1 -- 172.21.15.49:0/1150202900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 msgr2=0x557755996a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.578+0000 7fe223400640 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 0x557755996a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.578+0000 7fe223400640 1 -- 172.21.15.49:0/1150202900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577558ecc20 con 0x557755988af0 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.578+0000 7fe223400640 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755988af0 0x557755996500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x557755988820 tx=0x7fe21002c1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33ab0f617887244 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.578+0000 7fe20d600640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.579+0000 7fe20d600640 1 -- 172.21.15.49:0/1150202900 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210004b20 con 0x557755988af0 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.579+0000 7fe20d600640 1 -- 172.21.15.49:0/1150202900 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fe210000e40 con 0x557755988af0 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.582+0000 7fe22db93740 1 -- 172.21.15.49:0/1150202900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557755913d00 con 0x557755988af0 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.582+0000 7fe20d600640 1 -- 172.21.15.49:0/1150202900 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21002cb60 con 0x557755988af0 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.582+0000 7fe22db93740 10 client.27610 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.582+0000 7fe22db93740 1 -- 172.21.15.49:0/1150202900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5577558af290 con 0x557755988af0 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe20d600640 1 -- 172.21.15.49:0/1150202900 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557755913d00 con 0x557755988af0 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe20d600640 1 -- 172.21.15.49:0/1150202900 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe210064030 con 0x557755988af0 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe22db93740 10 client.27610 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe22db93740 10 client.27610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe22db93740 20 client.27610 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe22db93740 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5577559f88d0 0x5577559facf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe22db93740 4 client.27610 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:37:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe22db93740 1 -- 172.21.15.49:0/1150202900 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5577555ee600 con 0x5577559f88d0 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.583+0000 7fe223e00640 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5577559f88d0 0x5577559facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.584+0000 7fe223e00640 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5577559f88d0 0x5577559facf0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3682cc5d3063a6be server_cookie=c85bd42e68227ef3 in_seq=0 out_seq=0 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.584+0000 7fe20d600640 10 client.27610 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.597+0000 7fe20d600640 1 -- 172.21.15.49:0/1150202900 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5577555ee600 con 0x5577559f88d0 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.597+0000 7fe20d600640 10 client.27610 handle_command_reply: tid=0 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.597+0000 7fe22db93740 1 client.27610 shutdown 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.597+0000 7fe22db93740 1 -- 172.21.15.49:0/1150202900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755988af0 msgr2=0x557755996500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.597+0000 7fe22db93740 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755988af0 0x557755996500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x557755988820 tx=0x7fe21002c1d0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 1 -- 172.21.15.49:0/1150202900 shutdown_connections 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5577559f88d0 0x5577559facf0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557755991300 0x557755996a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557755988af0 0x557755996500 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 1 --2- 172.21.15.49:0/1150202900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557755987d80 0x557755998160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 1 -- 172.21.15.49:0/1150202900 >> 172.21.15.49:0/1150202900 conn(0x55775594f480 msgr2=0x55775598cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe20d600640 0 client.27610 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 1 -- 172.21.15.49:0/1150202900 shutdown_connections 2024-02-26T06:37:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 1 -- 172.21.15.49:0/1150202900 wait complete. 2024-02-26T06:37:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.598+0000 7fe22db93740 20 client.27610 trim_cache size 0 max 16384 2024-02-26T06:37:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:37:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:37:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:37:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:37:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 -- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe2080a22b0 msgr2=0x7fe2080a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe2080a22b0 0x7fe2080a4770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe210036040 tx=0x7fe21002c7c0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 -- 172.21.15.49:0/2990655984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe224106140 msgr2=0x7fe22419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe224106140 0x7fe22419daf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00be20 tx=0x7fe21c00bf10 comp rx=0 tx=0).stop 2024-02-26T06:37:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 -- 172.21.15.49:0/2990655984 shutdown_connections 2024-02-26T06:37:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe2080a22b0 0x7fe2080a4770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe224106140 0x7fe22419daf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe224103b90 0x7fe2241a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 --2- 172.21.15.49:0/2990655984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe224068d40 0x7fe2241a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 -- 172.21.15.49:0/2990655984 >> 172.21.15.49:0/2990655984 conn(0x7fe224076ed0 msgr2=0x7fe2240ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 -- 172.21.15.49:0/2990655984 shutdown_connections 2024-02-26T06:37:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:27.599+0000 7fe22ac00640 1 -- 172.21.15.49:0/2990655984 wait complete. 2024-02-26T06:37:28.300 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:27 smithi049 ceph-mon[31696]: pgmap v440: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-02-26T06:37:29.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:29 smithi080 ceph-mon[36206]: pgmap v441: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.6 MiB/s wr, 606 op/s 2024-02-26T06:37:29.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:29 smithi115 ceph-mon[36426]: pgmap v441: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.6 MiB/s wr, 606 op/s 2024-02-26T06:37:29.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:29 smithi049 ceph-mon[31696]: pgmap v441: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.6 MiB/s wr, 606 op/s 2024-02-26T06:37:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 820.0 s, 1598.7 tps, lat 63.275 ms stddev 81.752 2024-02-26T06:37:31.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:31 smithi080 ceph-mon[36206]: pgmap v442: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.7 MiB/s wr, 508 op/s 2024-02-26T06:37:31.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:31 smithi115 ceph-mon[36426]: pgmap v442: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.7 MiB/s wr, 508 op/s 2024-02-26T06:37:31.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:31 smithi049 ceph-mon[31696]: pgmap v442: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.7 MiB/s wr, 508 op/s 2024-02-26T06:37:33.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:33 smithi080 ceph-mon[36206]: pgmap v443: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-02-26T06:37:33.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:33 smithi115 ceph-mon[36426]: pgmap v443: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-02-26T06:37:33.677 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:33 smithi049 ceph-mon[31696]: pgmap v443: 97 pgs: 97 active+clean; 4.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-02-26T06:37:35.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:35 smithi080 ceph-mon[36206]: pgmap v444: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.6 MiB/s wr, 596 op/s 2024-02-26T06:37:35.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:35 smithi115 ceph-mon[36426]: pgmap v444: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.6 MiB/s wr, 596 op/s 2024-02-26T06:37:35.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:35 smithi049 ceph-mon[31696]: pgmap v444: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.6 MiB/s wr, 596 op/s 2024-02-26T06:37:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 825.0 s, 1160.1 tps, lat 86.518 ms stddev 101.258 2024-02-26T06:37:37.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:37 smithi049 ceph-mon[31696]: pgmap v445: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-02-26T06:37:37.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:37 smithi080 ceph-mon[36206]: pgmap v445: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-02-26T06:37:37.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:37 smithi115 ceph-mon[36426]: pgmap v445: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-02-26T06:37:39.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:39 smithi049 ceph-mon[31696]: pgmap v446: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-02-26T06:37:39.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:39 smithi080 ceph-mon[36206]: pgmap v446: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-02-26T06:37:39.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:39 smithi115 ceph-mon[36426]: pgmap v446: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-02-26T06:37:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 830.0 s, 909.4 tps, lat 106.588 ms stddev 137.650 2024-02-26T06:37:41.631 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:41 smithi080 ceph-mon[36206]: pgmap v447: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-02-26T06:37:41.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:41 smithi049 ceph-mon[31696]: pgmap v447: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-02-26T06:37:41.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:41 smithi115 ceph-mon[36426]: pgmap v447: 97 pgs: 97 active+clean; 4.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-02-26T06:37:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.167+0000 7f2447200640 1 -- 172.21.15.49:0/1558864597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440075fd0 msgr2=0x7f24401118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.167+0000 7f2447200640 1 --2- 172.21.15.49:0/1558864597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440075fd0 0x7f24401118d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2430009930 tx=0x7f243002bc30 comp rx=0 tx=0).stop 2024-02-26T06:37:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.169+0000 7f2447200640 1 -- 172.21.15.49:0/1558864597 shutdown_connections 2024-02-26T06:37:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.169+0000 7f2447200640 1 --2- 172.21.15.49:0/1558864597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2440111e10 0x7f2440114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.169+0000 7f2447200640 1 --2- 172.21.15.49:0/1558864597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440075fd0 0x7f24401118d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.169+0000 7f2447200640 1 --2- 172.21.15.49:0/1558864597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400756b0 0x7f2440075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.169+0000 7f2447200640 1 -- 172.21.15.49:0/1558864597 >> 172.21.15.49:0/1558864597 conn(0x7f24400fec60 msgr2=0x7f2440101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.169+0000 7f2447200640 1 -- 172.21.15.49:0/1558864597 shutdown_connections 2024-02-26T06:37:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.170+0000 7f2447200640 1 -- 172.21.15.49:0/1558864597 wait complete. 2024-02-26T06:37:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.170+0000 7f2447200640 1 Processor -- start 2024-02-26T06:37:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.171+0000 7f2447200640 1 -- start start 2024-02-26T06:37:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.171+0000 7f2447200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400756b0 0x7f24401a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.171+0000 7f2447200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2440075fd0 0x7f24401a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.172+0000 7f2445400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2440075fd0 0x7f24401a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.172+0000 7f2445400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2440075fd0 0x7f24401a3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38264/0 (socket says 172.21.15.49:38264) 2024-02-26T06:37:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.172+0000 7f2447200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440111e10 0x7f244019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.172+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019e0c0 con 0x7f24400756b0 2024-02-26T06:37:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.172+0000 7f2445400640 1 -- 172.21.15.49:0/2784403717 learned_addr learned my addr 172.21.15.49:0/2784403717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.172+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019e230 con 0x7f2440075fd0 2024-02-26T06:37:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.172+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019e3f0 con 0x7f2440111e10 2024-02-26T06:37:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.173+0000 7f2445e00640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400756b0 0x7f24401a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.173+0000 7f2446800640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440111e10 0x7f244019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.174+0000 7f2445e00640 1 -- 172.21.15.49:0/2784403717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440111e10 msgr2=0x7f244019da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.174+0000 7f2445e00640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440111e10 0x7f244019da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.174+0000 7f2445e00640 1 -- 172.21.15.49:0/2784403717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2440075fd0 msgr2=0x7f24401a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.174+0000 7f2445e00640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2440075fd0 0x7f24401a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.174+0000 7f2445e00640 1 -- 172.21.15.49:0/2784403717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430009560 con 0x7f24400756b0 2024-02-26T06:37:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.174+0000 7f2446800640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440111e10 0x7f244019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:37:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.175+0000 7f2445e00640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400756b0 0x7f24401a38c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f243400eab0 tx=0x7f243400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.176+0000 7f243aa00640 1 -- 172.21.15.49:0/2784403717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243400cc90 con 0x7f24400756b0 2024-02-26T06:37:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.176+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244019e6d0 con 0x7f24400756b0 2024-02-26T06:37:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.176+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401aa960 con 0x7f24400756b0 2024-02-26T06:37:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.178+0000 7f243aa00640 1 -- 172.21.15.49:0/2784403717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243400cdf0 con 0x7f24400756b0 2024-02-26T06:37:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.178+0000 7f243aa00640 1 -- 172.21.15.49:0/2784403717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434018860 con 0x7f24400756b0 2024-02-26T06:37:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.182+0000 7f244b826740 1 Processor -- start 2024-02-26T06:37:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.182+0000 7f243aa00640 1 -- 172.21.15.49:0/2784403717 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f24340189c0 con 0x7f24400756b0 2024-02-26T06:37:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.182+0000 7f244b826740 1 -- start start 2024-02-26T06:37:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.183+0000 7f244b826740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 0x561161c22a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.183+0000 7f244b826740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c22fc0 0x561161c23440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.183+0000 7f244b826740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c2e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.183+0000 7f244b826740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161bde5f0 con 0x561161c22fc0 2024-02-26T06:37:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.183+0000 7f244b826740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161b90890 con 0x561161c2c540 2024-02-26T06:37:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.183+0000 7f244b826740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161b8ddc0 con 0x561161c23d30 2024-02-26T06:37:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.183+0000 7f2445400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c2e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2446800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 0x561161c22a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2445400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c2e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38270/0 (socket says 172.21.15.49:38270) 2024-02-26T06:37:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2445400640 1 -- 172.21.15.49:0/3799730767 learned_addr learned my addr 172.21.15.49:0/3799730767 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f243aa00640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a22a0 0x7f24280a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2445e00640 1 --2- 172.21.15.49:0/3799730767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c22fc0 0x561161c23440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2445400640 1 -- 172.21.15.49:0/3799730767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 msgr2=0x561161c22a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2445400640 1 --2- 172.21.15.49:0/3799730767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 0x561161c22a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2445400640 1 -- 172.21.15.49:0/3799730767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c22fc0 msgr2=0x561161c23440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2445400640 1 --2- 172.21.15.49:0/3799730767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c22fc0 0x561161c23440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2445400640 1 -- 172.21.15.49:0/3799730767 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561161b0aad0 con 0x561161c2c540 2024-02-26T06:37:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f243aa00640 1 -- 172.21.15.49:0/2784403717 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f2434014070 con 0x7f24400756b0 2024-02-26T06:37:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.184+0000 7f2446800640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a22a0 0x7f24280a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.185+0000 7f2445400640 1 --2- 172.21.15.49:0/3799730767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c2e930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f243002c170 tx=0x7f243002c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5554e8e71e7513 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.185+0000 7f2446800640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a22a0 0x7f24280a4760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24400690b0 tx=0x7f243c004ab0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.186+0000 7f242f000640 1 -- 172.21.15.49:0/3799730767 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004880 con 0x561161c2c540 2024-02-26T06:37:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.186+0000 7f242f000640 1 -- 172.21.15.49:0/3799730767 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24300049e0 con 0x561161c2c540 2024-02-26T06:37:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.186+0000 7f242f000640 1 -- 172.21.15.49:0/3799730767 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430005170 con 0x561161c2c540 2024-02-26T06:37:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.187+0000 7f244b826740 1 -- 172.21.15.49:0/3799730767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 msgr2=0x561161c2e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.187+0000 7f244b826740 1 --2- 172.21.15.49:0/3799730767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c2e930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f243002c170 tx=0x7f243002c680 comp rx=0 tx=0).stop 2024-02-26T06:37:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.189+0000 7f244b826740 1 -- 172.21.15.49:0/3799730767 shutdown_connections 2024-02-26T06:37:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.189+0000 7f244b826740 1 --2- 172.21.15.49:0/3799730767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c2e930 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.189+0000 7f244b826740 1 --2- 172.21.15.49:0/3799730767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c22fc0 0x561161c23440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.189+0000 7f244b826740 1 --2- 172.21.15.49:0/3799730767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 0x561161c22a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.189+0000 7f244b826740 1 -- 172.21.15.49:0/3799730767 >> 172.21.15.49:0/3799730767 conn(0x561161bea480 msgr2=0x561161c1dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.190+0000 7f244b826740 1 -- 172.21.15.49:0/3799730767 shutdown_connections 2024-02-26T06:37:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.190+0000 7f244b826740 1 -- 172.21.15.49:0/3799730767 wait complete. 2024-02-26T06:37:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.190+0000 7f244b826740 1 -- start start 2024-02-26T06:37:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.191+0000 7f244b826740 1 Processor -- start 2024-02-26T06:37:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.195+0000 7f244b826740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 0x561161c294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.195+0000 7f244b826740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 0x561161c29a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.195+0000 7f244b826740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c2c540 0x561161c29f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.195+0000 7f244b826740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161b895a0 con 0x561161c23d30 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.195+0000 7f244b826740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161b2eb20 con 0x561161c22fc0 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.195+0000 7f244b826740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161b2e7c0 con 0x561161c2c540 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.195+0000 7f244b826740 10 client.0 _ll_register_callbacks cb 0x561161b42520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.196+0000 7f2446800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c2c540 0x561161c29f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.196+0000 7f2446800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c2c540 0x561161c29f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48400/0 (socket says 172.21.15.49:48400) 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.196+0000 7f2446800640 1 -- 172.21.15.49:0/304421248 learned_addr learned my addr 172.21.15.49:0/304421248 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.197+0000 7f2446800640 1 -- 172.21.15.49:0/304421248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 msgr2=0x561161c294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.197+0000 7f2446800640 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 0x561161c294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.197+0000 7f2446800640 1 -- 172.21.15.49:0/304421248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 msgr2=0x561161c29a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.197+0000 7f2446800640 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 0x561161c29a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.197+0000 7f2446800640 1 -- 172.21.15.49:0/304421248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2440114c90 con 0x561161c2c540 2024-02-26T06:37:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.198+0000 7f2446800640 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c2c540 0x561161c29f70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f243c013550 tx=0x7f243c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d03eca8653f0ecf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.199+0000 7f2423400640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:37:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.199+0000 7f2423400640 1 -- 172.21.15.49:0/304421248 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c010ea0 con 0x561161c2c540 2024-02-26T06:37:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.199+0000 7f2423400640 1 -- 172.21.15.49:0/304421248 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f243c031440 con 0x561161c2c540 2024-02-26T06:37:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.200+0000 7f244b826740 1 -- 172.21.15.49:0/304421248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561161b5bca0 con 0x561161c2c540 2024-02-26T06:37:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.203+0000 7f2423400640 1 -- 172.21.15.49:0/304421248 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f243c031600 con 0x561161c2c540 2024-02-26T06:37:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.203+0000 7f244b826740 10 client.27619 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.203+0000 7f244b826740 1 -- 172.21.15.49:0/304421248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=52}) v3 -- 0x561161b87c20 con 0x561161c2c540 2024-02-26T06:37:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.204+0000 7f2423400640 1 -- 172.21.15.49:0/304421248 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f243c037020 con 0x561161c2c540 2024-02-26T06:37:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.205+0000 7f244b826740 10 client.27619 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.205+0000 7f244b826740 10 client.27619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.205+0000 7f244b826740 20 client.27619 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.205+0000 7f244b826740 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561161c3c430 0x561161c3e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.205+0000 7f244b826740 4 client.27619 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:37:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.205+0000 7f244b826740 1 -- 172.21.15.49:0/304421248 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561161b7fcb0 con 0x561161c3c430 2024-02-26T06:37:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.206+0000 7f2445e00640 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561161c3c430 0x561161c3e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.209+0000 7f2445e00640 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561161c3c430 0x561161c3e8d0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d547f3fb7c080d server_cookie=a09e5aa2f62a0c6a in_seq=0 out_seq=0 2024-02-26T06:37:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.209+0000 7f2423400640 10 client.27619 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.210+0000 7f2423400640 1 -- 172.21.15.49:0/304421248 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561161b7fcb0 con 0x561161c3c430 2024-02-26T06:37:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.210+0000 7f2423400640 10 client.27619 handle_command_reply: tid=0 2024-02-26T06:37:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.210+0000 7f244b826740 1 client.27619 shutdown 2024-02-26T06:37:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.213+0000 7f244b826740 1 -- 172.21.15.49:0/304421248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c2c540 msgr2=0x561161c29f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.213+0000 7f244b826740 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c2c540 0x561161c29f70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f243c013550 tx=0x7f243c004a40 comp rx=0 tx=0).stop 2024-02-26T06:37:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.213+0000 7f244b826740 1 -- 172.21.15.49:0/304421248 shutdown_connections 2024-02-26T06:37:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.213+0000 7f244b826740 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561161c3c430 0x561161c3e8d0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.213+0000 7f244b826740 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c2c540 0x561161c29f70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.214+0000 7f2423400640 0 client.27619 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.214+0000 7f244b826740 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 0x561161c29a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.214+0000 7f244b826740 1 --2- 172.21.15.49:0/304421248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 0x561161c294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.216+0000 7f244b826740 1 -- 172.21.15.49:0/304421248 >> 172.21.15.49:0/304421248 conn(0x561161bea480 msgr2=0x561161c1fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.217+0000 7f244b826740 1 -- 172.21.15.49:0/304421248 shutdown_connections 2024-02-26T06:37:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.217+0000 7f244b826740 1 -- 172.21.15.49:0/304421248 wait complete. 2024-02-26T06:37:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.217+0000 7f244b826740 20 client.27619 trim_cache size 0 max 16384 2024-02-26T06:37:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.259+0000 7f244b826740 1 Processor -- start 2024-02-26T06:37:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.259+0000 7f244b826740 1 -- start start 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.260+0000 7f244b826740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161bea480 0x561161c2bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.260+0000 7f244b826740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 0x561161c267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.260+0000 7f244b826740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 0x561161c26cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.260+0000 7f244b826740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611618d1e60 con 0x561161bea480 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.260+0000 7f244b826740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161c2a4c0 con 0x561161c22fc0 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.260+0000 7f244b826740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161c1ecb0 con 0x561161c23d30 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.261+0000 7f2445400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 0x561161c267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.261+0000 7f2445e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161bea480 0x561161c2bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.261+0000 7f2445400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 0x561161c267b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:38278/0 (socket says 172.21.15.49:38278) 2024-02-26T06:37:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.261+0000 7f2445400640 1 -- 172.21.15.49:0/3435071056 learned_addr learned my addr 172.21.15.49:0/3435071056 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.261+0000 7f2445e00640 1 -- 172.21.15.49:0/3435071056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 msgr2=0x561161c26cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.261+0000 7f2445e00640 1 --2- 172.21.15.49:0/3435071056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 0x561161c26cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.262+0000 7f2445e00640 1 -- 172.21.15.49:0/3435071056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 msgr2=0x561161c267b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.262+0000 7f2445e00640 1 --2- 172.21.15.49:0/3435071056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 0x561161c267b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.262+0000 7f2445e00640 1 -- 172.21.15.49:0/3435071056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561161b24500 con 0x561161bea480 2024-02-26T06:37:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.262+0000 7f2445e00640 1 --2- 172.21.15.49:0/3435071056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161bea480 0x561161c2bc20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f24340cba90 tx=0x7f243406ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f1bb3faab24913 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.263+0000 7f242e600640 1 -- 172.21.15.49:0/3435071056 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340a34d0 con 0x561161bea480 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.263+0000 7f242e600640 1 -- 172.21.15.49:0/3435071056 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2434099030 con 0x561161bea480 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.263+0000 7f242e600640 1 -- 172.21.15.49:0/3435071056 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243409a3e0 con 0x561161bea480 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.264+0000 7f244b826740 1 -- 172.21.15.49:0/3435071056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161bea480 msgr2=0x561161c2bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.264+0000 7f244b826740 1 --2- 172.21.15.49:0/3435071056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161bea480 0x561161c2bc20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f24340cba90 tx=0x7f243406ccf0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.264+0000 7f244b826740 1 -- 172.21.15.49:0/3435071056 shutdown_connections 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.264+0000 7f244b826740 1 --2- 172.21.15.49:0/3435071056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c22fc0 0x561161c267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.264+0000 7f244b826740 1 --2- 172.21.15.49:0/3435071056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c23d30 0x561161c26cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.264+0000 7f244b826740 1 --2- 172.21.15.49:0/3435071056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161bea480 0x561161c2bc20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.264+0000 7f244b826740 1 -- 172.21.15.49:0/3435071056 >> 172.21.15.49:0/3435071056 conn(0x561161c19140 msgr2=0x561161c27e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.265+0000 7f244b826740 1 -- 172.21.15.49:0/3435071056 shutdown_connections 2024-02-26T06:37:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.265+0000 7f244b826740 1 -- 172.21.15.49:0/3435071056 wait complete. 2024-02-26T06:37:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.265+0000 7f244b826740 1 -- start start 2024-02-26T06:37:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.266+0000 7f244b826740 1 Processor -- start 2024-02-26T06:37:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.267+0000 7f244b826740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c22fc0 0x561161c8d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.267+0000 7f244b826740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 0x561161c8d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.267+0000 7f2445e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c22fc0 0x561161c8d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f244b826740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f2445e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c22fc0 0x561161c8d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48414/0 (socket says 172.21.15.49:48414) 2024-02-26T06:37:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f2445e00640 1 -- 172.21.15.49:0/3645639535 learned_addr learned my addr 172.21.15.49:0/3645639535 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161c33020 con 0x561161c23d30 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161c33190 con 0x561161c2c540 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561161c33350 con 0x561161c22fc0 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f244b826740 10 client.0 _ll_register_callbacks cb 0x7f243c013480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f2446800640 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.268+0000 7f2445400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 0x561161c8d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2446800640 1 -- 172.21.15.49:0/3645639535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c22fc0 msgr2=0x561161c8d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2446800640 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c22fc0 0x561161c8d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2446800640 1 -- 172.21.15.49:0/3645639535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 msgr2=0x561161c8d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2446800640 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 0x561161c8d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2446800640 1 -- 172.21.15.49:0/3645639535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401aa960 con 0x561161c2c540 2024-02-26T06:37:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2445e00640 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c22fc0 0x561161c8d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:37:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2446800640 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c32a60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f243c030b10 tx=0x7f243c009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8104a2b0aa99a867 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2423400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:37:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.269+0000 7f2423400640 1 -- 172.21.15.49:0/3645639535 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c031600 con 0x561161c2c540 2024-02-26T06:37:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.270+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561161baed00 con 0x561161c2c540 2024-02-26T06:37:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.270+0000 7f2423400640 1 -- 172.21.15.49:0/3645639535 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f243c031440 con 0x561161c2c540 2024-02-26T06:37:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.271+0000 7f2423400640 1 -- 172.21.15.49:0/3645639535 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f243c010ea0 con 0x561161c2c540 2024-02-26T06:37:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.271+0000 7f244b826740 10 client.27674 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.271+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x561161b4a290 con 0x561161c2c540 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.271+0000 7f2423400640 1 -- 172.21.15.49:0/3645639535 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561161baed00 con 0x561161c2c540 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.272+0000 7f2423400640 1 -- 172.21.15.49:0/3645639535 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f243c037070 con 0x561161c2c540 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.272+0000 7f244b826740 10 client.27674 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.272+0000 7f244b826740 10 client.27674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.272+0000 7f244b826740 20 client.27674 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.272+0000 7f244b826740 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561161c3c430 0x561161c97010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.272+0000 7f244b826740 4 client.27674 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.272+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561161889600 con 0x561161c3c430 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.272+0000 7f2445e00640 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561161c3c430 0x561161c97010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.273+0000 7f2445e00640 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561161c3c430 0x561161c97010 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7e4ef054e9a9f0 server_cookie=81857466bb266987 in_seq=0 out_seq=0 2024-02-26T06:37:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.273+0000 7f2423400640 10 client.27674 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f2423400640 1 -- 172.21.15.49:0/3645639535 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561161889600 con 0x561161c3c430 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f2423400640 10 client.27674 handle_command_reply: tid=0 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 client.27674 shutdown 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 msgr2=0x561161c32a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c32a60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f243c030b10 tx=0x7f243c009d70 comp rx=0 tx=0).stop 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 shutdown_connections 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x561161c3c430 0x561161c97010 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561161c2c540 0x561161c32a60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561161c23d30 0x561161c8d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 --2- 172.21.15.49:0/3645639535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561161c22fc0 0x561161c8d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f2423400640 0 client.27674 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 >> 172.21.15.49:0/3645639535 conn(0x561161c19140 msgr2=0x561161c294b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 shutdown_connections 2024-02-26T06:37:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.329+0000 7f244b826740 1 -- 172.21.15.49:0/3645639535 wait complete. 2024-02-26T06:37:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f244b826740 20 client.27674 trim_cache size 0 max 16384 2024-02-26T06:37:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:37:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:37:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:37:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a22a0 msgr2=0x7f24280a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a22a0 0x7f24280a4760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24400690b0 tx=0x7f243c004ab0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400756b0 msgr2=0x7f24401a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400756b0 0x7f24401a38c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f243400eab0 tx=0x7f243400ef70 comp rx=0 tx=0).stop 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 shutdown_connections 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f24280a22a0 0x7f24280a4760 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2440075fd0 0x7f24401a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2440111e10 0x7f244019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 --2- 172.21.15.49:0/2784403717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400756b0 0x7f24401a38c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.330+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 >> 172.21.15.49:0/2784403717 conn(0x7f24400fec60 msgr2=0x7f2440101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.331+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 shutdown_connections 2024-02-26T06:37:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:42.331+0000 7f2447200640 1 -- 172.21.15.49:0/2784403717 wait complete. 2024-02-26T06:37:43.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:43 smithi049 ceph-mon[31696]: pgmap v448: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 792 op/s 2024-02-26T06:37:43.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:43 smithi080 ceph-mon[36206]: pgmap v448: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 792 op/s 2024-02-26T06:37:43.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:43 smithi115 ceph-mon[36426]: pgmap v448: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 792 op/s 2024-02-26T06:37:45.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:45 smithi049 ceph-mon[31696]: pgmap v449: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 784 op/s 2024-02-26T06:37:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:45 smithi080 ceph-mon[36206]: pgmap v449: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 784 op/s 2024-02-26T06:37:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:45 smithi115 ceph-mon[36426]: pgmap v449: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 784 op/s 2024-02-26T06:37:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 835.0 s, 888.6 tps, lat 116.035 ms stddev 142.930 2024-02-26T06:37:47.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:47 smithi049 ceph-mon[31696]: pgmap v450: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-02-26T06:37:47.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:47 smithi080 ceph-mon[36206]: pgmap v450: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-02-26T06:37:47.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:47 smithi115 ceph-mon[36426]: pgmap v450: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-02-26T06:37:49.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:49 smithi049 ceph-mon[31696]: pgmap v451: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 970 op/s 2024-02-26T06:37:49.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:49 smithi080 ceph-mon[36206]: pgmap v451: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 970 op/s 2024-02-26T06:37:49.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:49 smithi115 ceph-mon[36426]: pgmap v451: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 970 op/s 2024-02-26T06:37:51.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 840.0 s, 1180.2 tps, lat 83.666 ms stddev 100.697 2024-02-26T06:37:51.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:51 smithi049 ceph-mon[31696]: pgmap v452: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 823 op/s 2024-02-26T06:37:51.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:51 smithi080 ceph-mon[36206]: pgmap v452: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 823 op/s 2024-02-26T06:37:51.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:51 smithi115 ceph-mon[36426]: pgmap v452: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 823 op/s 2024-02-26T06:37:53.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:53 smithi049 ceph-mon[31696]: pgmap v453: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-26T06:37:53.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:53 smithi080 ceph-mon[36206]: pgmap v453: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-26T06:37:53.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:53 smithi115 ceph-mon[36426]: pgmap v453: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-26T06:37:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 -- 172.21.15.49:0/124411806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974101d50 msgr2=0x7f6974105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 --2- 172.21.15.49:0/124411806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974101d50 0x7f6974105d90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6960009900 tx=0x7f696002bc30 comp rx=0 tx=0).stop 2024-02-26T06:37:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 -- 172.21.15.49:0/124411806 shutdown_connections 2024-02-26T06:37:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 --2- 172.21.15.49:0/124411806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6974106470 0x7f697410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 --2- 172.21.15.49:0/124411806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974101d50 0x7f6974105d90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 --2- 172.21.15.49:0/124411806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69741013a0 0x7f6974101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 -- 172.21.15.49:0/124411806 >> 172.21.15.49:0/124411806 conn(0x7f69740fcc40 msgr2=0x7f69740ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 -- 172.21.15.49:0/124411806 shutdown_connections 2024-02-26T06:37:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.664+0000 7f697ae00640 1 -- 172.21.15.49:0/124411806 wait complete. 2024-02-26T06:37:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.665+0000 7f697ae00640 1 Processor -- start 2024-02-26T06:37:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.665+0000 7f697ae00640 1 -- start start 2024-02-26T06:37:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f697ae00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69741013a0 0x7f69741954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f697ae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6974106470 0x7f6974195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f697ae00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974199db0 0x7f697419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f697ae00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697419a7a0 con 0x7f69741013a0 2024-02-26T06:37:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f697ae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697419a910 con 0x7f6974106470 2024-02-26T06:37:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f697ae00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697419aa80 con 0x7f6974199db0 2024-02-26T06:37:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f6973e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69741013a0 0x7f69741954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f6978e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974199db0 0x7f697419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f6978e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974199db0 0x7f697419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40022/0 (socket says 172.21.15.49:40022) 2024-02-26T06:37:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f6973400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6974106470 0x7f6974195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.666+0000 7f6978e00640 1 -- 172.21.15.49:0/2055245082 learned_addr learned my addr 172.21.15.49:0/2055245082 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.667+0000 7f6978e00640 1 -- 172.21.15.49:0/2055245082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6974106470 msgr2=0x7f6974195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.667+0000 7f6978e00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6974106470 0x7f6974195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.667+0000 7f6978e00640 1 -- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69741013a0 msgr2=0x7f69741954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.667+0000 7f6978e00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69741013a0 0x7f69741954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.667+0000 7f6978e00640 1 -- 172.21.15.49:0/2055245082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960009560 con 0x7f6974199db0 2024-02-26T06:37:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.667+0000 7f6978e00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974199db0 0x7f697419a260 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f696c00eb10 tx=0x7f696c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.668+0000 7f6970c00640 1 -- 172.21.15.49:0/2055245082 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c004b20 con 0x7f6974199db0 2024-02-26T06:37:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.668+0000 7f6970c00640 1 -- 172.21.15.49:0/2055245082 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696c004c80 con 0x7f6974199db0 2024-02-26T06:37:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.668+0000 7f6970c00640 1 -- 172.21.15.49:0/2055245082 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c010650 con 0x7f6974199db0 2024-02-26T06:37:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.669+0000 7f697ae00640 1 -- 172.21.15.49:0/2055245082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69741a2060 con 0x7f6974199db0 2024-02-26T06:37:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.669+0000 7f697ae00640 1 -- 172.21.15.49:0/2055245082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69741a2490 con 0x7f6974199db0 2024-02-26T06:37:55.673 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:55 smithi049 ceph-mon[31696]: pgmap v454: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 901 op/s 2024-02-26T06:37:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.674+0000 7f6970c00640 1 -- 172.21.15.49:0/2055245082 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f696c005020 con 0x7f6974199db0 2024-02-26T06:37:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.676+0000 7f6970c00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f69580a6840 0x7f69580a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.676+0000 7f6970c00640 1 -- 172.21.15.49:0/2055245082 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f696c014070 con 0x7f6974199db0 2024-02-26T06:37:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.677+0000 7f6973e00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f69580a6840 0x7f69580a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.679+0000 7f6973e00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f69580a6840 0x7f69580a8d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6964009bd0 tx=0x7f6964005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.680+0000 7f697dd6d740 1 Processor -- start 2024-02-26T06:37:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f697dd6d740 1 -- start start 2024-02-26T06:37:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392024af0 0x55d392023840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 0x55d392024200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d39202d300 0x55d39202f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f697dd6d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d391fdf5f0 con 0x55d392023d80 2024-02-26T06:37:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f697dd6d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d391f91890 con 0x55d39202d300 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f697dd6d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d391f8edc0 con 0x55d392024af0 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f6978e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d39202d300 0x55d39202f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f6978e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d39202d300 0x55d39202f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:40928/0 (socket says 172.21.15.49:40928) 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f6973e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392024af0 0x55d392023840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f6978e00640 1 -- 172.21.15.49:0/4118352387 learned_addr learned my addr 172.21.15.49:0/4118352387 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.681+0000 7f6973400640 1 --2- 172.21.15.49:0/4118352387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 0x55d392024200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6978e00640 1 -- 172.21.15.49:0/4118352387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392024af0 msgr2=0x55d392023840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6978e00640 1 --2- 172.21.15.49:0/4118352387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392024af0 0x55d392023840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6978e00640 1 -- 172.21.15.49:0/4118352387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 msgr2=0x55d392024200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6978e00640 1 --2- 172.21.15.49:0/4118352387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 0x55d392024200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6978e00640 1 -- 172.21.15.49:0/4118352387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d391f0bad0 con 0x55d39202d300 2024-02-26T06:37:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6978e00640 1 --2- 172.21.15.49:0/4118352387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d39202d300 0x55d39202f6f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f696c0a3120 tx=0x7f696c009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a01790f5d60038de server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6969600640 1 -- 172.21.15.49:0/4118352387 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c0a3650 con 0x55d39202d300 2024-02-26T06:37:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6969600640 1 -- 172.21.15.49:0/4118352387 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696c098f50 con 0x55d39202d300 2024-02-26T06:37:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.682+0000 7f6969600640 1 -- 172.21.15.49:0/4118352387 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c0a31c0 con 0x55d39202d300 2024-02-26T06:37:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.683+0000 7f697dd6d740 1 -- 172.21.15.49:0/4118352387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d39202d300 msgr2=0x55d39202f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.683+0000 7f697dd6d740 1 --2- 172.21.15.49:0/4118352387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d39202d300 0x55d39202f6f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f696c0a3120 tx=0x7f696c009a70 comp rx=0 tx=0).stop 2024-02-26T06:37:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.684+0000 7f697dd6d740 1 -- 172.21.15.49:0/4118352387 shutdown_connections 2024-02-26T06:37:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.684+0000 7f697dd6d740 1 --2- 172.21.15.49:0/4118352387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d39202d300 0x55d39202f6f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.684+0000 7f697dd6d740 1 --2- 172.21.15.49:0/4118352387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 0x55d392024200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.684+0000 7f697dd6d740 1 --2- 172.21.15.49:0/4118352387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392024af0 0x55d392023840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.684+0000 7f697dd6d740 1 -- 172.21.15.49:0/4118352387 >> 172.21.15.49:0/4118352387 conn(0x55d391feb480 msgr2=0x55d39201ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.684+0000 7f697dd6d740 1 -- 172.21.15.49:0/4118352387 shutdown_connections 2024-02-26T06:37:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.684+0000 7f697dd6d740 1 -- 172.21.15.49:0/4118352387 wait complete. 2024-02-26T06:37:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.685+0000 7f697dd6d740 1 -- start start 2024-02-26T06:37:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.685+0000 7f697dd6d740 1 Processor -- start 2024-02-26T06:37:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.686+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 0x55d39202a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.686+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39202a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.686+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d39202d300 0x55d39202ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.687+0000 7f697dd6d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d391f8a5a0 con 0x55d392023d80 2024-02-26T06:37:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.687+0000 7f697dd6d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d391f2fb20 con 0x55d392024af0 2024-02-26T06:37:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.687+0000 7f697dd6d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d391f2f7c0 con 0x55d39202d300 2024-02-26T06:37:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.687+0000 7f697dd6d740 10 client.0 _ll_register_callbacks cb 0x55d391f43520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.687+0000 7f6978e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d39202d300 0x55d39202ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.687+0000 7f6978e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d39202d300 0x55d39202ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40038/0 (socket says 172.21.15.49:40038) 2024-02-26T06:37:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.688+0000 7f6978e00640 1 -- 172.21.15.49:0/2257785292 learned_addr learned my addr 172.21.15.49:0/2257785292 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.688+0000 7f6973400640 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39202a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.688+0000 7f6973e00640 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 0x55d39202a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.688+0000 7f6978e00640 1 -- 172.21.15.49:0/2257785292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 msgr2=0x55d39202a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.688+0000 7f6978e00640 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39202a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.688+0000 7f6978e00640 1 -- 172.21.15.49:0/2257785292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 msgr2=0x55d39202a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.688+0000 7f6978e00640 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 0x55d39202a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.688+0000 7f6978e00640 1 -- 172.21.15.49:0/2257785292 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69741a2490 con 0x55d39202d300 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.689+0000 7f6978e00640 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d39202d300 0x55d39202ae10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f696c097880 tx=0x7f696c09f180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=985869194758d5af server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.689+0000 7f695e000640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.689+0000 7f695e000640 1 -- 172.21.15.49:0/2257785292 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c0a3650 con 0x55d39202d300 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.689+0000 7f695e000640 1 -- 172.21.15.49:0/2257785292 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f696c0cbe90 con 0x55d39202d300 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.689+0000 7f697dd6d740 1 -- 172.21.15.49:0/2257785292 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d391f5cca0 con 0x55d39202d300 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.689+0000 7f695e000640 1 -- 172.21.15.49:0/2257785292 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696c0991b0 con 0x55d39202d300 2024-02-26T06:37:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.689+0000 7f697dd6d740 10 client.27631 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.689+0000 7f697dd6d740 1 -- 172.21.15.49:0/2257785292 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55d391f88c20 con 0x55d39202d300 2024-02-26T06:37:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f695e000640 1 -- 172.21.15.49:0/2257785292 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696c07f020 con 0x55d39202d300 2024-02-26T06:37:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f695e000640 1 -- 172.21.15.49:0/2257785292 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f696c01f060 con 0x55d39202d300 2024-02-26T06:37:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f697dd6d740 10 client.27631 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f697dd6d740 10 client.27631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f697dd6d740 20 client.27631 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f697dd6d740 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d3920457c0 0x55d392047bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f697dd6d740 4 client.27631 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:37:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f697dd6d740 1 -- 172.21.15.49:0/2257785292 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d391f80cb0 con 0x55d3920457c0 2024-02-26T06:37:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.690+0000 7f6973e00640 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d3920457c0 0x55d392047bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.691+0000 7f6973e00640 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d3920457c0 0x55d392047bb0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd426d1d330c725a server_cookie=cdfad8d6e77c16ec in_seq=0 out_seq=0 2024-02-26T06:37:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.691+0000 7f695e000640 10 client.27631 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.692+0000 7f695e000640 1 -- 172.21.15.49:0/2257785292 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d391f80cb0 con 0x55d3920457c0 2024-02-26T06:37:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.692+0000 7f695e000640 10 client.27631 handle_command_reply: tid=0 2024-02-26T06:37:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.693+0000 7f697dd6d740 1 client.27631 shutdown 2024-02-26T06:37:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.693+0000 7f697dd6d740 1 -- 172.21.15.49:0/2257785292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d39202d300 msgr2=0x55d39202ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.693+0000 7f697dd6d740 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d39202d300 0x55d39202ae10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f696c097880 tx=0x7f696c09f180 comp rx=0 tx=0).stop 2024-02-26T06:37:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 1 -- 172.21.15.49:0/2257785292 shutdown_connections 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d3920457c0 0x55d392047bb0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f695e000640 0 client.27631 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39202a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d39202d300 0x55d39202ae10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 1 --2- 172.21.15.49:0/2257785292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d392023d80 0x55d39202a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 1 -- 172.21.15.49:0/2257785292 >> 172.21.15.49:0/2257785292 conn(0x55d391feb480 msgr2=0x55d3920207c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 1 -- 172.21.15.49:0/2257785292 shutdown_connections 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 1 -- 172.21.15.49:0/2257785292 wait complete. 2024-02-26T06:37:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.694+0000 7f697dd6d740 20 client.27631 trim_cache size 0 max 16384 2024-02-26T06:37:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.714+0000 7f697dd6d740 1 Processor -- start 2024-02-26T06:37:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.714+0000 7f697dd6d740 1 -- start start 2024-02-26T06:37:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.714+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d39202cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.714+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39203c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.714+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d39203cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.714+0000 7f697dd6d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d391cd2e60 con 0x55d39202d300 2024-02-26T06:37:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.714+0000 7f697dd6d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39201fa70 con 0x55d392024af0 2024-02-26T06:37:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.714+0000 7f697dd6d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392028900 con 0x55d392023d80 2024-02-26T06:37:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.715+0000 7f6973400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39203c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.715+0000 7f6973400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39203c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:40942/0 (socket says 172.21.15.49:40942) 2024-02-26T06:37:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.715+0000 7f6973400640 1 -- 172.21.15.49:0/3644727785 learned_addr learned my addr 172.21.15.49:0/3644727785 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.715+0000 7f6978e00640 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d39203cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.715+0000 7f6973e00640 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d39202cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.715+0000 7f6973e00640 1 -- 172.21.15.49:0/3644727785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 msgr2=0x55d39203c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.715+0000 7f6973e00640 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39203c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.715+0000 7f6973e00640 1 -- 172.21.15.49:0/3644727785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 msgr2=0x55d39203cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.716+0000 7f6973e00640 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d39203cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.716+0000 7f6973e00640 1 -- 172.21.15.49:0/3644727785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d391f25500 con 0x55d392023d80 2024-02-26T06:37:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.716+0000 7f6978e00640 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d39203cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:37:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.717+0000 7f6973e00640 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d39202cac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6964004650 tx=0x7f6964014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0d73a53c112dc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.718+0000 7f6968c00640 1 -- 172.21.15.49:0/3644727785 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964022030 con 0x55d392023d80 2024-02-26T06:37:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.718+0000 7f6968c00640 1 -- 172.21.15.49:0/3644727785 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6964023040 con 0x55d392023d80 2024-02-26T06:37:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.718+0000 7f6968c00640 1 -- 172.21.15.49:0/3644727785 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964033700 con 0x55d392023d80 2024-02-26T06:37:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 -- 172.21.15.49:0/3644727785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 msgr2=0x55d39202cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d39202cac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6964004650 tx=0x7f6964014000 comp rx=0 tx=0).stop 2024-02-26T06:37:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 -- 172.21.15.49:0/3644727785 shutdown_connections 2024-02-26T06:37:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d39203c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d39203cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 --2- 172.21.15.49:0/3644727785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d39202cac0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 -- 172.21.15.49:0/3644727785 >> 172.21.15.49:0/3644727785 conn(0x55d391feb480 msgr2=0x55d39201e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 -- 172.21.15.49:0/3644727785 shutdown_connections 2024-02-26T06:37:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.719+0000 7f697dd6d740 1 -- 172.21.15.49:0/3644727785 wait complete. 2024-02-26T06:37:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 1 -- start start 2024-02-26T06:37:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 1 Processor -- start 2024-02-26T06:37:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d3920270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d392027600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d392027b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39209bc90 con 0x55d39202d300 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39209bdd0 con 0x55d392024af0 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39209bf40 con 0x55d392023d80 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.720+0000 7f697dd6d740 10 client.0 _ll_register_callbacks cb 0x7f696c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d3920270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d3920270c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40050/0 (socket says 172.21.15.49:40050) 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973e00640 1 -- 172.21.15.49:0/67213718 learned_addr learned my addr 172.21.15.49:0/67213718 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973400640 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d392027600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6978e00640 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d392027b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973400640 1 -- 172.21.15.49:0/67213718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 msgr2=0x55d3920270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973400640 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d3920270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973400640 1 -- 172.21.15.49:0/67213718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 msgr2=0x55d392027b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973400640 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d392027b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6973400640 1 -- 172.21.15.49:0/67213718 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d391f88c20 con 0x55d392024af0 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.721+0000 7f6978e00640 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d392027b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.722+0000 7f6973400640 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d392027600 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6960000c00 tx=0x7f696002cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b8da2937136546 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.722+0000 7f695e000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.722+0000 7f695e000640 1 -- 172.21.15.49:0/67213718 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960045040 con 0x55d392024af0 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.722+0000 7f695e000640 1 -- 172.21.15.49:0/67213718 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f6960004310 con 0x55d392024af0 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.722+0000 7f697dd6d740 1 -- 172.21.15.49:0/67213718 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d391fafd00 con 0x55d392024af0 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.722+0000 7f695e000640 1 -- 172.21.15.49:0/67213718 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696002dea0 con 0x55d392024af0 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.722+0000 7f697dd6d740 10 client.27692 fetch_fsmap learned FSMap version 52 2024-02-26T06:37:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.722+0000 7f697dd6d740 1 -- 172.21.15.49:0/67213718 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f6964017070 con 0x55d392024af0 2024-02-26T06:37:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.723+0000 7f695e000640 1 -- 172.21.15.49:0/67213718 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d391fafd00 con 0x55d392024af0 2024-02-26T06:37:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.723+0000 7f695e000640 1 -- 172.21.15.49:0/67213718 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6960048030 con 0x55d392024af0 2024-02-26T06:37:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.724+0000 7f697dd6d740 10 client.27692 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:37:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.724+0000 7f697dd6d740 10 client.27692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:37:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.724+0000 7f697dd6d740 20 client.27692 populate_metadata read hostname 'smithi049' 2024-02-26T06:37:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.724+0000 7f697dd6d740 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d3920457c0 0x55d39209f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:37:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.724+0000 7f697dd6d740 4 client.27692 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:37:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.724+0000 7f697dd6d740 1 -- 172.21.15.49:0/67213718 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d391c8a600 con 0x55d3920457c0 2024-02-26T06:37:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.724+0000 7f6973e00640 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d3920457c0 0x55d39209f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:37:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.725+0000 7f6973e00640 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d3920457c0 0x55d39209f8a0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ead08a2297170a8 server_cookie=843d57e61db86dac in_seq=0 out_seq=0 2024-02-26T06:37:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.725+0000 7f695e000640 10 client.27692 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.740+0000 7f695e000640 1 -- 172.21.15.49:0/67213718 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d391c8a600 con 0x55d3920457c0 2024-02-26T06:37:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.740+0000 7f695e000640 10 client.27692 handle_command_reply: tid=0 2024-02-26T06:37:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.740+0000 7f697dd6d740 1 client.27692 shutdown 2024-02-26T06:37:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 -- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 msgr2=0x55d392027600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d392027600 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6960000c00 tx=0x7f696002cc70 comp rx=0 tx=0).stop 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 -- 172.21.15.49:0/67213718 shutdown_connections 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55d3920457c0 0x55d39209f8a0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d392024af0 0x55d392027600 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d39202d300 0x55d392027b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f695e000640 0 client.27692 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 --2- 172.21.15.49:0/67213718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d392023d80 0x55d3920270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 -- 172.21.15.49:0/67213718 >> 172.21.15.49:0/67213718 conn(0x55d391feb480 msgr2=0x55d392028c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 -- 172.21.15.49:0/67213718 shutdown_connections 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 1 -- 172.21.15.49:0/67213718 wait complete. 2024-02-26T06:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.741+0000 7f697dd6d740 20 client.27692 trim_cache size 0 max 16384 2024-02-26T06:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 -- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f69580a6840 msgr2=0x7f69580a8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f69580a6840 0x7f69580a8d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6964009bd0 tx=0x7f6964005f90 comp rx=0 tx=0).stop 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 -- 172.21.15.49:0/2055245082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974199db0 msgr2=0x7f697419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974199db0 0x7f697419a260 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f696c00eb10 tx=0x7f696c00c2d0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 -- 172.21.15.49:0/2055245082 shutdown_connections 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f69580a6840 0x7f69580a8d00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6974106470 0x7f6974195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6974199db0 0x7f697419a260 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 --2- 172.21.15.49:0/2055245082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69741013a0 0x7f69741954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 -- 172.21.15.49:0/2055245082 >> 172.21.15.49:0/2055245082 conn(0x7f69740fcc40 msgr2=0x7f69740fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 -- 172.21.15.49:0/2055245082 shutdown_connections 2024-02-26T06:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:37:55.742+0000 7f697ae00640 1 -- 172.21.15.49:0/2055245082 wait complete. 2024-02-26T06:37:55.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:55 smithi080 ceph-mon[36206]: pgmap v454: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 901 op/s 2024-02-26T06:37:55.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:55 smithi115 ceph-mon[36426]: pgmap v454: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 901 op/s 2024-02-26T06:37:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 845.0 s, 1221.8 tps, lat 81.454 ms stddev 93.632 2024-02-26T06:37:56.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:56 smithi080 ceph-mon[36206]: pgmap v455: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-02-26T06:37:56.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:56 smithi115 ceph-mon[36426]: pgmap v455: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-02-26T06:37:56.918 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:56 smithi049 ceph-mon[31696]: pgmap v455: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-02-26T06:37:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:37:58 smithi080 ceph-mon[36206]: pgmap v456: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-02-26T06:37:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:37:58 smithi115 ceph-mon[36426]: pgmap v456: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-02-26T06:37:59.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:37:58 smithi049 ceph-mon[31696]: pgmap v456: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-02-26T06:38:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:00 smithi080 ceph-mon[36206]: pgmap v457: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-02-26T06:38:01.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:00 smithi115 ceph-mon[36426]: pgmap v457: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-02-26T06:38:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:00 smithi049 ceph-mon[31696]: pgmap v457: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-02-26T06:38:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 850.0 s, 1133.6 tps, lat 88.269 ms stddev 109.313 2024-02-26T06:38:03.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:02 smithi080 ceph-mon[36206]: pgmap v458: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-02-26T06:38:03.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:02 smithi115 ceph-mon[36426]: pgmap v458: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-02-26T06:38:03.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:02 smithi049 ceph-mon[31696]: pgmap v458: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-02-26T06:38:04.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:03 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:38:04.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:03 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:38:04.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:03 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:38:05.087 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:04 smithi080 ceph-mon[36206]: pgmap v459: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 597 op/s 2024-02-26T06:38:05.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:04 smithi115 ceph-mon[36426]: pgmap v459: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 597 op/s 2024-02-26T06:38:05.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:04 smithi049 ceph-mon[31696]: pgmap v459: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 597 op/s 2024-02-26T06:38:06.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:05 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:38:06.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:05 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:38:06.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:05 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:38:06.104 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:05 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:38:06.104 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:05 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:38:06.104 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:05 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:38:06.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:05 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:38:06.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:05 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:38:06.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:05 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:38:06.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 855.0 s, 391.8 tps, lat 255.135 ms stddev 273.461 2024-02-26T06:38:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:07 smithi049 ceph-mon[31696]: pgmap v460: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2024-02-26T06:38:07.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:07 smithi049 ceph-mon[31696]: osdmap e106: 12 total, 12 up, 12 in 2024-02-26T06:38:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:07 smithi115 ceph-mon[36426]: pgmap v460: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2024-02-26T06:38:07.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:07 smithi115 ceph-mon[36426]: osdmap e106: 12 total, 12 up, 12 in 2024-02-26T06:38:07.687 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:07 smithi080 ceph-mon[36206]: pgmap v460: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2024-02-26T06:38:07.687 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:07 smithi080 ceph-mon[36206]: osdmap e106: 12 total, 12 up, 12 in 2024-02-26T06:38:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.993+0000 7ff67e000640 1 -- 172.21.15.49:0/1645849836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678105080 msgr2=0x7ff678105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.993+0000 7ff67e000640 1 --2- 172.21.15.49:0/1645849836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678105080 0x7ff678105460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff668009930 tx=0x7ff66802bc30 comp rx=0 tx=0).stop 2024-02-26T06:38:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.993+0000 7ff67e000640 1 -- 172.21.15.49:0/1645849836 shutdown_connections 2024-02-26T06:38:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.993+0000 7ff67e000640 1 --2- 172.21.15.49:0/1645849836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6781015b0 0x7ff67810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.993+0000 7ff67e000640 1 --2- 172.21.15.49:0/1645849836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff678100bf0 0x7ff678101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.993+0000 7ff67e000640 1 --2- 172.21.15.49:0/1645849836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678105080 0x7ff678105460 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.993+0000 7ff67e000640 1 -- 172.21.15.49:0/1645849836 >> 172.21.15.49:0/1645849836 conn(0x7ff678076210 msgr2=0x7ff6780fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.994+0000 7ff67e000640 1 -- 172.21.15.49:0/1645849836 shutdown_connections 2024-02-26T06:38:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.994+0000 7ff67e000640 1 -- 172.21.15.49:0/1645849836 wait complete. 2024-02-26T06:38:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.995+0000 7ff67e000640 1 Processor -- start 2024-02-26T06:38:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.995+0000 7ff67e000640 1 -- start start 2024-02-26T06:38:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff67e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678100bf0 0x7ff67819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff67e000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6781015b0 0x7ff67819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff67e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff678105080 0x7ff678199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff67e000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff678199eb0 con 0x7ff6781015b0 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff67e000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67819a020 con 0x7ff678100bf0 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff67e000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67819a1e0 con 0x7ff678105080 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678100bf0 0x7ff67819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff676a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6781015b0 0x7ff67819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678100bf0 0x7ff67819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60680/0 (socket says 172.21.15.49:60680) 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677400640 1 -- 172.21.15.49:0/2896110701 learned_addr learned my addr 172.21.15.49:0/2896110701 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677e00640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff678105080 0x7ff678199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677400640 1 -- 172.21.15.49:0/2896110701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff678105080 msgr2=0x7ff678199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677400640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff678105080 0x7ff678199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677400640 1 -- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6781015b0 msgr2=0x7ff67819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677400640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6781015b0 0x7ff67819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.996+0000 7ff677400640 1 -- 172.21.15.49:0/2896110701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668009560 con 0x7ff678100bf0 2024-02-26T06:38:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.997+0000 7ff677400640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678100bf0 0x7ff67819f660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff668000c00 tx=0x7ff66802cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.997+0000 7ff66fe00640 1 -- 172.21.15.49:0/2896110701 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668004350 con 0x7ff678100bf0 2024-02-26T06:38:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.997+0000 7ff66fe00640 1 -- 172.21.15.49:0/2896110701 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6680044b0 con 0x7ff678100bf0 2024-02-26T06:38:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.997+0000 7ff66fe00640 1 -- 172.21.15.49:0/2896110701 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668003d30 con 0x7ff678100bf0 2024-02-26T06:38:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.997+0000 7ff67e000640 1 -- 172.21.15.49:0/2896110701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67819a460 con 0x7ff678100bf0 2024-02-26T06:38:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:08.997+0000 7ff67e000640 1 -- 172.21.15.49:0/2896110701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6781a65e0 con 0x7ff678100bf0 2024-02-26T06:38:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.002+0000 7ff66fe00640 1 -- 172.21.15.49:0/2896110701 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff668004df0 con 0x7ff678100bf0 2024-02-26T06:38:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.004+0000 7ff66fe00640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff65c0a2350 0x7ff65c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.004+0000 7ff66fe00640 1 -- 172.21.15.49:0/2896110701 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8594+0+0 (secure 0 0 0) 0x7ff6680f0400 con 0x7ff678100bf0 2024-02-26T06:38:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.005+0000 7ff676a00640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff65c0a2350 0x7ff65c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.006+0000 7ff676a00640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff65c0a2350 0x7ff65c0a4810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff67819af70 tx=0x7ff66400b410 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff680fb8740 1 Processor -- start 2024-02-26T06:38:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff680fb8740 1 -- start start 2024-02-26T06:38:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445819af0 0x557445818840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445818d80 0x557445819200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x5574458246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff680fb8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574457d45f0 con 0x557445819af0 2024-02-26T06:38:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff680fb8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557445786890 con 0x557445822300 2024-02-26T06:38:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff680fb8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557445783dc0 con 0x557445818d80 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff676a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445818d80 0x557445819200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff677400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445819af0 0x557445818840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff677e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x5574458246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff676a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445818d80 0x557445819200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53224/0 (socket says 172.21.15.49:53224) 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff677e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x5574458246f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60694/0 (socket says 172.21.15.49:60694) 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.013+0000 7ff676a00640 1 -- 172.21.15.49:0/2896589644 learned_addr learned my addr 172.21.15.49:0/2896589644 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff676a00640 1 -- 172.21.15.49:0/2896589644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 msgr2=0x5574458246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff676a00640 1 --2- 172.21.15.49:0/2896589644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x5574458246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff676a00640 1 -- 172.21.15.49:0/2896589644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445819af0 msgr2=0x557445818840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff676a00640 1 --2- 172.21.15.49:0/2896589644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445819af0 0x557445818840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff676a00640 1 -- 172.21.15.49:0/2896589644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557445700ad0 con 0x557445818d80 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff677400640 1 --2- 172.21.15.49:0/2896589644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445819af0 0x557445818840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff676a00640 1 --2- 172.21.15.49:0/2896589644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445818d80 0x557445819200 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff66400bfd0 tx=0x7ff664013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=127a377de1b9d907 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff663e00640 1 -- 172.21.15.49:0/2896589644 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66401e620 con 0x557445818d80 2024-02-26T06:38:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff663e00640 1 -- 172.21.15.49:0/2896589644 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff664015070 con 0x557445818d80 2024-02-26T06:38:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.014+0000 7ff663e00640 1 -- 172.21.15.49:0/2896589644 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66401cab0 con 0x557445818d80 2024-02-26T06:38:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.015+0000 7ff680fb8740 1 -- 172.21.15.49:0/2896589644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445818d80 msgr2=0x557445819200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.015+0000 7ff680fb8740 1 --2- 172.21.15.49:0/2896589644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445818d80 0x557445819200 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff66400bfd0 tx=0x7ff664013000 comp rx=0 tx=0).stop 2024-02-26T06:38:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.016+0000 7ff680fb8740 1 -- 172.21.15.49:0/2896589644 shutdown_connections 2024-02-26T06:38:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.016+0000 7ff680fb8740 1 --2- 172.21.15.49:0/2896589644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x5574458246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.016+0000 7ff680fb8740 1 --2- 172.21.15.49:0/2896589644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445818d80 0x557445819200 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.016+0000 7ff680fb8740 1 --2- 172.21.15.49:0/2896589644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445819af0 0x557445818840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.016+0000 7ff680fb8740 1 -- 172.21.15.49:0/2896589644 >> 172.21.15.49:0/2896589644 conn(0x5574457e0480 msgr2=0x557445813a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.016+0000 7ff680fb8740 1 -- 172.21.15.49:0/2896589644 shutdown_connections 2024-02-26T06:38:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.016+0000 7ff680fb8740 1 -- 172.21.15.49:0/2896589644 wait complete. 2024-02-26T06:38:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.017+0000 7ff680fb8740 1 -- start start 2024-02-26T06:38:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.017+0000 7ff680fb8740 1 Processor -- start 2024-02-26T06:38:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x55744581f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445819af0 0x55744581f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445822300 0x55744581fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff680fb8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744577f5a0 con 0x557445818d80 2024-02-26T06:38:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff680fb8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557445724b20 con 0x557445819af0 2024-02-26T06:38:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff680fb8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574457247c0 con 0x557445822300 2024-02-26T06:38:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff680fb8740 10 client.0 _ll_register_callbacks cb 0x557445738520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff677e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445822300 0x55744581fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff677400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x55744581f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff677e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445822300 0x55744581fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53230/0 (socket says 172.21.15.49:53230) 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff677e00640 1 -- 172.21.15.49:0/4149323540 learned_addr learned my addr 172.21.15.49:0/4149323540 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.018+0000 7ff676a00640 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445819af0 0x55744581f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff677e00640 1 -- 172.21.15.49:0/4149323540 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445819af0 msgr2=0x55744581f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff677e00640 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445819af0 0x55744581f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff677e00640 1 -- 172.21.15.49:0/4149323540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 msgr2=0x55744581f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff677e00640 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x55744581f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff677e00640 1 -- 172.21.15.49:0/4149323540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66400bd00 con 0x557445822300 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff677400640 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x55744581f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff677e00640 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445822300 0x55744581fde0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff670011d20 tx=0x7ff67000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50cf53cdf3d1fd09 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff660c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff660c00640 1 -- 172.21.15.49:0/4149323540 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670016070 con 0x557445822300 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff660c00640 1 -- 172.21.15.49:0/4149323540 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8594+0+0 (secure 0 0 0) 0x7ff670003970 con 0x557445822300 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff680fb8740 1 -- 172.21.15.49:0/4149323540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557445751ca0 con 0x557445822300 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff660c00640 1 -- 172.21.15.49:0/4149323540 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff670003b30 con 0x557445822300 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff680fb8740 10 client.27649 fetch_fsmap learned FSMap version 52 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.019+0000 7ff680fb8740 1 -- 172.21.15.49:0/4149323540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55744577dc20 con 0x557445822300 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff660c00640 1 -- 172.21.15.49:0/4149323540 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff670026020 con 0x557445822300 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff660c00640 1 -- 172.21.15.49:0/4149323540 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff67001b540 con 0x557445822300 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff680fb8740 10 client.27649 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff680fb8740 10 client.27649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:38:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff680fb8740 20 client.27649 populate_metadata read hostname 'smithi049' 2024-02-26T06:38:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff680fb8740 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55744583ace0 0x55744583d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff680fb8740 4 client.27649 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:38:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff680fb8740 1 -- 172.21.15.49:0/4149323540 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557445775cb0 con 0x55744583ace0 2024-02-26T06:38:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.020+0000 7ff677400640 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55744583ace0 0x55744583d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.021+0000 7ff677400640 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55744583ace0 0x55744583d100 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f713595e537010 server_cookie=1ae6ab22d040be5d in_seq=0 out_seq=0 2024-02-26T06:38:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.021+0000 7ff660c00640 10 client.27649 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.022+0000 7ff660c00640 1 -- 172.21.15.49:0/4149323540 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557445775cb0 con 0x55744583ace0 2024-02-26T06:38:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.022+0000 7ff660c00640 10 client.27649 handle_command_reply: tid=0 2024-02-26T06:38:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.022+0000 7ff680fb8740 1 client.27649 shutdown 2024-02-26T06:38:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.023+0000 7ff680fb8740 1 -- 172.21.15.49:0/4149323540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445822300 msgr2=0x55744581fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.023+0000 7ff680fb8740 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445822300 0x55744581fde0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff670011d20 tx=0x7ff67000f790 comp rx=0 tx=0).stop 2024-02-26T06:38:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.023+0000 7ff680fb8740 1 -- 172.21.15.49:0/4149323540 shutdown_connections 2024-02-26T06:38:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.023+0000 7ff680fb8740 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55744583ace0 0x55744583d100 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.024+0000 7ff680fb8740 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445819af0 0x55744581f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.024+0000 7ff660c00640 0 client.27649 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.024+0000 7ff680fb8740 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445822300 0x55744581fde0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.024+0000 7ff680fb8740 1 --2- 172.21.15.49:0/4149323540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x55744581f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.024+0000 7ff680fb8740 1 -- 172.21.15.49:0/4149323540 >> 172.21.15.49:0/4149323540 conn(0x5574457e0480 msgr2=0x5574458157c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.024+0000 7ff680fb8740 1 -- 172.21.15.49:0/4149323540 shutdown_connections 2024-02-26T06:38:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.024+0000 7ff680fb8740 1 -- 172.21.15.49:0/4149323540 wait complete. 2024-02-26T06:38:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.024+0000 7ff680fb8740 20 client.27649 trim_cache size 0 max 16384 2024-02-26T06:38:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.044+0000 7ff680fb8740 1 Processor -- start 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff680fb8740 1 -- start start 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x557445821a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x557445831a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x557445831f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff680fb8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574454c7e60 con 0x557445818d80 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff680fb8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574458206b0 con 0x557445822300 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff680fb8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557445824f10 con 0x557445819af0 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff677400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x557445821a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff677e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x557445831f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff676a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x557445831a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff676a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x557445831a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53232/0 (socket says 172.21.15.49:53232) 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff677e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x557445831f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60716/0 (socket says 172.21.15.49:60716) 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.045+0000 7ff676a00640 1 -- 172.21.15.49:0/2241755155 learned_addr learned my addr 172.21.15.49:0/2241755155 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff677e00640 1 -- 172.21.15.49:0/2241755155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 msgr2=0x557445831a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff677e00640 1 --2- 172.21.15.49:0/2241755155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x557445831a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff677e00640 1 -- 172.21.15.49:0/2241755155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 msgr2=0x557445821a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff677e00640 1 --2- 172.21.15.49:0/2241755155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x557445821a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff677e00640 1 -- 172.21.15.49:0/2241755155 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55744571a500 con 0x557445822300 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff676a00640 1 --2- 172.21.15.49:0/2241755155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x557445831a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff677e00640 1 --2- 172.21.15.49:0/2241755155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x557445831f50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff67001b920 tx=0x7ff670011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b873c1d10da6c3ce server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff663400640 1 -- 172.21.15.49:0/2241755155 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670016070 con 0x557445822300 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff663400640 1 -- 172.21.15.49:0/2241755155 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff670026070 con 0x557445822300 2024-02-26T06:38:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.046+0000 7ff663400640 1 -- 172.21.15.49:0/2241755155 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670005310 con 0x557445822300 2024-02-26T06:38:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.047+0000 7ff680fb8740 1 -- 172.21.15.49:0/2241755155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 msgr2=0x557445831f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.047+0000 7ff680fb8740 1 --2- 172.21.15.49:0/2241755155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x557445831f50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff67001b920 tx=0x7ff670011900 comp rx=0 tx=0).stop 2024-02-26T06:38:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.047+0000 7ff680fb8740 1 -- 172.21.15.49:0/2241755155 shutdown_connections 2024-02-26T06:38:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.047+0000 7ff680fb8740 1 --2- 172.21.15.49:0/2241755155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x557445831f50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.047+0000 7ff680fb8740 1 --2- 172.21.15.49:0/2241755155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x557445831a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.047+0000 7ff680fb8740 1 --2- 172.21.15.49:0/2241755155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x557445821a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.047+0000 7ff680fb8740 1 -- 172.21.15.49:0/2241755155 >> 172.21.15.49:0/2241755155 conn(0x5574457e0480 msgr2=0x5574458135a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.048+0000 7ff680fb8740 1 -- 172.21.15.49:0/2241755155 shutdown_connections 2024-02-26T06:38:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.048+0000 7ff680fb8740 1 -- 172.21.15.49:0/2241755155 wait complete. 2024-02-26T06:38:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.048+0000 7ff680fb8740 1 -- start start 2024-02-26T06:38:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.049+0000 7ff680fb8740 1 Processor -- start 2024-02-26T06:38:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x557445831ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x55744582e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff680fb8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x55744582e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff680fb8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744582ef30 con 0x557445818d80 2024-02-26T06:38:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff680fb8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744582f0a0 con 0x557445822300 2024-02-26T06:38:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff680fb8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574458911f0 con 0x557445819af0 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff680fb8740 10 client.0 _ll_register_callbacks cb 0x7ff670009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff676a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x55744582e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff677e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x55744582e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff677400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x557445831ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff676a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x55744582e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53248/0 (socket says 172.21.15.49:53248) 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff676a00640 1 -- 172.21.15.49:0/912890773 learned_addr learned my addr 172.21.15.49:0/912890773 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.050+0000 7ff677e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x55744582e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:60732/0 (socket says 172.21.15.49:60732) 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.051+0000 7ff676a00640 1 -- 172.21.15.49:0/912890773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 msgr2=0x55744582e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.051+0000 7ff676a00640 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x55744582e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.051+0000 7ff676a00640 1 -- 172.21.15.49:0/912890773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 msgr2=0x557445831ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.051+0000 7ff676a00640 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x557445831ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.051+0000 7ff676a00640 1 -- 172.21.15.49:0/912890773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557445700ad0 con 0x557445819af0 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.051+0000 7ff676a00640 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x55744582e220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff66400baa0 tx=0x7ff664004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7506b85a44ac57 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.051+0000 7ff660c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.051+0000 7ff660c00640 1 -- 172.21.15.49:0/912890773 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66401ccd0 con 0x557445819af0 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff660c00640 1 -- 172.21.15.49:0/912890773 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8594+0+0 (secure 0 0 0) 0x7ff664017070 con 0x557445819af0 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 1 -- 172.21.15.49:0/912890773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574457a4d00 con 0x557445819af0 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff660c00640 1 -- 172.21.15.49:0/912890773 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff66401ce30 con 0x557445819af0 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 10 client.27655 fetch_fsmap learned FSMap version 52 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 1 -- 172.21.15.49:0/912890773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7ff67001be70 con 0x557445819af0 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff660c00640 1 -- 172.21.15.49:0/912890773 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5574457a4d00 con 0x557445819af0 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff660c00640 1 -- 172.21.15.49:0/912890773 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff664030040 con 0x557445819af0 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 10 client.27655 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:38:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 10 client.27655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:38:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 20 client.27655 populate_metadata read hostname 'smithi049' 2024-02-26T06:38:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557445892980 0x557445894da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 4 client.27655 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:38:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.052+0000 7ff680fb8740 1 -- 172.21.15.49:0/912890773 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55744547f600 con 0x557445892980 2024-02-26T06:38:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.053+0000 7ff677400640 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557445892980 0x557445894da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.053+0000 7ff677400640 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557445892980 0x557445894da0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c971cd5176f96d2e server_cookie=8337058a15bf8a07 in_seq=0 out_seq=0 2024-02-26T06:38:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.053+0000 7ff660c00640 10 client.27655 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.064+0000 7ff660c00640 1 -- 172.21.15.49:0/912890773 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55744547f600 con 0x557445892980 2024-02-26T06:38:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.064+0000 7ff660c00640 10 client.27655 handle_command_reply: tid=0 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.064+0000 7ff680fb8740 1 client.27655 shutdown 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.064+0000 7ff680fb8740 1 -- 172.21.15.49:0/912890773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 msgr2=0x55744582e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.064+0000 7ff680fb8740 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x55744582e220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff66400baa0 tx=0x7ff664004730 comp rx=0 tx=0).stop 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.064+0000 7ff680fb8740 1 -- 172.21.15.49:0/912890773 shutdown_connections 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.064+0000 7ff680fb8740 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x557445892980 0x557445894da0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.065+0000 7ff680fb8740 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557445822300 0x55744582e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.065+0000 7ff680fb8740 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557445819af0 0x55744582e220 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.065+0000 7ff680fb8740 1 --2- 172.21.15.49:0/912890773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557445818d80 0x557445831ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.065+0000 7ff660c00640 0 client.27655 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.065+0000 7ff680fb8740 1 -- 172.21.15.49:0/912890773 >> 172.21.15.49:0/912890773 conn(0x5574457e0480 msgr2=0x55744581d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.065+0000 7ff680fb8740 1 -- 172.21.15.49:0/912890773 shutdown_connections 2024-02-26T06:38:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.065+0000 7ff680fb8740 1 -- 172.21.15.49:0/912890773 wait complete. 2024-02-26T06:38:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.065+0000 7ff680fb8740 20 client.27655 trim_cache size 0 max 16384 2024-02-26T06:38:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:38:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:38:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:38:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:38:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 -- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff65c0a2350 msgr2=0x7ff65c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff65c0a2350 0x7ff65c0a4810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff67819af70 tx=0x7ff66400b410 comp rx=0 tx=0).stop 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 -- 172.21.15.49:0/2896110701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678100bf0 msgr2=0x7ff67819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678100bf0 0x7ff67819f660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff668000c00 tx=0x7ff66802cd80 comp rx=0 tx=0).stop 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 -- 172.21.15.49:0/2896110701 shutdown_connections 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff65c0a2350 0x7ff65c0a4810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff678105080 0x7ff678199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6781015b0 0x7ff67819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 --2- 172.21.15.49:0/2896110701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff678100bf0 0x7ff67819f660 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 -- 172.21.15.49:0/2896110701 >> 172.21.15.49:0/2896110701 conn(0x7ff678076210 msgr2=0x7ff678100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 -- 172.21.15.49:0/2896110701 shutdown_connections 2024-02-26T06:38:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:09.066+0000 7ff67e000640 1 -- 172.21.15.49:0/2896110701 wait complete. 2024-02-26T06:38:09.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:09 smithi049 ceph-mon[31696]: pgmap v462: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-02-26T06:38:09.579 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:09 smithi080 ceph-mon[36206]: pgmap v462: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-02-26T06:38:09.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:09 smithi115 ceph-mon[36426]: pgmap v462: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-02-26T06:38:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 860.0 s, 1480.8 tps, lat 68.974 ms stddev 94.965 2024-02-26T06:38:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:11 smithi080 ceph-mon[36206]: pgmap v463: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-02-26T06:38:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:11 smithi115 ceph-mon[36426]: pgmap v463: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-02-26T06:38:11.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:11 smithi049 ceph-mon[31696]: pgmap v463: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-02-26T06:38:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:13 smithi080 ceph-mon[36206]: pgmap v464: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2024-02-26T06:38:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:13 smithi115 ceph-mon[36426]: pgmap v464: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2024-02-26T06:38:13.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:13 smithi049 ceph-mon[31696]: pgmap v464: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2024-02-26T06:38:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:15 smithi080 ceph-mon[36206]: pgmap v465: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 1.47k op/s 2024-02-26T06:38:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:15 smithi080 ceph-mon[36206]: osdmap e107: 12 total, 12 up, 12 in 2024-02-26T06:38:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:15 smithi115 ceph-mon[36426]: pgmap v465: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 1.47k op/s 2024-02-26T06:38:15.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:15 smithi115 ceph-mon[36426]: osdmap e107: 12 total, 12 up, 12 in 2024-02-26T06:38:15.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:15 smithi049 ceph-mon[31696]: pgmap v465: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 1.47k op/s 2024-02-26T06:38:15.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:15 smithi049 ceph-mon[31696]: osdmap e107: 12 total, 12 up, 12 in 2024-02-26T06:38:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 865.0 s, 972.0 tps, lat 100.346 ms stddev 141.887 2024-02-26T06:38:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:17 smithi080 ceph-mon[36206]: pgmap v467: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.31k op/s 2024-02-26T06:38:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:17 smithi115 ceph-mon[36426]: pgmap v467: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.31k op/s 2024-02-26T06:38:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:17 smithi049 ceph-mon[31696]: pgmap v467: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.31k op/s 2024-02-26T06:38:19.588 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:19 smithi080 ceph-mon[36206]: pgmap v468: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-26T06:38:19.593 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:19 smithi115 ceph-mon[36426]: pgmap v468: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-26T06:38:19.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:19 smithi049 ceph-mon[31696]: pgmap v468: 97 pgs: 97 active+clean; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-26T06:38:21.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 870.0 s, 1263.0 tps, lat 81.155 ms stddev 108.414 2024-02-26T06:38:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:21 smithi080 ceph-mon[36206]: pgmap v469: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-02-26T06:38:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:21 smithi115 ceph-mon[36426]: pgmap v469: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-02-26T06:38:21.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:21 smithi049 ceph-mon[31696]: pgmap v469: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-02-26T06:38:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.287+0000 7f94d6000640 1 -- 172.21.15.49:0/4072191428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01035c0 msgr2=0x7f94d01039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.287+0000 7f94d6000640 1 --2- 172.21.15.49:0/4072191428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01035c0 0x7f94d01039a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009900 tx=0x7f94c002bc40 comp rx=0 tx=0).stop 2024-02-26T06:38:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.288+0000 7f94d6000640 1 -- 172.21.15.49:0/4072191428 shutdown_connections 2024-02-26T06:38:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.288+0000 7f94d6000640 1 --2- 172.21.15.49:0/4072191428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94d0108690 0x7f94d0110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.288+0000 7f94d6000640 1 --2- 172.21.15.49:0/4072191428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94d0103f70 0x7f94d0107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.288+0000 7f94d6000640 1 --2- 172.21.15.49:0/4072191428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01035c0 0x7f94d01039a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.288+0000 7f94d6000640 1 -- 172.21.15.49:0/4072191428 >> 172.21.15.49:0/4072191428 conn(0x7f94d00fee40 msgr2=0x7f94d0101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.288+0000 7f94d6000640 1 -- 172.21.15.49:0/4072191428 shutdown_connections 2024-02-26T06:38:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.288+0000 7f94d6000640 1 -- 172.21.15.49:0/4072191428 wait complete. 2024-02-26T06:38:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.289+0000 7f94d6000640 1 Processor -- start 2024-02-26T06:38:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.289+0000 7f94d6000640 1 -- start start 2024-02-26T06:38:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.290+0000 7f94d6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94d01035c0 0x7f94d019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.290+0000 7f94d6000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94d0108690 0x7f94d019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.290+0000 7f94d6000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01997a0 0x7f94d0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.290+0000 7f94d6000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d019a250 con 0x7f94d0108690 2024-02-26T06:38:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.290+0000 7f94d6000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d019a3c0 con 0x7f94d01035c0 2024-02-26T06:38:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.290+0000 7f94d6000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d019a530 con 0x7f94d01997a0 2024-02-26T06:38:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.290+0000 7f94cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01997a0 0x7f94d0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.290+0000 7f94cf400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94d01035c0 0x7f94d019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.291+0000 7f94cfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01997a0 0x7f94d0199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49400/0 (socket says 172.21.15.49:49400) 2024-02-26T06:38:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.291+0000 7f94cfe00640 1 -- 172.21.15.49:0/1966814905 learned_addr learned my addr 172.21.15.49:0/1966814905 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.291+0000 7f94cea00640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94d0108690 0x7f94d019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.292+0000 7f94cfe00640 1 -- 172.21.15.49:0/1966814905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94d01035c0 msgr2=0x7f94d019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.292+0000 7f94cfe00640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94d01035c0 0x7f94d019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.292+0000 7f94cfe00640 1 -- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94d0108690 msgr2=0x7f94d019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.292+0000 7f94cfe00640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94d0108690 0x7f94d019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.292+0000 7f94cfe00640 1 -- 172.21.15.49:0/1966814905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c8009840 con 0x7f94d01997a0 2024-02-26T06:38:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.293+0000 7f94cf400640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94d01035c0 0x7f94d019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.293+0000 7f94cfe00640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01997a0 0x7f94d0199c50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f94c800efd0 tx=0x7f94c800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.293+0000 7f94c7e00640 1 -- 172.21.15.49:0/1966814905 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8009b40 con 0x7f94d01997a0 2024-02-26T06:38:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.293+0000 7f94c7e00640 1 -- 172.21.15.49:0/1966814905 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94c8003b70 con 0x7f94d01997a0 2024-02-26T06:38:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.293+0000 7f94c7e00640 1 -- 172.21.15.49:0/1966814905 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c80108f0 con 0x7f94d01997a0 2024-02-26T06:38:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.293+0000 7f94d6000640 1 -- 172.21.15.49:0/1966814905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c0009560 con 0x7f94d01997a0 2024-02-26T06:38:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.293+0000 7f94d6000640 1 -- 172.21.15.49:0/1966814905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d0075950 con 0x7f94d01997a0 2024-02-26T06:38:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.297+0000 7f94c7e00640 1 -- 172.21.15.49:0/1966814905 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f94c8003ce0 con 0x7f94d01997a0 2024-02-26T06:38:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94d8e39740 1 Processor -- start 2024-02-26T06:38:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94d8e39740 1 -- start start 2024-02-26T06:38:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b7209840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b720a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7213300 0x5634b72156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94d8e39740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b71c55f0 con 0x5634b7209d80 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94d8e39740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b7177890 con 0x5634b720aaf0 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94d8e39740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b7174dc0 con 0x5634b7213300 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.298+0000 7f94c7e00640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f94b40a2350 0x7f94b40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.299+0000 7f94c7e00640 1 -- 172.21.15.49:0/1966814905 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f94c8014070 con 0x7f94d01997a0 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.299+0000 7f94cea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b7209840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.299+0000 7f94cf400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b720a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.299+0000 7f94cfe00640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f94b40a2350 0x7f94b40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.299+0000 7f94cea00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b7209840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:43890/0 (socket says 172.21.15.49:43890) 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.299+0000 7f94cea00640 1 -- 172.21.15.49:0/912887748 learned_addr learned my addr 172.21.15.49:0/912887748 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.299+0000 7f94cea00640 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7213300 0x5634b72156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.300+0000 7f94cfe00640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f94b40a2350 0x7f94b40a4810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f94c8003f90 tx=0x7f94c8004a40 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.300+0000 7f94cea00640 1 -- 172.21.15.49:0/912887748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 msgr2=0x5634b7209840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.300+0000 7f94cea00640 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b7209840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.300+0000 7f94cea00640 1 -- 172.21.15.49:0/912887748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 msgr2=0x5634b720a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.300+0000 7f94cea00640 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b720a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.300+0000 7f94cea00640 1 -- 172.21.15.49:0/912887748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634b70f1ad0 con 0x5634b7213300 2024-02-26T06:38:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.300+0000 7f94cf400640 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b720a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.301+0000 7f94cea00640 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7213300 0x5634b72156f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f94bc012750 tx=0x7f94bc00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da37006739277e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.302+0000 7f94bbe00640 1 -- 172.21.15.49:0/912887748 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc004030 con 0x5634b7213300 2024-02-26T06:38:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.302+0000 7f94bbe00640 1 -- 172.21.15.49:0/912887748 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94bc005020 con 0x5634b7213300 2024-02-26T06:38:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.302+0000 7f94bbe00640 1 -- 172.21.15.49:0/912887748 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc004b20 con 0x5634b7213300 2024-02-26T06:38:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.302+0000 7f94d8e39740 1 -- 172.21.15.49:0/912887748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7213300 msgr2=0x5634b72156f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.302+0000 7f94d8e39740 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7213300 0x5634b72156f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f94bc012750 tx=0x7f94bc00b390 comp rx=0 tx=0).stop 2024-02-26T06:38:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.303+0000 7f94d8e39740 1 -- 172.21.15.49:0/912887748 shutdown_connections 2024-02-26T06:38:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.303+0000 7f94d8e39740 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7213300 0x5634b72156f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.303+0000 7f94d8e39740 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b720a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.303+0000 7f94d8e39740 1 --2- 172.21.15.49:0/912887748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b7209840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.303+0000 7f94d8e39740 1 -- 172.21.15.49:0/912887748 >> 172.21.15.49:0/912887748 conn(0x5634b71d1480 msgr2=0x5634b7204a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.303+0000 7f94d8e39740 1 -- 172.21.15.49:0/912887748 shutdown_connections 2024-02-26T06:38:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.303+0000 7f94d8e39740 1 -- 172.21.15.49:0/912887748 wait complete. 2024-02-26T06:38:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.304+0000 7f94d8e39740 1 -- start start 2024-02-26T06:38:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.304+0000 7f94d8e39740 1 Processor -- start 2024-02-26T06:38:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7209d80 0x5634b72103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b7210920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 0x5634b7210e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94d8e39740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b71705a0 con 0x5634b7213300 2024-02-26T06:38:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94d8e39740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b7115b20 con 0x5634b7209d80 2024-02-26T06:38:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94d8e39740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b71157c0 con 0x5634b720aaf0 2024-02-26T06:38:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94d8e39740 10 client.0 _ll_register_callbacks cb 0x5634b7129520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:38:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94cf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b7210920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94cf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b7210920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49422/0 (socket says 172.21.15.49:49422) 2024-02-26T06:38:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94cf400640 1 -- 172.21.15.49:0/429138060 learned_addr learned my addr 172.21.15.49:0/429138060 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.305+0000 7f94cea00640 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7209d80 0x5634b72103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94cea00640 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 0x5634b7210e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94cea00640 1 -- 172.21.15.49:0/429138060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 msgr2=0x5634b7210920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94cea00640 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b7210920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94cea00640 1 -- 172.21.15.49:0/429138060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 msgr2=0x5634b7210e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94cea00640 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 0x5634b7210e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94cea00640 1 -- 172.21.15.49:0/429138060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc01a040 con 0x5634b7209d80 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94cf400640 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b7210920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94cea00640 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7209d80 0x5634b72103e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f94bc001eb0 tx=0x7f94bc012510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca7056c200e23d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.306+0000 7f94b8c00640 1 -- 172.21.15.49:0/429138060 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00bc40 con 0x5634b7209d80 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.307+0000 7f94d8e39740 1 -- 172.21.15.49:0/429138060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634b7142ca0 con 0x5634b7209d80 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.307+0000 7f94b8c00640 1 -- 172.21.15.49:0/429138060 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f94bc000e40 con 0x5634b7209d80 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.307+0000 7f94b8c00640 1 -- 172.21.15.49:0/429138060 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94bc004250 con 0x5634b7209d80 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.307+0000 7f94d8e39740 10 client.27722 fetch_fsmap learned FSMap version 52 2024-02-26T06:38:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.307+0000 7f94d8e39740 1 -- 172.21.15.49:0/429138060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5634b716ec20 con 0x5634b7209d80 2024-02-26T06:38:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.307+0000 7f94b8c00640 1 -- 172.21.15.49:0/429138060 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5634b7142ca0 con 0x5634b7209d80 2024-02-26T06:38:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.307+0000 7f94b8c00640 1 -- 172.21.15.49:0/429138060 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f94bc009810 con 0x5634b7209d80 2024-02-26T06:38:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.308+0000 7f94d8e39740 10 client.27722 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:38:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.308+0000 7f94d8e39740 10 client.27722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:38:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.308+0000 7f94d8e39740 20 client.27722 populate_metadata read hostname 'smithi049' 2024-02-26T06:38:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.308+0000 7f94d8e39740 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5634b7223210 0x5634b7225630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.308+0000 7f94d8e39740 4 client.27722 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:38:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.308+0000 7f94d8e39740 1 -- 172.21.15.49:0/429138060 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634b7166cb0 con 0x5634b7223210 2024-02-26T06:38:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.308+0000 7f94cf400640 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5634b7223210 0x5634b7225630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.310+0000 7f94cf400640 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5634b7223210 0x5634b7225630 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c22da6cd7db45cc server_cookie=1caad4f5b45bd02b in_seq=0 out_seq=0 2024-02-26T06:38:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.310+0000 7f94b8c00640 10 client.27722 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.312+0000 7f94b8c00640 1 -- 172.21.15.49:0/429138060 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634b7166cb0 con 0x5634b7223210 2024-02-26T06:38:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.312+0000 7f94b8c00640 10 client.27722 handle_command_reply: tid=0 2024-02-26T06:38:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.312+0000 7f94d8e39740 1 client.27722 shutdown 2024-02-26T06:38:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94d8e39740 1 -- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7209d80 msgr2=0x5634b72103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94d8e39740 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7209d80 0x5634b72103e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f94bc001eb0 tx=0x7f94bc012510 comp rx=0 tx=0).stop 2024-02-26T06:38:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94d8e39740 1 -- 172.21.15.49:0/429138060 shutdown_connections 2024-02-26T06:38:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94d8e39740 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5634b7223210 0x5634b7225630 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94d8e39740 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 0x5634b7210e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94d8e39740 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b7210920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94d8e39740 1 --2- 172.21.15.49:0/429138060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7209d80 0x5634b72103e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94b8c00640 0 client.27722 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.315+0000 7f94d8e39740 1 -- 172.21.15.49:0/429138060 >> 172.21.15.49:0/429138060 conn(0x5634b71d1480 msgr2=0x5634b72067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.316+0000 7f94d8e39740 1 -- 172.21.15.49:0/429138060 shutdown_connections 2024-02-26T06:38:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.317+0000 7f94d8e39740 1 -- 172.21.15.49:0/429138060 wait complete. 2024-02-26T06:38:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.317+0000 7f94d8e39740 20 client.27722 trim_cache size 0 max 16384 2024-02-26T06:38:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.363+0000 7f94d8e39740 1 Processor -- start 2024-02-26T06:38:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.364+0000 7f94d8e39740 1 -- start start 2024-02-26T06:38:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.364+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b7212b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.364+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b721a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.364+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7213300 0x5634b721a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.364+0000 7f94d8e39740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b6eb8e60 con 0x5634b7209d80 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.364+0000 7f94d8e39740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b7166cb0 con 0x5634b7213300 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.364+0000 7f94d8e39740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b72113b0 con 0x5634b720aaf0 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b721a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cf400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b721a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49432/0 (socket says 172.21.15.49:49432) 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cf400640 1 -- 172.21.15.49:0/2774556615 learned_addr learned my addr 172.21.15.49:0/2774556615 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cea00640 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b7212b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cea00640 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7213300 0x5634b721a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cf400640 1 -- 172.21.15.49:0/2774556615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7213300 msgr2=0x5634b721a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cf400640 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7213300 0x5634b721a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cf400640 1 -- 172.21.15.49:0/2774556615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 msgr2=0x5634b7212b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cf400640 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b7212b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cf400640 1 -- 172.21.15.49:0/2774556615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634b710b500 con 0x5634b720aaf0 2024-02-26T06:38:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.365+0000 7f94cea00640 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b7212b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:38:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.366+0000 7f94cf400640 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b721a120 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f94c0005ab0 tx=0x7f94c0005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=749095d1ae778484 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.368+0000 7f94bb400640 1 -- 172.21.15.49:0/2774556615 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0045040 con 0x5634b720aaf0 2024-02-26T06:38:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.368+0000 7f94bb400640 1 -- 172.21.15.49:0/2774556615 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94c0047040 con 0x5634b720aaf0 2024-02-26T06:38:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.368+0000 7f94bb400640 1 -- 172.21.15.49:0/2774556615 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0005150 con 0x5634b720aaf0 2024-02-26T06:38:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.369+0000 7f94d8e39740 1 -- 172.21.15.49:0/2774556615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 msgr2=0x5634b721a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.369+0000 7f94d8e39740 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b721a120 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f94c0005ab0 tx=0x7f94c0005a20 comp rx=0 tx=0).stop 2024-02-26T06:38:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.369+0000 7f94d8e39740 1 -- 172.21.15.49:0/2774556615 shutdown_connections 2024-02-26T06:38:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.369+0000 7f94d8e39740 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b7213300 0x5634b721a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.369+0000 7f94d8e39740 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b720aaf0 0x5634b721a120 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.369+0000 7f94d8e39740 1 --2- 172.21.15.49:0/2774556615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7209d80 0x5634b7212b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.369+0000 7f94d8e39740 1 -- 172.21.15.49:0/2774556615 >> 172.21.15.49:0/2774556615 conn(0x5634b71d1480 msgr2=0x5634b72045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.370+0000 7f94d8e39740 1 -- 172.21.15.49:0/2774556615 shutdown_connections 2024-02-26T06:38:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.370+0000 7f94d8e39740 1 -- 172.21.15.49:0/2774556615 wait complete. 2024-02-26T06:38:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.371+0000 7f94d8e39740 1 -- start start 2024-02-26T06:38:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.371+0000 7f94d8e39740 1 Processor -- start 2024-02-26T06:38:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7209d80 0x5634b720cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b720d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94d8e39740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 0x5634b720d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94d8e39740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b721aa30 con 0x5634b7213300 2024-02-26T06:38:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94d8e39740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b7279710 con 0x5634b720aaf0 2024-02-26T06:38:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94d8e39740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b7279850 con 0x5634b7209d80 2024-02-26T06:38:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94d8e39740 10 client.0 _ll_register_callbacks cb 0x7f94c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:38:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94cea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7209d80 0x5634b720cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94cea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7209d80 0x5634b720cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:49436/0 (socket says 172.21.15.49:49436) 2024-02-26T06:38:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.374+0000 7f94cea00640 1 -- 172.21.15.49:0/3155474985 learned_addr learned my addr 172.21.15.49:0/3155474985 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.375+0000 7f94cf400640 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b720d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.376+0000 7f94cea00640 1 -- 172.21.15.49:0/3155474985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 msgr2=0x5634b720d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.376+0000 7f94cea00640 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b720d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.376+0000 7f94cea00640 1 -- 172.21.15.49:0/3155474985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 msgr2=0x5634b720d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.376+0000 7f94cea00640 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 0x5634b720d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.376+0000 7f94cea00640 1 -- 172.21.15.49:0/3155474985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c0030070 con 0x5634b7209d80 2024-02-26T06:38:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.376+0000 7f94cea00640 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7209d80 0x5634b720cf40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009bd0 tx=0x7f94bc00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f27d11461825c39 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.376+0000 7f94b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-02-26T06:38:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.377+0000 7f94b8c00640 1 -- 172.21.15.49:0/3155474985 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc004bd0 con 0x5634b7209d80 2024-02-26T06:38:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.377+0000 7f94b8c00640 1 -- 172.21.15.49:0/3155474985 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f94bc000e40 con 0x5634b7209d80 2024-02-26T06:38:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.377+0000 7f94d8e39740 1 -- 172.21.15.49:0/3155474985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634b7195d00 con 0x5634b7209d80 2024-02-26T06:38:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.377+0000 7f94b8c00640 1 -- 172.21.15.49:0/3155474985 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94bc005850 con 0x5634b7209d80 2024-02-26T06:38:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.377+0000 7f94d8e39740 10 client.27673 fetch_fsmap learned FSMap version 52 2024-02-26T06:38:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.377+0000 7f94d8e39740 1 -- 172.21.15.49:0/3155474985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5634b7131290 con 0x5634b7209d80 2024-02-26T06:38:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94b8c00640 1 -- 172.21.15.49:0/3155474985 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5634b7195d00 con 0x5634b7209d80 2024-02-26T06:38:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94b8c00640 1 -- 172.21.15.49:0/3155474985 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f94bc003880 con 0x5634b7209d80 2024-02-26T06:38:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94d8e39740 10 client.27673 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:38:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94d8e39740 10 client.27673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:38:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94d8e39740 20 client.27673 populate_metadata read hostname 'smithi049' 2024-02-26T06:38:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94d8e39740 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5634b727ae10 0x5634b727d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94d8e39740 4 client.27673 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:38:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94d8e39740 1 -- 172.21.15.49:0/3155474985 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5634b6e70600 con 0x5634b727ae10 2024-02-26T06:38:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.378+0000 7f94cf400640 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5634b727ae10 0x5634b727d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.379+0000 7f94b8c00640 10 client.27673 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.379+0000 7f94cf400640 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5634b727ae10 0x5634b727d230 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0ce1c7c945baef server_cookie=95c2e59985d65008 in_seq=0 out_seq=0 2024-02-26T06:38:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94b8c00640 1 -- 172.21.15.49:0/3155474985 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5634b6e70600 con 0x5634b727ae10 2024-02-26T06:38:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94b8c00640 10 client.27673 handle_command_reply: tid=0 2024-02-26T06:38:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 client.27673 shutdown 2024-02-26T06:38:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 -- 172.21.15.49:0/3155474985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7209d80 msgr2=0x5634b720cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7209d80 0x5634b720cf40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009bd0 tx=0x7f94bc00bd60 comp rx=0 tx=0).stop 2024-02-26T06:38:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 -- 172.21.15.49:0/3155474985 shutdown_connections 2024-02-26T06:38:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x5634b727ae10 0x5634b727d230 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5634b720aaf0 0x5634b720d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94b8c00640 0 client.27673 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5634b7213300 0x5634b720d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 --2- 172.21.15.49:0/3155474985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634b7209d80 0x5634b720cf40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.391+0000 7f94d8e39740 1 -- 172.21.15.49:0/3155474985 >> 172.21.15.49:0/3155474985 conn(0x5634b71d1480 msgr2=0x5634b720ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.392+0000 7f94d8e39740 1 -- 172.21.15.49:0/3155474985 shutdown_connections 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.392+0000 7f94d8e39740 1 -- 172.21.15.49:0/3155474985 wait complete. 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.392+0000 7f94d8e39740 20 client.27673 trim_cache size 0 max 16384 2024-02-26T06:38:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:38:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:38:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:38:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:38:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.392+0000 7f94d6000640 1 -- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f94b40a2350 msgr2=0x7f94b40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.392+0000 7f94d6000640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f94b40a2350 0x7f94b40a4810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f94c8003f90 tx=0x7f94c8004a40 comp rx=0 tx=0).stop 2024-02-26T06:38:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.392+0000 7f94d6000640 1 -- 172.21.15.49:0/1966814905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01997a0 msgr2=0x7f94d0199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.392+0000 7f94d6000640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01997a0 0x7f94d0199c50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f94c800efd0 tx=0x7f94c800c6a0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.393+0000 7f94d6000640 1 -- 172.21.15.49:0/1966814905 shutdown_connections 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.393+0000 7f94d6000640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f94b40a2350 0x7f94b40a4810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.393+0000 7f94d6000640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94d01997a0 0x7f94d0199c50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.393+0000 7f94d6000640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94d0108690 0x7f94d019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.393+0000 7f94d6000640 1 --2- 172.21.15.49:0/1966814905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94d01035c0 0x7f94d019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.393+0000 7f94d6000640 1 -- 172.21.15.49:0/1966814905 >> 172.21.15.49:0/1966814905 conn(0x7f94d00fee40 msgr2=0x7f94d0101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.393+0000 7f94d6000640 1 -- 172.21.15.49:0/1966814905 shutdown_connections 2024-02-26T06:38:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:22.393+0000 7f94d6000640 1 -- 172.21.15.49:0/1966814905 wait complete. 2024-02-26T06:38:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:23 smithi080 ceph-mon[36206]: pgmap v470: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-02-26T06:38:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:23 smithi115 ceph-mon[36426]: pgmap v470: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-02-26T06:38:23.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:23 smithi049 ceph-mon[31696]: pgmap v470: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-02-26T06:38:25.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:25 smithi080 ceph-mon[36206]: pgmap v471: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-02-26T06:38:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:25 smithi115 ceph-mon[36426]: pgmap v471: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-02-26T06:38:25.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:25 smithi049 ceph-mon[31696]: pgmap v471: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-02-26T06:38:26.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 875.0 s, 1408.0 tps, lat 70.914 ms stddev 84.564 2024-02-26T06:38:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:27 smithi115 ceph-mon[36426]: pgmap v472: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 865 op/s 2024-02-26T06:38:27.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:27 smithi049 ceph-mon[31696]: pgmap v472: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 865 op/s 2024-02-26T06:38:27.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:27 smithi080 ceph-mon[36206]: pgmap v472: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 865 op/s 2024-02-26T06:38:29.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:29 smithi049 ceph-mon[31696]: pgmap v473: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 10 MiB/s wr, 694 op/s 2024-02-26T06:38:29.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:29 smithi080 ceph-mon[36206]: pgmap v473: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 10 MiB/s wr, 694 op/s 2024-02-26T06:38:29.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:29 smithi115 ceph-mon[36426]: pgmap v473: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 10 MiB/s wr, 694 op/s 2024-02-26T06:38:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 880.0 s, 1281.8 tps, lat 78.354 ms stddev 97.204 2024-02-26T06:38:31.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:31 smithi049 ceph-mon[31696]: pgmap v474: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.7 MiB/s wr, 557 op/s 2024-02-26T06:38:31.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:31 smithi080 ceph-mon[36206]: pgmap v474: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.7 MiB/s wr, 557 op/s 2024-02-26T06:38:31.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:31 smithi115 ceph-mon[36426]: pgmap v474: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.7 MiB/s wr, 557 op/s 2024-02-26T06:38:33.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:33 smithi049 ceph-mon[31696]: pgmap v475: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.8 MiB/s wr, 654 op/s 2024-02-26T06:38:33.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:33 smithi080 ceph-mon[36206]: pgmap v475: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.8 MiB/s wr, 654 op/s 2024-02-26T06:38:33.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:33 smithi115 ceph-mon[36426]: pgmap v475: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.8 MiB/s wr, 654 op/s 2024-02-26T06:38:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.572+0000 7f103cc00640 1 -- 172.21.15.49:0/1185856617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10381013a0 msgr2=0x7f1038101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.572+0000 7f103cc00640 1 --2- 172.21.15.49:0/1185856617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10381013a0 0x7f1038101780 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1028009930 tx=0x7f102802bc20 comp rx=0 tx=0).stop 2024-02-26T06:38:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.573+0000 7f103cc00640 1 -- 172.21.15.49:0/1185856617 shutdown_connections 2024-02-26T06:38:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.573+0000 7f103cc00640 1 --2- 172.21.15.49:0/1185856617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1038106470 0x7f103810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.573+0000 7f103cc00640 1 --2- 172.21.15.49:0/1185856617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1038101d50 0x7f1038105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.573+0000 7f103cc00640 1 --2- 172.21.15.49:0/1185856617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10381013a0 0x7f1038101780 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.573+0000 7f103cc00640 1 -- 172.21.15.49:0/1185856617 >> 172.21.15.49:0/1185856617 conn(0x7f10380fcc40 msgr2=0x7f10380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.573+0000 7f103cc00640 1 -- 172.21.15.49:0/1185856617 shutdown_connections 2024-02-26T06:38:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.573+0000 7f103cc00640 1 -- 172.21.15.49:0/1185856617 wait complete. 2024-02-26T06:38:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.575+0000 7f103cc00640 1 Processor -- start 2024-02-26T06:38:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.575+0000 7f103cc00640 1 -- start start 2024-02-26T06:38:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.575+0000 7f103cc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10381013a0 0x7f103810a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.575+0000 7f103cc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1038106470 0x7f103810a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.575+0000 7f103cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f103810ac10 0x7f10381a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.575+0000 7f103cc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103810e320 con 0x7f103810ac10 2024-02-26T06:38:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f103cc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10381aa520 con 0x7f10381013a0 2024-02-26T06:38:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f103cc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10381aa660 con 0x7f1038106470 2024-02-26T06:38:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1038106470 0x7f103810a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10381013a0 0x7f103810a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10381013a0 0x7f103810a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33376/0 (socket says 172.21.15.49:33376) 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035e00640 1 -- 172.21.15.49:0/3504189052 learned_addr learned my addr 172.21.15.49:0/3504189052 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1036800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f103810ac10 0x7f10381a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035e00640 1 -- 172.21.15.49:0/3504189052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1038106470 msgr2=0x7f103810a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035e00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1038106470 0x7f103810a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035e00640 1 -- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f103810ac10 msgr2=0x7f10381a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035e00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f103810ac10 0x7f10381a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035e00640 1 -- 172.21.15.49:0/3504189052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028009560 con 0x7f10381013a0 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.576+0000 7f1035400640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1038106470 0x7f103810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.577+0000 7f1035e00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10381013a0 0x7f103810a190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1028005ab0 tx=0x7f1028004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.577+0000 7f102ea00640 1 -- 172.21.15.49:0/3504189052 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102802dea0 con 0x7f10381013a0 2024-02-26T06:38:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.577+0000 7f102ea00640 1 -- 172.21.15.49:0/3504189052 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1028004440 con 0x7f10381013a0 2024-02-26T06:38:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.577+0000 7f103cc00640 1 -- 172.21.15.49:0/3504189052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10381aa860 con 0x7f10381013a0 2024-02-26T06:38:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.577+0000 7f102ea00640 1 -- 172.21.15.49:0/3504189052 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028003c90 con 0x7f10381013a0 2024-02-26T06:38:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.577+0000 7f103cc00640 1 -- 172.21.15.49:0/3504189052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10381aadc0 con 0x7f10381013a0 2024-02-26T06:38:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.580+0000 7f103fc02740 1 Processor -- start 2024-02-26T06:38:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.581+0000 7f102ea00640 1 -- 172.21.15.49:0/3504189052 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1028004df0 con 0x7f10381013a0 2024-02-26T06:38:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.581+0000 7f103fc02740 1 -- start start 2024-02-26T06:38:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.581+0000 7f103fc02740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786eb9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.581+0000 7f103fc02740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786eba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.581+0000 7f103fc02740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ec56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.581+0000 7f103fc02740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786e755f0 con 0x55b786ec3300 2024-02-26T06:38:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.581+0000 7f103fc02740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786e27890 con 0x55b786eb9d80 2024-02-26T06:38:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.581+0000 7f103fc02740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786e24dc0 con 0x55b786ebaaf0 2024-02-26T06:38:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.582+0000 7f1036800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786eb9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.582+0000 7f1036800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786eb9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48310/0 (socket says 172.21.15.49:48310) 2024-02-26T06:38:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.582+0000 7f1036800640 1 -- 172.21.15.49:0/2899963490 learned_addr learned my addr 172.21.15.49:0/2899963490 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.583+0000 7f1035e00640 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786eba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.583+0000 7f1035400640 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ec56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.583+0000 7f1036800640 1 -- 172.21.15.49:0/2899963490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 msgr2=0x55b786eba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.583+0000 7f1036800640 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786eba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.583+0000 7f1036800640 1 -- 172.21.15.49:0/2899963490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 msgr2=0x55b786ec56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.583+0000 7f1036800640 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ec56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.584+0000 7f1036800640 1 -- 172.21.15.49:0/2899963490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b786da1ad0 con 0x55b786ebaaf0 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.584+0000 7f1035400640 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ec56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.584+0000 7f102ea00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f101c0a2350 0x7f101c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.584+0000 7f1036800640 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786eb9840 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1030009f70 tx=0x7f1030010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d20ea6303763236 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.584+0000 7f1023000640 1 -- 172.21.15.49:0/2899963490 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030004d20 con 0x55b786ebaaf0 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.584+0000 7f102ea00640 1 -- 172.21.15.49:0/3504189052 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f10280eff70 con 0x7f10381013a0 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.585+0000 7f1023000640 1 -- 172.21.15.49:0/2899963490 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1030004e80 con 0x55b786ebaaf0 2024-02-26T06:38:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.585+0000 7f1023000640 1 -- 172.21.15.49:0/2899963490 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030012990 con 0x55b786ebaaf0 2024-02-26T06:38:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.585+0000 7f1035e00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f101c0a2350 0x7f101c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.586+0000 7f103fc02740 1 -- 172.21.15.49:0/2899963490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 msgr2=0x55b786eb9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.586+0000 7f103fc02740 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786eb9840 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1030009f70 tx=0x7f1030010800 comp rx=0 tx=0).stop 2024-02-26T06:38:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.586+0000 7f103fc02740 1 -- 172.21.15.49:0/2899963490 shutdown_connections 2024-02-26T06:38:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.586+0000 7f103fc02740 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786eba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.586+0000 7f103fc02740 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ec56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.586+0000 7f103fc02740 1 --2- 172.21.15.49:0/2899963490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786eb9840 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.586+0000 7f103fc02740 1 -- 172.21.15.49:0/2899963490 >> 172.21.15.49:0/2899963490 conn(0x55b786e81480 msgr2=0x55b786eb4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.587+0000 7f103fc02740 1 -- 172.21.15.49:0/2899963490 shutdown_connections 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.587+0000 7f1035e00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f101c0a2350 0x7f101c0a4810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f10280ed860 tx=0x7f10280a5f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.587+0000 7f103fc02740 1 -- 172.21.15.49:0/2899963490 wait complete. 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.587+0000 7f103fc02740 1 -- start start 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.587+0000 7f103fc02740 1 Processor -- start 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f103fc02740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786eb9d80 0x55b786ec0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f103fc02740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786ec08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f103fc02740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ec3300 0x55b786ec0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f103fc02740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786e205a0 con 0x55b786eb9d80 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f103fc02740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786dc5b20 con 0x55b786ebaaf0 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f103fc02740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786dc57c0 con 0x55b786ec3300 2024-02-26T06:38:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f103fc02740 10 client.0 _ll_register_callbacks cb 0x55b786dd9520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:38:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f1035400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786ec08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f1035400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786ec08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33394/0 (socket says 172.21.15.49:33394) 2024-02-26T06:38:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f1035400640 1 -- 172.21.15.49:0/4088002435 learned_addr learned my addr 172.21.15.49:0/4088002435 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.588+0000 7f1036800640 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786eb9d80 0x55b786ec0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1036800640 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ec3300 0x55b786ec0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1035400640 1 -- 172.21.15.49:0/4088002435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ec3300 msgr2=0x55b786ec0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1035400640 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ec3300 0x55b786ec0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1035400640 1 -- 172.21.15.49:0/4088002435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786eb9d80 msgr2=0x55b786ec0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1035400640 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786eb9d80 0x55b786ec0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1035400640 1 -- 172.21.15.49:0/4088002435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1030009bd0 con 0x55b786ebaaf0 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1036800640 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ec3300 0x55b786ec0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1035400640 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786ec08d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f10240119b0 tx=0x7f1024011e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df07b66c6a97b620 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1017400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.589+0000 7f1017400640 1 -- 172.21.15.49:0/4088002435 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102400fed0 con 0x55b786ebaaf0 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.590+0000 7f103fc02740 1 -- 172.21.15.49:0/4088002435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b786df2ca0 con 0x55b786ebaaf0 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.590+0000 7f1017400640 1 -- 172.21.15.49:0/4088002435 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f1024012450 con 0x55b786ebaaf0 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.590+0000 7f1017400640 1 -- 172.21.15.49:0/4088002435 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1024012610 con 0x55b786ebaaf0 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.590+0000 7f103fc02740 10 client.27740 fetch_fsmap learned FSMap version 52 2024-02-26T06:38:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.590+0000 7f103fc02740 1 -- 172.21.15.49:0/4088002435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55b786e1ec20 con 0x55b786ebaaf0 2024-02-26T06:38:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.591+0000 7f1017400640 1 -- 172.21.15.49:0/4088002435 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1024027020 con 0x55b786ebaaf0 2024-02-26T06:38:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.591+0000 7f1017400640 1 -- 172.21.15.49:0/4088002435 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f10240229e0 con 0x55b786ebaaf0 2024-02-26T06:38:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.591+0000 7f103fc02740 10 client.27740 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:38:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.591+0000 7f103fc02740 10 client.27740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:38:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.591+0000 7f103fc02740 20 client.27740 populate_metadata read hostname 'smithi049' 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.591+0000 7f103fc02740 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b786ed33b0 0x55b786ed57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.591+0000 7f103fc02740 4 client.27740 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.592+0000 7f103fc02740 1 -- 172.21.15.49:0/4088002435 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b786e16cb0 con 0x55b786ed33b0 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.592+0000 7f1036800640 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b786ed33b0 0x55b786ed57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.592+0000 7f1036800640 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b786ed33b0 0x55b786ed57d0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7d9fd85b1ebb2e server_cookie=95f28a01f4bd4fee in_seq=0 out_seq=0 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.592+0000 7f1017400640 10 client.27740 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.593+0000 7f1017400640 1 -- 172.21.15.49:0/4088002435 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b786e16cb0 con 0x55b786ed33b0 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.593+0000 7f1017400640 10 client.27740 handle_command_reply: tid=0 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.593+0000 7f103fc02740 1 client.27740 shutdown 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 -- 172.21.15.49:0/4088002435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 msgr2=0x55b786ec08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786ec08d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f10240119b0 tx=0x7f1024011e70 comp rx=0 tx=0).stop 2024-02-26T06:38:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 -- 172.21.15.49:0/4088002435 shutdown_connections 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b786ed33b0 0x55b786ed57d0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786ec08d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f1017400640 0 client.27740 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ec3300 0x55b786ec0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 --2- 172.21.15.49:0/4088002435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786eb9d80 0x55b786ec0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 -- 172.21.15.49:0/4088002435 >> 172.21.15.49:0/4088002435 conn(0x55b786e81480 msgr2=0x55b786eb67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 -- 172.21.15.49:0/4088002435 shutdown_connections 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 1 -- 172.21.15.49:0/4088002435 wait complete. 2024-02-26T06:38:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.594+0000 7f103fc02740 20 client.27740 trim_cache size 0 max 16384 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.618+0000 7f103fc02740 1 Processor -- start 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.618+0000 7f103fc02740 1 -- start start 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.618+0000 7f103fc02740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786ec08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.618+0000 7f103fc02740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786ec0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.618+0000 7f103fc02740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ebec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.618+0000 7f103fc02740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786b68e60 con 0x55b786ec3300 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.618+0000 7f103fc02740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786ec16e0 con 0x55b786eb9d80 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.618+0000 7f103fc02740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786eb5a70 con 0x55b786ebaaf0 2024-02-26T06:38:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1036800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786ec08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786ec0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786ec0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48334/0 (socket says 172.21.15.49:48334) 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1036800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786ec08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33396/0 (socket says 172.21.15.49:33396) 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 -- 172.21.15.49:0/1595619281 learned_addr learned my addr 172.21.15.49:0/1595619281 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1036800640 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ebec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 -- 172.21.15.49:0/1595619281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 msgr2=0x55b786ec08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786ec08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 -- 172.21.15.49:0/1595619281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 msgr2=0x55b786ebec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ebec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 -- 172.21.15.49:0/1595619281 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b786dbb500 con 0x55b786ebaaf0 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1036800640 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ebec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.619+0000 7f1035400640 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786ec0e20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1024022dc0 tx=0x7f102400aa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e1d1242f4d9c70 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.620+0000 7f1022600640 1 -- 172.21.15.49:0/1595619281 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024005d60 con 0x55b786ebaaf0 2024-02-26T06:38:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.620+0000 7f1022600640 1 -- 172.21.15.49:0/1595619281 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1024027040 con 0x55b786ebaaf0 2024-02-26T06:38:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.620+0000 7f1022600640 1 -- 172.21.15.49:0/1595619281 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024022630 con 0x55b786ebaaf0 2024-02-26T06:38:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 -- 172.21.15.49:0/1595619281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 msgr2=0x55b786ec0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786ec0e20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1024022dc0 tx=0x7f102400aa70 comp rx=0 tx=0).stop 2024-02-26T06:38:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 -- 172.21.15.49:0/1595619281 shutdown_connections 2024-02-26T06:38:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ebec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786ebaaf0 0x55b786ec0e20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 --2- 172.21.15.49:0/1595619281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786eb9d80 0x55b786ec08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 -- 172.21.15.49:0/1595619281 >> 172.21.15.49:0/1595619281 conn(0x55b786e81480 msgr2=0x55b786eb45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 -- 172.21.15.49:0/1595619281 shutdown_connections 2024-02-26T06:38:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.621+0000 7f103fc02740 1 -- 172.21.15.49:0/1595619281 wait complete. 2024-02-26T06:38:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.622+0000 7f103fc02740 1 -- start start 2024-02-26T06:38:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.623+0000 7f103fc02740 1 Processor -- start 2024-02-26T06:38:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f103fc02740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786eb9d80 0x55b786eca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f103fc02740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786eca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f103fc02740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ec86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f103fc02740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786ec8db0 con 0x55b786ec3300 2024-02-26T06:38:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f103fc02740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786ec8f20 con 0x55b786ebaaf0 2024-02-26T06:38:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f103fc02740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b786ec90e0 con 0x55b786eb9d80 2024-02-26T06:38:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f103fc02740 10 client.0 _ll_register_callbacks cb 0x7f102400a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f1035400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786eca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f1035400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786eca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:33408/0 (socket says 172.21.15.49:33408) 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.624+0000 7f1035400640 1 -- 172.21.15.49:0/2882218883 learned_addr learned my addr 172.21.15.49:0/2882218883 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1036800640 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786eb9d80 0x55b786eca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1035400640 1 -- 172.21.15.49:0/2882218883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786eb9d80 msgr2=0x55b786eca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1035400640 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786eb9d80 0x55b786eca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1035400640 1 -- 172.21.15.49:0/2882218883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 msgr2=0x55b786ec86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1035400640 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ec86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1035400640 1 -- 172.21.15.49:0/2882218883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b786e1ec20 con 0x55b786ebaaf0 2024-02-26T06:38:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1036800640 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786eb9d80 0x55b786eca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1035400640 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786eca680 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f102402e040 tx=0x7f102400ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a1408f98aaad2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1017400640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.625+0000 7f1017400640 1 -- 172.21.15.49:0/2882218883 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024004430 con 0x55b786ebaaf0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 1 -- 172.21.15.49:0/2882218883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b786e45d00 con 0x55b786ebaaf0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f1017400640 1 -- 172.21.15.49:0/2882218883 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f1024028080 con 0x55b786ebaaf0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f1017400640 1 -- 172.21.15.49:0/2882218883 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10240223d0 con 0x55b786ebaaf0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 10 client.27746 fetch_fsmap learned FSMap version 52 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 1 -- 172.21.15.49:0/2882218883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55b786de1290 con 0x55b786ebaaf0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f1017400640 1 -- 172.21.15.49:0/2882218883 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1024042020 con 0x55b786ebaaf0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f1017400640 1 -- 172.21.15.49:0/2882218883 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1024003a00 con 0x55b786ebaaf0 2024-02-26T06:38:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 10 client.27746 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:38:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 10 client.27746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:38:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 20 client.27746 populate_metadata read hostname 'smithi049' 2024-02-26T06:38:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b786ed33b0 0x55b786f2c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 4 client.27746 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:38:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.626+0000 7f103fc02740 1 -- 172.21.15.49:0/2882218883 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b786b20600 con 0x55b786ed33b0 2024-02-26T06:38:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.627+0000 7f1036800640 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b786ed33b0 0x55b786f2c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.627+0000 7f1017400640 10 client.27746 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.627+0000 7f1036800640 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b786ed33b0 0x55b786f2c9c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6fe53368a1cc0be server_cookie=fa8e4364399fbbb2 in_seq=0 out_seq=0 2024-02-26T06:38:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.636+0000 7f1017400640 1 -- 172.21.15.49:0/2882218883 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b786b20600 con 0x55b786ed33b0 2024-02-26T06:38:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.636+0000 7f1017400640 10 client.27746 handle_command_reply: tid=0 2024-02-26T06:38:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 client.27746 shutdown 2024-02-26T06:38:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 -- 172.21.15.49:0/2882218883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 msgr2=0x55b786eca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786eca680 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f102402e040 tx=0x7f102400ff00 comp rx=0 tx=0).stop 2024-02-26T06:38:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 -- 172.21.15.49:0/2882218883 shutdown_connections 2024-02-26T06:38:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55b786ed33b0 0x55b786f2c9c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b786ebaaf0 0x55b786eca680 secure :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f102402e040 tx=0x7f102400ff00 comp rx=0 tx=0).stop 2024-02-26T06:38:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b786ec3300 0x55b786ec86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 --2- 172.21.15.49:0/2882218883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b786eb9d80 0x55b786eca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.637+0000 7f103fc02740 1 -- 172.21.15.49:0/2882218883 >> 172.21.15.49:0/2882218883 conn(0x55b786e81480 msgr2=0x55b786ebebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.638+0000 7f1017400640 0 client.27746 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.639+0000 7f103fc02740 1 -- 172.21.15.49:0/2882218883 shutdown_connections 2024-02-26T06:38:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.639+0000 7f103fc02740 1 -- 172.21.15.49:0/2882218883 wait complete. 2024-02-26T06:38:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.639+0000 7f103fc02740 20 client.27746 trim_cache size 0 max 16384 2024-02-26T06:38:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:38:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:38:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:38:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:38:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 -- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f101c0a2350 msgr2=0x7f101c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f101c0a2350 0x7f101c0a4810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f10280ed860 tx=0x7f10280a5f90 comp rx=0 tx=0).stop 2024-02-26T06:38:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 -- 172.21.15.49:0/3504189052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10381013a0 msgr2=0x7f103810a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10381013a0 0x7f103810a190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1028005ab0 tx=0x7f1028004040 comp rx=0 tx=0).stop 2024-02-26T06:38:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 -- 172.21.15.49:0/3504189052 shutdown_connections 2024-02-26T06:38:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f101c0a2350 0x7f101c0a4810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f103810ac10 0x7f10381a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1038106470 0x7f103810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 --2- 172.21.15.49:0/3504189052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10381013a0 0x7f103810a190 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.640+0000 7f103cc00640 1 -- 172.21.15.49:0/3504189052 >> 172.21.15.49:0/3504189052 conn(0x7f10380fcc40 msgr2=0x7f10380fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.641+0000 7f103cc00640 1 -- 172.21.15.49:0/3504189052 shutdown_connections 2024-02-26T06:38:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:35.641+0000 7f103cc00640 1 -- 172.21.15.49:0/3504189052 wait complete. 2024-02-26T06:38:35.675 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:35 smithi049 ceph-mon[31696]: pgmap v476: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 9.8 MiB/s wr, 607 op/s 2024-02-26T06:38:35.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:35 smithi080 ceph-mon[36206]: pgmap v476: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 9.8 MiB/s wr, 607 op/s 2024-02-26T06:38:35.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:35 smithi115 ceph-mon[36426]: pgmap v476: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 9.8 MiB/s wr, 607 op/s 2024-02-26T06:38:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 885.0 s, 1468.0 tps, lat 67.717 ms stddev 79.518 2024-02-26T06:38:37.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:37 smithi049 ceph-mon[31696]: pgmap v477: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 696 op/s 2024-02-26T06:38:37.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:37 smithi080 ceph-mon[36206]: pgmap v477: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 696 op/s 2024-02-26T06:38:37.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:37 smithi115 ceph-mon[36426]: pgmap v477: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 696 op/s 2024-02-26T06:38:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:39 smithi049 ceph-mon[31696]: pgmap v478: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.3 MiB/s wr, 570 op/s 2024-02-26T06:38:39.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:39 smithi080 ceph-mon[36206]: pgmap v478: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.3 MiB/s wr, 570 op/s 2024-02-26T06:38:39.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:39 smithi115 ceph-mon[36426]: pgmap v478: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.3 MiB/s wr, 570 op/s 2024-02-26T06:38:41.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 890.0 s, 1084.2 tps, lat 92.122 ms stddev 145.388 2024-02-26T06:38:41.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:41 smithi049 ceph-mon[31696]: pgmap v479: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 628 op/s 2024-02-26T06:38:41.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:41 smithi080 ceph-mon[36206]: pgmap v479: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 628 op/s 2024-02-26T06:38:41.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:41 smithi115 ceph-mon[36426]: pgmap v479: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 628 op/s 2024-02-26T06:38:43.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:43 smithi049 ceph-mon[31696]: pgmap v480: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.33k op/s 2024-02-26T06:38:43.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:43 smithi080 ceph-mon[36206]: pgmap v480: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.33k op/s 2024-02-26T06:38:43.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:43 smithi115 ceph-mon[36426]: pgmap v480: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.33k op/s 2024-02-26T06:38:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:45 smithi049 ceph-mon[31696]: pgmap v481: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2024-02-26T06:38:45.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:45 smithi080 ceph-mon[36206]: pgmap v481: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2024-02-26T06:38:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:45 smithi115 ceph-mon[36426]: pgmap v481: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2024-02-26T06:38:46.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 895.0 s, 1402.8 tps, lat 71.846 ms stddev 89.198 2024-02-26T06:38:46.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:46 smithi080 ceph-mon[36206]: pgmap v482: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 20 MiB/s wr, 1.59k op/s 2024-02-26T06:38:46.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:46 smithi115 ceph-mon[36426]: pgmap v482: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 20 MiB/s wr, 1.59k op/s 2024-02-26T06:38:46.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:46 smithi049 ceph-mon[31696]: pgmap v482: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 20 MiB/s wr, 1.59k op/s 2024-02-26T06:38:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.817+0000 7f4b8ee00640 1 -- 172.21.15.49:0/365255686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88108cf0 msgr2=0x7f4b88113d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.817+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/365255686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88108cf0 0x7f4b88113d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009890 tx=0x7f4b7402bb70 comp rx=0 tx=0).stop 2024-02-26T06:38:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.820+0000 7f4b8ee00640 1 -- 172.21.15.49:0/365255686 shutdown_connections 2024-02-26T06:38:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.820+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/365255686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b881142d0 0x7f4b881166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.820+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/365255686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88108cf0 0x7f4b88113d90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.820+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/365255686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b881083d0 0x7f4b881087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.820+0000 7f4b8ee00640 1 -- 172.21.15.49:0/365255686 >> 172.21.15.49:0/365255686 conn(0x7f4b8806c220 msgr2=0x7f4b8806c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.820+0000 7f4b8ee00640 1 -- 172.21.15.49:0/365255686 shutdown_connections 2024-02-26T06:38:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.820+0000 7f4b8ee00640 1 -- 172.21.15.49:0/365255686 wait complete. 2024-02-26T06:38:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.823+0000 7f4b8ee00640 1 Processor -- start 2024-02-26T06:38:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.824+0000 7f4b8ee00640 1 -- start start 2024-02-26T06:38:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.824+0000 7f4b8ee00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b881083d0 0x7f4b88109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.824+0000 7f4b8ee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b88108cf0 0x7f4b8810a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.824+0000 7f4b8ee00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b881142d0 0x7f4b8810a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.824+0000 7f4b8ee00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8810bb40 con 0x7f4b881142d0 2024-02-26T06:38:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.824+0000 7f4b8ee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b881b2810 con 0x7f4b88108cf0 2024-02-26T06:38:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.824+0000 7f4b8ee00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b881b2950 con 0x7f4b881083d0 2024-02-26T06:38:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.825+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b881083d0 0x7f4b88109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.825+0000 7f4b87400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b88108cf0 0x7f4b8810a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.825+0000 7f4b87400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b88108cf0 0x7f4b8810a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:48754/0 (socket says 172.21.15.49:48754) 2024-02-26T06:38:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.825+0000 7f4b87400640 1 -- 172.21.15.49:0/3227990027 learned_addr learned my addr 172.21.15.49:0/3227990027 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.825+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b881083d0 0x7f4b88109bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:50976/0 (socket says 172.21.15.49:50976) 2024-02-26T06:38:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.825+0000 7f4b8ce00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b881142d0 0x7f4b8810a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.826+0000 7f4b87e00640 1 -- 172.21.15.49:0/3227990027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b88108cf0 msgr2=0x7f4b8810a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.826+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b88108cf0 0x7f4b8810a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.826+0000 7f4b87e00640 1 -- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b881142d0 msgr2=0x7f4b8810a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.827+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b881142d0 0x7f4b8810a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.827+0000 7f4b87e00640 1 -- 172.21.15.49:0/3227990027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b74009490 con 0x7f4b881083d0 2024-02-26T06:38:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.827+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b881083d0 0x7f4b88109bf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800c970 tx=0x7f4b7800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.827+0000 7f4b84c00640 1 -- 172.21.15.49:0/3227990027 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78007bb0 con 0x7f4b881083d0 2024-02-26T06:38:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.828+0000 7f4b84c00640 1 -- 172.21.15.49:0/3227990027 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b78007d10 con 0x7f4b881083d0 2024-02-26T06:38:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.828+0000 7f4b84c00640 1 -- 172.21.15.49:0/3227990027 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78018760 con 0x7f4b881083d0 2024-02-26T06:38:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.828+0000 7f4b8ee00640 1 -- 172.21.15.49:0/3227990027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b881b2c30 con 0x7f4b881083d0 2024-02-26T06:38:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.828+0000 7f4b8ee00640 1 -- 172.21.15.49:0/3227990027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b881b3190 con 0x7f4b881083d0 2024-02-26T06:38:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.833+0000 7f4b91c67740 1 Processor -- start 2024-02-26T06:38:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.833+0000 7f4b91c67740 1 -- start start 2024-02-26T06:38:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.833+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 0x560d57942840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.833+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 0x560d57943200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.833+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d5794e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b91c67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d578fe5f0 con 0x560d57943af0 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b91c67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d578b0890 con 0x560d5794c300 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b91c67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d578addc0 con 0x560d57942d80 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b87400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d5794e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b87400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d5794e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:48770/0 (socket says 172.21.15.49:48770) 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b87400640 1 -- 172.21.15.49:0/337625627 learned_addr learned my addr 172.21.15.49:0/337625627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b8ce00640 1 --2- 172.21.15.49:0/337625627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 0x560d57942840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b87400640 1 -- 172.21.15.49:0/337625627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 msgr2=0x560d57943200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:38:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.834+0000 7f4b87400640 1 --2- 172.21.15.49:0/337625627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 0x560d57943200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.835+0000 7f4b87400640 1 -- 172.21.15.49:0/337625627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 msgr2=0x560d57942840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.835+0000 7f4b87400640 1 --2- 172.21.15.49:0/337625627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 0x560d57942840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.835+0000 7f4b87400640 1 -- 172.21.15.49:0/337625627 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d5782aad0 con 0x560d5794c300 2024-02-26T06:38:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.837+0000 7f4b87400640 1 --2- 172.21.15.49:0/337625627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d5794e6f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4b881a3fa0 tx=0x7f4b74034c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8030b1d1f94cd81 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.837+0000 7f4b84c00640 1 -- 172.21.15.49:0/3227990027 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4b7800f450 con 0x7f4b881083d0 2024-02-26T06:38:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.837+0000 7f4b7d600640 1 -- 172.21.15.49:0/337625627 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74004030 con 0x560d5794c300 2024-02-26T06:38:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.837+0000 7f4b7d600640 1 -- 172.21.15.49:0/337625627 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b74034440 con 0x560d5794c300 2024-02-26T06:38:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.837+0000 7f4b7d600640 1 -- 172.21.15.49:0/337625627 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7402cab0 con 0x560d5794c300 2024-02-26T06:38:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.839+0000 7f4b91c67740 1 -- 172.21.15.49:0/337625627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 msgr2=0x560d5794e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.839+0000 7f4b91c67740 1 --2- 172.21.15.49:0/337625627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d5794e6f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4b881a3fa0 tx=0x7f4b74034c90 comp rx=0 tx=0).stop 2024-02-26T06:38:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.839+0000 7f4b91c67740 1 -- 172.21.15.49:0/337625627 shutdown_connections 2024-02-26T06:38:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.839+0000 7f4b91c67740 1 --2- 172.21.15.49:0/337625627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d5794e6f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.839+0000 7f4b91c67740 1 --2- 172.21.15.49:0/337625627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 0x560d57943200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.839+0000 7f4b91c67740 1 --2- 172.21.15.49:0/337625627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 0x560d57942840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.839+0000 7f4b91c67740 1 -- 172.21.15.49:0/337625627 >> 172.21.15.49:0/337625627 conn(0x560d5790a480 msgr2=0x560d5793da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.843+0000 7f4b91c67740 1 -- 172.21.15.49:0/337625627 shutdown_connections 2024-02-26T06:38:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.843+0000 7f4b91c67740 1 -- 172.21.15.49:0/337625627 wait complete. 2024-02-26T06:38:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.843+0000 7f4b84c00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4b6c0a22e0 0x7f4b6c0a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.843+0000 7f4b91c67740 1 -- start start 2024-02-26T06:38:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.844+0000 7f4b91c67740 1 Processor -- start 2024-02-26T06:38:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.844+0000 7f4b8ce00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4b6c0a22e0 0x7f4b6c0a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.845+0000 7f4b84c00640 1 -- 172.21.15.49:0/3227990027 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f4b780cdff0 con 0x7f4b881083d0 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.845+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 0x560d57949380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.845+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d57943af0 0x560d579498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.845+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d5794c300 0x560d57949e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.845+0000 7f4b91c67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d578a95a0 con 0x560d57942d80 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.845+0000 7f4b91c67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5784eb20 con 0x560d57943af0 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.845+0000 7f4b91c67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5784e7c0 con 0x560d5794c300 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.846+0000 7f4b91c67740 10 client.0 _ll_register_callbacks cb 0x560d57862520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.846+0000 7f4b8ce00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4b6c0a22e0 0x7f4b6c0a47a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b80005c40 tx=0x7f4b80005bd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.846+0000 7f4b87400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d57943af0 0x560d579498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.846+0000 7f4b87400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d57943af0 0x560d579498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:48786/0 (socket says 172.21.15.49:48786) 2024-02-26T06:38:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.846+0000 7f4b87400640 1 -- 172.21.15.49:0/4064345816 learned_addr learned my addr 172.21.15.49:0/4064345816 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.846+0000 7f4b8ce00640 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d5794c300 0x560d57949e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.847+0000 7f4b87400640 1 -- 172.21.15.49:0/4064345816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d5794c300 msgr2=0x560d57949e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.847+0000 7f4b87400640 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d5794c300 0x560d57949e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.847+0000 7f4b87400640 1 -- 172.21.15.49:0/4064345816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 msgr2=0x560d57949380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.847+0000 7f4b87400640 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 0x560d57949380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.847+0000 7f4b87400640 1 -- 172.21.15.49:0/4064345816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b88117050 con 0x560d57943af0 2024-02-26T06:38:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.848+0000 7f4b87400640 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d57943af0 0x560d579498c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4b7402c180 tx=0x7f4b740042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15782b1732c9d2f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.848+0000 7f4b72000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:38:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.848+0000 7f4b72000640 1 -- 172.21.15.49:0/4064345816 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b740044e0 con 0x560d57943af0 2024-02-26T06:38:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.848+0000 7f4b72000640 1 -- 172.21.15.49:0/4064345816 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f4b74034650 con 0x560d57943af0 2024-02-26T06:38:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.848+0000 7f4b91c67740 1 -- 172.21.15.49:0/4064345816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d5787bca0 con 0x560d57943af0 2024-02-26T06:38:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.849+0000 7f4b72000640 1 -- 172.21.15.49:0/4064345816 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b74034810 con 0x560d57943af0 2024-02-26T06:38:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.850+0000 7f4b91c67740 10 client.27758 fetch_fsmap learned FSMap version 52 2024-02-26T06:38:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.850+0000 7f4b91c67740 1 -- 172.21.15.49:0/4064345816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x560d578a7c20 con 0x560d57943af0 2024-02-26T06:38:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.851+0000 7f4b72000640 1 -- 172.21.15.49:0/4064345816 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560d5787bca0 con 0x560d57943af0 2024-02-26T06:38:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.852+0000 7f4b72000640 1 -- 172.21.15.49:0/4064345816 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4b74032600 con 0x560d57943af0 2024-02-26T06:38:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.853+0000 7f4b91c67740 10 client.27758 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:38:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.853+0000 7f4b91c67740 10 client.27758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:38:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.853+0000 7f4b91c67740 20 client.27758 populate_metadata read hostname 'smithi049' 2024-02-26T06:38:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.853+0000 7f4b91c67740 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560d5795bfe0 0x560d5795e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.853+0000 7f4b91c67740 4 client.27758 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:38:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.853+0000 7f4b91c67740 1 -- 172.21.15.49:0/4064345816 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d5789fcb0 con 0x560d5795bfe0 2024-02-26T06:38:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.853+0000 7f4b87e00640 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560d5795bfe0 0x560d5795e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.854+0000 7f4b87e00640 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560d5795bfe0 0x560d5795e400 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5ea04695627d81 server_cookie=c335e74a82f353c3 in_seq=0 out_seq=0 2024-02-26T06:38:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b72000640 10 client.27758 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b72000640 1 -- 172.21.15.49:0/4064345816 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d5789fcb0 con 0x560d5795bfe0 2024-02-26T06:38:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b72000640 10 client.27758 handle_command_reply: tid=0 2024-02-26T06:38:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 client.27758 shutdown 2024-02-26T06:38:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 -- 172.21.15.49:0/4064345816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d57943af0 msgr2=0x560d579498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d57943af0 0x560d579498c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4b7402c180 tx=0x7f4b740042f0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 -- 172.21.15.49:0/4064345816 shutdown_connections 2024-02-26T06:38:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560d5795bfe0 0x560d5795e400 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d57943af0 0x560d579498c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d5794c300 0x560d57949e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 --2- 172.21.15.49:0/4064345816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 0x560d57949380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b91c67740 1 -- 172.21.15.49:0/4064345816 >> 172.21.15.49:0/4064345816 conn(0x560d5790a480 msgr2=0x560d5793f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.855+0000 7f4b72000640 0 client.27758 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.856+0000 7f4b91c67740 1 -- 172.21.15.49:0/4064345816 shutdown_connections 2024-02-26T06:38:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.856+0000 7f4b91c67740 1 -- 172.21.15.49:0/4064345816 wait complete. 2024-02-26T06:38:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.856+0000 7f4b91c67740 20 client.27758 trim_cache size 0 max 16384 2024-02-26T06:38:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.906+0000 7f4b91c67740 1 Processor -- start 2024-02-26T06:38:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.906+0000 7f4b91c67740 1 -- start start 2024-02-26T06:38:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 0x560d5794bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 0x560d57949c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57953120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b91c67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d575f1e60 con 0x560d57943af0 2024-02-26T06:38:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b91c67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d57944fe0 con 0x560d5794c300 2024-02-26T06:38:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b91c67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d57942080 con 0x560d57942d80 2024-02-26T06:38:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b8ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57953120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 0x560d5794bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b8ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57953120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:48792/0 (socket says 172.21.15.49:48792) 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b8ce00640 1 -- 172.21.15.49:0/3171050376 learned_addr learned my addr 172.21.15.49:0/3171050376 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.907+0000 7f4b87400640 1 --2- 172.21.15.49:0/3171050376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 0x560d57949c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b87e00640 1 -- 172.21.15.49:0/3171050376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 msgr2=0x560d57953120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3171050376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57953120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b87e00640 1 -- 172.21.15.49:0/3171050376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 msgr2=0x560d57949c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3171050376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 0x560d57949c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b87e00640 1 -- 172.21.15.49:0/3171050376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d57844500 con 0x560d57942d80 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3171050376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 0x560d5794bab0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4b780cb6c0 tx=0x7f4b7800a360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d999b0268badf4d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b7cc00640 1 -- 172.21.15.49:0/3171050376 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780a3100 con 0x560d57942d80 2024-02-26T06:38:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b7cc00640 1 -- 172.21.15.49:0/3171050376 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b78098c60 con 0x560d57942d80 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.908+0000 7f4b7cc00640 1 -- 172.21.15.49:0/3171050376 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780cb940 con 0x560d57942d80 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 -- 172.21.15.49:0/3171050376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 msgr2=0x560d5794bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3171050376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 0x560d5794bab0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4b780cb6c0 tx=0x7f4b7800a360 comp rx=0 tx=0).stop 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 -- 172.21.15.49:0/3171050376 shutdown_connections 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3171050376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57953120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3171050376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57943af0 0x560d57949c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3171050376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57942d80 0x560d5794bab0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 -- 172.21.15.49:0/3171050376 >> 172.21.15.49:0/3171050376 conn(0x560d5790a480 msgr2=0x560d57947c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 -- 172.21.15.49:0/3171050376 shutdown_connections 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.909+0000 7f4b91c67740 1 -- 172.21.15.49:0/3171050376 wait complete. 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.910+0000 7f4b91c67740 1 -- start start 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.910+0000 7f4b91c67740 1 Processor -- start 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.910+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 0x560d579534d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.910+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57943af0 0x560d57945cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.910+0000 7f4b91c67740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57946230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.910+0000 7f4b91c67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d57946930 con 0x560d57942d80 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.910+0000 7f4b91c67740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d57946aa0 con 0x560d5794c300 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.911+0000 7f4b91c67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d579c3510 con 0x560d57943af0 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.911+0000 7f4b91c67740 10 client.0 _ll_register_callbacks cb 0x7f4b74034be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.911+0000 7f4b8ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57946230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.911+0000 7f4b8ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57946230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:48802/0 (socket says 172.21.15.49:48802) 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.911+0000 7f4b8ce00640 1 -- 172.21.15.49:0/3313461231 learned_addr learned my addr 172.21.15.49:0/3313461231 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.912+0000 7f4b87400640 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57943af0 0x560d57945cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.912+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 0x560d579534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.912+0000 7f4b8ce00640 1 -- 172.21.15.49:0/3313461231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57943af0 msgr2=0x560d57945cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.912+0000 7f4b8ce00640 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57943af0 0x560d57945cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.912+0000 7f4b8ce00640 1 -- 172.21.15.49:0/3313461231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 msgr2=0x560d579534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.912+0000 7f4b8ce00640 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 0x560d579534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.912+0000 7f4b8ce00640 1 -- 172.21.15.49:0/3313461231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b881b3190 con 0x560d5794c300 2024-02-26T06:38:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.913+0000 7f4b8ce00640 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57946230 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4b8001bc00 tx=0x7f4b80004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637d0ad0beaf8010 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.913+0000 7f4b72000640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.913+0000 7f4b72000640 1 -- 172.21.15.49:0/3313461231 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80021040 con 0x560d5794c300 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.914+0000 7f4b72000640 1 -- 172.21.15.49:0/3313461231 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f4b8000e870 con 0x560d5794c300 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.914+0000 7f4b91c67740 1 -- 172.21.15.49:0/3313461231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d578ced00 con 0x560d5794c300 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.914+0000 7f4b72000640 1 -- 172.21.15.49:0/3313461231 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b80015070 con 0x560d5794c300 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.914+0000 7f4b91c67740 10 client.27764 fetch_fsmap learned FSMap version 52 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.914+0000 7f4b91c67740 1 -- 172.21.15.49:0/3313461231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x560d5786a290 con 0x560d5794c300 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.914+0000 7f4b72000640 1 -- 172.21.15.49:0/3313461231 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560d578ced00 con 0x560d5794c300 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.915+0000 7f4b72000640 1 -- 172.21.15.49:0/3313461231 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 52) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4b80028d20 con 0x560d5794c300 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.915+0000 7f4b91c67740 10 client.27764 fetch_fsmap finished waiting for FSMap version 52 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.915+0000 7f4b91c67740 10 client.27764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:38:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.915+0000 7f4b91c67740 20 client.27764 populate_metadata read hostname 'smithi049' 2024-02-26T06:38:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.915+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560d579c4ca0 0x560d579c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.915+0000 7f4b91c67740 4 client.27764 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:38:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.915+0000 7f4b91c67740 1 -- 172.21.15.49:0/3313461231 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560d575a9600 con 0x560d579c4ca0 2024-02-26T06:38:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.915+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560d579c4ca0 0x560d579c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.916+0000 7f4b87e00640 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560d579c4ca0 0x560d579c70c0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad632b69ddf0d8d server_cookie=ed6f8885351d4b21 in_seq=0 out_seq=0 2024-02-26T06:38:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.916+0000 7f4b72000640 10 client.27764 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b72000640 1 -- 172.21.15.49:0/3313461231 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560d575a9600 con 0x560d579c4ca0 2024-02-26T06:38:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b72000640 10 client.27764 handle_command_reply: tid=0 2024-02-26T06:38:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 client.27764 shutdown 2024-02-26T06:38:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 -- 172.21.15.49:0/3313461231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 msgr2=0x560d57946230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57946230 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4b8001bc00 tx=0x7f4b80004ab0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 -- 172.21.15.49:0/3313461231 shutdown_connections 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x560d579c4ca0 0x560d579c70c0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560d5794c300 0x560d57946230 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d57943af0 0x560d57945cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 --2- 172.21.15.49:0/3313461231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560d57942d80 0x560d579534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b91c67740 1 -- 172.21.15.49:0/3313461231 >> 172.21.15.49:0/3313461231 conn(0x560d5790a480 msgr2=0x560d5793c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.929+0000 7f4b72000640 0 client.27764 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.930+0000 7f4b91c67740 1 -- 172.21.15.49:0/3313461231 shutdown_connections 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.930+0000 7f4b91c67740 1 -- 172.21.15.49:0/3313461231 wait complete. 2024-02-26T06:38:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.930+0000 7f4b91c67740 20 client.27764 trim_cache size 0 max 16384 2024-02-26T06:38:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:38:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:38:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:38:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:38:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 -- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4b6c0a22e0 msgr2=0x7f4b6c0a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4b6c0a22e0 0x7f4b6c0a47a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b80005c40 tx=0x7f4b80005bd0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 -- 172.21.15.49:0/3227990027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b881083d0 msgr2=0x7f4b88109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b881083d0 0x7f4b88109bf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800c970 tx=0x7f4b7800ce30 comp rx=0 tx=0).stop 2024-02-26T06:38:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 -- 172.21.15.49:0/3227990027 shutdown_connections 2024-02-26T06:38:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f4b6c0a22e0 0x7f4b6c0a47a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b88108cf0 0x7f4b8810a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b881142d0 0x7f4b8810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 --2- 172.21.15.49:0/3227990027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b881083d0 0x7f4b88109bf0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 -- 172.21.15.49:0/3227990027 >> 172.21.15.49:0/3227990027 conn(0x7f4b8806c220 msgr2=0x7f4b8806fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 -- 172.21.15.49:0/3227990027 shutdown_connections 2024-02-26T06:38:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:38:48.931+0000 7f4b8ee00640 1 -- 172.21.15.49:0/3227990027 wait complete. 2024-02-26T06:38:49.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:48 smithi080 ceph-mon[36206]: pgmap v483: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-02-26T06:38:49.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:48 smithi115 ceph-mon[36426]: pgmap v483: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-02-26T06:38:49.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:48 smithi049 ceph-mon[31696]: pgmap v483: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 18 MiB/s wr, 1.51k op/s 2024-02-26T06:38:51.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:50 smithi080 ceph-mon[36206]: pgmap v484: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-02-26T06:38:51.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:50 smithi115 ceph-mon[36426]: pgmap v484: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-02-26T06:38:51.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:50 smithi049 ceph-mon[31696]: pgmap v484: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-02-26T06:38:51.303 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 900.0 s, 1549.0 tps, lat 63.507 ms stddev 72.684 2024-02-26T06:38:51.575 INFO:teuthology.orchestra.run.smithi049.stdout:transaction type: 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:scaling factor: 32 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:query mode: simple 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:number of clients: 100 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:number of threads: 4 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:duration: 900 s 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:number of transactions actually processed: 1594976 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:latency average = 56.400 ms 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:latency stddev = 101.728 ms 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:tps = 1771.658585 (including connections establishing) 2024-02-26T06:38:51.576 INFO:teuthology.orchestra.run.smithi049.stdout:tps = 1771.671508 (excluding connections establishing) 2024-02-26T06:38:51.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-02-26T06:38:53.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:52 smithi080 ceph-mon[36206]: pgmap v485: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-02-26T06:38:53.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:52 smithi115 ceph-mon[36426]: pgmap v485: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-02-26T06:38:53.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:52 smithi049 ceph-mon[31696]: pgmap v485: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-02-26T06:38:55.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:54 smithi080 ceph-mon[36206]: pgmap v486: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 11 MiB/s wr, 863 op/s 2024-02-26T06:38:55.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:54 smithi115 ceph-mon[36426]: pgmap v486: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 11 MiB/s wr, 863 op/s 2024-02-26T06:38:55.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:54 smithi049 ceph-mon[31696]: pgmap v486: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 11 MiB/s wr, 863 op/s 2024-02-26T06:38:56.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:55 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:38:56.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:55 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:38:56.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:55 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:38:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:56 smithi115 ceph-mon[36426]: pgmap v487: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 937 op/s 2024-02-26T06:38:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:56 smithi049 ceph-mon[31696]: pgmap v487: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 937 op/s 2024-02-26T06:38:57.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:56 smithi080 ceph-mon[36206]: pgmap v487: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 937 op/s 2024-02-26T06:38:59.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:38:58 smithi115 ceph-mon[36426]: pgmap v488: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.8 MiB/s wr, 763 op/s 2024-02-26T06:38:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:38:58 smithi049 ceph-mon[31696]: pgmap v488: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.8 MiB/s wr, 763 op/s 2024-02-26T06:38:59.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:38:58 smithi080 ceph-mon[36206]: pgmap v488: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.8 MiB/s wr, 763 op/s 2024-02-26T06:39:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:00 smithi049 ceph-mon[31696]: pgmap v489: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.0 MiB/s wr, 788 op/s 2024-02-26T06:39:01.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:00 smithi080 ceph-mon[36206]: pgmap v489: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.0 MiB/s wr, 788 op/s 2024-02-26T06:39:01.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:00 smithi115 ceph-mon[36426]: pgmap v489: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.0 MiB/s wr, 788 op/s 2024-02-26T06:39:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 -- 172.21.15.49:0/2037518041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c81035a0 msgr2=0x7f42c8103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 --2- 172.21.15.49:0/2037518041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c81035a0 0x7f42c8103980 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009930 tx=0x7f42b802bc30 comp rx=0 tx=0).stop 2024-02-26T06:39:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 -- 172.21.15.49:0/2037518041 shutdown_connections 2024-02-26T06:39:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 --2- 172.21.15.49:0/2037518041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c8103f50 0x7f42c8107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 --2- 172.21.15.49:0/2037518041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42c8108670 0x7f42c8110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 --2- 172.21.15.49:0/2037518041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c81035a0 0x7f42c8103980 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 -- 172.21.15.49:0/2037518041 >> 172.21.15.49:0/2037518041 conn(0x7f42c80fee00 msgr2=0x7f42c8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 -- 172.21.15.49:0/2037518041 shutdown_connections 2024-02-26T06:39:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.080+0000 7f42cdc00640 1 -- 172.21.15.49:0/2037518041 wait complete. 2024-02-26T06:39:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.081+0000 7f42cdc00640 1 Processor -- start 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.081+0000 7f42cdc00640 1 -- start start 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42cdc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42c81035a0 0x7f42c819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42cdc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c8108670 0x7f42c819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42cdc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c81997d0 0x7f42c8199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42cdc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c819a280 con 0x7f42c8108670 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42cdc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c819a3f0 con 0x7f42c81997d0 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42cdc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c819a560 con 0x7f42c81035a0 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c8108670 0x7f42c819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42c81035a0 0x7f42c819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c81997d0 0x7f42c8199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42c81035a0 0x7f42c819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41884/0 (socket says 172.21.15.49:41884) 2024-02-26T06:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6a00640 1 -- 172.21.15.49:0/626946476 learned_addr learned my addr 172.21.15.49:0/626946476 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6a00640 1 -- 172.21.15.49:0/626946476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c81997d0 msgr2=0x7f42c8199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6a00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c81997d0 0x7f42c8199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6a00640 1 -- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c8108670 msgr2=0x7f42c819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6a00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c8108670 0x7f42c819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.082+0000 7f42c6a00640 1 -- 172.21.15.49:0/626946476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b8009560 con 0x7f42c81035a0 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.083+0000 7f42c6a00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42c81035a0 0x7f42c819f660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009230 tx=0x7f42b802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.083+0000 7f42bf400640 1 -- 172.21.15.49:0/626946476 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8004350 con 0x7f42c81035a0 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.083+0000 7f42bf400640 1 -- 172.21.15.49:0/626946476 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42b80044b0 con 0x7f42c81035a0 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.083+0000 7f42bf400640 1 -- 172.21.15.49:0/626946476 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8003d00 con 0x7f42c81035a0 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.083+0000 7f42cdc00640 1 -- 172.21.15.49:0/626946476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c81a64b0 con 0x7f42c81035a0 2024-02-26T06:39:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.083+0000 7f42cdc00640 1 -- 172.21.15.49:0/626946476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c81a6990 con 0x7f42c81035a0 2024-02-26T06:39:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.086+0000 7f42bf400640 1 -- 172.21.15.49:0/626946476 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f42b8004df0 con 0x7f42c81035a0 2024-02-26T06:39:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.086+0000 7f42d0be9740 1 Processor -- start 2024-02-26T06:39:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42d0be9740 1 -- start start 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b69840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b6a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42d0be9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832b255f0 con 0x55c832b73300 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42d0be9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832ad7890 con 0x55c832b6aaf0 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42d0be9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832ad4dc0 con 0x55c832b69d80 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42c7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b69840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42c7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b69840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:40626/0 (socket says 172.21.15.49:40626) 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42c7400640 1 -- 172.21.15.49:0/4257680205 learned_addr learned my addr 172.21.15.49:0/4257680205 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42c6a00640 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b6a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.087+0000 7f42c6000640 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42c7400640 1 -- 172.21.15.49:0/4257680205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 msgr2=0x55c832b6a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42c7400640 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b6a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42c7400640 1 -- 172.21.15.49:0/4257680205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 msgr2=0x55c832b756f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42c7400640 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b756f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42c7400640 1 -- 172.21.15.49:0/4257680205 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c832a51ad0 con 0x55c832b6aaf0 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42c6000640 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42bf400640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42ac0a22d0 0x7f42ac0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42bf400640 1 -- 172.21.15.49:0/626946476 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f42b80effe0 con 0x7f42c81035a0 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42c7400640 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b69840 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009f50 tx=0x7f42c00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a704edd5cd3ea5d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.088+0000 7f42c6a00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42ac0a22d0 0x7f42ac0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42b3e00640 1 -- 172.21.15.49:0/4257680205 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0004d20 con 0x55c832b6aaf0 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42b3e00640 1 -- 172.21.15.49:0/4257680205 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42c0004e80 con 0x55c832b6aaf0 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42b3e00640 1 -- 172.21.15.49:0/4257680205 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0005280 con 0x55c832b6aaf0 2024-02-26T06:39:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42d0be9740 1 -- 172.21.15.49:0/4257680205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 msgr2=0x55c832b69840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42d0be9740 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b69840 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009f50 tx=0x7f42c00118d0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42c6a00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42ac0a22d0 0x7f42ac0a4790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55c832b6b250 tx=0x7f42b80f3a70 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42d0be9740 1 -- 172.21.15.49:0/4257680205 shutdown_connections 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42d0be9740 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42d0be9740 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b6a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42d0be9740 1 --2- 172.21.15.49:0/4257680205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b69840 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.089+0000 7f42d0be9740 1 -- 172.21.15.49:0/4257680205 >> 172.21.15.49:0/4257680205 conn(0x55c832b31480 msgr2=0x55c832b64a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.090+0000 7f42d0be9740 1 -- 172.21.15.49:0/4257680205 shutdown_connections 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.090+0000 7f42d0be9740 1 -- 172.21.15.49:0/4257680205 wait complete. 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.090+0000 7f42d0be9740 1 -- start start 2024-02-26T06:39:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.090+0000 7f42d0be9740 1 Processor -- start 2024-02-26T06:39:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b70900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b70e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42d0be9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832ad05a0 con 0x55c832b73300 2024-02-26T06:39:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42d0be9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832a75b20 con 0x55c832b6aaf0 2024-02-26T06:39:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42d0be9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832a757c0 con 0x55c832b69d80 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42d0be9740 10 client.0 _ll_register_callbacks cb 0x55c832a89520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42c7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b70e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42c6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b70900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42c7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b70e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35962/0 (socket says 172.21.15.49:35962) 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42c7400640 1 -- 172.21.15.49:0/1698190511 learned_addr learned my addr 172.21.15.49:0/1698190511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.091+0000 7f42c7400640 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42c6000640 1 -- 172.21.15.49:0/1698190511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 msgr2=0x55c832b703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42c6000640 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42c6000640 1 -- 172.21.15.49:0/1698190511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 msgr2=0x55c832b70e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42c6000640 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b70e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42c6000640 1 -- 172.21.15.49:0/1698190511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c0009c00 con 0x55c832b6aaf0 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42c6000640 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b70900 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55c832b6a820 tx=0x7f42b4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d0bef8e2131187 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42b0c00640 1 -- 172.21.15.49:0/1698190511 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4017070 con 0x55c832b6aaf0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42d0be9740 1 -- 172.21.15.49:0/1698190511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c832aa2ca0 con 0x55c832b6aaf0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.092+0000 7f42b0c00640 1 -- 172.21.15.49:0/1698190511 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f42b4004050 con 0x55c832b6aaf0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42b0c00640 1 -- 172.21.15.49:0/1698190511 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42b4004210 con 0x55c832b6aaf0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42d0be9740 10 client.27776 fetch_fsmap learned FSMap version 53 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42d0be9740 1 -- 172.21.15.49:0/1698190511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55c832acec20 con 0x55c832b6aaf0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42b0c00640 1 -- 172.21.15.49:0/1698190511 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c832aa2ca0 con 0x55c832b6aaf0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42b0c00640 1 -- 172.21.15.49:0/1698190511 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 53) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f42b4003870 con 0x55c832b6aaf0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42d0be9740 10 client.27776 fetch_fsmap finished waiting for FSMap version 53 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42d0be9740 10 client.27776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42d0be9740 20 client.27776 populate_metadata read hostname 'smithi049' 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c832b834d0 0x55c832b858f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42d0be9740 4 client.27776 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.093+0000 7f42d0be9740 1 -- 172.21.15.49:0/1698190511 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c832ac6cb0 con 0x55c832b834d0 2024-02-26T06:39:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.094+0000 7f42c7400640 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c832b834d0 0x55c832b858f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.094+0000 7f42c7400640 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c832b834d0 0x55c832b858f0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b8706efcdd1d52 server_cookie=38405176ac482e66 in_seq=0 out_seq=0 2024-02-26T06:39:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.094+0000 7f42b0c00640 10 client.27776 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.095+0000 7f42b0c00640 1 -- 172.21.15.49:0/1698190511 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c832ac6cb0 con 0x55c832b834d0 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.095+0000 7f42b0c00640 10 client.27776 handle_command_reply: tid=0 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.095+0000 7f42d0be9740 1 client.27776 shutdown 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.095+0000 7f42d0be9740 1 -- 172.21.15.49:0/1698190511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 msgr2=0x55c832b70900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.095+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b70900 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55c832b6a820 tx=0x7f42b4012b60 comp rx=0 tx=0).stop 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 1 -- 172.21.15.49:0/1698190511 shutdown_connections 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c832b834d0 0x55c832b858f0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b70900 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b70e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1698190511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42b0c00640 0 client.27776 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:39:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 1 -- 172.21.15.49:0/1698190511 >> 172.21.15.49:0/1698190511 conn(0x55c832b31480 msgr2=0x55c832b667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 1 -- 172.21.15.49:0/1698190511 shutdown_connections 2024-02-26T06:39:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 1 -- 172.21.15.49:0/1698190511 wait complete. 2024-02-26T06:39:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.096+0000 7f42d0be9740 20 client.27776 trim_cache size 0 max 16384 2024-02-26T06:39:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.110+0000 7f42d0be9740 1 Processor -- start 2024-02-26T06:39:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.110+0000 7f42d0be9740 1 -- start start 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.110+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b69d80 0x55c832b70910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.110+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b6aaf0 0x55c832b70e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.110+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b73300 0x55c832b6d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.110+0000 7f42d0be9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832818e60 con 0x55c832b69d80 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.110+0000 7f42d0be9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832b6c010 con 0x55c832b73300 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.110+0000 7f42d0be9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832b56670 con 0x55c832b6aaf0 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b69d80 0x55c832b70910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c6000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b6aaf0 0x55c832b70e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b69d80 0x55c832b70910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35964/0 (socket says 172.21.15.49:35964) 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c7400640 1 -- 172.21.15.49:0/1983127852 learned_addr learned my addr 172.21.15.49:0/1983127852 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c7400640 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b73300 0x55c832b6d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c6000640 1 -- 172.21.15.49:0/1983127852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b73300 msgr2=0x55c832b6d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c6000640 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b73300 0x55c832b6d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c6000640 1 -- 172.21.15.49:0/1983127852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b69d80 msgr2=0x55c832b70910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:39:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c6000640 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b69d80 0x55c832b70910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c6000640 1 -- 172.21.15.49:0/1983127852 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c832a6b500 con 0x55c832b6aaf0 2024-02-26T06:39:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c7400640 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b73300 0x55c832b6d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:39:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.111+0000 7f42c6000640 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b6aaf0 0x55c832b70e50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f42b4003cd0 tx=0x7f42b40092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1499c18ea2aad45 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42b3400640 1 -- 172.21.15.49:0/1983127852 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4017040 con 0x55c832b6aaf0 2024-02-26T06:39:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42b3400640 1 -- 172.21.15.49:0/1983127852 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42b40055a0 con 0x55c832b6aaf0 2024-02-26T06:39:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42b3400640 1 -- 172.21.15.49:0/1983127852 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b400fcf0 con 0x55c832b6aaf0 2024-02-26T06:39:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42d0be9740 1 -- 172.21.15.49:0/1983127852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b6aaf0 msgr2=0x55c832b70e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b6aaf0 0x55c832b70e50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f42b4003cd0 tx=0x7f42b40092e0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42d0be9740 1 -- 172.21.15.49:0/1983127852 shutdown_connections 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b73300 0x55c832b6d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b6aaf0 0x55c832b70e50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42d0be9740 1 --2- 172.21.15.49:0/1983127852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b69d80 0x55c832b70910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.112+0000 7f42d0be9740 1 -- 172.21.15.49:0/1983127852 >> 172.21.15.49:0/1983127852 conn(0x55c832b31480 msgr2=0x55c832b6ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.113+0000 7f42d0be9740 1 -- 172.21.15.49:0/1983127852 shutdown_connections 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.113+0000 7f42d0be9740 1 -- 172.21.15.49:0/1983127852 wait complete. 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.113+0000 7f42d0be9740 1 -- start start 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.113+0000 7f42d0be9740 1 Processor -- start 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b78690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42d0be9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b78bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42d0be9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832b79110 con 0x55c832b73300 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42d0be9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832b79250 con 0x55c832b6aaf0 2024-02-26T06:39:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42d0be9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c832b79410 con 0x55c832b69d80 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42d0be9740 10 client.0 _ll_register_callbacks cb 0x7f42b40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b78690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b78690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:40652/0 (socket says 172.21.15.49:40652) 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c6000640 1 -- 172.21.15.49:0/2117267821 learned_addr learned my addr 172.21.15.49:0/2117267821 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c7400640 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c7400640 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b78bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c6000640 1 -- 172.21.15.49:0/2117267821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 msgr2=0x55c832b7a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c6000640 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b7a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c6000640 1 -- 172.21.15.49:0/2117267821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 msgr2=0x55c832b78bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c6000640 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b78bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c6000640 1 -- 172.21.15.49:0/2117267821 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c832acec20 con 0x55c832b6aaf0 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.114+0000 7f42c7400640 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b78bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42c6000640 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b78690 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f42b4003e10 tx=0x7f42b400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d19a2bb87e84a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-02-26T06:39:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42b0c00640 1 -- 172.21.15.49:0/2117267821 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4017070 con 0x55c832b6aaf0 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42d0be9740 1 -- 172.21.15.49:0/2117267821 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c832af5d00 con 0x55c832b6aaf0 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42b0c00640 1 -- 172.21.15.49:0/2117267821 <== mon.2 v2:172.21.15.80:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f42b4000e40 con 0x55c832b6aaf0 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42b0c00640 1 -- 172.21.15.49:0/2117267821 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42b400f970 con 0x55c832b6aaf0 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42d0be9740 10 client.27779 fetch_fsmap learned FSMap version 53 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42d0be9740 1 -- 172.21.15.49:0/2117267821 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55c832a91290 con 0x55c832b6aaf0 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.115+0000 7f42b0c00640 1 -- 172.21.15.49:0/2117267821 <== mon.2 v2:172.21.15.80:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c832af5d00 con 0x55c832b6aaf0 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.116+0000 7f42b0c00640 1 -- 172.21.15.49:0/2117267821 <== mon.2 v2:172.21.15.80:3300/0 5 ==== fsmap(e 53) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f42b402b020 con 0x55c832b6aaf0 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.116+0000 7f42d0be9740 10 client.27779 fetch_fsmap finished waiting for FSMap version 53 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.116+0000 7f42d0be9740 10 client.27779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.116+0000 7f42d0be9740 20 client.27779 populate_metadata read hostname 'smithi049' 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.116+0000 7f42d0be9740 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c832bdac30 0x55c832bdd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.116+0000 7f42d0be9740 4 client.27779 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:39:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.116+0000 7f42d0be9740 1 -- 172.21.15.49:0/2117267821 --> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c8327d0600 con 0x55c832bdac30 2024-02-26T06:39:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.116+0000 7f42c7400640 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c832bdac30 0x55c832bdd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.117+0000 7f42c7400640 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c832bdac30 0x55c832bdd050 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc6e1228cb58fde server_cookie=cc8aa3b611dcd6e8 in_seq=0 out_seq=0 2024-02-26T06:39:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.117+0000 7f42b0c00640 10 client.27779 ms_handle_connect on v2:172.21.15.80:6838/3160003047 2024-02-26T06:39:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42b0c00640 1 -- 172.21.15.49:0/2117267821 <== mds.0 v2:172.21.15.80:6838/3160003047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c8327d0600 con 0x55c832bdac30 2024-02-26T06:39:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42b0c00640 10 client.27779 handle_command_reply: tid=0 2024-02-26T06:39:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 client.27779 shutdown 2024-02-26T06:39:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 -- 172.21.15.49:0/2117267821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 msgr2=0x55c832b78690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b78690 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f42b4003e10 tx=0x7f42b400f790 comp rx=0 tx=0).stop 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 -- 172.21.15.49:0/2117267821 shutdown_connections 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.80:6838/3160003047,v1:172.21.15.80:6839/3160003047] conn(0x55c832bdac30 0x55c832bdd050 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c832b6aaf0 0x55c832b78690 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c832b73300 0x55c832b78bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 --2- 172.21.15.49:0/2117267821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c832b69d80 0x55c832b7a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42b0c00640 0 client.27779 ms_handle_reset on v2:172.21.15.80:6838/3160003047 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.162+0000 7f42d0be9740 1 -- 172.21.15.49:0/2117267821 >> 172.21.15.49:0/2117267821 conn(0x55c832b31480 msgr2=0x55c832b63480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42d0be9740 1 -- 172.21.15.49:0/2117267821 shutdown_connections 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42d0be9740 1 -- 172.21.15.49:0/2117267821 wait complete. 2024-02-26T06:39:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42d0be9740 20 client.27779 trim_cache size 0 max 16384 2024-02-26T06:39:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-02-26T06:39:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-02-26T06:39:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-02-26T06:39:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-02-26T06:39:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42cdc00640 1 -- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42ac0a22d0 msgr2=0x7f42ac0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42cdc00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42ac0a22d0 0x7f42ac0a4790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55c832b6b250 tx=0x7f42b80f3a70 comp rx=0 tx=0).stop 2024-02-26T06:39:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42cdc00640 1 -- 172.21.15.49:0/626946476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42c81035a0 msgr2=0x7f42c819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42cdc00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42c81035a0 0x7f42c819f660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009230 tx=0x7f42b802c650 comp rx=0 tx=0).stop 2024-02-26T06:39:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42cdc00640 1 -- 172.21.15.49:0/626946476 shutdown_connections 2024-02-26T06:39:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.163+0000 7f42cdc00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f42ac0a22d0 0x7f42ac0a4790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.164+0000 7f42cdc00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c81997d0 0x7f42c8199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.164+0000 7f42cdc00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c8108670 0x7f42c819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.164+0000 7f42cdc00640 1 --2- 172.21.15.49:0/626946476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42c81035a0 0x7f42c819f660 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.164+0000 7f42cdc00640 1 -- 172.21.15.49:0/626946476 >> 172.21.15.49:0/626946476 conn(0x7f42c80fee00 msgr2=0x7f42c8101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.164+0000 7f42cdc00640 1 -- 172.21.15.49:0/626946476 shutdown_connections 2024-02-26T06:39:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-02-26T06:39:02.164+0000 7f42cdc00640 1 -- 172.21.15.49:0/626946476 wait complete. 2024-02-26T06:39:03.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:02 smithi049 ceph-mon[31696]: pgmap v490: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-02-26T06:39:03.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:02 smithi080 ceph-mon[36206]: pgmap v490: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-02-26T06:39:03.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:02 smithi115 ceph-mon[36426]: pgmap v490: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-02-26T06:39:05.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:04 smithi049 ceph-mon[31696]: pgmap v491: 97 pgs: 97 active+clean; 4.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-02-26T06:39:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:04 smithi080 ceph-mon[36206]: pgmap v491: 97 pgs: 97 active+clean; 4.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-02-26T06:39:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:04 smithi115 ceph-mon[36426]: pgmap v491: 97 pgs: 97 active+clean; 4.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-02-26T06:39:06.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:05 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:06.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:05 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:06.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:05 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:06 smithi049 ceph-mon[31696]: pgmap v492: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 1.92k op/s 2024-02-26T06:39:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:06 smithi049 ceph-mon[31696]: osdmap e108: 12 total, 12 up, 12 in 2024-02-26T06:39:07.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:06 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:39:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:06 smithi080 ceph-mon[36206]: pgmap v492: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 1.92k op/s 2024-02-26T06:39:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:06 smithi080 ceph-mon[36206]: osdmap e108: 12 total, 12 up, 12 in 2024-02-26T06:39:07.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:06 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:39:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:06 smithi115 ceph-mon[36426]: pgmap v492: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 1.92k op/s 2024-02-26T06:39:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:06 smithi115 ceph-mon[36426]: osdmap e108: 12 total, 12 up, 12 in 2024-02-26T06:39:07.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:06 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:39:08.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:07 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:39:08.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:07 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:39:08.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:07 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:39:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:07 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:39:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:07 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:39:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:07 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:39:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:07 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:39:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:07 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:39:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:07 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:39:09.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:08 smithi049 ceph-mon[31696]: pgmap v494: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-02-26T06:39:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:08 smithi080 ceph-mon[36206]: pgmap v494: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-02-26T06:39:09.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:08 smithi115 ceph-mon[36426]: pgmap v494: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-02-26T06:39:10.120 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-02-26T06:39:10.340 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2024-02-26T06:39:10.349 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:10.349 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 .. 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 data 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:total 36 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 . 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 .. 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 26 06:23 base 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Feb 26 06:23 current_logfiles 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Feb 26 06:23 global 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 log 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_commit_ts 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_dynshmem 2024-02-26T06:39:10.350 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Feb 26 06:23 pg_hba.conf 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Feb 26 06:23 pg_ident.conf 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:39 pg_logical 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 pg_multixact 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_notify 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_replslot 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_serial 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_snapshots 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:39 pg_stat 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:39 pg_stat_tmp 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:38 pg_subtrans 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_tblspc 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pg_twophase 2024-02-26T06:39:10.351 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 PG_VERSION 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Feb 26 06:39 pg_wal 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:32 pg_xact 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Feb 26 06:23 postgresql.auto.conf 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Feb 26 06:23 postgresql.conf 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Feb 26 06:23 postmaster.opts 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base: 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 26 06:23 . 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 1 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 13493 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Feb 26 06:28 13494 2024-02-26T06:39:10.352 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 pgsql_tmp 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/1: 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 . 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 26 06:23 .. 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 112 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 113 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 1247 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1247_fsm 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1247_vm 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 26 06:23 1249 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1249_fsm 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1249_vm 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 26 06:23 1255 2024-02-26T06:39:10.353 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1255_fsm 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1255_vm 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 26 06:23 1259 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1259_fsm 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1259_vm 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 26 06:23 13344 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13344_fsm 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13344_vm 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13346 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13348 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13349_fsm 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349_vm 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13351 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13353 2024-02-26T06:39:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13354_fsm 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354_vm 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13356 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13358 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13359_fsm 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359_vm 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13361 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13363 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1417 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1418 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 174 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 175 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2187 2024-02-26T06:39:10.355 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2224 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2328 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2336 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2337 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2579 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2600 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2600_fsm 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2600_vm 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2601_fsm 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601_vm 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2602 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2602_fsm 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2602_vm 2024-02-26T06:39:10.356 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2603 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2603_fsm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2603_vm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2604 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2605 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2605_fsm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2605_vm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2606_fsm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606_vm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2607 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2607_fsm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2607_vm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 26 06:23 2608 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2608_fsm 2024-02-26T06:39:10.357 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2608_vm 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 26 06:23 2609 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2609_fsm 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2609_vm 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2610 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2610_fsm 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2610_vm 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2611 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2612_fsm 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612_vm 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2613 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2615_fsm 2024-02-26T06:39:10.358 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615_vm 2024-02-26T06:39:10.359 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2616 2024-02-26T06:39:10.359 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2616_fsm 2024-02-26T06:39:10.359 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2616_vm 2024-02-26T06:39:10.359 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 26 06:23 2617 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2617_fsm 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2617_vm 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 26 06:23 2618 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2618_fsm 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2618_vm 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 26 06:23 2619 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2619_fsm 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2619_vm 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2620 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2650 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2651 2024-02-26T06:39:10.360 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2652 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2653 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2654 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2655 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2656 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2657 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 26 06:23 2658 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 2659 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2660 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2661 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2662 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2663 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2664 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2665 2024-02-26T06:39:10.361 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2666 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2667 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2668 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2669 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2670 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 26 06:23 2673 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 26 06:23 2674 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 26 06:23 2675 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2678 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2679 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2680 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2681 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2682 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2683 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2684 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2685 2024-02-26T06:39:10.362 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2686 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2687 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2688 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2689 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 26 06:23 2690 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 26 06:23 2691 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2692 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2693 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2696 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2699 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2701 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2702 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2703 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2704 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2753 2024-02-26T06:39:10.363 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2753_fsm 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2753_vm 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2754 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2755 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2756 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2757 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2830 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2831 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2832 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2833 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2834 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2835 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2836 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2837 2024-02-26T06:39:10.364 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 26 06:23 2838 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2838_fsm 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2838_vm 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2839 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840_fsm 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2840_vm 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2841 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2995 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2996 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3079_fsm 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079_vm 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3080 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3081 2024-02-26T06:39:10.365 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3085 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3118 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3119 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 26 06:23 3164 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3256 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3257 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3258 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3350 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3351 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3379 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3380 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3381 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3394 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3394_fsm 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3394_vm 2024-02-26T06:39:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3395 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3429 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3430 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3431 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3433 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3439 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3440 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3455 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 26 06:23 3456 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3456_fsm 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3456_vm 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3466 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3467 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3468 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3501 2024-02-26T06:39:10.367 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3502 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3503 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3534 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3541_fsm 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541_vm 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3542 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3574 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3575 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3576 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3596 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3597 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3598 2024-02-26T06:39:10.368 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3599 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3600_fsm 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600_vm 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3601_fsm 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601_vm 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3602_fsm 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602_vm 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603_fsm 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3603_vm 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3604 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3605 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3606 2024-02-26T06:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3607 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3608 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 3609 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3712 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3764_fsm 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764_vm 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3766 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3767 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3997 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4143 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4144 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4145 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4146 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4147 2024-02-26T06:39:10.370 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4148 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4149 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4150 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4151 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4152 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4153 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4154 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4155 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4156 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4157 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4158 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4159 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4160 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4161 2024-02-26T06:39:10.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4162 2024-02-26T06:39:10.405 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4163 2024-02-26T06:39:10.405 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4164 2024-02-26T06:39:10.405 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4165 2024-02-26T06:39:10.405 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4166 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4167 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4168 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4169 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4170 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4171 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4172 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4173 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4174 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 5002 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 548 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 549 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6102 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6104 2024-02-26T06:39:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6106 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6110 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6111 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6112 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6113 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6117 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 826 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 827 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 828 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 26 06:23 pg_filenode.map 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 PG_VERSION 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 26 06:23 . 2024-02-26T06:39:10.407 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 26 06:23 .. 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 112 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 113 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 1247 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1247_fsm 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1247_vm 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 26 06:23 1249 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1249_fsm 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1249_vm 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 26 06:23 1255 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1255_fsm 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1255_vm 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 26 06:23 1259 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1259_fsm 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1259_vm 2024-02-26T06:39:10.408 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 26 06:23 13344 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13344_fsm 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13344_vm 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13346 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13348 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13349_fsm 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349_vm 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13351 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13353 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13354_fsm 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354_vm 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13356 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13358 2024-02-26T06:39:10.409 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13359_fsm 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359_vm 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13361 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13363 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1417 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1418 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 174 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 175 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2187 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2224 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2328 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2336 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2337 2024-02-26T06:39:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2579 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2600 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2600_fsm 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2600_vm 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2601_fsm 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601_vm 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2602 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2602_fsm 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2602_vm 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2603 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2603_fsm 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2603_vm 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2604 2024-02-26T06:39:10.411 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2605 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2605_fsm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2605_vm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2606_fsm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606_vm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2607 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2607_fsm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2607_vm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 26 06:23 2608 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2608_fsm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2608_vm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 26 06:23 2609 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2609_fsm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2609_vm 2024-02-26T06:39:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2610 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2610_fsm 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2610_vm 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2611 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2612_fsm 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612_vm 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2613 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2615_fsm 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615_vm 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2616 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2616_fsm 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2616_vm 2024-02-26T06:39:10.413 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 26 06:23 2617 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2617_fsm 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2617_vm 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 26 06:23 2618 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2618_fsm 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2618_vm 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 26 06:23 2619 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2619_fsm 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2619_vm 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2620 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2650 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2651 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2652 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2653 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2654 2024-02-26T06:39:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2655 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2656 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2657 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 26 06:23 2658 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 2659 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2660 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2661 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2662 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2663 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2664 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2665 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2666 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2667 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2668 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2669 2024-02-26T06:39:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2670 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 26 06:23 2673 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 26 06:23 2674 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 26 06:23 2675 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2678 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2679 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2680 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2681 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2682 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2683 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2684 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2685 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2686 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2687 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2688 2024-02-26T06:39:10.416 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2689 2024-02-26T06:39:10.417 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 26 06:23 2690 2024-02-26T06:39:10.419 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 26 06:23 2691 2024-02-26T06:39:10.419 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2692 2024-02-26T06:39:10.419 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2693 2024-02-26T06:39:10.419 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2696 2024-02-26T06:39:10.420 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2699 2024-02-26T06:39:10.420 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2701 2024-02-26T06:39:10.420 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2702 2024-02-26T06:39:10.420 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2703 2024-02-26T06:39:10.420 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2704 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2753 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2753_fsm 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2753_vm 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2754 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2755 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2756 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2757 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2830 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2831 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2832 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2833 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2834 2024-02-26T06:39:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2835 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2836 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2837 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 26 06:23 2838 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2838_fsm 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2838_vm 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2839 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840_fsm 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2840_vm 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2841 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2995 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2996 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3079_fsm 2024-02-26T06:39:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079_vm 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3080 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3081 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3085 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3118 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3119 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 26 06:23 3164 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3256 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3257 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3258 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3350 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3351 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3379 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3380 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3381 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3394 2024-02-26T06:39:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3394_fsm 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3394_vm 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3395 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3429 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3430 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3431 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3433 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3439 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3440 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3455 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 26 06:23 3456 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3456_fsm 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3456_vm 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3466 2024-02-26T06:39:10.424 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3467 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3468 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3501 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3502 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3503 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3534 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3541_fsm 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541_vm 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3542 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3574 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3575 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3576 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3596 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3597 2024-02-26T06:39:10.425 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3598 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3599 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3600_fsm 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600_vm 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3601_fsm 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601_vm 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3602_fsm 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602_vm 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603_fsm 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3603_vm 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3604 2024-02-26T06:39:10.426 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3605 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3606 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3607 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3608 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 3609 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3712 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3764_fsm 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764_vm 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3766 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3767 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3997 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4143 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4144 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4145 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4146 2024-02-26T06:39:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4147 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4148 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4149 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4150 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4151 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4152 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4153 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4154 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4155 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4156 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4157 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4158 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4159 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4160 2024-02-26T06:39:10.428 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4161 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4162 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4163 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4164 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4165 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4166 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4167 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4168 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4169 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4170 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4171 2024-02-26T06:39:10.429 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4172 2024-02-26T06:39:11.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:11 smithi080 ceph-mon[36206]: pgmap v495: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 MiB/s wr, 1.73k op/s 2024-02-26T06:39:11.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:11 smithi115 ceph-mon[36426]: pgmap v495: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 MiB/s wr, 1.73k op/s 2024-02-26T06:39:11.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:11 smithi049 ceph-mon[31696]: pgmap v495: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 MiB/s wr, 1.73k op/s 2024-02-26T06:39:13.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:13 smithi080 ceph-mon[36206]: pgmap v496: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-02-26T06:39:13.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:13 smithi115 ceph-mon[36426]: pgmap v496: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-02-26T06:39:13.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:13 smithi049 ceph-mon[31696]: pgmap v496: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-02-26T06:39:14.459 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4173 2024-02-26T06:39:14.459 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4174 2024-02-26T06:39:14.459 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 5002 2024-02-26T06:39:14.459 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 548 2024-02-26T06:39:14.459 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 549 2024-02-26T06:39:14.459 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6102 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6104 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6106 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6110 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6111 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6112 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6113 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6117 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 826 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 827 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 828 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 26 06:23 pg_filenode.map 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 PG_VERSION 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-02-26T06:39:14.460 INFO:teuthology.orchestra.run.smithi049.stdout:total 594254 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Feb 26 06:28 . 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 26 06:23 .. 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 112 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 113 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 1247 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1247_fsm 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1247_vm 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 26 06:23 1249 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1249_fsm 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1249_vm 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 26 06:23 1255 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1255_fsm 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1255_vm 2024-02-26T06:39:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 26 06:38 1259 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1259_fsm 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1259_vm 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 26 06:23 13344 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13344_fsm 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13344_vm 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13346 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13348 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13349_fsm 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13349_vm 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13351 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13353 2024-02-26T06:39:14.462 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13354_fsm 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13354_vm 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13356 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13358 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 13359_fsm 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13359_vm 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 13361 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 13363 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1417 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 1418 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442433536 Feb 26 06:38 16396 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Feb 26 06:38 16396_fsm 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:27 16396_vm 2024-02-26T06:39:14.463 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 704512 Feb 26 06:38 16397 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:38 16397_fsm 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:38 16397_vm 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1351680 Feb 26 06:38 16399 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:38 16399_fsm 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:38 16399_vm 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:38 16400 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 26 06:38 16402 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Feb 26 06:38 16404 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 83632128 Feb 26 06:38 16406 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:38 16406_fsm 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:38 16406_vm 2024-02-26T06:39:14.464 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 174 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 175 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2187 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2224 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2328 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2336 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2337 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2579 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2600 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2600_fsm 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2600_vm 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2601_fsm 2024-02-26T06:39:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2601_vm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2602 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2602_fsm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2602_vm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2603 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2603_fsm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2603_vm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2604 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2605 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2605_fsm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2605_vm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2606_fsm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2606_vm 2024-02-26T06:39:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2607 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2607_fsm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2607_vm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 26 06:23 2608 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2608_fsm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2608_vm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 26 06:23 2609 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2609_fsm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2609_vm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2610 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2610_fsm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2610_vm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2611 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2612_fsm 2024-02-26T06:39:14.467 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2612_vm 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2613 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2615_fsm 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2615_vm 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2616 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2616_fsm 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2616_vm 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 26 06:23 2617 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2617_fsm 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2617_vm 2024-02-26T06:39:14.468 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 26 06:23 2618 2024-02-26T06:39:14.469 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2618_fsm 2024-02-26T06:39:14.469 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2618_vm 2024-02-26T06:39:14.469 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Feb 26 06:38 2619 2024-02-26T06:39:14.469 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:26 2619_fsm 2024-02-26T06:39:14.469 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2619_vm 2024-02-26T06:39:14.469 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2620 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2650 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2651 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2652 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2653 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2654 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2655 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2656 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2657 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 26 06:23 2658 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 26 06:23 2659 2024-02-26T06:39:14.470 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2660 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2661 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2662 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2663 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2664 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2665 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2666 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2667 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2668 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2669 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2670 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 26 06:23 2673 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 26 06:23 2674 2024-02-26T06:39:14.471 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 26 06:23 2675 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2678 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2679 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2680 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2681 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2682 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2683 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2684 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2685 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2686 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2687 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2688 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 26 06:23 2689 2024-02-26T06:39:14.472 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 26 06:23 2690 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 26 06:23 2691 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2692 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2693 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:28 2696 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2699 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2701 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2702 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2703 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 2704 2024-02-26T06:39:14.473 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2753 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2753_fsm 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2753_vm 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2754 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2755 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2756 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 2757 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2830 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2831 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2832 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2833 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2834 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2835 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2836 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2837 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 26 06:23 2838 2024-02-26T06:39:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2838_fsm 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2838_vm 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2839 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2840_fsm 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2840_vm 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2841 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2995 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2996 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3079_fsm 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3079_vm 2024-02-26T06:39:14.475 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3080 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3081 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3085 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3118 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3119 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 26 06:23 3164 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3256 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3257 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3258 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3350 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3351 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3379 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3380 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3381 2024-02-26T06:39:14.476 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3394 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3394_fsm 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3394_vm 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3395 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3429 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3430 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3431 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3433 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3439 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3440 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 26 06:23 3455 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 26 06:23 3456 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3456_fsm 2024-02-26T06:39:14.477 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3456_vm 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3466 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3467 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3468 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3501 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3502 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3503 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3534 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541 2024-02-26T06:39:14.478 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3541_fsm 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3541_vm 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3542 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3574 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3575 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3576 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3596 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3597 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3598 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3599 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3600_fsm 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3600_vm 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3601_fsm 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3601_vm 2024-02-26T06:39:14.479 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3602_fsm 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3602_vm 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3603_fsm 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3603_vm 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3604 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3605 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3606 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3607 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3608 2024-02-26T06:39:14.480 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 26 06:23 3609 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3712 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 3764_fsm 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3764_vm 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3766 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 3767 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3997 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4143 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4144 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4145 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4146 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4147 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4148 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4149 2024-02-26T06:39:14.481 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4150 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4151 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4152 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4153 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4154 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4155 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4156 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4157 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4158 2024-02-26T06:39:14.482 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4159 2024-02-26T06:39:14.501 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4160 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4161 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4162 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4163 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4164 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4165 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4166 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4167 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4168 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4169 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4170 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4171 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4172 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4173 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4174 2024-02-26T06:39:14.502 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 5002 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 548 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 549 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6102 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6104 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6106 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6110 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6111 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6112 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6113 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6117 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 826 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 827 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 828 2024-02-26T06:39:14.503 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 26 06:23 pg_filenode.map 2024-02-26T06:39:14.504 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Feb 26 06:23 pg_internal.init 2024-02-26T06:39:14.504 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:23 PG_VERSION 2024-02-26T06:39:14.504 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.504 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-02-26T06:39:14.504 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.504 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.504 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 26 06:23 .. 2024-02-26T06:39:14.504 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.507 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/global: 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:total 560 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Feb 26 06:23 . 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1213 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1213_fsm 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1213_vm 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1214 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1214_fsm 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1214_vm 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 1232 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 1233 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1260 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1260_fsm 2024-02-26T06:39:14.508 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1260_vm 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1261 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1261_fsm 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1261_vm 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1262 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 1262_fsm 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 1262_vm 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2396 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 26 06:23 2396_fsm 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2396_vm 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2397 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2671 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2672 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2676 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2677 2024-02-26T06:39:14.509 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2694 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2695 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2697 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 26 06:23 2698 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2846 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2847 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2964 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2965 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 2966 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 2967 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 3592 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 3593 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4060 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4061 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4175 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4176 2024-02-26T06:39:14.510 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4177 2024-02-26T06:39:14.511 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4178 2024-02-26T06:39:14.511 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4181 2024-02-26T06:39:14.511 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4182 2024-02-26T06:39:14.511 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4183 2024-02-26T06:39:14.511 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4184 2024-02-26T06:39:14.511 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 4185 2024-02-26T06:39:14.512 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 4186 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6000 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6001 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6002 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 6100 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6114 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 6115 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:39 pg_control 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 26 06:23 pg_filenode.map 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Feb 26 06:23 pg_internal.init 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/log: 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:total 2 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:39:14.513 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Feb 26 06:39 postgresql-Mon.log 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:total 1 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:39 . 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 mappings 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Feb 26 06:39 replorigin_checkpoint 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 snapshots 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:39 .. 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:39 .. 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.515 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 . 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 members 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 offsets 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 .. 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:23 0000 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:23 . 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:23 .. 2024-02-26T06:39:14.516 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 26 06:28 0000 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.517 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.537 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.537 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.537 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-02-26T06:39:14.537 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.537 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:total 15 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 26 06:39 . 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Feb 26 06:39 db_0.stat 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Feb 26 06:39 db_13494.stat 2024-02-26T06:39:14.538 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Feb 26 06:39 global.stat 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:39 . 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:total 96 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 26 06:38 . 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 26 06:38 0018 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.539 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-02-26T06:39:14.540 INFO:teuthology.orchestra.run.smithi049.stdout:total 1048576 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Feb 26 06:39 . 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:39 000000010000000000000094 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 000000010000000000000095 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 000000010000000000000096 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 000000010000000000000097 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:34 000000010000000000000098 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:36 000000010000000000000099 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:35 00000001000000000000009A 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:35 00000001000000000000009B 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 00000001000000000000009C 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 00000001000000000000009D 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 00000001000000000000009E 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:35 00000001000000000000009F 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000A0 2024-02-26T06:39:14.541 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000A1 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000A2 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:34 0000000100000000000000A3 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000A4 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000A5 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000A6 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000A7 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000A8 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:34 0000000100000000000000A9 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000AA 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000AB 2024-02-26T06:39:14.542 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000AC 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000AD 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000AE 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000AF 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000B0 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000B1 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000B2 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000B3 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:33 0000000100000000000000B4 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000B5 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:36 0000000100000000000000B6 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:34 0000000100000000000000B7 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000B8 2024-02-26T06:39:14.543 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:36 0000000100000000000000B9 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:35 0000000100000000000000BA 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:32 0000000100000000000000BB 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000BC 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000BD 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000BE 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000BF 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000C0 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000C1 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000C2 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000C3 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000C4 2024-02-26T06:39:14.544 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000C5 2024-02-26T06:39:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000C6 2024-02-26T06:39:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000C7 2024-02-26T06:39:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000C8 2024-02-26T06:39:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000C9 2024-02-26T06:39:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000CA 2024-02-26T06:39:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000CB 2024-02-26T06:39:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:37 0000000100000000000000CC 2024-02-26T06:39:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000CD 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000CE 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000CF 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000D0 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000D1 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000D2 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 26 06:38 0000000100000000000000D3 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 archive_status 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 26 06:23 . 2024-02-26T06:39:14.546 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Feb 26 06:39 .. 2024-02-26T06:39:14.547 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:14.547 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-02-26T06:39:14.547 INFO:teuthology.orchestra.run.smithi049.stdout:total 392 2024-02-26T06:39:14.547 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 26 06:32 . 2024-02-26T06:39:14.547 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 26 06:39 .. 2024-02-26T06:39:14.547 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Feb 26 06:32 0000 2024-02-26T06:39:14.547 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 Feb 26 06:38 0001 2024-02-26T06:39:14.549 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-02-26T06:39:14.561 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:39:14.581 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-26T06:39:14.595 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-02-26T06:39:14.595 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T06:39:14.596 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:39:14.596 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-02-26T06:39:14.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-02-26T06:39:14.891 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.120+0000 7fb975000640 1 -- 172.21.15.49:0/4164706435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9701035d0 msgr2=0x7fb9701039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.120+0000 7fb975000640 1 --2- 172.21.15.49:0/4164706435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9701035d0 0x7fb9701039b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb960009930 tx=0x7fb96002bc20 comp rx=0 tx=0).stop 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.120+0000 7fb975000640 1 -- 172.21.15.49:0/4164706435 shutdown_connections 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.120+0000 7fb975000640 1 --2- 172.21.15.49:0/4164706435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9701086a0 0x7fb970110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.120+0000 7fb975000640 1 --2- 172.21.15.49:0/4164706435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb970103f80 0x7fb970107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.120+0000 7fb975000640 1 --2- 172.21.15.49:0/4164706435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9701035d0 0x7fb9701039b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.121+0000 7fb975000640 1 -- 172.21.15.49:0/4164706435 >> 172.21.15.49:0/4164706435 conn(0x7fb9700fee30 msgr2=0x7fb970101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.121+0000 7fb975000640 1 -- 172.21.15.49:0/4164706435 shutdown_connections 2024-02-26T06:39:15.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.121+0000 7fb975000640 1 -- 172.21.15.49:0/4164706435 wait complete. 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.122+0000 7fb975000640 1 Processor -- start 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.122+0000 7fb975000640 1 -- start start 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.122+0000 7fb975000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9701035d0 0x7fb97019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.122+0000 7fb975000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9701086a0 0x7fb97019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.122+0000 7fb975000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970199800 0x7fb970199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.122+0000 7fb975000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97019a2b0 con 0x7fb970199800 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.122+0000 7fb975000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97019a420 con 0x7fb9701035d0 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.122+0000 7fb975000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97019a590 con 0x7fb9701086a0 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb96e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970199800 0x7fb970199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb96de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9701035d0 0x7fb97019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb967e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9701086a0 0x7fb97019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb96de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9701035d0 0x7fb97019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50320/0 (socket says 172.21.15.49:50320) 2024-02-26T06:39:15.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb96de00640 1 -- 172.21.15.49:0/2860068922 learned_addr learned my addr 172.21.15.49:0/2860068922 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:15.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb967e00640 1 -- 172.21.15.49:0/2860068922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9701035d0 msgr2=0x7fb97019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb967e00640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9701035d0 0x7fb97019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb967e00640 1 -- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970199800 msgr2=0x7fb970199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb967e00640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970199800 0x7fb970199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb967e00640 1 -- 172.21.15.49:0/2860068922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960009560 con 0x7fb9701086a0 2024-02-26T06:39:15.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb96de00640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9701035d0 0x7fb97019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:39:15.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.123+0000 7fb96e800640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970199800 0x7fb970199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:39:15.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.124+0000 7fb967e00640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9701086a0 0x7fb97019fb80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb97006a860 tx=0x7fb95800dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:15.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.124+0000 7fb966a00640 1 -- 172.21.15.49:0/2860068922 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958004020 con 0x7fb9701086a0 2024-02-26T06:39:15.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.124+0000 7fb966a00640 1 -- 172.21.15.49:0/2860068922 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb958004180 con 0x7fb9701086a0 2024-02-26T06:39:15.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.124+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9701a64f0 con 0x7fb9701086a0 2024-02-26T06:39:15.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.124+0000 7fb966a00640 1 -- 172.21.15.49:0/2860068922 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958004a00 con 0x7fb9701086a0 2024-02-26T06:39:15.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.124+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9701a6980 con 0x7fb9701086a0 2024-02-26T06:39:15.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.125+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb970104aa0 con 0x7fb9701086a0 2024-02-26T06:39:15.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.127+0000 7fb966a00640 1 -- 172.21.15.49:0/2860068922 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb9580042f0 con 0x7fb9701086a0 2024-02-26T06:39:15.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.128+0000 7fb966a00640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb9540a22d0 0x7fb9540a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:15.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.129+0000 7fb966a00640 1 -- 172.21.15.49:0/2860068922 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fb9580cdd30 con 0x7fb9701086a0 2024-02-26T06:39:15.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.129+0000 7fb96de00640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb9540a22d0 0x7fb9540a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:15.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.130+0000 7fb96de00640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb9540a22d0 0x7fb9540a4790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb960000c00 tx=0x7fb9600058d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:15.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.133+0000 7fb966a00640 1 -- 172.21.15.49:0/2860068922 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb958096910 con 0x7fb9701086a0 2024-02-26T06:39:15.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.362+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 --> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fb97010b940 con 0x7fb9540a22d0 2024-02-26T06:39:15.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.386+0000 7fb966a00640 1 -- 172.21.15.49:0/2860068922 <== mgr.25384 v2:172.21.15.49:6800/2212175168 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fb97010b940 con 0x7fb9540a22d0 2024-02-26T06:39:15.386 INFO:teuthology.orchestra.run.smithi049.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-26T00:00:00", "created": "2024-02-26T06:23:24", "first": "2024-02-26T06:24:00", "last": "2024-02-26T06:39:00", "last_pruned": "2024-02-26T06:39:01", "created_count": 16, "pruned_count": 10, "active": true} 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb9540a22d0 msgr2=0x7fb9540a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb9540a22d0 0x7fb9540a4790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb960000c00 tx=0x7fb9600058d0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9701086a0 msgr2=0x7fb97019fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9701086a0 0x7fb97019fb80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb97006a860 tx=0x7fb95800dbb0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 shutdown_connections 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb9540a22d0 0x7fb9540a4790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970199800 0x7fb970199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9701086a0 0x7fb97019fb80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 --2- 172.21.15.49:0/2860068922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9701035d0 0x7fb97019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 >> 172.21.15.49:0/2860068922 conn(0x7fb9700fee30 msgr2=0x7fb970101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 shutdown_connections 2024-02-26T06:39:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.390+0000 7fb975000640 1 -- 172.21.15.49:0/2860068922 wait complete. 2024-02-26T06:39:15.409 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-26T06:39:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:15 smithi080 ceph-mon[36206]: pgmap v497: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 11 MiB/s wr, 697 op/s 2024-02-26T06:39:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:15 smithi080 ceph-mon[36206]: osdmap e109: 12 total, 12 up, 12 in 2024-02-26T06:39:15.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:15 smithi115 ceph-mon[36426]: pgmap v497: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 11 MiB/s wr, 697 op/s 2024-02-26T06:39:15.591 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:15 smithi115 ceph-mon[36426]: osdmap e109: 12 total, 12 up, 12 in 2024-02-26T06:39:15.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.637+0000 7f3be8400640 1 -- 172.21.15.49:0/3953878966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3be01086a0 msgr2=0x7f3be0110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.637+0000 7f3be8400640 1 --2- 172.21.15.49:0/3953878966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3be01086a0 0x7f3be0110270 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009890 tx=0x7f3bd802bb70 comp rx=0 tx=0).stop 2024-02-26T06:39:15.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.637+0000 7f3be8400640 1 -- 172.21.15.49:0/3953878966 shutdown_connections 2024-02-26T06:39:15.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.637+0000 7f3be8400640 1 --2- 172.21.15.49:0/3953878966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3be01086a0 0x7f3be0110270 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.637+0000 7f3be8400640 1 --2- 172.21.15.49:0/3953878966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be0103f80 0x7f3be0107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.637+0000 7f3be8400640 1 --2- 172.21.15.49:0/3953878966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3be01035d0 0x7f3be01039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.637+0000 7f3be8400640 1 -- 172.21.15.49:0/3953878966 >> 172.21.15.49:0/3953878966 conn(0x7f3be00fee30 msgr2=0x7f3be0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:15.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.638+0000 7f3be8400640 1 -- 172.21.15.49:0/3953878966 shutdown_connections 2024-02-26T06:39:15.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.638+0000 7f3be8400640 1 -- 172.21.15.49:0/3953878966 wait complete. 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be8400640 1 Processor -- start 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be8400640 1 -- start start 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be8400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be01035d0 0x7f3be019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be8400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3be01086a0 0x7f3be019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be8400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3be0199750 0x7f3be0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be8400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019a220 con 0x7f3be0199750 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be8400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019a390 con 0x7f3be01086a0 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be8400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019a550 con 0x7f3be01035d0 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be6400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3be0199750 0x7f3be0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be5a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be01035d0 0x7f3be019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be5a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be01035d0 0x7f3be019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53556/0 (socket says 172.21.15.49:53556) 2024-02-26T06:39:15.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3be01086a0 0x7f3be019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.639+0000 7f3be5a00640 1 -- 172.21.15.49:0/4155186894 learned_addr learned my addr 172.21.15.49:0/4155186894 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3be5a00640 1 -- 172.21.15.49:0/4155186894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3be01086a0 msgr2=0x7f3be019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3be5a00640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3be01086a0 0x7f3be019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3be5a00640 1 -- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3be0199750 msgr2=0x7f3be0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3be5a00640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3be0199750 0x7f3be0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3be5a00640 1 -- 172.21.15.49:0/4155186894 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd8009490 con 0x7f3be01035d0 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3be5a00640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be01035d0 0x7f3be019f690 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400efb0 tx=0x7f3bd400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3bde000640 1 -- 172.21.15.49:0/4155186894 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400b570 con 0x7f3be01035d0 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be01a64f0 con 0x7f3be01035d0 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3bde000640 1 -- 172.21.15.49:0/4155186894 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bd400bbb0 con 0x7f3be01035d0 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.640+0000 7f3bde000640 1 -- 172.21.15.49:0/4155186894 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4016d20 con 0x7f3be01035d0 2024-02-26T06:39:15.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.641+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be01a69d0 con 0x7f3be01035d0 2024-02-26T06:39:15.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.642+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be0104aa0 con 0x7f3be01035d0 2024-02-26T06:39:15.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.644+0000 7f3bde000640 1 -- 172.21.15.49:0/4155186894 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3bd400a040 con 0x7f3be01035d0 2024-02-26T06:39:15.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.645+0000 7f3bde000640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3bc80a22d0 0x7f3bc80a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:15.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.645+0000 7f3bde000640 1 -- 172.21.15.49:0/4155186894 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f3bd40ce160 con 0x7f3be01035d0 2024-02-26T06:39:15.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.645+0000 7f3be5000640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3bc80a22d0 0x7f3bc80a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:15.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.646+0000 7f3be5000640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3bc80a22d0 0x7f3bc80a4790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3be019afa0 tx=0x7f3bcc005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:15.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.647+0000 7f3bde000640 1 -- 172.21.15.49:0/4155186894 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3bd4096ea0 con 0x7f3be01035d0 2024-02-26T06:39:15.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:15 smithi049 ceph-mon[31696]: pgmap v497: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 11 MiB/s wr, 697 op/s 2024-02-26T06:39:15.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:15 smithi049 ceph-mon[31696]: osdmap e109: 12 total, 12 up, 12 in 2024-02-26T06:39:15.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.871+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 --> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3be010b9a0 con 0x7f3bc80a22d0 2024-02-26T06:39:15.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.886+0000 7f3bde000640 1 -- 172.21.15.49:0/4155186894 <== mgr.25384 v2:172.21.15.49:6800/2212175168 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f3be010b9a0 con 0x7f3bc80a22d0 2024-02-26T06:39:15.886 INFO:teuthology.orchestra.run.smithi049.stdout:/ 1m 6m3h 2024-02-26T06:39:15.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.888+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3bc80a22d0 msgr2=0x7f3bc80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.888+0000 7f3be8400640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3bc80a22d0 0x7f3bc80a4790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3be019afa0 tx=0x7f3bcc005f90 comp rx=0 tx=0).stop 2024-02-26T06:39:15.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.888+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be01035d0 msgr2=0x7f3be019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:15.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.888+0000 7f3be8400640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be01035d0 0x7f3be019f690 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400efb0 tx=0x7f3bd400c800 comp rx=0 tx=0).stop 2024-02-26T06:39:15.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.889+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 shutdown_connections 2024-02-26T06:39:15.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.889+0000 7f3be8400640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3bc80a22d0 0x7f3bc80a4790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.889+0000 7f3be8400640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3be01086a0 0x7f3be019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.889+0000 7f3be8400640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3be0199750 0x7f3be0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.889+0000 7f3be8400640 1 --2- 172.21.15.49:0/4155186894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be01035d0 0x7f3be019f690 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:15.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.889+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 >> 172.21.15.49:0/4155186894 conn(0x7f3be00fee30 msgr2=0x7f3be00ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:15.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.889+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 shutdown_connections 2024-02-26T06:39:15.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:15.889+0000 7f3be8400640 1 -- 172.21.15.49:0/4155186894 wait complete. 2024-02-26T06:39:15.911 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-26T06:39:15.925 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-26T06:39:15.925 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-02-26T06:39:15.926 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-26T06:39:15.942 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:39:16.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 -- 172.21.15.49:0/2495880293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d81080f0 msgr2=0x7fc9d8114600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2495880293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d81080f0 0x7fc9d8114600 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009c10 tx=0x7fc9d002bef0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 -- 172.21.15.49:0/2495880293 shutdown_connections 2024-02-26T06:39:16.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2495880293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d81080f0 0x7fc9d8114600 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2495880293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d8107730 0x7fc9d8107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2495880293 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9d810d740 0x7fc9d810db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 -- 172.21.15.49:0/2495880293 >> 172.21.15.49:0/2495880293 conn(0x7fc9d80756d0 msgr2=0x7fc9d8077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:16.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 -- 172.21.15.49:0/2495880293 shutdown_connections 2024-02-26T06:39:16.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.207+0000 7fc9dce00640 1 -- 172.21.15.49:0/2495880293 wait complete. 2024-02-26T06:39:16.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.209+0000 7fc9dce00640 1 Processor -- start 2024-02-26T06:39:16.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.209+0000 7fc9dce00640 1 -- start start 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9dce00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d8107730 0x7fc9d819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9dce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9d81080f0 0x7fc9d819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9dce00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d810d740 0x7fc9d81a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9dce00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d81a2c40 con 0x7fc9d8107730 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9dce00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d81a2db0 con 0x7fc9d81080f0 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9dce00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d81a2f70 con 0x7fc9d810d740 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d8107730 0x7fc9d819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9d81080f0 0x7fc9d819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d8107730 0x7fc9d819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42362/0 (socket says 172.21.15.49:42362) 2024-02-26T06:39:16.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5e00640 1 -- 172.21.15.49:0/2998844342 learned_addr learned my addr 172.21.15.49:0/2998844342 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d6800640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d810d740 0x7fc9d81a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5400640 1 -- 172.21.15.49:0/2998844342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d810d740 msgr2=0x7fc9d81a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5400640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d810d740 0x7fc9d81a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5400640 1 -- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d8107730 msgr2=0x7fc9d819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5400640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d8107730 0x7fc9d819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.210+0000 7fc9d5400640 1 -- 172.21.15.49:0/2998844342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d0009840 con 0x7fc9d81080f0 2024-02-26T06:39:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.211+0000 7fc9d5400640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9d81080f0 0x7fc9d819e310 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e9f0 tx=0x7fc9c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.211+0000 7fc9cea00640 1 -- 172.21.15.49:0/2998844342 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000ccc0 con 0x7fc9d81080f0 2024-02-26T06:39:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.211+0000 7fc9cea00640 1 -- 172.21.15.49:0/2998844342 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9c000ce20 con 0x7fc9d81080f0 2024-02-26T06:39:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.211+0000 7fc9cea00640 1 -- 172.21.15.49:0/2998844342 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0018920 con 0x7fc9d81080f0 2024-02-26T06:39:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.211+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d81a3250 con 0x7fc9d81080f0 2024-02-26T06:39:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.211+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d81aaad0 con 0x7fc9d81080f0 2024-02-26T06:39:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.212+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d80683c0 con 0x7fc9d81080f0 2024-02-26T06:39:16.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.214+0000 7fc9cea00640 1 -- 172.21.15.49:0/2998844342 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc9c0018a80 con 0x7fc9d81080f0 2024-02-26T06:39:16.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.216+0000 7fc9cea00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc9bc0a2230 0x7fc9bc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:16.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.216+0000 7fc9cea00640 1 -- 172.21.15.49:0/2998844342 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fc9c0014070 con 0x7fc9d81080f0 2024-02-26T06:39:16.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.216+0000 7fc9d5e00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc9bc0a2230 0x7fc9bc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:16.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.217+0000 7fc9d5e00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc9bc0a2230 0x7fc9bc0a46f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80058c0 tx=0x7fc9c8009260 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:16.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.220+0000 7fc9cea00640 1 -- 172.21.15.49:0/2998844342 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9c0052d20 con 0x7fc9d81080f0 2024-02-26T06:39:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.420+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9d810fd30 con 0x7fc9d81080f0 2024-02-26T06:39:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.423+0000 7fc9cea00640 1 -- 172.21.15.49:0/2998844342 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+23176 (secure 0 0 0) 0x7fc9d810fd30 con 0x7fc9d81080f0 2024-02-26T06:39:16.423 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:16.423 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":109,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:39:15.282063+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3617020911":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/3210390195":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2527769498":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/3477045895":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/1431851577":"2024-02-27T06:23:23.289826+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/1431851577":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:39:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.425+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc9bc0a2230 msgr2=0x7fc9bc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.425+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc9bc0a2230 0x7fc9bc0a46f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80058c0 tx=0x7fc9c8009260 comp rx=0 tx=0).stop 2024-02-26T06:39:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.425+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9d81080f0 msgr2=0x7fc9d819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.425+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9d81080f0 0x7fc9d819e310 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e9f0 tx=0x7fc9c000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.426+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 shutdown_connections 2024-02-26T06:39:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.426+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc9bc0a2230 0x7fc9bc0a46f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.426+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9d81080f0 0x7fc9d819e310 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.426+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d810d740 0x7fc9d81a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.426+0000 7fc9dce00640 1 --2- 172.21.15.49:0/2998844342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d8107730 0x7fc9d819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.426+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 >> 172.21.15.49:0/2998844342 conn(0x7fc9d80756d0 msgr2=0x7fc9d8075ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.426+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 shutdown_connections 2024-02-26T06:39:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.426+0000 7fc9dce00640 1 -- 172.21.15.49:0/2998844342 wait complete. 2024-02-26T06:39:16.446 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:39:16.446 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:39:16.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:16 smithi080 ceph-mon[36206]: from='client.27718 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:39:16.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:16 smithi115 ceph-mon[36426]: from='client.27718 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:39:16.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:16 smithi049 ceph-mon[31696]: from='client.27718 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:39:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 -- 172.21.15.49:0/2978879899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534108540 msgr2=0x7f2534110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 --2- 172.21.15.49:0/2978879899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534108540 0x7f2534110110 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f252c009c10 tx=0x7f252c02bef0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 -- 172.21.15.49:0/2978879899 shutdown_connections 2024-02-26T06:39:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 --2- 172.21.15.49:0/2978879899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534108540 0x7f2534110110 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 --2- 172.21.15.49:0/2978879899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2534103e20 0x7f2534107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 --2- 172.21.15.49:0/2978879899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534103470 0x7f2534103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 -- 172.21.15.49:0/2978879899 >> 172.21.15.49:0/2978879899 conn(0x7f25340fec90 msgr2=0x7f25341010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 -- 172.21.15.49:0/2978879899 shutdown_connections 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.681+0000 7f2538c00640 1 -- 172.21.15.49:0/2978879899 wait complete. 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.682+0000 7f2538c00640 1 Processor -- start 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.682+0000 7f2538c00640 1 -- start start 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f2538c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534103470 0x7f253419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f2538c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2534108540 0x7f253419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f2538c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25341997b0 0x7f2534199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f2538c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253419a280 con 0x7f25341997b0 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f2538c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253419a3f0 con 0x7f2534103470 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f2538c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253419a5b0 con 0x7f2534108540 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f252be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2534108540 0x7f253419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f2533e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25341997b0 0x7f2534199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f252be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2534108540 0x7f253419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:53594/0 (socket says 172.21.15.49:53594) 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f252be00640 1 -- 172.21.15.49:0/1138430925 learned_addr learned my addr 172.21.15.49:0/1138430925 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:16.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.683+0000 7f2533400640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534103470 0x7f253419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2533e00640 1 -- 172.21.15.49:0/1138430925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2534108540 msgr2=0x7f253419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2533e00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2534108540 0x7f253419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2533e00640 1 -- 172.21.15.49:0/1138430925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534103470 msgr2=0x7f253419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2533e00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534103470 0x7f253419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2533e00640 1 -- 172.21.15.49:0/1138430925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524009560 con 0x7f25341997b0 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2533400640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534103470 0x7f253419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f252be00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2534108540 0x7f253419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2533e00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25341997b0 0x7f2534199c30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f252c02c430 tx=0x7f252c02cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2530c00640 1 -- 172.21.15.49:0/1138430925 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c003970 con 0x7f25341997b0 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252c009840 con 0x7f25341997b0 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2530c00640 1 -- 172.21.15.49:0/1138430925 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f252c004040 con 0x7f25341997b0 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.684+0000 7f2530c00640 1 -- 172.21.15.49:0/1138430925 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c03d760 con 0x7f25341997b0 2024-02-26T06:39:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.685+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25341a6790 con 0x7f25341997b0 2024-02-26T06:39:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.686+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2534104940 con 0x7f25341997b0 2024-02-26T06:39:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.688+0000 7f2530c00640 1 -- 172.21.15.49:0/1138430925 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f252c0041b0 con 0x7f25341997b0 2024-02-26T06:39:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.689+0000 7f2530c00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f25180a2230 0x7f25180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.689+0000 7f2530c00640 1 -- 172.21.15.49:0/1138430925 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f252c0f04b0 con 0x7f25341997b0 2024-02-26T06:39:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.689+0000 7f2533400640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f25180a2230 0x7f25180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.690+0000 7f2533400640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f25180a2230 0x7f25180a46f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2524009530 tx=0x7f2524009260 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.694+0000 7f2530c00640 1 -- 172.21.15.49:0/1138430925 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f252c0b9280 con 0x7f25341997b0 2024-02-26T06:39:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.896+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f253410b7d0 con 0x7f25341997b0 2024-02-26T06:39:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.897+0000 7f2530c00640 1 -- 172.21.15.49:0/1138430925 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f252c0bc940 con 0x7f25341997b0 2024-02-26T06:39:16.897 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:39:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.899+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f25180a2230 msgr2=0x7f25180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.899+0000 7f2538c00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f25180a2230 0x7f25180a46f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2524009530 tx=0x7f2524009260 comp rx=0 tx=0).stop 2024-02-26T06:39:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25341997b0 msgr2=0x7f2534199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25341997b0 0x7f2534199c30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f252c02c430 tx=0x7f252c02cb70 comp rx=0 tx=0).stop 2024-02-26T06:39:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 shutdown_connections 2024-02-26T06:39:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f25180a2230 0x7f25180a46f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25341997b0 0x7f2534199c30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2534108540 0x7f253419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 --2- 172.21.15.49:0/1138430925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2534103470 0x7f253419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 >> 172.21.15.49:0/1138430925 conn(0x7f25340fec90 msgr2=0x7f2534100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 shutdown_connections 2024-02-26T06:39:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:16.900+0000 7f2538c00640 1 -- 172.21.15.49:0/1138430925 wait complete. 2024-02-26T06:39:16.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:39:17.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 -- 172.21.15.49:0/4041582655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb89c108520 msgr2=0x7fb89c1100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 --2- 172.21.15.49:0/4041582655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb89c108520 0x7fb89c1100f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb894009be0 tx=0x7fb89402bef0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 -- 172.21.15.49:0/4041582655 shutdown_connections 2024-02-26T06:39:17.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 --2- 172.21.15.49:0/4041582655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb89c103e00 0x7fb89c107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 --2- 172.21.15.49:0/4041582655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb89c108520 0x7fb89c1100f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 --2- 172.21.15.49:0/4041582655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89c103450 0x7fb89c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 -- 172.21.15.49:0/4041582655 >> 172.21.15.49:0/4041582655 conn(0x7fb89c0fecd0 msgr2=0x7fb89c1010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 -- 172.21.15.49:0/4041582655 shutdown_connections 2024-02-26T06:39:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.164+0000 7fb8a2800640 1 -- 172.21.15.49:0/4041582655 wait complete. 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.166+0000 7fb8a2800640 1 Processor -- start 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.166+0000 7fb8a2800640 1 -- start start 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.166+0000 7fb8a2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb89c103450 0x7fb89c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.166+0000 7fb8a2800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89c108520 0x7fb89c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.166+0000 7fb8a2800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb89c199700 0x7fb89c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.166+0000 7fb8a2800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c19a1d0 con 0x7fb89c108520 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.166+0000 7fb8a2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c19a340 con 0x7fb89c103450 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.166+0000 7fb8a2800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c19a500 con 0x7fb89c199700 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89c108520 0x7fb89c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89c108520 0x7fb89c19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42394/0 (socket says 172.21.15.49:42394) 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a0a00640 1 -- 172.21.15.49:0/1879161990 learned_addr learned my addr 172.21.15.49:0/1879161990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a1400640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb89c103450 0x7fb89c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a1e00640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb89c199700 0x7fb89c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a0a00640 1 -- 172.21.15.49:0/1879161990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb89c199700 msgr2=0x7fb89c199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a0a00640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb89c199700 0x7fb89c199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a0a00640 1 -- 172.21.15.49:0/1879161990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb89c103450 msgr2=0x7fb89c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a0a00640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb89c103450 0x7fb89c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.167+0000 7fb8a0a00640 1 -- 172.21.15.49:0/1879161990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb894009840 con 0x7fb89c108520 2024-02-26T06:39:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.168+0000 7fb8a0a00640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89c108520 0x7fb89c19fb30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00e9f0 tx=0x7fb88c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.168+0000 7fb89a000640 1 -- 172.21.15.49:0/1879161990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0096c0 con 0x7fb89c108520 2024-02-26T06:39:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.168+0000 7fb89a000640 1 -- 172.21.15.49:0/1879161990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb88c005500 con 0x7fb89c108520 2024-02-26T06:39:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.168+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c1a6450 con 0x7fb89c108520 2024-02-26T06:39:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.168+0000 7fb89a000640 1 -- 172.21.15.49:0/1879161990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c010430 con 0x7fb89c108520 2024-02-26T06:39:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.168+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c1a6930 con 0x7fb89c108520 2024-02-26T06:39:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.169+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c104920 con 0x7fb89c108520 2024-02-26T06:39:17.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.173+0000 7fb89a000640 1 -- 172.21.15.49:0/1879161990 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb88c005000 con 0x7fb89c108520 2024-02-26T06:39:17.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.174+0000 7fb89a000640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8840a22b0 0x7fb8840a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:17.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.174+0000 7fb89a000640 1 -- 172.21.15.49:0/1879161990 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fb88c014070 con 0x7fb89c108520 2024-02-26T06:39:17.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.175+0000 7fb8a1400640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8840a22b0 0x7fb8840a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:17.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.175+0000 7fb8a1400640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8840a22b0 0x7fb8840a4770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb890005fd0 tx=0x7fb890009260 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:17.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.178+0000 7fb89a000640 1 -- 172.21.15.49:0/1879161990 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb88c0971a0 con 0x7fb89c108520 2024-02-26T06:39:17.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.381+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb89c10b750 con 0x7fb89c108520 2024-02-26T06:39:17.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.382+0000 7fb89a000640 1 -- 172.21.15.49:0/1879161990 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb88c09a860 con 0x7fb89c108520 2024-02-26T06:39:17.382 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:39:17.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.384+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8840a22b0 msgr2=0x7fb8840a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.384+0000 7fb8a2800640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8840a22b0 0x7fb8840a4770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb890005fd0 tx=0x7fb890009260 comp rx=0 tx=0).stop 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.384+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89c108520 msgr2=0x7fb89c19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.384+0000 7fb8a2800640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89c108520 0x7fb89c19fb30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00e9f0 tx=0x7fb88c00edf0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.385+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 shutdown_connections 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.385+0000 7fb8a2800640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb8840a22b0 0x7fb8840a4770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.385+0000 7fb8a2800640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb89c199700 0x7fb89c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.385+0000 7fb8a2800640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb89c108520 0x7fb89c19fb30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.385+0000 7fb8a2800640 1 --2- 172.21.15.49:0/1879161990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb89c103450 0x7fb89c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.385+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 >> 172.21.15.49:0/1879161990 conn(0x7fb89c0fecd0 msgr2=0x7fb89c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.385+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 shutdown_connections 2024-02-26T06:39:17.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.385+0000 7fb8a2800640 1 -- 172.21.15.49:0/1879161990 wait complete. 2024-02-26T06:39:17.403 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:39:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:17 smithi080 ceph-mon[36206]: pgmap v499: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 150 KiB/s wr, 45 op/s 2024-02-26T06:39:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:17 smithi080 ceph-mon[36206]: from='client.27724 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:39:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:17 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2998844342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:39:17.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:17 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1138430925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:39:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:17 smithi115 ceph-mon[36426]: pgmap v499: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 150 KiB/s wr, 45 op/s 2024-02-26T06:39:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:17 smithi115 ceph-mon[36426]: from='client.27724 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:39:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:17 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2998844342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:39:17.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:17 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1138430925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:39:17.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 -- 172.21.15.49:0/3439103079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc103e20 msgr2=0x7f5afc107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 --2- 172.21.15.49:0/3439103079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc103e20 0x7f5afc107e60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009890 tx=0x7f5ae802bb70 comp rx=0 tx=0).stop 2024-02-26T06:39:17.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 -- 172.21.15.49:0/3439103079 shutdown_connections 2024-02-26T06:39:17.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 --2- 172.21.15.49:0/3439103079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc108540 0x7f5afc110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 --2- 172.21.15.49:0/3439103079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc103e20 0x7f5afc107e60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 --2- 172.21.15.49:0/3439103079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5afc103470 0x7f5afc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 -- 172.21.15.49:0/3439103079 >> 172.21.15.49:0/3439103079 conn(0x7f5afc0fec90 msgr2=0x7f5afc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:17.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 -- 172.21.15.49:0/3439103079 shutdown_connections 2024-02-26T06:39:17.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.636+0000 7f5b01400640 1 -- 172.21.15.49:0/3439103079 wait complete. 2024-02-26T06:39:17.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.637+0000 7f5b01400640 1 Processor -- start 2024-02-26T06:39:17.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.637+0000 7f5b01400640 1 -- start start 2024-02-26T06:39:17.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.637+0000 7f5b01400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc103470 0x7f5afc19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:17.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.637+0000 7f5b01400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5afc108540 0x7f5afc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.637+0000 7f5b01400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc199720 0x7f5afc199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.637+0000 7f5b01400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc19a1f0 con 0x7f5afc199720 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.637+0000 7f5b01400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc19a360 con 0x7f5afc103470 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.637+0000 7f5b01400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc19a520 con 0x7f5afc108540 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5b00a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc199720 0x7f5afc199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc103470 0x7f5afc19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc103470 0x7f5afc19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:50392/0 (socket says 172.21.15.49:50392) 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afbe00640 1 -- 172.21.15.49:0/3775420711 learned_addr learned my addr 172.21.15.49:0/3775420711 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afb400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5afc108540 0x7f5afc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afbe00640 1 -- 172.21.15.49:0/3775420711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5afc108540 msgr2=0x7f5afc19fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afbe00640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5afc108540 0x7f5afc19fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afbe00640 1 -- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc199720 msgr2=0x7f5afc199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afbe00640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc199720 0x7f5afc199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5afbe00640 1 -- 172.21.15.49:0/3775420711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8009490 con 0x7f5afc103470 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.638+0000 7f5b00a00640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc199720 0x7f5afc199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.639+0000 7f5afbe00640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc103470 0x7f5afc19f660 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c420 tx=0x7f5aec00c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.639+0000 7f5af8c00640 1 -- 172.21.15.49:0/3775420711 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec009b50 con 0x7f5afc103470 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.639+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc1a64c0 con 0x7f5afc103470 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.639+0000 7f5af8c00640 1 -- 172.21.15.49:0/3775420711 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5aec003970 con 0x7f5afc103470 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.639+0000 7f5af8c00640 1 -- 172.21.15.49:0/3775420711 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00b630 con 0x7f5afc103470 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.639+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc1a69a0 con 0x7f5afc103470 2024-02-26T06:39:17.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.640+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc104940 con 0x7f5afc103470 2024-02-26T06:39:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.642+0000 7f5af8c00640 1 -- 172.21.15.49:0/3775420711 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5aec00a040 con 0x7f5afc103470 2024-02-26T06:39:17.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.644+0000 7f5af8c00640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5ae00a22b0 0x7f5ae00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:17.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.644+0000 7f5af8c00640 1 -- 172.21.15.49:0/3775420711 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f5aec0cdff0 con 0x7f5afc103470 2024-02-26T06:39:17.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.644+0000 7f5afb400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5ae00a22b0 0x7f5ae00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:17.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.645+0000 7f5afb400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5ae00a22b0 0x7f5ae00a4770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009860 tx=0x7f5ae80056d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:17.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.648+0000 7f5af8c00640 1 -- 172.21.15.49:0/3775420711 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5aec101d20 con 0x7f5afc103470 2024-02-26T06:39:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:17 smithi049 ceph-mon[31696]: pgmap v499: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 150 KiB/s wr, 45 op/s 2024-02-26T06:39:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:17 smithi049 ceph-mon[31696]: from='client.27724 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:39:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:17 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2998844342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:39:17.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:17 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1138430925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:39:17.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.850+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5afc10b770 con 0x7f5afc103470 2024-02-26T06:39:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.851+0000 7f5af8c00640 1 -- 172.21.15.49:0/3775420711 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5aec096c40 con 0x7f5afc103470 2024-02-26T06:39:17.851 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.853+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5ae00a22b0 msgr2=0x7f5ae00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.853+0000 7f5b01400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5ae00a22b0 0x7f5ae00a4770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009860 tx=0x7f5ae80056d0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.853+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc103470 msgr2=0x7f5afc19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.853+0000 7f5b01400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc103470 0x7f5afc19f660 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c420 tx=0x7f5aec00c8e0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.854+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 shutdown_connections 2024-02-26T06:39:17.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.854+0000 7f5b01400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f5ae00a22b0 0x7f5ae00a4770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.854+0000 7f5b01400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5afc199720 0x7f5afc199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.854+0000 7f5b01400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5afc108540 0x7f5afc19fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.854+0000 7f5b01400640 1 --2- 172.21.15.49:0/3775420711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc103470 0x7f5afc19f660 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:17.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.854+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 >> 172.21.15.49:0/3775420711 conn(0x7f5afc0fec90 msgr2=0x7f5afc0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:17.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.854+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 shutdown_connections 2024-02-26T06:39:17.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:17.854+0000 7f5b01400640 1 -- 172.21.15.49:0/3775420711 wait complete. 2024-02-26T06:39:17.872 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 -- 172.21.15.49:0/4147527960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1274101d40 msgr2=0x7f1274105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 --2- 172.21.15.49:0/4147527960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1274101d40 0x7f1274105d80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1260009860 tx=0x7f126002bb70 comp rx=0 tx=0).stop 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 -- 172.21.15.49:0/4147527960 shutdown_connections 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 --2- 172.21.15.49:0/4147527960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1274106460 0x7f127410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 --2- 172.21.15.49:0/4147527960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1274101d40 0x7f1274105d80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 --2- 172.21.15.49:0/4147527960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1274101390 0x7f1274101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 -- 172.21.15.49:0/4147527960 >> 172.21.15.49:0/4147527960 conn(0x7f12740785d0 msgr2=0x7f12740789e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 -- 172.21.15.49:0/4147527960 shutdown_connections 2024-02-26T06:39:18.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.103+0000 7f1279c00640 1 -- 172.21.15.49:0/4147527960 wait complete. 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.104+0000 7f1279c00640 1 Processor -- start 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.104+0000 7f1279c00640 1 -- start start 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.104+0000 7f1279c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1274101390 0x7f127419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1279c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1274106460 0x7f127419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1279c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12741975e0 0x7f1274197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1279c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1274197fd0 con 0x7f12741975e0 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1279c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1274198140 con 0x7f1274106460 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1279c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1274198300 con 0x7f1274101390 2024-02-26T06:39:18.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1273400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12741975e0 0x7f1274197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1273400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12741975e0 0x7f1274197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42428/0 (socket says 172.21.15.49:42428) 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1273400640 1 -- 172.21.15.49:0/942796285 learned_addr learned my addr 172.21.15.49:0/942796285 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1272000640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1274106460 0x7f127419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1272a00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1274101390 0x7f127419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1273400640 1 -- 172.21.15.49:0/942796285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1274101390 msgr2=0x7f127419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1273400640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1274101390 0x7f127419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1273400640 1 -- 172.21.15.49:0/942796285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1274106460 msgr2=0x7f127419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1273400640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1274106460 0x7f127419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:18.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.105+0000 7f1273400640 1 -- 172.21.15.49:0/942796285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c009900 con 0x7f12741975e0 2024-02-26T06:39:18.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.106+0000 7f1273400640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12741975e0 0x7f1274197a90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f126c00efc0 tx=0x7f126c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:18.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.106+0000 7f126b400640 1 -- 172.21.15.49:0/942796285 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c003970 con 0x7f12741975e0 2024-02-26T06:39:18.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.106+0000 7f126b400640 1 -- 172.21.15.49:0/942796285 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f126c004020 con 0x7f12741975e0 2024-02-26T06:39:18.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.106+0000 7f126b400640 1 -- 172.21.15.49:0/942796285 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c00b9b0 con 0x7f12741975e0 2024-02-26T06:39:18.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.106+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1260009490 con 0x7f12741975e0 2024-02-26T06:39:18.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.106+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12741a4640 con 0x7f12741975e0 2024-02-26T06:39:18.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.107+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1274068430 con 0x7f12741975e0 2024-02-26T06:39:18.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.110+0000 7f126b400640 1 -- 172.21.15.49:0/942796285 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f126c00a040 con 0x7f12741975e0 2024-02-26T06:39:18.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.111+0000 7f126b400640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f12580a22b0 0x7f12580a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:18.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.111+0000 7f126b400640 1 -- 172.21.15.49:0/942796285 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f126c0cdff0 con 0x7f12741975e0 2024-02-26T06:39:18.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.111+0000 7f1272a00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f12580a22b0 0x7f12580a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:18.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.112+0000 7f1272a00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f12580a22b0 0x7f12580a4770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1264009bd0 tx=0x7f1264005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:18.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.115+0000 7f126b400640 1 -- 172.21.15.49:0/942796285 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f126c096c60 con 0x7f12741975e0 2024-02-26T06:39:18.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.368+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f1274109690 con 0x7f12741975e0 2024-02-26T06:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.388+0000 7f126b400640 1 -- 172.21.15.49:0/942796285 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) v1 ==== 81+0+736942 (secure 0 0 0) 0x7f126c004190 con 0x7f12741975e0 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 27, 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: "active_gid": 25384, 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: "active_name": "x", 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addrs": { 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-02-26T06:39:18.392 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2212175168 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2212175168 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addr": "172.21.15.49:6801/2212175168", 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "active_change": "2024-02-26T06:23:23.290316+0000", 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "active_mgr_features": 4540701547738038271, 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "available": true, 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "standbys": [ 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.393 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 25375, 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "z", 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540701547738038271, 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:18.394 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.395 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.396 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.398 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.399 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.400 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.401 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-02-26T06:39:18.402 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-02-26T06:39:18.403 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.404 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.407 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.408 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-02-26T06:39:18.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-02-26T06:39:18.410 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.412 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.413 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-02-26T06:39:18.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.415 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.417 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.419 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.421 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.422 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.423 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.424 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.425 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.427 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.429 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.430 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.431 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-02-26T06:39:18.432 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.433 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T06:39:18.434 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.435 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.436 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.438 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T06:39:18.439 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.440 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.441 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.442 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.443 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.444 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.449 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.450 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.451 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.453 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.454 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.455 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.456 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T06:39:18.457 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.459 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.460 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.461 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.462 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-02-26T06:39:18.464 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.466 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.467 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.468 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.469 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.471 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.472 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.473 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.473 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.473 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.473 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.473 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.473 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.474 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.475 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-02-26T06:39:18.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-02-26T06:39:18.478 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.479 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-02-26T06:39:18.480 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.481 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.483 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.484 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.485 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.486 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-02-26T06:39:18.487 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-02-26T06:39:18.488 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-02-26T06:39:18.489 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-02-26T06:39:18.490 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-02-26T06:39:18.491 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-02-26T06:39:18.492 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-02-26T06:39:18.493 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-02-26T06:39:18.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.495 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.496 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.497 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.498 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.499 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.500 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.501 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T06:39:18.502 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.505 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.506 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-02-26T06:39:18.507 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T06:39:18.508 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.514 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T06:39:18.516 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T06:39:18.517 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T06:39:18.518 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.519 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.520 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-02-26T06:39:18.521 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.523 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.524 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.526 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.527 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.528 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.529 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.530 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.532 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.533 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.534 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.535 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.536 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.539 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.540 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T06:39:18.541 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T06:39:18.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.543 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T06:39:18.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.547 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.548 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.549 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-02-26T06:39:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.553 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.554 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-02-26T06:39:18.554 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-02-26T06:39:18.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.554 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.554 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.560 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.560 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.560 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.560 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.560 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.560 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.560 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.561 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.562 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.563 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.564 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-02-26T06:39:18.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.566 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:18.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.569 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.572 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.573 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.574 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.576 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.577 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.580 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-02-26T06:39:18.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.584 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-02-26T06:39:18.588 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-02-26T06:39:18.588 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.588 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.588 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.588 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-02-26T06:39:18.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.590 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.592 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-02-26T06:39:18.593 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.594 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.597 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.598 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T06:39:18.603 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.605 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.607 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.608 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.612 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.613 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.614 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.616 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-02-26T06:39:18.617 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.618 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.619 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.620 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.620 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.620 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.620 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.620 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.620 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.620 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.621 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.622 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.622 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.622 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.623 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.625 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.626 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.630 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.631 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.632 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.632 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.632 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.632 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.632 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.633 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.638 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.640 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.641 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.643 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.644 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.645 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-02-26T06:39:18.646 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.647 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.649 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-02-26T06:39:18.650 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.651 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.652 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.653 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.654 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.656 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.657 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.658 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.659 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.661 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.662 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.663 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.665 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.666 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.667 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.669 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.671 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.672 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.673 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.674 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.676 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-02-26T06:39:18.677 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-02-26T06:39:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-02-26T06:39:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-02-26T06:39:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "drive_group_interval": { 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "drive_group_interval", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300.0", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-02-26T06:39:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-02-26T06:39:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.760 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.763 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.765 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-02-26T06:39:18.768 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.769 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.770 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.771 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.772 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-02-26T06:39:18.773 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.774 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.775 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.776 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.777 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.778 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-02-26T06:39:18.779 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.779 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.779 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.779 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-02-26T06:39:18.779 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-02-26T06:39:18.779 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.779 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.780 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-02-26T06:39:18.781 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.782 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.783 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.785 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.786 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-02-26T06:39:18.787 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.788 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.789 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.790 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.791 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.793 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.794 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.795 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.798 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.800 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.801 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.803 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.804 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.805 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T06:39:18.807 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.809 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-02-26T06:39:18.811 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.812 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.813 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.814 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.814 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.814 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.814 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.814 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-02-26T06:39:18.814 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-02-26T06:39:18.814 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.815 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.816 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.817 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.818 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.819 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-02-26T06:39:18.820 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.821 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.822 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.825 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.826 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.827 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-02-26T06:39:18.828 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.829 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-02-26T06:39:18.830 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.831 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.832 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.833 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.834 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-02-26T06:39:18.835 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.836 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.837 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.838 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.839 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.840 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-02-26T06:39:18.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.842 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.843 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-02-26T06:39:18.844 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.848 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.849 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.850 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.851 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.853 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.854 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.855 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.856 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 25381, 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "y", 2024-02-26T06:39:18.857 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540701547738038271, 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.858 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.860 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.862 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-02-26T06:39:18.866 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.867 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.869 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-02-26T06:39:18.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.871 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-02-26T06:39:18.872 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.873 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.874 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-02-26T06:39:18.875 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.876 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.877 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-02-26T06:39:18.878 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T06:39:18.879 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.880 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.881 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.883 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.884 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.885 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T06:39:18.886 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-02-26T06:39:18.887 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.888 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:18.889 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.890 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.891 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.892 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-02-26T06:39:18.894 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.895 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T06:39:18.896 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.897 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.899 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.900 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.901 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.902 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-02-26T06:39:18.903 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.904 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T06:39:18.906 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.907 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.908 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.909 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.910 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T06:39:18.911 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T06:39:18.912 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.913 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.914 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.915 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.916 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.917 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.918 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.919 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T06:39:18.920 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.921 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.923 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.924 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:18.925 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-02-26T06:39:18.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.927 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-02-26T06:39:18.928 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.930 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.933 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.934 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.935 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.936 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.938 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-02-26T06:39:18.939 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-02-26T06:39:18.940 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.941 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.942 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.943 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-02-26T06:39:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-02-26T06:39:18.945 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-02-26T06:39:18.946 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-02-26T06:39:18.949 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.950 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.950 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.950 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-02-26T06:39:18.950 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.950 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.950 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.952 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-02-26T06:39:18.952 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-02-26T06:39:18.952 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.952 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.952 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.952 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.952 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-02-26T06:39:18.953 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.954 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.955 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.956 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-02-26T06:39:18.957 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T06:39:18.958 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.959 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.960 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:18.961 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.962 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.963 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.964 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T06:39:18.965 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T06:39:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T06:39:18.967 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T06:39:18.968 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.969 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.970 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.971 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T06:39:18.972 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.973 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.974 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.975 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.976 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.978 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.979 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.980 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.981 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.982 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.983 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.984 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.984 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.984 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.984 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.984 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.984 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.984 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.985 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:18.986 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T06:39:18.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T06:39:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T06:39:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T06:39:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T06:39:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.000 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-02-26T06:39:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-02-26T06:39:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-02-26T06:39:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-02-26T06:39:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-02-26T06:39:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-02-26T06:39:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T06:39:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-02-26T06:39:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-02-26T06:39:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-02-26T06:39:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-02-26T06:39:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-02-26T06:39:19.081 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.082 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.083 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.084 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.085 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.086 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.087 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.092 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.092 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.092 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.092 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.092 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.092 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.095 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T06:39:19.096 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.096 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.096 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.096 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.097 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.100 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-02-26T06:39:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-02-26T06:39:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-02-26T06:39:19.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.114 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.115 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.118 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.119 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.121 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.122 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.125 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-02-26T06:39:19.126 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.127 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.128 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.128 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.128 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.128 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.129 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.131 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.132 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.132 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.132 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.132 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.132 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.132 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.132 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.133 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-02-26T06:39:19.134 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.135 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.136 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T06:39:19.136 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.136 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.136 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.136 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.136 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.136 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.137 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.139 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.140 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.140 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.140 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.140 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.140 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.141 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.142 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.144 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.150 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.150 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.150 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.150 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.150 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.150 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.150 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.151 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.153 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.154 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.155 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.156 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.157 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.158 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.159 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.160 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.161 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-02-26T06:39:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-02-26T06:39:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-02-26T06:39:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T06:39:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.163 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.167 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.168 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.170 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.171 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.173 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.175 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.176 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-02-26T06:39:19.177 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.179 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.180 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.181 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.182 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-02-26T06:39:19.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T06:39:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-02-26T06:39:19.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.186 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.187 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-02-26T06:39:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.200 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.201 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.203 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-02-26T06:39:19.204 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-02-26T06:39:19.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.204 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.204 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.204 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.204 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.204 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.204 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.205 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.207 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.208 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.209 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.210 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.211 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.213 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.213 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.213 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.213 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.213 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.213 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:19.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.215 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.215 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.215 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.403+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f12580a22b0 msgr2=0x7f12580a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.403+0000 7f1279c00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f12580a22b0 0x7f12580a4770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1264009bd0 tx=0x7f1264005f90 comp rx=0 tx=0).stop 2024-02-26T06:39:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.403+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12741975e0 msgr2=0x7f1274197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.403+0000 7f1279c00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12741975e0 0x7f1274197a90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f126c00efc0 tx=0x7f126c00c800 comp rx=0 tx=0).stop 2024-02-26T06:39:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.404+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 shutdown_connections 2024-02-26T06:39:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.404+0000 7f1279c00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f12580a22b0 0x7f12580a4770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.404+0000 7f1279c00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1274106460 0x7f127419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.404+0000 7f1279c00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12741975e0 0x7f1274197a90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.404+0000 7f1279c00640 1 --2- 172.21.15.49:0/942796285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1274101390 0x7f127419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.404+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 >> 172.21.15.49:0/942796285 conn(0x7f12740785d0 msgr2=0x7f12740ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.404+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 shutdown_connections 2024-02-26T06:39:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:39:18.404+0000 7f1279c00640 1 -- 172.21.15.49:0/942796285 wait complete. 2024-02-26T06:39:19.218 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:18 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1879161990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:39:19.218 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:18 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3775420711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:39:19.218 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:18 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1879161990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:39:19.218 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:18 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3775420711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:39:19.220 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.220 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.220 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.220 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.220 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.220 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.221 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.224 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.226 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.227 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi049.stdout: "drive_group_interval": { 2024-02-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "drive_group_interval", 2024-02-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300.0", 2024-02-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.231 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.231 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.231 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.231 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.231 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.231 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.231 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-02-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.239 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.241 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.242 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.243 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.244 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.246 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.248 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-02-26T06:39:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T06:39:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-02-26T06:39:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-02-26T06:39:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T06:39:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-02-26T06:39:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-02-26T06:39:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-02-26T06:39:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-02-26T06:39:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-02-26T06:39:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-02-26T06:39:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T06:39:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-02-26T06:39:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-02-26T06:39:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-02-26T06:39:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-02-26T06:39:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-02-26T06:39:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-02-26T06:39:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-02-26T06:39:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-02-26T06:39:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-02-26T06:39:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-02-26T06:39:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-02-26T06:39:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.331 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.331 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.331 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.331 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.331 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.331 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.332 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.332 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.332 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.333 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-02-26T06:39:19.333 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-02-26T06:39:19.333 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.333 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.333 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.333 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.334 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.334 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.334 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.334 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.334 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.334 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.335 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.335 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-02-26T06:39:19.335 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-02-26T06:39:19.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.335 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.336 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.336 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.336 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.336 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.337 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.337 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.337 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.337 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.337 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-02-26T06:39:19.338 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-02-26T06:39:19.338 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.338 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.339 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.339 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T06:39:19.339 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.339 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.339 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.340 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.340 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.340 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.340 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.341 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.341 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.342 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-02-26T06:39:19.342 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.342 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.342 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.342 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.343 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.343 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.344 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.344 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.344 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.344 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.345 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.345 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.345 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.345 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.346 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.346 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.346 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.346 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.346 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.346 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.347 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.347 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.347 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.348 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.348 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.348 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.348 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.348 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.349 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.349 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.349 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.349 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.349 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.350 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.350 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.350 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.350 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.351 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.351 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.351 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.351 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.351 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.352 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.352 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.352 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.352 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.353 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.353 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.353 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.353 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.354 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.354 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.354 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.355 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.355 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.355 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.355 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.355 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.355 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-02-26T06:39:19.356 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.357 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.358 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.359 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.360 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.361 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.362 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.363 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.364 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-02-26T06:39:19.365 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-02-26T06:39:19.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.369 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.370 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.371 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.372 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.373 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.375 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.376 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: "modules": [ 2024-02-26T06:39:19.377 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard", 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "iostat", 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "nfs", 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "restful", 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_schedule" 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.378 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T06:39:19.379 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.380 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.381 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.382 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.383 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-02-26T06:39:19.385 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.386 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-02-26T06:39:19.387 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-02-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.389 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-02-26T06:39:19.390 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-02-26T06:39:19.391 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.392 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-02-26T06:39:19.393 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:39:19.396 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.398 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-02-26T06:39:19.399 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-02-26T06:39:19.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-02-26T06:39:19.401 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.403 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.404 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.405 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.407 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.410 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.411 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.412 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.413 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-02-26T06:39:19.414 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T06:39:19.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-02-26T06:39:19.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.418 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-02-26T06:39:19.418 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.418 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.418 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T06:39:19.418 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.418 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.418 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.419 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.420 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.420 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T06:39:19.420 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T06:39:19.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.420 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.420 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-02-26T06:39:19.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.422 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.423 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T06:39:19.424 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T06:39:19.425 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T06:39:19.426 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:19.428 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-02-26T06:39:19.429 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.430 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.431 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.432 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-02-26T06:39:19.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-02-26T06:39:19.434 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-02-26T06:39:19.435 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.436 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-02-26T06:39:19.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.438 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.439 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.440 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.442 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.443 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.444 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.445 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.448 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.450 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:19.451 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-02-26T06:39:19.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.458 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.459 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.462 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-02-26T06:39:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.464 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.466 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-02-26T06:39:19.467 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.468 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-02-26T06:39:19.469 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.472 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.473 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.474 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.476 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.477 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.478 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.480 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.481 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T06:39:19.482 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.483 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.484 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.485 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.485 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.485 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.485 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.485 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.485 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.485 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.486 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.487 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T06:39:19.490 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.491 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.492 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.492 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T06:39:19.492 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.492 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.494 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-02-26T06:39:19.496 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.497 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.498 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.502 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.503 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.504 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.506 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.507 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.508 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.509 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.510 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.511 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.514 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T06:39:19.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.521 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.522 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.523 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.523 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.523 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.523 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.523 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T06:39:19.523 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T06:39:19.523 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.524 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.525 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.526 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.528 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.529 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.530 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.532 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.534 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.538 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:19.539 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.540 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.541 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.542 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-02-26T06:39:19.543 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.545 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.548 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.549 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.555 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.556 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.558 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.560 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-02-26T06:39:19.561 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.562 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.564 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.565 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.566 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.568 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.569 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-02-26T06:39:19.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-02-26T06:39:19.571 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.572 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.573 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.574 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.575 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-02-26T06:39:19.576 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.577 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.578 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.579 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.580 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.582 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.585 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.586 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.587 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-02-26T06:39:19.588 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.589 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.592 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.593 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.594 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-02-26T06:39:19.595 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.596 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.597 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.598 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.599 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T06:39:19.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:39:19.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.604 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.605 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.606 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.607 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.608 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.609 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.610 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.611 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.612 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.613 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-02-26T06:39:19.614 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-02-26T06:39:19.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.618 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.619 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.620 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.621 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.622 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.623 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.624 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.625 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-02-26T06:39:19.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.630 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-02-26T06:39:19.632 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.633 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.635 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.638 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.639 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.640 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.641 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.642 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.643 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.644 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.645 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.647 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.648 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.649 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.650 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.652 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.653 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.655 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.657 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.658 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.659 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.660 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.660 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.660 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.660 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.660 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.660 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.660 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.661 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.662 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.663 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.664 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.665 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.667 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.668 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.668 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.668 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.668 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.668 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.670 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.671 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.672 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.673 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.674 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.675 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.676 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.677 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.680 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.681 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.683 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.684 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.685 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.688 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.690 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T06:39:19.691 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.692 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.693 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.694 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.695 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.696 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-02-26T06:39:19.697 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T06:39:19.698 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-02-26T06:39:19.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:19.700 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:19.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.702 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.704 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.707 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.708 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.709 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.710 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.711 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.712 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.713 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.714 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-02-26T06:39:19.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.717 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.718 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.719 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.720 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.721 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.724 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.726 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.728 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.729 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.730 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "drive_group_interval": { 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "drive_group_interval", 2024-02-26T06:39:19.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300.0", 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.732 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.733 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.737 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.740 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.741 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.742 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.743 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.744 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.745 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.747 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.748 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.751 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.753 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.754 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.757 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.761 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.762 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.764 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.765 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.766 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.767 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.768 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.769 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.770 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.771 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.772 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.774 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.775 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.776 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.777 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-02-26T06:39:19.778 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.779 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.781 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.782 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.783 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.784 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.785 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.786 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.787 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-02-26T06:39:19.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-02-26T06:39:19.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.790 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.791 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.792 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.793 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.795 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.797 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-02-26T06:39:19.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-02-26T06:39:19.799 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.800 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.801 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.802 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.803 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.805 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.806 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.807 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.808 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.811 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.812 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.813 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.815 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.816 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.817 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.819 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.821 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.822 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.823 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.824 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.825 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.826 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.827 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-02-26T06:39:19.828 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.829 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-02-26T06:39:19.830 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.831 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.832 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.833 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-02-26T06:39:19.834 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "services": { 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard": "https://172.21.15.49:8443/" 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "always_on_modules": { 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "octopus": [ 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-02-26T06:39:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "pacific": [ 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-02-26T06:39:19.836 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "quincy": [ 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-02-26T06:39:19.837 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "reef": [ 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-02-26T06:39:19.838 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "squid": [ 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.839 INFO:teuthology.orchestra.run.smithi049.stdout: "last_failure_osd_epoch": 89, 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: "active_clients": [ 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 283586734 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.840 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "libcephsqlite", 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4089829258 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-02-26T06:39:19.841 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 505394890 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2238333605 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 676543922 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1015910381 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:19.845 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-02-26T06:39:19.845 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:39:19.888 DEBUG:tasks.check_counter:Getting stats from a 2024-02-26T06:39:19.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-26T06:39:19.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:18 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1879161990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:39:19.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:18 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3775420711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:39:19.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:19 smithi049 ceph-mon[31696]: pgmap v500: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 40 op/s 2024-02-26T06:39:19.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:19 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/942796285' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T06:39:19.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:19 smithi115 ceph-mon[36426]: pgmap v500: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 40 op/s 2024-02-26T06:39:19.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:19 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/942796285' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T06:39:19.891 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:19 smithi080 ceph-mon[36206]: pgmap v500: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 40 op/s 2024-02-26T06:39:19.891 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:19 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/942796285' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T06:39:20.411 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:39:21.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:21 smithi080 ceph-mon[36206]: pgmap v501: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 121 KiB/s wr, 38 op/s 2024-02-26T06:39:21.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:21 smithi115 ceph-mon[36426]: pgmap v501: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 121 KiB/s wr, 38 op/s 2024-02-26T06:39:21.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:21 smithi049 ceph-mon[31696]: pgmap v501: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 121 KiB/s wr, 38 op/s 2024-02-26T06:39:23.337 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 3, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 4, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 2688, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 448, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.005612450, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000068580, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.001885194, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.001675998, 2024-02-26T06:39:23.338 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000558666 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 2688, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 3, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 233, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 480, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 273472, 2024-02-26T06:39:23.339 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 6, 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.049889871, 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.038506580, 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.006009543, 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 230, 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.062796010, 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000273026 2024-02-26T06:39:23.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 273472 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 316, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 304, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1732672, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 183328, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-02-26T06:39:23.341 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.116689692, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.046872370, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.071935671, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.014366853, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 303, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.066959443, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000220988 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.342 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1732672, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 183328 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:23.343 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:23.344 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:23.345 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 149992, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 30, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:23.346 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 319, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1688073, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 319, 2024-02-26T06:39:23.347 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 319, 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1688073, 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:23.348 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:39:23.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:23 smithi080 ceph-mon[36206]: pgmap v502: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 KiB/s wr, 26 op/s 2024-02-26T06:39:23.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:23 smithi115 ceph-mon[36426]: pgmap v502: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 KiB/s wr, 26 op/s 2024-02-26T06:39:23.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:23 smithi049 ceph-mon[31696]: pgmap v502: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 KiB/s wr, 26 op/s 2024-02-26T06:39:24.378 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-02-26T06:39:24.378 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-02-26T06:39:24.378 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-02-26T06:39:24.378 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-02-26T06:39:24.378 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-02-26T06:39:24.378 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-02-26T06:39:24.378 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-02-26T06:39:24.378 DEBUG:tasks.check_counter:Getting stats from d 2024-02-26T06:39:24.378 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-26T06:39:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:39:25.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:25 smithi115 ceph-mon[36426]: pgmap v503: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-02-26T06:39:25.602 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:25 smithi049 ceph-mon[31696]: pgmap v503: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-02-26T06:39:25.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:25 smithi080 ceph-mon[36206]: pgmap v503: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-02-26T06:39:27.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:27 smithi115 ceph-mon[36426]: pgmap v504: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-02-26T06:39:27.617 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:27 smithi049 ceph-mon[31696]: pgmap v504: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-02-26T06:39:27.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:27 smithi080 ceph-mon[36206]: pgmap v504: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-02-26T06:39:27.955 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:39:27.955 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:27.955 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.005182844, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.001358161, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 3, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 238, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 480, 2024-02-26T06:39:27.957 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 274352, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 6, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.041748604, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.030019645, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.004964073, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 235, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.056236503, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000239304 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.958 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 274352 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 327, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 313, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1737600, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 188144, 2024-02-26T06:39:27.959 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.114307087, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.026227741, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.105403869, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.014875570, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 313, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.067895972, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000216920 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:27.960 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1737600, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 188144 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:27.961 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:27.962 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 147930, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 28, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-02-26T06:39:27.964 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-02-26T06:39:27.965 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 326, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1688983, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 326, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 326, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1688983, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:27.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:27.967 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:39:29.147 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-26T06:39:29.147 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-26T06:39:29.148 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-02-26T06:39:29.148 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-02-26T06:39:29.148 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-02-26T06:39:29.148 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-02-26T06:39:29.148 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-02-26T06:39:29.148 DEBUG:tasks.check_counter:Getting stats from g 2024-02-26T06:39:29.148 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-26T06:39:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:39:29.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:29 smithi049 ceph-mon[31696]: pgmap v505: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-02-26T06:39:29.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:29 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:29.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:29 smithi080 ceph-mon[36206]: pgmap v505: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-02-26T06:39:29.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:29 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:29.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:29 smithi115 ceph-mon[36426]: pgmap v505: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-02-26T06:39:29.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:29 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:31.642 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:31 smithi115 ceph-mon[36426]: pgmap v506: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-02-26T06:39:31.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:31 smithi049 ceph-mon[31696]: pgmap v506: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-02-26T06:39:31.831 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:31 smithi080 ceph-mon[36206]: pgmap v506: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-02-26T06:39:32.405 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 148, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 147, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 182656, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 86640, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.044513592, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.016969776, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.013207176, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.006005650, 2024-02-26T06:39:32.406 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 146, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.042108809, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000288416 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 182656, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 86640 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 179, 2024-02-26T06:39:32.407 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 183, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1693200, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 179632, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.089451173, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.032307610, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.071704775, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.007201213, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 180, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.038555739, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000214198 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1693200, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 179632 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 221, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 320, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 195904, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.037285173, 2024-02-26T06:39:32.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.027998426, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.003890958, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 218, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.057868744, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000265452 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 195904 2024-02-26T06:39:32.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:32.411 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:32.412 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 147930, 2024-02-26T06:39:32.413 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 28, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-02-26T06:39:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 325, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1827243, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 325, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-02-26T06:39:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 325, 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1827243, 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:32.416 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:39:33.399 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-02-26T06:39:33.399 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-02-26T06:39:33.399 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-02-26T06:39:33.399 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-02-26T06:39:33.399 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-02-26T06:39:33.400 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-02-26T06:39:33.400 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-02-26T06:39:33.400 DEBUG:tasks.check_counter:Getting stats from j 2024-02-26T06:39:33.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-26T06:39:33.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:33 smithi049 ceph-mon[31696]: pgmap v507: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-02-26T06:39:33.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:33 smithi080 ceph-mon[36206]: pgmap v507: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-02-26T06:39:33.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:33 smithi115 ceph-mon[36426]: pgmap v507: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-02-26T06:39:33.885 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:39:34.577 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:34 smithi049 ceph-mon[31696]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:34.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:34 smithi080 ceph-mon[36206]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:34.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:34 smithi115 ceph-mon[36426]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T06:39:35.665 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:35 smithi049 ceph-mon[31696]: pgmap v508: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-02-26T06:39:35.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:35 smithi080 ceph-mon[36206]: pgmap v508: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-02-26T06:39:35.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:35 smithi115 ceph-mon[36426]: pgmap v508: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-02-26T06:39:36.295 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.006231091, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.001169551, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 7, 2024-02-26T06:39:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 243, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3504, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 275328, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 6, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.038250349, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.027808254, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.004979859, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 241, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.050141506, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000208056 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 275328 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 326, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 314, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1735392, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 189568, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.124278819, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.047513069, 2024-02-26T06:39:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.087012436, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.013729251, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 313, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.063749020, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000203670 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1735392, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 189568 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 147970, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 29, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-02-26T06:39:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 329, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1689373, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 329, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-02-26T06:39:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 329, 2024-02-26T06:39:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1689373, 2024-02-26T06:39:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-02-26T06:39:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-02-26T06:39:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-02-26T06:39:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-02-26T06:39:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-02-26T06:39:36.307 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-02-26T06:39:37.359 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-26T06:39:37.360 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-26T06:39:37.360 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-02-26T06:39:37.360 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-02-26T06:39:37.360 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-02-26T06:39:37.360 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-02-26T06:39:37.360 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-02-26T06:39:37.364 DEBUG:tasks.check_counter:Getting stats from b 2024-02-26T06:39:37.365 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-26T06:39:37.566 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:37 smithi049 ceph-mon[31696]: pgmap v509: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-02-26T06:39:37.786 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:39:37.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:37 smithi080 ceph-mon[36206]: pgmap v509: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-02-26T06:39:37.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:37 smithi115 ceph-mon[36426]: pgmap v509: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-02-26T06:39:39.648 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:39 smithi080 ceph-mon[36206]: pgmap v510: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-02-26T06:39:39.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:39 smithi049 ceph-mon[31696]: pgmap v510: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-02-26T06:39:39.775 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 10888, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 12972, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 4404812, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 21436786, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 241, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 7, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.837355184, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.288567713, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.417829729, 2024-02-26T06:39:39.776 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.161675110, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 12969, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.770246455, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000059391 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 10358, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.001623227, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000156 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 2106528, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 4066720 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:39.777 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 8283, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 9227, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 4301790, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 15559619, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 11, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 5, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.482772573, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.171303433, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.210544542, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.096158140, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 9226, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 1.898008850, 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000205723 2024-02-26T06:39:39.778 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 9147, 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.001318766, 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000144 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 584608 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 8888, 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 9073, 2024-02-26T06:39:39.779 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 6140676, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 13853570, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 20, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 7, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.624930002, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.207336829, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.290083963, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.148646579, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 9072, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 1.181057048, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000130187 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.780 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 9963, 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.001479604, 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000148 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 2420624, 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 369824 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 1, 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-MDSRank": { 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-02-26T06:39:39.781 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 10306, 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 3.448892138, 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000334648 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-PurgeQueue": { 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 161, 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.008519429, 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000052915 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": { 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "request": 2341, 2024-02-26T06:39:39.782 INFO:teuthology.orchestra.run.smithi080.stdout: "reply": 2147, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "reply_latency": { 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 2147, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 27.313081525, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.012721509 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_reply": 0, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "forward": 194, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_complete": 6, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_keys": 0, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_commit": 1093, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_split": 4, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_merge": 0, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes": 596, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_top": 202, 2024-02-26T06:39:39.783 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_bottom": 380, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pin_tail": 14, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pinned": 307, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_expired": 2207, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_with_caps": 138, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": 138, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "subtrees": 50, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse": 3391, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_hit": 2520, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_forward": 191, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_discover": 31, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_dir_fetch": 1, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_remote_ino": 0, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_lock": 76, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "load_cent": 5017, 2024-02-26T06:39:39.784 INFO:teuthology.orchestra.run.smithi080.stdout: "q": 0, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "exported": 60, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "exported_inodes": 2082, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "imported": 60, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "imported_inodes": 1375, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_dir_fetch": 0, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_backtrace_fetch": 6908, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_peer_discover": 18, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rfiles": 1008, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rbytes": 1397316801, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rsnaps": 6, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_fetch": 6908, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_set_tag": 0, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:39:39.785 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_inodes": 344, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_base_inodes": 12, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dirfrag_rstats": 332, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_file_inodes": 6564, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_inode_file_caps": 24, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_revoke": 321, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_grant": 1560, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flush_ack": 1, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps": 1772, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps_dirty": 1088, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_cap_release": 45, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "process_request_cap_release": 297 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.786 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_cache": { 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update": 81, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update_receipt": 116, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_try_discover": 59, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_send_discover": 153, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_handle_discover": 223, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays": 2, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_delayed": 0, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_created": 87, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_enqueued": 36, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_reintegrated": 47, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_migrated": 9, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_processing": 0, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:39:39.787 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_started": 0, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_completed": 0, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_enqueue_scrub": 12, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_exportdir": 60, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_flush": 0, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragmentdir": 4, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragstats": 0, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_inodestats": 0 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_log": { 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "evlrg": 0, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "evadd": 4311, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "evex": 4114, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "evtrm": 4114, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "ev": 197, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "evexg": 0, 2024-02-26T06:39:39.788 INFO:teuthology.orchestra.run.smithi080.stdout: "evexd": 30, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "segadd": 80, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "segex": 79, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "segtrm": 79, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "seg": 1, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "segmjr": 1, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "segexg": 0, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "segexd": 2, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "expos": 33817342, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "wrpos": 34602240, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "rdpos": 0, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "jlat": { 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 2277, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 13.366579074, 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.005870258 2024-02-26T06:39:39.789 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "replayed": 0 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_mem": { 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "ino": 437, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "ino+": 2528, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "ino-": 2091, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dir": 104, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dir+": 213, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dir-": 109, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dn": 601, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dn+": 3230, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dn-": 2629, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "cap": 138, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "cap+": 1307, 2024-02-26T06:39:39.790 INFO:teuthology.orchestra.run.smithi080.stdout: "cap-": 1169, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "rss": 69492, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "heap": 215420 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_server": { 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_client_request": 3028, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_server_request": 373, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_request": 2341, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_session": 287, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_peer_request": 595, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "req_create_latency": { 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 1022, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 3.622614435, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.003544632 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getattr_latency": { 2024-02-26T06:39:39.791 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 182, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.073975805, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000406460 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getfilelock_latency": { 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "req_link_latency": { 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 53, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.234164713, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.004418202 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookup_latency": { 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 183, 2024-02-26T06:39:39.792 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 1.215349679, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.006641255 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookuphash_latency": { 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupino_latency": { 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupname_latency": { 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.793 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupparent_latency": { 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupsnap_latency": { 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.794 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lssnap_latency": { 2024-02-26T06:39:39.795 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 16, 2024-02-26T06:39:39.795 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.002706556, 2024-02-26T06:39:39.795 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000169159 2024-02-26T06:39:39.795 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.795 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mkdir_latency": { 2024-02-26T06:39:39.795 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 37, 2024-02-26T06:39:39.795 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.706949616, 2024-02-26T06:39:39.795 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.019106746 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mknod_latency": { 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mksnap_latency": { 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 16, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.150200996, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.009387562 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "req_open_latency": { 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 408, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.143460794, 2024-02-26T06:39:39.796 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000351619 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "req_readdir_latency": { 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 60, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 2.008760467, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.033479341 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rename_latency": { 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 77, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 18.917272980, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.245678869 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "req_renamesnap_latency": { 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "req_snapdiff_latency": { 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmdir_latency": { 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 2, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.013768499, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.006884249 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmsnap_latency": { 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 10, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.052624025, 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.005262402 2024-02-26T06:39:39.798 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmxattr_latency": { 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setattr_latency": { 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 1, 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.001614982, 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.001614982 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.799 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setdirlayout_latency": { 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setfilelock_latency": { 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setlayout_latency": { 2024-02-26T06:39:39.800 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setxattr_latency": { 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 7, 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.020883795, 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.002983399 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "req_symlink_latency": { 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.801 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "req_unlink_latency": { 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 73, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.148734183, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.002037454 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getvxattr_latency": { 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_sessions": { 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "session_count": 2, 2024-02-26T06:39:39.802 INFO:teuthology.orchestra.run.smithi080.stdout: "session_add": 5, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "session_remove": 3, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_open": 2, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_stale": 0, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "total_load": 120, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "average_load": 60, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "avg_session_uptime": 964, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "mdthresh_evicted": 0 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 240, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 240, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:39.803 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:39.804 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:39.805 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 594288, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 659, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 88, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 1, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 8136, 2024-02-26T06:39:39.806 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 170, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 1828842, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 49184, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.807 INFO:teuthology.orchestra.run.smithi080.stdout: "objecter": { 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "op_active": 0, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "op_laggy": 0, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send": 12544, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send_bytes": 35545350, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "op_resend": 31, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "op_reply": 12544, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "op_latency": { 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 12544, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 138.240885169, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.011020478 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "op_inflight": 0, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "oplen_avg": { 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 12544, 2024-02-26T06:39:39.808 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 16735 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "op": 12544, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "op_r": 6956, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "op_w": 5588, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "op_rmw": 0, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "op_pg": 0, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_stat": 1014, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_create": 1130, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_read": 30, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_write": 2295, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writefull": 279, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writesame": 0, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_append": 0, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_zero": 0, 2024-02-26T06:39:39.809 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_truncate": 3, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_delete": 577, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_mapext": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_sparse_read": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_clonerange": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_getxattr": 6922, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_setxattr": 2260, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_rmxattr": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_call": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_watch": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_notify": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls": 0, 2024-02-26T06:39:39.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_other": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_active": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_send": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_resend": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_ping": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_active": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_send": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_resend": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_active": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_send": 0, 2024-02-26T06:39:39.811 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_resend": 0, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_active": 0, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_send": 0, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_resend": 0, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "command_active": 0, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "command_send": 0, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "command_resend": 0, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "map_epoch": 109, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "map_full": 0, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "map_inc": 24, 2024-02-26T06:39:39.812 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_sessions": 12, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_open": 12, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_close": 0, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_laggy": 0, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_wr": 2115, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_rd": 12, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_del": 98 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "oft": { 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_objs": 1, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_updates": 0, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_removes": 0 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "osd-slow-ops": { 2024-02-26T06:39:39.813 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_ops_count": 1 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "purge_queue": { 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops": 0, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops_high_water": 17, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing": 0, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_high_water": 9, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed_ops": 50, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed": 36, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_item_in_journal": 0 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:39.814 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 28055, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 11982019, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 28055, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 28055, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 11982019, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_bytes": { 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:39.815 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 12544, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 35549138, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 12544, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 35549138, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.816 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_ops": { 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 1024, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 12544, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 12544, 2024-02-26T06:39:39.817 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 12544, 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 12544, 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:39.818 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 36, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 3884, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 36, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 18, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 3884, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle-0x558251dec6e0": { 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 2413, 2024-02-26T06:39:39.819 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 4311, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 30407936, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 4311, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 2277, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 30405523, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.820 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:39.821 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-02-26T06:39:39.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:39 smithi115 ceph-mon[36426]: pgmap v510: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-02-26T06:39:40.745 INFO:tasks.check_counter:Daemon mds.b mds.exported=60 2024-02-26T06:39:40.745 INFO:tasks.check_counter:Daemon mds.b mds.imported=60 2024-02-26T06:39:40.745 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=81 2024-02-26T06:39:40.745 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=116 2024-02-26T06:39:40.745 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-02-26T06:39:40.745 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-02-26T06:39:40.745 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-02-26T06:39:40.745 DEBUG:tasks.check_counter:Getting stats from e 2024-02-26T06:39:40.746 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-26T06:39:40.993 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:39:41.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:41 smithi049 ceph-mon[31696]: pgmap v511: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-02-26T06:39:41.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:41 smithi080 ceph-mon[36206]: pgmap v511: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-02-26T06:39:41.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:41 smithi115 ceph-mon[36426]: pgmap v511: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 965, 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 967, 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 4444797, 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 300179, 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 8, 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.073655410, 2024-02-26T06:39:43.389 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.026386314, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.019748115, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.020225942, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 965, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.037277073, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000038629 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.390 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1513, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 1468, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 6376649, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 544488, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 4, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.146729284, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.040286965, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.068735877, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.040623210, 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:43.391 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 1467, 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.080914929, 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000055156 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 1899136, 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 194608 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:43.392 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1510, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 1747, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 1049356, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 1125769, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 8, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 5, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.156514192, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.049530702, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.040247931, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.052212753, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 1744, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.106996166, 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000061351 2024-02-26T06:39:43.393 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 658336 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 1, 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-MDSRank": { 2024-02-26T06:39:43.394 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 6108, 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.155596210, 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000025474 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-PurgeQueue": { 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.395 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": { 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "request": 0, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "reply": 0, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "reply_latency": { 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_reply": 0, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "forward": 0, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_complete": 0, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_keys": 0, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_commit": 0, 2024-02-26T06:39:43.396 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_split": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_merge": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes": 1440, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_top": 676, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_bottom": 764, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pin_tail": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pinned": 475, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_expired": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_with_caps": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "subtrees": 51, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_hit": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_forward": 0, 2024-02-26T06:39:43.397 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_discover": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_remote_ino": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_lock": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "load_cent": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "q": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "exported": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "exported_inodes": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "imported": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "imported_inodes": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_dir_fetch": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_peer_discover": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rfiles": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rbytes": 0, 2024-02-26T06:39:43.398 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rsnaps": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_set_tag": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_file_inodes": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:39:43.399 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_cap_release": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "process_request_cap_release": 0 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_cache": { 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update_receipt": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_try_discover": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_send_discover": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_handle_discover": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_delayed": 0, 2024-02-26T06:39:43.400 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_created": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_enqueued": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_reintegrated": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_migrated": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_processing": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_started": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_completed": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_exportdir": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_flush": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragstats": 0, 2024-02-26T06:39:43.401 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_inodestats": 0 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_log": { 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "evlrg": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "evadd": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "evex": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "evtrm": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "ev": 3846, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "evexg": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "evexd": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "segadd": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "segex": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "segtrm": 0, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "seg": 7, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "segmjr": 2, 2024-02-26T06:39:43.402 INFO:teuthology.orchestra.run.smithi080.stdout: "segexg": 0, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "segexd": 0, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "expos": 4194304, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "wrpos": 13156528, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "rdpos": 13154862, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "jlat": { 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "replayed": 3846 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_mem": { 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "ino": 1176, 2024-02-26T06:39:43.403 INFO:teuthology.orchestra.run.smithi080.stdout: "ino+": 2209, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "ino-": 1033, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "dir": 113, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "dir+": 150, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "dir-": 37, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "dn": 1440, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "dn+": 2374, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "dn-": 934, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "cap": 0, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "cap+": 0, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "cap-": 0, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "rss": 54832, 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: "heap": 215420 2024-02-26T06:39:43.404 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_server": { 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_client_request": 0, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_server_request": 0, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_request": 0, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_session": 0, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_peer_request": 0, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "req_create_latency": { 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getattr_latency": { 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.405 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getfilelock_latency": { 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "req_link_latency": { 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookup_latency": { 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.406 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookuphash_latency": { 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupino_latency": { 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupname_latency": { 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.407 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupparent_latency": { 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupsnap_latency": { 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lssnap_latency": { 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.408 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.409 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.409 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.409 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mkdir_latency": { 2024-02-26T06:39:43.409 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.409 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.409 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.409 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mknod_latency": { 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mksnap_latency": { 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "req_open_latency": { 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.410 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "req_readdir_latency": { 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rename_latency": { 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "req_renamesnap_latency": { 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.411 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: "req_snapdiff_latency": { 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmdir_latency": { 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.412 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmsnap_latency": { 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmxattr_latency": { 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setattr_latency": { 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.413 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setdirlayout_latency": { 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setfilelock_latency": { 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setlayout_latency": { 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.414 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setxattr_latency": { 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "req_symlink_latency": { 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "req_unlink_latency": { 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.415 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getvxattr_latency": { 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_sessions": { 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "session_count": 2, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "session_add": 4, 2024-02-26T06:39:43.416 INFO:teuthology.orchestra.run.smithi080.stdout: "session_remove": 2, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_open": 3, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_stale": 0, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "total_load": 0, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "average_load": 0, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "avg_session_uptime": 3975, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "mdthresh_evicted": 0 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:43.417 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:43.418 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:43.419 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 253542, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 1299, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 8136, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 170, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:43.420 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:43.421 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-02-26T06:39:43.421 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 3873600, 2024-02-26T06:39:43.421 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 32661, 2024-02-26T06:39:43.421 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:43.421 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-02-26T06:39:43.423 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:43.423 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-02-26T06:39:43.423 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "objecter": { 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "op_active": 0, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "op_laggy": 0, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send": 3593, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send_bytes": 21, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "op_resend": 3, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "op_reply": 3593, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "op_latency": { 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3593, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 12.690722357, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.003532068 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.424 INFO:teuthology.orchestra.run.smithi080.stdout: "op_inflight": 0, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "oplen_avg": { 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3593, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 3594 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "op": 3593, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "op_r": 3593, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "op_w": 0, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "op_rmw": 0, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "op_pg": 0, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_stat": 2186, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_create": 0, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_read": 1405, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_write": 0, 2024-02-26T06:39:43.425 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writefull": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writesame": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_append": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_zero": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_truncate": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_delete": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_mapext": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_sparse_read": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_clonerange": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_getxattr": 1, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_setxattr": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_rmxattr": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_call": 0, 2024-02-26T06:39:43.426 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_watch": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_notify": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_other": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_active": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_send": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_resend": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_ping": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_active": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_send": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_resend": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_active": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_send": 0, 2024-02-26T06:39:43.427 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_resend": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_active": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_send": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_resend": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "command_active": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "command_send": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "command_resend": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "map_epoch": 109, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "map_full": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "map_inc": 23, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_sessions": 7, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_open": 8, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_close": 1, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_laggy": 0, 2024-02-26T06:39:43.428 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_wr": 0, 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_rd": 2, 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_del": 0 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "oft": { 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_objs": 0, 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_updates": 0, 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_removes": 0 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "osd-slow-ops": { 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_ops_count": 0 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "purge_queue": { 2024-02-26T06:39:43.429 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops": 0, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing": 0, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_high_water": 0, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed_ops": 0, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed": 0, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_item_in_journal": 0 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 3984, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 11489521, 2024-02-26T06:39:43.430 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 3984, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 3984, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 11489521, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_bytes": { 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-02-26T06:39:43.431 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 3593, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 8963354, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 3593, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 8963354, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.432 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_ops": { 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 1024, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 3593, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 3593, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 3593, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 3593, 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:43.433 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:43.434 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 0, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 0, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle-0x55d08348c960": { 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:43.435 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 0, 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 0, 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:43.436 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-02-26T06:39:43.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:43 smithi049 ceph-mon[31696]: pgmap v512: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-02-26T06:39:43.767 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:43 smithi080 ceph-mon[36206]: pgmap v512: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-02-26T06:39:43.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:43 smithi115 ceph-mon[36426]: pgmap v512: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-02-26T06:39:44.215 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-02-26T06:39:44.215 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-02-26T06:39:44.215 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-02-26T06:39:44.216 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-02-26T06:39:44.216 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-02-26T06:39:44.216 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:39:44.216 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:39:44.216 DEBUG:tasks.check_counter:Getting stats from h 2024-02-26T06:39:44.216 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-26T06:39:44.455 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:39:45.638 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:45 smithi080 ceph-mon[36206]: pgmap v513: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-02-26T06:39:45.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:45 smithi049 ceph-mon[31696]: pgmap v513: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-02-26T06:39:45.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:45 smithi115 ceph-mon[36426]: pgmap v513: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-02-26T06:39:46.224 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-02-26T06:39:46.329 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1531, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 1486, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 2283312, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 549455, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 5, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 1, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.166495853, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.057650955, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.068639698, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.040517682, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 1485, 2024-02-26T06:39:46.330 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.106086135, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000071438 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 1898592, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 193360 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 5, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 243, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 1036, 2024-02-26T06:39:46.331 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 745340, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 8, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 4, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.012320499, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.007910618, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.002196890, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000076255, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 240, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.030680033, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000127833 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.332 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 744656 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 2460, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 2460, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 6675365, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 764271, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 5, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.141390208, 2024-02-26T06:39:46.333 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.032151046, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.043044518, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.057805793, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 2459, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.089956902, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000036582 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T06:39:46.334 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 1, 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-MDSRank": { 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 6076, 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.130042911, 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000021402 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "finisher-PurgeQueue": { 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "queue_len": 0, 2024-02-26T06:39:46.335 INFO:teuthology.orchestra.run.smithi080.stdout: "complete_latency": { 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": { 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "request": 0, 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "reply": 0, 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "reply_latency": { 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.336 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_reply": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "forward": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_complete": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_fetch_keys": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_commit": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_split": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_merge": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes": 772, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_top": 105, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_bottom": 667, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pin_tail": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_pinned": 623, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_expired": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "inodes_with_caps": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": 0, 2024-02-26T06:39:46.337 INFO:teuthology.orchestra.run.smithi080.stdout: "subtrees": 30, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_hit": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_forward": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_discover": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_remote_ino": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "traverse_lock": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "load_cent": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "q": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "exported": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "exported_inodes": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "imported": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "imported_inodes": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_dir_fetch": 0, 2024-02-26T06:39:46.338 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "openino_peer_discover": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rfiles": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rbytes": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "root_rsnaps": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_set_tag": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_file_inodes": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_cap_release": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "process_request_cap_release": 0 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_cache": { 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_update_receipt": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_try_discover": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_send_discover": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "dir_handle_discover": 0, 2024-02-26T06:39:46.340 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_delayed": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_created": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_enqueued": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_reintegrated": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "strays_migrated": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_processing": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_started": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_completed": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_exportdir": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_flush": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:39:46.341 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_fragstats": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "ireq_inodestats": 0 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_log": { 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "evlrg": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "evadd": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "evex": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "evtrm": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "ev": 3487, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "evexg": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "evexd": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "segadd": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "segex": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "segtrm": 0, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "seg": 5, 2024-02-26T06:39:46.342 INFO:teuthology.orchestra.run.smithi080.stdout: "segmjr": 2, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "segexg": 0, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "segexd": 0, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "expos": 4194304, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "wrpos": 10242606, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "rdpos": 10242032, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "jlat": { 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "replayed": 3487 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_mem": { 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "ino": 585, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "ino+": 1226, 2024-02-26T06:39:46.343 INFO:teuthology.orchestra.run.smithi080.stdout: "ino-": 641, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "dir": 45, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "dir+": 79, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "dir-": 34, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "dn": 772, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "dn+": 1570, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "dn-": 798, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "cap": 0, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "cap+": 0, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "cap-": 0, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "rss": 53420, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "heap": 215420 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.344 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_server": { 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_client_request": 0, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "dispatch_server_request": 0, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_request": 0, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_client_session": 0, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "handle_peer_request": 0, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "req_create_latency": { 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getattr_latency": { 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.345 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getfilelock_latency": { 2024-02-26T06:39:46.346 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.346 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.346 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.346 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.346 INFO:teuthology.orchestra.run.smithi080.stdout: "req_link_latency": { 2024-02-26T06:39:46.346 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.346 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.347 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.347 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookup_latency": { 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookuphash_latency": { 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupino_latency": { 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.348 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupname_latency": { 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupparent_latency": { 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lookupsnap_latency": { 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.350 INFO:teuthology.orchestra.run.smithi080.stdout: "req_lssnap_latency": { 2024-02-26T06:39:46.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.350 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.350 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.350 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mkdir_latency": { 2024-02-26T06:39:46.350 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.350 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mknod_latency": { 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: "req_mksnap_latency": { 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.351 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "req_open_latency": { 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "req_readdir_latency": { 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rename_latency": { 2024-02-26T06:39:46.352 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "req_renamesnap_latency": { 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "req_snapdiff_latency": { 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmdir_latency": { 2024-02-26T06:39:46.353 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmsnap_latency": { 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "req_rmxattr_latency": { 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setattr_latency": { 2024-02-26T06:39:46.354 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setdirlayout_latency": { 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setfilelock_latency": { 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setlayout_latency": { 2024-02-26T06:39:46.355 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "req_setxattr_latency": { 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "req_symlink_latency": { 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.356 INFO:teuthology.orchestra.run.smithi080.stdout: "req_unlink_latency": { 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "req_getvxattr_latency": { 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_sessions": { 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "session_count": 2, 2024-02-26T06:39:46.357 INFO:teuthology.orchestra.run.smithi080.stdout: "session_add": 4, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "session_remove": 2, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_open": 4, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "sessions_stale": 0, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "total_load": 0, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "average_load": 0, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "avg_session_uptime": 4979, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "mdthresh_evicted": 0 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:46.358 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:46.359 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:46.360 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 205534, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 683, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 8136, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 170, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:46.361 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 2004408, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 30578, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "objecter": { 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "op_active": 0, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "op_laggy": 0, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send": 3603, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "op_send_bytes": 21, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "op_resend": 3, 2024-02-26T06:39:46.362 INFO:teuthology.orchestra.run.smithi080.stdout: "op_reply": 3603, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "op_latency": { 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3603, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 3.742917360, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.001038833 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "op_inflight": 0, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "oplen_avg": { 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3603, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 3604 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "op": 3603, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "op_r": 3603, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "op_w": 0, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "op_rmw": 0, 2024-02-26T06:39:46.363 INFO:teuthology.orchestra.run.smithi080.stdout: "op_pg": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_stat": 2145, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_create": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_read": 1456, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_write": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writefull": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_writesame": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_append": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_zero": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_truncate": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_delete": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_mapext": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_sparse_read": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_clonerange": 0, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_getxattr": 1, 2024-02-26T06:39:46.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_setxattr": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_rmxattr": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_call": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_watch": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_notify": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "osdop_other": 0, 2024-02-26T06:39:46.365 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_active": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_send": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_resend": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "linger_ping": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_active": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_send": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "poolop_resend": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_active": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_send": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "poolstat_resend": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_active": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_send": 0, 2024-02-26T06:39:46.366 INFO:teuthology.orchestra.run.smithi080.stdout: "statfs_resend": 0, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "command_active": 0, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "command_send": 0, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "command_resend": 0, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "map_epoch": 109, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "map_full": 0, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "map_inc": 23, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_sessions": 4, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_open": 5, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_session_close": 1, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_laggy": 0, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_wr": 0, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_rd": 2, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_del": 0 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.367 INFO:teuthology.orchestra.run.smithi080.stdout: "oft": { 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_objs": 0, 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_updates": 0, 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: "omap_total_removes": 0 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: "osd-slow-ops": { 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: "slow_ops_count": 0 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: "purge_queue": { 2024-02-26T06:39:46.368 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops": 0, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing": 0, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executing_high_water": 0, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed_ops": 0, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_executed": 0, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "pq_item_in_journal": 0 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:46.369 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 3992, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 8577816, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 3992, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 3992, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 8577816, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_bytes": { 2024-02-26T06:39:46.370 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 3603, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 6049045, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 3603, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 6049045, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.371 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-objecter_ops": { 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 1024, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 3603, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 3603, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 3603, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 3603, 2024-02-26T06:39:46.372 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:46.373 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 0, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 0, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-write_buf_throttle-0x564c80de0960": { 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 3758096384, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 0, 2024-02-26T06:39:46.374 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 0, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 0, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 0, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:46.375 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-02-26T06:39:47.300 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-02-26T06:39:47.301 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-02-26T06:39:47.301 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-02-26T06:39:47.301 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-02-26T06:39:47.301 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-02-26T06:39:47.301 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:39:47.301 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:39:47.301 DEBUG:tasks.check_counter:Getting stats from k 2024-02-26T06:39:47.301 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-26T06:39:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:39:47.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:47 smithi080 ceph-mon[36206]: pgmap v514: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-02-26T06:39:47.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:47 smithi049 ceph-mon[31696]: pgmap v514: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-02-26T06:39:47.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:47 smithi115 ceph-mon[36426]: pgmap v514: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-02-26T06:39:49.288 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-02-26T06:39:49.288 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:49.288 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 335, 2024-02-26T06:39:49.288 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 321, 2024-02-26T06:39:49.288 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 1739840, 2024-02-26T06:39:49.288 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 192672, 2024-02-26T06:39:49.288 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 4, 2024-02-26T06:39:49.288 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 0, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.056139235, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.016078841, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.042304445, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.007170424, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 321, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.013794477, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000042973 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:49.289 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 1739840, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 192672 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 2, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 243, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 320, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 197568, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 5, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.012254726, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.008672991, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.001170328, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 240, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.023567302, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000098197 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 197568 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 0, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 2, 2024-02-26T06:39:49.291 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 0, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 0, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 2, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 1, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.000171679, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.000114030, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:49.292 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 0, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:49.293 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:49.294 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:49.295 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 148670, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 30, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 0, 2024-02-26T06:39:49.296 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 928, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 2, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-02-26T06:39:49.297 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 333, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 1690060, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 333, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 333, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 1690060, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-02-26T06:39:49.298 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-02-26T06:39:49.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:49 smithi080 ceph-mon[36206]: pgmap v515: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-02-26T06:39:49.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:49 smithi049 ceph-mon[31696]: pgmap v515: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-02-26T06:39:49.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:49 smithi115 ceph-mon[36426]: pgmap v515: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-02-26T06:39:49.999 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-02-26T06:39:49.999 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-02-26T06:39:49.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-02-26T06:39:49.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-02-26T06:39:49.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-02-26T06:39:49.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-02-26T06:39:49.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-02-26T06:39:49.999 DEBUG:tasks.check_counter:Getting stats from c 2024-02-26T06:39:50.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-26T06:39:50.246 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:39:51.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:51 smithi049 ceph-mon[31696]: pgmap v516: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-02-26T06:39:51.675 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:51 smithi115 ceph-mon[36426]: pgmap v516: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-02-26T06:39:51.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:51 smithi080 ceph-mon[36206]: pgmap v516: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 5331, 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 5398, 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2126517, 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 6055597, 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 18, 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.308219350, 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.134113544, 2024-02-26T06:39:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.172266804, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.025577573, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5396, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.168403415, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000031208 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 8082, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001217929, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000150 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 5360, 2024-02-26T06:39:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 19712 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1590, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1831, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 719217, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 5281312, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 33, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.131221369, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.050085610, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.034868331, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.035130576, 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1828, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.095143824, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000052048 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 36, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000007835, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000217 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2656, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 663168 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 9696, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 10764, 2024-02-26T06:39:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 6974083, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 6698190, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.591250247, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.237928351, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.399578316, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.032853759, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 10764, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.337486034, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000031353 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 15535, 2024-02-26T06:39:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.002442735, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000157 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3066448, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 3368080 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3657, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.182632592, 2024-02-26T06:39:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000323388 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 676, 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.025206092, 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000037287 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 1574, 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 1476, 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1476, 2024-02-26T06:39:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 11.620795869, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.007873167 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 98, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 17, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 2, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 35, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 3, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 1319, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 319, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 985, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 15, 2024-02-26T06:39:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 862, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 1056, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 722, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 722, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 53, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 3755, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 3206, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 84, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 98, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 6, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 118, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 249, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-02-26T06:39:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 37, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 885, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 37, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 1536, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 2051, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 14, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 1011, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 1397347374, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 2051, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 20, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:39:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 20, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 2031, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 71, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 356, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 1075, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 1, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 2, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 1246, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 636, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 176, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 1018 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-02-26T06:39:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 58, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 128, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 74, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 195, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 182, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 5, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 261, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 217, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 37, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 25, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:39:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 37, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 3, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 1, 2024-02-26T06:39:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 3859, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 3859, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 7, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 7, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-02-26T06:39:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 13246238, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 902, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 6.135565275, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.006802178 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 1264, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 2323, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 1059, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 126, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 173, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 47, 2024-02-26T06:39:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 1336, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 2629, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 1293, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 644, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 1412, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 768, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 64000, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 207228 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 2643, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 811, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 1574, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 178, 2024-02-26T06:39:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 1160, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 221, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.363716043, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001645773 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 51, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.070297677, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001378385 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 44, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.473047165, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.010751071 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 622, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 8.492035440, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.013652790 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-02-26T06:39:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000120361, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000120361 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-02-26T06:39:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-02-26T06:39:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 163, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.183854333, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001127940 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 49, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.074075517, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001511745 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 275, 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.743487380, 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.006339954 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.003622982, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003622982 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 49, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.216538971, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004419162 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-02-26T06:39:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 4, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 293, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 146, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 965, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-02-26T06:39:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 739952, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 1527, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 264, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 3, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 8136, 2024-02-26T06:39:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 170, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 4426782, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 56400, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-02-26T06:39:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 3730, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 9284345, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 33, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 3730, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3730, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 15.336703127, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004111716 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3730, 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4003 2024-02-26T06:39:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 3730, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 2149, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 1581, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 79, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 981, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 136, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-02-26T06:39:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 234, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2056, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:39:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 1, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-02-26T06:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 109, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 23, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 358, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 38, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 90 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-02-26T06:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 10, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 10, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 217, 2024-02-26T06:39:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 217, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 16613, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 8156616, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 16613, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 16613, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 8156616, 2024-02-26T06:39:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3730, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 9306534, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3730, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 9306534, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-02-26T06:39:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3730, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 3730, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3730, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3730, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-02-26T06:39:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 217, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 22469, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 217, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 79, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 22469, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x560db6c7c6e0": { 2024-02-26T06:39:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 87470, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 3859, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 9051934, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 3859, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 902, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 8964464, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:52.109 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-02-26T06:39:52.956 INFO:tasks.check_counter:Daemon mds.c mds.exported=37 2024-02-26T06:39:52.956 INFO:tasks.check_counter:Daemon mds.c mds.imported=37 2024-02-26T06:39:52.957 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=58 2024-02-26T06:39:52.957 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=128 2024-02-26T06:39:52.957 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-02-26T06:39:52.957 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:39:52.957 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:39:52.957 DEBUG:tasks.check_counter:Getting stats from f 2024-02-26T06:39:52.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-26T06:39:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:39:53.584 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:53 smithi115 ceph-mon[36426]: pgmap v517: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-02-26T06:39:53.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:53 smithi080 ceph-mon[36206]: pgmap v517: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-02-26T06:39:53.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:53 smithi049 ceph-mon[31696]: pgmap v517: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-02-26T06:39:54.692 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:54 smithi115 ceph-mon[36426]: pgmap v518: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-02-26T06:39:54.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:54 smithi080 ceph-mon[36206]: pgmap v518: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-02-26T06:39:54.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:54 smithi049 ceph-mon[31696]: pgmap v518: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-02-26T06:39:54.943 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2015, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2016, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 13190497, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 626364, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.185464818, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.049643531, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.056343467, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.064104588, 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2014, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.112178512, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000055699 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1367, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1322, 2024-02-26T06:39:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 11762750, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 499618, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 5, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.130529383, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.037042990, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.067621901, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.032062357, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1321, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.080938306, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000061270 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1899840, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 196016 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 878, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1116, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 8643594, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 855241, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.073673003, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.028442718, 2024-02-26T06:39:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.019063630, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.019482345, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1114, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.060668434, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000054459 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 584048 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6510, 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.188509741, 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000028956 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-02-26T06:39:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-02-26T06:39:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 751, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 388, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 363, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 196, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 23, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-02-26T06:39:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 1008, 2024-02-26T06:39:54.952 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 1397317236, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:39:54.953 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-02-26T06:39:54.954 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.955 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 4341, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 72, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-02-26T06:39:54.956 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 33817342, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 34782190, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 34754945, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 4341 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 426, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 2488, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 2062, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 40, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 153, 2024-02-26T06:39:54.957 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 113, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 753, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 2878, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 2125, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 57396, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215420 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-02-26T06:39:54.958 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-02-26T06:39:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.960 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.961 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-02-26T06:39:54.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.963 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.964 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.965 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.967 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.968 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 5, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 3, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 4, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-02-26T06:39:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 4866, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:54.970 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:54.971 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 234826, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 598, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:39:54.972 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 8136, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 170, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 1697616, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 47699, 2024-02-26T06:39:54.973 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 3862, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 26, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 4, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 3862, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3862, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 6.494873720, 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001681738 2024-02-26T06:39:54.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3862, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3863 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 3862, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 3862, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 2289, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1570, 2024-02-26T06:39:54.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-02-26T06:39:54.976 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-02-26T06:39:54.977 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 109, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 23, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 8, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 8, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-02-26T06:39:54.978 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-02-26T06:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 4256, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 33190973, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:54.980 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 4256, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4256, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 33190973, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-02-26T06:39:54.981 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3862, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 30619387, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3862, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 30619387, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3862, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 3862, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3862, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3862, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.983 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:54.984 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55eba0cee960": { 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:54.986 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-02-26T06:39:55.784 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-02-26T06:39:55.784 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-02-26T06:39:55.785 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-02-26T06:39:55.785 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-02-26T06:39:55.785 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-02-26T06:39:55.785 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:39:55.785 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:39:55.785 DEBUG:tasks.check_counter:Getting stats from i 2024-02-26T06:39:55.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-26T06:39:56.024 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:39:57.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:56 smithi080 ceph-mon[36206]: pgmap v519: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-02-26T06:39:57.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:56 smithi115 ceph-mon[36426]: pgmap v519: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-02-26T06:39:57.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:56 smithi049 ceph-mon[31696]: pgmap v519: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-02-26T06:39:57.793 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-02-26T06:39:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 9675, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 10783, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 3827306, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 11171589, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 19, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.610523126, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.267835965, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.361069158, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.052690401, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 10781, 2024-02-26T06:39:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.288806037, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000026788 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14428, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.002163737, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000149 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1221296, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 2599056 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 6172, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 5252, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 3056297, 2024-02-26T06:39:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 3372235, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 29, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.325230259, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.124150919, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.208929468, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.024410949, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5252, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.196633731, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000037439 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 8743, 2024-02-26T06:39:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001400122, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000160 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 5488, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 587232 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1258, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1250, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2324013, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 656768, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 11, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.118080407, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.039076180, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.069489914, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.024541049, 2024-02-26T06:39:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1247, 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.056541563, 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000045342 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 60, 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000009956, 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000165 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1918544, 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 225808 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-02-26T06:39:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4418, 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.142192582, 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000258531 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-02-26T06:39:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 79, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.002782965, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000035227 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 1511, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 1394, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1394, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 8.797016029, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.006310628 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-02-26T06:39:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 117, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 15, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 1, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 43, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 4, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 782, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 193, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 482, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 107, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 500, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 929, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 61, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 61, 2024-02-26T06:39:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 51, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 4213, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 3369, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 109, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 79, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 1, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 161, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 248, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 28, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 729, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 28, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 785, 2024-02-26T06:39:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 2472, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 1011, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 1397347374, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 2472, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 38, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 38, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 2434, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 31, 2024-02-26T06:39:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 473, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 857, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 1, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 1, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 1449, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 789, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 71, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 527 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 60, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 197, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 109, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 206, 2024-02-26T06:39:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 149, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 11, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 134, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 18, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 109, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 20, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 28, 2024-02-26T06:39:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 4, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 3519, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 3519, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 6, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-02-26T06:39:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 6, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 10387242, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1106, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.974372905, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003593465 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 615, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 1303, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 688, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 113, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 194, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 81, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 784, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 1770, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 986, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 61, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 570, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 509, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 60652, 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 207228 2024-02-26T06:39:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 2979, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 670, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 1511, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 269, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 1107, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 55, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.103901984, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001889126 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 41, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.035048636, 2024-02-26T06:39:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000854844 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 114, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.537529976, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004715175 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 783, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4.364520597, 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005574100 2024-02-26T06:39:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000102102, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000102102 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 83, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.154392582, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001860151 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 49, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.538957864, 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.031407303 2024-02-26T06:39:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 135, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.741192587, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.012897722 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.004039791, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004039791 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-02-26T06:39:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 132, 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.317329910, 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002404014 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-02-26T06:39:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 4, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 383, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 191, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 982, 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-02-26T06:39:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:39:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-02-26T06:39:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 709330, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 680, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 528, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 6, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 8136, 2024-02-26T06:39:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 170, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 2334748, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 45828, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-02-26T06:39:57.824 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 4006, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 6545332, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 28, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 4006, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4006, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 17.121669410, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004274006 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4006, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4328 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 4006, 2024-02-26T06:39:57.825 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 2497, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 1509, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 9, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 1115, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 76, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 2, 2024-02-26T06:39:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 33, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2474, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-02-26T06:39:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 1, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-02-26T06:39:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 109, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 23, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 467, 2024-02-26T06:39:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 32, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 89 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 6, 2024-02-26T06:39:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 6, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 18, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 18, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 17101, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 7492132, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 17101, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 17101, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 7492132, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-02-26T06:39:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4006, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 6547330, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4006, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 6547330, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-02-26T06:39:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4006, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 4006, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4006, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4006, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 18, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 2242, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 18, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 9, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2242, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x561aea2c86e0": { 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 3519, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 6192938, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 3519, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 1106, 2024-02-26T06:39:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 6192938, 2024-02-26T06:39:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:39:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:39:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:39:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:39:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:39:57.837 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-02-26T06:39:58.626 INFO:tasks.check_counter:Daemon mds.i mds.exported=28 2024-02-26T06:39:58.627 INFO:tasks.check_counter:Daemon mds.i mds.imported=28 2024-02-26T06:39:58.627 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=60 2024-02-26T06:39:58.627 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=197 2024-02-26T06:39:58.627 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-02-26T06:39:58.627 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:39:58.627 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:39:58.627 DEBUG:tasks.check_counter:Getting stats from l 2024-02-26T06:39:58.627 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-26T06:39:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:39:58.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:39:58 smithi115 ceph-mon[36426]: pgmap v520: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 7 op/s 2024-02-26T06:39:59.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:39:58 smithi080 ceph-mon[36206]: pgmap v520: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 7 op/s 2024-02-26T06:39:59.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:39:58 smithi049 ceph-mon[31696]: pgmap v520: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 7 op/s 2024-02-26T06:40:00.419 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31692]: 2024-02-26T06:39:59.999+0000 7f751c200700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:40:00.604 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-02-26T06:40:00.604 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:40:00.604 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 338, 2024-02-26T06:40:00.604 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 325, 2024-02-26T06:40:00.604 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1740592, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 195168, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 4, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 0, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.053560531, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.015691346, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.040171104, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.007163769, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 325, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.013813825, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000042504 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:40:00.605 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1740592, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 195168 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 247, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 320, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 198272, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 5, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.013108931, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.009283668, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.001393190, 2024-02-26T06:40:00.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 244, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.033573012, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000137594 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 198272 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:40:00.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 0, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 0, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 0, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 2, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 1, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.000267511, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.000194330, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 0, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-02-26T06:40:00.609 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:40:00.610 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-02-26T06:40:00.611 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 148692, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 30, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-02-26T06:40:00.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 928, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 2, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-02-26T06:40:00.613 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 337, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1690580, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 337, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 337, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1690580, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-02-26T06:40:00.614 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-02-26T06:40:01.021 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:00 smithi115 ceph-mon[36426]: pgmap v521: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 6 op/s 2024-02-26T06:40:01.021 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:00 smithi115 ceph-mon[36426]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:40:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:00 smithi080 ceph-mon[36206]: pgmap v521: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 6 op/s 2024-02-26T06:40:01.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:00 smithi080 ceph-mon[36206]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:40:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:00 smithi049 ceph-mon[31696]: pgmap v521: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 6 op/s 2024-02-26T06:40:01.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:00 smithi049 ceph-mon[31696]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:40:01.324 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-02-26T06:40:01.324 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-02-26T06:40:01.324 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-02-26T06:40:01.324 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-02-26T06:40:01.324 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-02-26T06:40:01.324 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-02-26T06:40:01.324 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-02-26T06:40:01.325 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-26T06:40:01.336 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-26T06:40:01.336 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:40:01.336 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T06:40:01.428 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:40:01.428 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T06:40:01.491 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:40:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 -- 172.21.15.49:0/1957360056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c068c80 msgr2=0x7f893c10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 --2- 172.21.15.49:0/1957360056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c068c80 0x7f893c10de20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8934009860 tx=0x7f893402bb70 comp rx=0 tx=0).stop 2024-02-26T06:40:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 -- 172.21.15.49:0/1957360056 shutdown_connections 2024-02-26T06:40:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 --2- 172.21.15.49:0/1957360056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c068c80 0x7f893c10de20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 --2- 172.21.15.49:0/1957360056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f893c0682e0 0x7f893c068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 --2- 172.21.15.49:0/1957360056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f893c107010 0x7f893c1073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 -- 172.21.15.49:0/1957360056 >> 172.21.15.49:0/1957360056 conn(0x7f893c075a90 msgr2=0x7f893c075ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:01.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 -- 172.21.15.49:0/1957360056 shutdown_connections 2024-02-26T06:40:01.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.733+0000 7f893be00640 1 -- 172.21.15.49:0/1957360056 wait complete. 2024-02-26T06:40:01.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.734+0000 7f893be00640 1 Processor -- start 2024-02-26T06:40:01.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893be00640 1 -- start start 2024-02-26T06:40:01.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0682e0 0x7f893c100200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:01.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f893c068c80 0x7f893c100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:01.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f893c107010 0x7f893c100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:01.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1a5910 con 0x7f893c068c80 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1a5a80 con 0x7f893c0682e0 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893be00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1a5c40 con 0x7f893c107010 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f893c068c80 0x7f893c100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f893c068c80 0x7f893c100740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51778/0 (socket says 172.21.15.49:51778) 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893a000640 1 -- 172.21.15.49:0/2486010641 learned_addr learned my addr 172.21.15.49:0/2486010641 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893aa00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0682e0 0x7f893c100200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.735+0000 7f893b400640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f893c107010 0x7f893c100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f893a000640 1 -- 172.21.15.49:0/2486010641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f893c107010 msgr2=0x7f893c100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f893a000640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f893c107010 0x7f893c100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f893a000640 1 -- 172.21.15.49:0/2486010641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0682e0 msgr2=0x7f893c100200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f893a000640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0682e0 0x7f893c100200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f893a000640 1 -- 172.21.15.49:0/2486010641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934009490 con 0x7f893c068c80 2024-02-26T06:40:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f893a000640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f893c068c80 0x7f893c100740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f892400ebc0 tx=0x7f892400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f8933400640 1 -- 172.21.15.49:0/2486010641 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924010430 con 0x7f893c068c80 2024-02-26T06:40:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f8933400640 1 -- 172.21.15.49:0/2486010641 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8924010a70 con 0x7f893c068c80 2024-02-26T06:40:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c1a5f20 con 0x7f893c068c80 2024-02-26T06:40:01.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.736+0000 7f8933400640 1 -- 172.21.15.49:0/2486010641 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924018aa0 con 0x7f893c068c80 2024-02-26T06:40:01.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.737+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c1a63a0 con 0x7f893c068c80 2024-02-26T06:40:01.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.738+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c067e90 con 0x7f893c068c80 2024-02-26T06:40:01.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.740+0000 7f8933400640 1 -- 172.21.15.49:0/2486010641 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8924010be0 con 0x7f893c068c80 2024-02-26T06:40:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.741+0000 7f8933400640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f891c0a2230 0x7f891c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.742+0000 7f8933400640 1 -- 172.21.15.49:0/2486010641 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f8924014070 con 0x7f893c068c80 2024-02-26T06:40:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.742+0000 7f893aa00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f891c0a2230 0x7f891c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:01.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.742+0000 7f893aa00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f891c0a2230 0x7f891c0a46f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f892c005fd0 tx=0x7f892c009260 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:01.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.746+0000 7f8933400640 1 -- 172.21.15.49:0/2486010641 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8924052d20 con 0x7f893c068c80 2024-02-26T06:40:01.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.950+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f893c109470 con 0x7f893c068c80 2024-02-26T06:40:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.952+0000 7f8933400640 1 -- 172.21.15.49:0/2486010641 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+23176 (secure 0 0 0) 0x7f89240973d0 con 0x7f893c068c80 2024-02-26T06:40:01.953 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:40:01.954 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":109,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:39:15.282063+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3617020911":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/3210390195":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2527769498":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/3477045895":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/1431851577":"2024-02-27T06:23:23.289826+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/1431851577":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:40:01.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f891c0a2230 msgr2=0x7f891c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f891c0a2230 0x7f891c0a46f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f892c005fd0 tx=0x7f892c009260 comp rx=0 tx=0).stop 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f893c068c80 msgr2=0x7f893c100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f893c068c80 0x7f893c100740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f892400ebc0 tx=0x7f892400c2d0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 shutdown_connections 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f891c0a2230 0x7f891c0a46f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f893c107010 0x7f893c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f893c068c80 0x7f893c100740 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 --2- 172.21.15.49:0/2486010641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0682e0 0x7f893c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.954+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 >> 172.21.15.49:0/2486010641 conn(0x7f893c075a90 msgr2=0x7f893c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.955+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 shutdown_connections 2024-02-26T06:40:01.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:01.955+0000 7f893be00640 1 -- 172.21.15.49:0/2486010641 wait complete. 2024-02-26T06:40:01.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:15:16.187766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:20:29.814371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:20:31.033937+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:40:01.978 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:40:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 -- 172.21.15.49:0/2236805479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0103470 msgr2=0x7f3da0103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 --2- 172.21.15.49:0/2236805479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0103470 0x7f3da0103850 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009930 tx=0x7f3d9002bc20 comp rx=0 tx=0).stop 2024-02-26T06:40:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 -- 172.21.15.49:0/2236805479 shutdown_connections 2024-02-26T06:40:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 --2- 172.21.15.49:0/2236805479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3da0103e20 0x7f3da0107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 --2- 172.21.15.49:0/2236805479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da0108540 0x7f3da0110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 --2- 172.21.15.49:0/2236805479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0103470 0x7f3da0103850 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 -- 172.21.15.49:0/2236805479 >> 172.21.15.49:0/2236805479 conn(0x7f3da00fecb0 msgr2=0x7f3da01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 -- 172.21.15.49:0/2236805479 shutdown_connections 2024-02-26T06:40:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.217+0000 7f3d9f200640 1 -- 172.21.15.49:0/2236805479 wait complete. 2024-02-26T06:40:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.218+0000 7f3d9f200640 1 Processor -- start 2024-02-26T06:40:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9f200640 1 -- start start 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3da0103470 0x7f3da019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0108540 0x7f3da019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9f200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da0199710 0x7f3da0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da019a1e0 con 0x7f3da0108540 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da019a350 con 0x7f3da0103470 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9f200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da019a510 con 0x7f3da0199710 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0108540 0x7f3da019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0108540 0x7f3da019fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51798/0 (socket says 172.21.15.49:51798) 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3da0103470 0x7f3da019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.219+0000 7f3d9d400640 1 -- 172.21.15.49:0/721022125 learned_addr learned my addr 172.21.15.49:0/721022125 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d9e800640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da0199710 0x7f3da0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d9d400640 1 -- 172.21.15.49:0/721022125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da0199710 msgr2=0x7f3da0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d9d400640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da0199710 0x7f3da0199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d9d400640 1 -- 172.21.15.49:0/721022125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3da0103470 msgr2=0x7f3da019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d9d400640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3da0103470 0x7f3da019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d9d400640 1 -- 172.21.15.49:0/721022125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d90009560 con 0x7f3da0108540 2024-02-26T06:40:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d9de00640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3da0103470 0x7f3da019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:40:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d9d400640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0108540 0x7f3da019fb90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800e9c0 tx=0x7f3d8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d96a00640 1 -- 172.21.15.49:0/721022125 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88009700 con 0x7f3da0108540 2024-02-26T06:40:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.220+0000 7f3d96a00640 1 -- 172.21.15.49:0/721022125 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d8800cd80 con 0x7f3da0108540 2024-02-26T06:40:02.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.221+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da01a64b0 con 0x7f3da0108540 2024-02-26T06:40:02.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.221+0000 7f3d96a00640 1 -- 172.21.15.49:0/721022125 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88010430 con 0x7f3da0108540 2024-02-26T06:40:02.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.221+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da01a6990 con 0x7f3da0108540 2024-02-26T06:40:02.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.222+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da0104940 con 0x7f3da0108540 2024-02-26T06:40:02.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.224+0000 7f3d96a00640 1 -- 172.21.15.49:0/721022125 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3d88005000 con 0x7f3da0108540 2024-02-26T06:40:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.225+0000 7f3d96a00640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3d840a2230 0x7f3d840a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.225+0000 7f3d96a00640 1 -- 172.21.15.49:0/721022125 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f3d88014070 con 0x7f3da0108540 2024-02-26T06:40:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.226+0000 7f3d9de00640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3d840a2230 0x7f3d840a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:02.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.226+0000 7f3d9de00640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3d840a2230 0x7f3d840a46f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d90005ab0 tx=0x7f3d90005fd0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.229+0000 7f3d96a00640 1 -- 172.21.15.49:0/721022125 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3d88097140 con 0x7f3da0108540 2024-02-26T06:40:02.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.430+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3da010b770 con 0x7f3da0108540 2024-02-26T06:40:02.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.431+0000 7f3d96a00640 1 -- 172.21.15.49:0/721022125 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d8809a800 con 0x7f3da0108540 2024-02-26T06:40:02.432 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3d840a2230 msgr2=0x7f3d840a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3d840a2230 0x7f3d840a46f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d90005ab0 tx=0x7f3d90005fd0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0108540 msgr2=0x7f3da019fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0108540 0x7f3da019fb90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800e9c0 tx=0x7f3d8800ee80 comp rx=0 tx=0).stop 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 shutdown_connections 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f3d840a2230 0x7f3d840a46f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da0199710 0x7f3da0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0108540 0x7f3da019fb90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 --2- 172.21.15.49:0/721022125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3da0103470 0x7f3da019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 >> 172.21.15.49:0/721022125 conn(0x7f3da00fecb0 msgr2=0x7f3da0100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:02.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 shutdown_connections 2024-02-26T06:40:02.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.434+0000 7f3d9f200640 1 -- 172.21.15.49:0/721022125 wait complete. 2024-02-26T06:40:02.457 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:40:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 -- 172.21.15.49:0/2002058999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2c4107720 msgr2=0x7fc2c4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 --2- 172.21.15.49:0/2002058999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2c4107720 0x7fc2c4107ba0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009890 tx=0x7fc2b002bb70 comp rx=0 tx=0).stop 2024-02-26T06:40:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 -- 172.21.15.49:0/2002058999 shutdown_connections 2024-02-26T06:40:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 --2- 172.21.15.49:0/2002058999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2c41080e0 0x7fc2c41145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 --2- 172.21.15.49:0/2002058999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2c4107720 0x7fc2c4107ba0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 --2- 172.21.15.49:0/2002058999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c410d730 0x7fc2c410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 -- 172.21.15.49:0/2002058999 >> 172.21.15.49:0/2002058999 conn(0x7fc2c40756c0 msgr2=0x7fc2c4077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 -- 172.21.15.49:0/2002058999 shutdown_connections 2024-02-26T06:40:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.699+0000 7fc2cc400640 1 -- 172.21.15.49:0/2002058999 wait complete. 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.700+0000 7fc2cc400640 1 Processor -- start 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2cc400640 1 -- start start 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2cc400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2c4107720 0x7fc2c419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2cc400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c41080e0 0x7fc2c419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2cc400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2c410d730 0x7fc2c41a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2cc400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41a2d20 con 0x7fc2c4107720 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2cc400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41a2e90 con 0x7fc2c41080e0 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2cc400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41a3050 con 0x7fc2c410d730 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2c9a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2c4107720 0x7fc2c419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2c9a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2c4107720 0x7fc2c419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51812/0 (socket says 172.21.15.49:51812) 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2c9a00640 1 -- 172.21.15.49:0/3037371227 learned_addr learned my addr 172.21.15.49:0/3037371227 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2ca400640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2c410d730 0x7fc2c41a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.701+0000 7fc2c9000640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c41080e0 0x7fc2c419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c9a00640 1 -- 172.21.15.49:0/3037371227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2c410d730 msgr2=0x7fc2c41a2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c9a00640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2c410d730 0x7fc2c41a2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c9a00640 1 -- 172.21.15.49:0/3037371227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c41080e0 msgr2=0x7fc2c419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c9a00640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c41080e0 0x7fc2c419e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c9a00640 1 -- 172.21.15.49:0/3037371227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0009490 con 0x7fc2c4107720 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c9a00640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2c4107720 0x7fc2c419de60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800e9c0 tx=0x7fc2b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c2000640 1 -- 172.21.15.49:0/3037371227 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800ccc0 con 0x7fc2c4107720 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c2000640 1 -- 172.21.15.49:0/3037371227 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2b800ce20 con 0x7fc2c4107720 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c41a3330 con 0x7fc2c4107720 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.702+0000 7fc2c2000640 1 -- 172.21.15.49:0/3037371227 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8010430 con 0x7fc2c4107720 2024-02-26T06:40:02.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.703+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c41aabb0 con 0x7fc2c4107720 2024-02-26T06:40:02.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.704+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c40683c0 con 0x7fc2c4107720 2024-02-26T06:40:02.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.706+0000 7fc2c2000640 1 -- 172.21.15.49:0/3037371227 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc2b8004f80 con 0x7fc2c4107720 2024-02-26T06:40:02.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.707+0000 7fc2c2000640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc2ac0a2230 0x7fc2ac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:02.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.707+0000 7fc2c2000640 1 -- 172.21.15.49:0/3037371227 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fc2b8014070 con 0x7fc2c4107720 2024-02-26T06:40:02.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.707+0000 7fc2c9000640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc2ac0a2230 0x7fc2ac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:02.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.708+0000 7fc2c9000640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc2ac0a2230 0x7fc2ac0a46f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00097b0 tx=0x7fc2b00058d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:02.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.712+0000 7fc2c2000640 1 -- 172.21.15.49:0/3037371227 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2b8096fb0 con 0x7fc2c4107720 2024-02-26T06:40:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.918+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2c410fce0 con 0x7fc2c4107720 2024-02-26T06:40:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.919+0000 7fc2c2000640 1 -- 172.21.15.49:0/3037371227 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2b809a670 con 0x7fc2c4107720 2024-02-26T06:40:02.920 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-02-26T06:40:02.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc2ac0a2230 msgr2=0x7fc2ac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc2ac0a2230 0x7fc2ac0a46f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00097b0 tx=0x7fc2b00058d0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2c4107720 msgr2=0x7fc2c419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:02.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2c4107720 0x7fc2c419de60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800e9c0 tx=0x7fc2b800ee80 comp rx=0 tx=0).stop 2024-02-26T06:40:02.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 shutdown_connections 2024-02-26T06:40:02.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fc2ac0a2230 0x7fc2ac0a46f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c41080e0 0x7fc2c419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2c410d730 0x7fc2c41a2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 --2- 172.21.15.49:0/3037371227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2c4107720 0x7fc2c419de60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:02.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 >> 172.21.15.49:0/3037371227 conn(0x7fc2c40756c0 msgr2=0x7fc2c4075e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:02.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 shutdown_connections 2024-02-26T06:40:02.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:02.922+0000 7fc2cc400640 1 -- 172.21.15.49:0/3037371227 wait complete. 2024-02-26T06:40:02.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:40:03.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:02 smithi080 ceph-mon[36206]: pgmap v522: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-02-26T06:40:03.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:02 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2486010641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:40:03.081 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:02 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/721022125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:40:03.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:02 smithi115 ceph-mon[36426]: pgmap v522: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-02-26T06:40:03.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:02 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2486010641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:40:03.091 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:02 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/721022125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:40:03.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:02 smithi049 ceph-mon[31696]: pgmap v522: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-02-26T06:40:03.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:02 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2486010641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:40:03.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:02 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/721022125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:40:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 -- 172.21.15.49:0/2358418609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc111fd0 msgr2=0x7fd7dc1143c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2358418609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc111fd0 0x7fd7dc1143c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009ca0 tx=0x7fd7d402c100 comp rx=0 tx=0).stop 2024-02-26T06:40:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 -- 172.21.15.49:0/2358418609 shutdown_connections 2024-02-26T06:40:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2358418609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc111fd0 0x7fd7dc1143c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2358418609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7dc075fd0 0x7fd7dc111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2358418609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7dc0756b0 0x7fd7dc075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 -- 172.21.15.49:0/2358418609 >> 172.21.15.49:0/2358418609 conn(0x7fd7dc0fee00 msgr2=0x7fd7dc101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:03.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 -- 172.21.15.49:0/2358418609 shutdown_connections 2024-02-26T06:40:03.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.182+0000 7fd7e3600640 1 -- 172.21.15.49:0/2358418609 wait complete. 2024-02-26T06:40:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.183+0000 7fd7e3600640 1 Processor -- start 2024-02-26T06:40:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e3600640 1 -- start start 2024-02-26T06:40:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e3600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc0756b0 0x7fd7dc19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e3600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7dc075fd0 0x7fd7dc19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e3600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7dc111fd0 0x7fd7dc1a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e3600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1a2cf0 con 0x7fd7dc0756b0 2024-02-26T06:40:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e3600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1a2e60 con 0x7fd7dc075fd0 2024-02-26T06:40:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e3600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1a3020 con 0x7fd7dc111fd0 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e0c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc0756b0 0x7fd7dc19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e0c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc0756b0 0x7fd7dc19de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51820/0 (socket says 172.21.15.49:51820) 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e0c00640 1 -- 172.21.15.49:0/2986237426 learned_addr learned my addr 172.21.15.49:0/2986237426 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7e1600640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7dc111fd0 0x7fd7dc1a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.184+0000 7fd7dbe00640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7dc075fd0 0x7fd7dc19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7e0c00640 1 -- 172.21.15.49:0/2986237426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7dc111fd0 msgr2=0x7fd7dc1a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7e0c00640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7dc111fd0 0x7fd7dc1a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7e0c00640 1 -- 172.21.15.49:0/2986237426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7dc075fd0 msgr2=0x7fd7dc19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7e0c00640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7dc075fd0 0x7fd7dc19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7e0c00640 1 -- 172.21.15.49:0/2986237426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d4009900 con 0x7fd7dc0756b0 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7dbe00640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7dc075fd0 0x7fd7dc19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:40:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7e0c00640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc0756b0 0x7fd7dc19de30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000d950 tx=0x7fd7d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7d9600640 1 -- 172.21.15.49:0/2986237426 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0009c30 con 0x7fd7dc0756b0 2024-02-26T06:40:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc1a3300 con 0x7fd7dc0756b0 2024-02-26T06:40:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7d9600640 1 -- 172.21.15.49:0/2986237426 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7d0009d90 con 0x7fd7dc0756b0 2024-02-26T06:40:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.185+0000 7fd7d9600640 1 -- 172.21.15.49:0/2986237426 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d000f7e0 con 0x7fd7dc0756b0 2024-02-26T06:40:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.186+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc1aab80 con 0x7fd7dc0756b0 2024-02-26T06:40:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.187+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7dc0683c0 con 0x7fd7dc0756b0 2024-02-26T06:40:03.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.189+0000 7fd7d9600640 1 -- 172.21.15.49:0/2986237426 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd7d0010ad0 con 0x7fd7dc0756b0 2024-02-26T06:40:03.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.190+0000 7fd7d9600640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd7c40a2230 0x7fd7c40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:03.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.191+0000 7fd7d9600640 1 -- 172.21.15.49:0/2986237426 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7fd7d00cdcd0 con 0x7fd7dc0756b0 2024-02-26T06:40:03.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.191+0000 7fd7dbe00640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd7c40a2230 0x7fd7c40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.191+0000 7fd7dbe00640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd7c40a2230 0x7fd7c40a46f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8005fd0 tx=0x7fd7c800b5d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:03.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.193+0000 7fd7d9600640 1 -- 172.21.15.49:0/2986237426 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7d0096940 con 0x7fd7dc0756b0 2024-02-26T06:40:03.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.395+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7dc19f180 con 0x7fd7dc0756b0 2024-02-26T06:40:03.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.396+0000 7fd7d9600640 1 -- 172.21.15.49:0/2986237426 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7d00200a0 con 0x7fd7dc0756b0 2024-02-26T06:40:03.397 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-02-26T06:40:03.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd7c40a2230 msgr2=0x7fd7c40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd7c40a2230 0x7fd7c40a46f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8005fd0 tx=0x7fd7c800b5d0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc0756b0 msgr2=0x7fd7dc19de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc0756b0 0x7fd7dc19de30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000d950 tx=0x7fd7d000de10 comp rx=0 tx=0).stop 2024-02-26T06:40:03.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 shutdown_connections 2024-02-26T06:40:03.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fd7c40a2230 0x7fd7c40a46f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7dc075fd0 0x7fd7dc19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7dc111fd0 0x7fd7dc1a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 --2- 172.21.15.49:0/2986237426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc0756b0 0x7fd7dc19de30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 >> 172.21.15.49:0/2986237426 conn(0x7fd7dc0fee00 msgr2=0x7fd7dc1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:03.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 shutdown_connections 2024-02-26T06:40:03.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.399+0000 7fd7e3600640 1 -- 172.21.15.49:0/2986237426 wait complete. 2024-02-26T06:40:03.420 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T06:40:03.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 -- 172.21.15.49:0/3290289993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c10d730 msgr2=0x7f654c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 --2- 172.21.15.49:0/3290289993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c10d730 0x7f654c10db10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6544009cd0 tx=0x7f654402c0e0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 -- 172.21.15.49:0/3290289993 shutdown_connections 2024-02-26T06:40:03.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 --2- 172.21.15.49:0/3290289993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f654c1080e0 0x7f654c1145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 --2- 172.21.15.49:0/3290289993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f654c107720 0x7f654c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 --2- 172.21.15.49:0/3290289993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c10d730 0x7f654c10db10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 -- 172.21.15.49:0/3290289993 >> 172.21.15.49:0/3290289993 conn(0x7f654c0756c0 msgr2=0x7f654c077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:03.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 -- 172.21.15.49:0/3290289993 shutdown_connections 2024-02-26T06:40:03.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.699+0000 7f6554000640 1 -- 172.21.15.49:0/3290289993 wait complete. 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.700+0000 7f6554000640 1 Processor -- start 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.700+0000 7f6554000640 1 -- start start 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6554000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c107720 0x7f654c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6554000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f654c1080e0 0x7f654c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6554000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f654c10d730 0x7f654c1a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6554000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c1a2ca0 con 0x7f654c10d730 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6554000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c1a2e10 con 0x7f654c1080e0 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6554000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c1a2fd0 con 0x7f654c107720 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6552000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f654c10d730 0x7f654c1a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6551600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c107720 0x7f654c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6551600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c107720 0x7f654c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41194/0 (socket says 172.21.15.49:41194) 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6552000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f654c10d730 0x7f654c1a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51832/0 (socket says 172.21.15.49:51832) 2024-02-26T06:40:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6551600640 1 -- 172.21.15.49:0/4169938203 learned_addr learned my addr 172.21.15.49:0/4169938203 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6550c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f654c1080e0 0x7f654c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6551600640 1 -- 172.21.15.49:0/4169938203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f654c1080e0 msgr2=0x7f654c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6551600640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f654c1080e0 0x7f654c19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6551600640 1 -- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f654c10d730 msgr2=0x7f654c1a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6551600640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f654c10d730 0x7f654c1a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.701+0000 7f6551600640 1 -- 172.21.15.49:0/4169938203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538009490 con 0x7f654c107720 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.702+0000 7f6551600640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c107720 0x7f654c19dde0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6544004af0 tx=0x7f65440325e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.702+0000 7f654a000640 1 -- 172.21.15.49:0/4169938203 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544034e20 con 0x7f654c107720 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.702+0000 7f654a000640 1 -- 172.21.15.49:0/4169938203 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6544003e50 con 0x7f654c107720 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.702+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6544009900 con 0x7f654c107720 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.702+0000 7f654a000640 1 -- 172.21.15.49:0/4169938203 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544004760 con 0x7f654c107720 2024-02-26T06:40:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.702+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654c1aa950 con 0x7f654c107720 2024-02-26T06:40:03.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.703+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654c0683c0 con 0x7f654c107720 2024-02-26T06:40:03.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.705+0000 7f654a000640 1 -- 172.21.15.49:0/4169938203 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f654400a040 con 0x7f654c107720 2024-02-26T06:40:03.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.707+0000 7f654a000640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65300a2230 0x7f65300a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:03.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.707+0000 7f654a000640 1 -- 172.21.15.49:0/4169938203 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7f65440f0820 con 0x7f654c107720 2024-02-26T06:40:03.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.707+0000 7f6550c00640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65300a2230 0x7f65300a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:03.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.708+0000 7f6550c00640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65300a2230 0x7f65300a46f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f654c077da0 tx=0x7f6538005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:03.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.711+0000 7f654a000640 1 -- 172.21.15.49:0/4169938203 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65440b9540 con 0x7f654c107720 2024-02-26T06:40:03.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.930+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f654c10fc90 con 0x7f654c107720 2024-02-26T06:40:03.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.931+0000 7f654a000640 1 -- 172.21.15.49:0/4169938203 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v109) v1 ==== 81+0+943 (secure 0 0 0) 0x7f65440bcc00 con 0x7f654c107720 2024-02-26T06:40:03.933 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3617020911 2024-02-27T06:23:23.289826+0000 2024-02-26T06:40:03.933 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3210390195 2024-02-27T06:23:23.289826+0000 2024-02-26T06:40:03.933 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2527769498 2024-02-27T06:23:23.289826+0000 2024-02-26T06:40:03.933 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3477045895 2024-02-27T06:23:23.289826+0000 2024-02-26T06:40:03.933 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2180727249 2024-02-27T06:12:27.274814+0000 2024-02-26T06:40:03.933 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2102614714 2024-02-27T06:12:27.274814+0000 2024-02-26T06:40:03.933 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/1431851577 2024-02-27T06:23:23.289826+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/4089389406 2024-02-27T06:12:27.274814+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3504875682 2024-02-27T06:12:27.274814+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/746416011 2024-02-27T06:11:51.411945+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/765116076 2024-02-27T06:11:51.411945+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/1431851577 2024-02-27T06:23:23.289826+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2937664554 2024-02-27T06:11:51.411945+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/765116076 2024-02-27T06:11:51.411945+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2168062918 2024-02-27T06:11:51.411945+0000 2024-02-26T06:40:03.934 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/4089389406 2024-02-27T06:12:27.274814+0000 2024-02-26T06:40:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:listed 16 entries 2024-02-26T06:40:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65300a2230 msgr2=0x7f65300a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65300a2230 0x7f65300a46f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f654c077da0 tx=0x7f6538005f90 comp rx=0 tx=0).stop 2024-02-26T06:40:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c107720 msgr2=0x7f654c19dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c107720 0x7f654c19dde0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6544004af0 tx=0x7f65440325e0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 shutdown_connections 2024-02-26T06:40:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f65300a2230 0x7f65300a46f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f654c1080e0 0x7f654c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f654c10d730 0x7f654c1a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 --2- 172.21.15.49:0/4169938203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f654c107720 0x7f654c19dde0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.934+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 >> 172.21.15.49:0/4169938203 conn(0x7f654c0756c0 msgr2=0x7f654c075f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.935+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 shutdown_connections 2024-02-26T06:40:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:03.935+0000 7f6554000640 1 -- 172.21.15.49:0/4169938203 wait complete. 2024-02-26T06:40:03.958 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-26T06:40:03.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:40:03.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /tmp/cephfs 2024-02-26T06:40:04.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:03 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3037371227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:40:04.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:03 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2986237426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:40:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:03 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3037371227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:40:04.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:03 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2986237426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:40:04.214 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:40:04.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:40:04.214 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-02-26T06:40:04.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:40:04.325 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-02-26T06:40:04.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:03 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3037371227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:40:04.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:03 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2986237426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:40:04.453 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-02-26T06:40:04.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:40:04.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-02-26T06:40:04.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:40:04.598 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-02-26T06:40:04.804 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-26T06:40:04.804 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-02-26T06:40:04.804 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T06:40:04.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053411,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/68ba21dd65729f3cbc24d11cfef07b34b51128e07de162247ba9572806cc546a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/68ba21dd65729f3cbc24d11cfef07b34b51128e07de162247ba9572806cc546a/diff,workdir=/var/lib/containers/storage/overlay/68ba21dd65729f3cbc24d11cfef07b34b51128e07de162247ba9572806cc546a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f1682739d815a9379c3fd0e49686d80b4df2f149f19c15a049890f40e2859aff/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/f1682739d815a9379c3fd0e49686d80b4df2f149f19c15a049890f40e2859aff/diff,workdir=/var/lib/containers/storage/overlay/f1682739d815a9379c3fd0e49686d80b4df2f149f19c15a049890f40e2859aff/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/34827bc72fb45092da5a6a4de92fa485e87593667ff8aca2aaba7e87a6fb00fb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/34827bc72fb45092da5a6a4de92fa485e87593667ff8aca2aaba7e87a6fb00fb/diff,workdir=/var/lib/containers/storage/overlay/34827bc72fb45092da5a6a4de92fa485e87593667ff8aca2aaba7e87a6fb00fb/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/03739fd2320473fbbdd6663ec1beceb08ad9bb85dac74b27a315090fe2a9af65/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/03739fd2320473fbbdd6663ec1beceb08ad9bb85dac74b27a315090fe2a9af65/diff,workdir=/var/lib/containers/storage/overlay/03739fd2320473fbbdd6663ec1beceb08ad9bb85dac74b27a315090fe2a9af65/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5ff2f676491eb937e63fd1e2bc3e20e26249bb7c95e1d7c6defe64a002d08982/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/5ff2f676491eb937e63fd1e2bc3e20e26249bb7c95e1d7c6defe64a002d08982/diff,workdir=/var/lib/containers/storage/overlay/5ff2f676491eb937e63fd1e2bc3e20e26249bb7c95e1d7c6defe64a002d08982/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4be2dc69b7f199e54ce033fcf9c90753ebac5b80999e92cc21d85dd1dfba8286/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/4be2dc69b7f199e54ce033fcf9c90753ebac5b80999e92cc21d85dd1dfba8286/diff,workdir=/var/lib/containers/storage/overlay/4be2dc69b7f199e54ce033fcf9c90753ebac5b80999e92cc21d85dd1dfba8286/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/96331eeec3bc8f5d0975e577dd42c5e7e115b2e2317a0770663c5cf38d8fa95e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/96331eeec3bc8f5d0975e577dd42c5e7e115b2e2317a0770663c5cf38d8fa95e/diff,workdir=/var/lib/containers/storage/overlay/96331eeec3bc8f5d0975e577dd42c5e7e115b2e2317a0770663c5cf38d8fa95e/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dd68b84ff27d4ba4116daa1d48426e8a62e3ba16a3f045cf6eddbf8b60a41fac/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/dd68b84ff27d4ba4116daa1d48426e8a62e3ba16a3f045cf6eddbf8b60a41fac/diff,workdir=/var/lib/containers/storage/overlay/dd68b84ff27d4ba4116daa1d48426e8a62e3ba16a3f045cf6eddbf8b60a41fac/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/925367ad3eed1d669845144f845176417ee64e4c9099c568bad912f31890ca37/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/925367ad3eed1d669845144f845176417ee64e4c9099c568bad912f31890ca37/diff,workdir=/var/lib/containers/storage/overlay/925367ad3eed1d669845144f845176417ee64e4c9099c568bad912f31890ca37/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/110f21e2b9e31a174744c2fcdedc8e38f5efffabcedfc55edb0556ae98f66898/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ITBWDYFWUMWIRBMHMWLKL7L4XB:/var/lib/containers/storage/overlay/l/666H4S7HOV7ZS5XHS675JVPXDL,upperdir=/var/lib/containers/storage/overlay/110f21e2b9e31a174744c2fcdedc8e38f5efffabcedfc55edb0556ae98f66898/diff,workdir=/var/lib/containers/storage/overlay/110f21e2b9e31a174744c2fcdedc8e38f5efffabcedfc55edb0556ae98f66898/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 2024-02-26T06:40:04.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:40:04.881 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-02-26T06:40:04.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:40:04.966 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-02-26T06:40:05.168 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:04 smithi049 ceph-mon[31696]: pgmap v523: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-02-26T06:40:05.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:04 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4169938203' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T06:40:05.172 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-02-26T06:40:05.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:40:05.177 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-26T06:40:05.189 INFO:tasks.cephadm:Teardown begin 2024-02-26T06:40:05.189 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T06:40:05.189 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:40:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:04 smithi080 ceph-mon[36206]: pgmap v523: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-02-26T06:40:05.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:04 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4169938203' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T06:40:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:04 smithi115 ceph-mon[36426]: pgmap v523: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-02-26T06:40:05.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:04 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4169938203' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T06:40:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 -- 172.21.15.49:0/2389393377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8068820 msgr2=0x7ff9b8068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 --2- 172.21.15.49:0/2389393377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8068820 0x7ff9b8068c80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009860 tx=0x7ff9a802bb70 comp rx=0 tx=0).stop 2024-02-26T06:40:05.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 -- 172.21.15.49:0/2389393377 shutdown_connections 2024-02-26T06:40:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 --2- 172.21.15.49:0/2389393377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b810d830 0x7ff9b810fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 --2- 172.21.15.49:0/2389393377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8068820 0x7ff9b8068c80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 --2- 172.21.15.49:0/2389393377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b8101e90 0x7ff9b80682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 -- 172.21.15.49:0/2389393377 >> 172.21.15.49:0/2389393377 conn(0x7ff9b8075a90 msgr2=0x7ff9b8075ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 -- 172.21.15.49:0/2389393377 shutdown_connections 2024-02-26T06:40:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.386+0000 7ff9bf200640 1 -- 172.21.15.49:0/2389393377 wait complete. 2024-02-26T06:40:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.388+0000 7ff9bf200640 1 Processor -- start 2024-02-26T06:40:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.388+0000 7ff9bf200640 1 -- start start 2024-02-26T06:40:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bf200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b8068820 0x7ff9b819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bf200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8101e90 0x7ff9b819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bf200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b810d830 0x7ff9b8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:05.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bf200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8199c70 con 0x7ff9b8068820 2024-02-26T06:40:05.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bf200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8199de0 con 0x7ff9b810d830 2024-02-26T06:40:05.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bf200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8199fa0 con 0x7ff9b8101e90 2024-02-26T06:40:05.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9be800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b810d830 0x7ff9b8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:05.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bd400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8101e90 0x7ff9b819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:05.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bde00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b8068820 0x7ff9b819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:05.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9be800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b810d830 0x7ff9b8199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:55912/0 (socket says 172.21.15.49:55912) 2024-02-26T06:40:05.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9be800640 1 -- 172.21.15.49:0/455153866 learned_addr learned my addr 172.21.15.49:0/455153866 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:05.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bd400640 1 -- 172.21.15.49:0/455153866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b810d830 msgr2=0x7ff9b8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:05.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bd400640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b810d830 0x7ff9b8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bd400640 1 -- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b8068820 msgr2=0x7ff9b819f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:05.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bd400640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b8068820 0x7ff9b819f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9bd400640 1 -- 172.21.15.49:0/455153866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4009900 con 0x7ff9b8101e90 2024-02-26T06:40:05.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.389+0000 7ff9be800640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b810d830 0x7ff9b8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:40:05.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.390+0000 7ff9bd400640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8101e90 0x7ff9b819f9b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff9a802c0b0 tx=0x7ff9a80045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:05.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.390+0000 7ff9aea00640 1 -- 172.21.15.49:0/455153866 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80349d0 con 0x7ff9b8101e90 2024-02-26T06:40:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.390+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a8009490 con 0x7ff9b8101e90 2024-02-26T06:40:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.390+0000 7ff9aea00640 1 -- 172.21.15.49:0/455153866 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9a802c9f0 con 0x7ff9b8101e90 2024-02-26T06:40:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.390+0000 7ff9aea00640 1 -- 172.21.15.49:0/455153866 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80035f0 con 0x7ff9b8101e90 2024-02-26T06:40:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.390+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b819a450 con 0x7ff9b8101e90 2024-02-26T06:40:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.391+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b8067e90 con 0x7ff9b8101e90 2024-02-26T06:40:05.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.393+0000 7ff9aea00640 1 -- 172.21.15.49:0/455153866 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff9a803b040 con 0x7ff9b8101e90 2024-02-26T06:40:05.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.394+0000 7ff9aea00640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff9a40a2230 0x7ff9a40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:05.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.395+0000 7ff9aea00640 1 -- 172.21.15.49:0/455153866 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8550+0+0 (secure 0 0 0) 0x7ff9a80efa60 con 0x7ff9b8101e90 2024-02-26T06:40:05.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.395+0000 7ff9bde00640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff9a40a2230 0x7ff9a40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:05.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.395+0000 7ff9bde00640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff9a40a2230 0x7ff9a40a46f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005fd0 tx=0x7ff9b4010040 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:05.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.397+0000 7ff9aea00640 1 -- 172.21.15.49:0/455153866 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9a80b8830 con 0x7ff9b8101e90 2024-02-26T06:40:05.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.627+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9b819ab50 con 0x7ff9b8101e90 2024-02-26T06:40:05.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.628+0000 7ff9aea00640 1 -- 172.21.15.49:0/455153866 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+11342 (secure 0 0 0) 0x7ff9a8041090 con 0x7ff9b8101e90 2024-02-26T06:40:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:40:05.629 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:39:33.249841+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":144,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:40:05.630 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 56 2024-02-26T06:40:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.630+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff9a40a2230 msgr2=0x7ff9a40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.630+0000 7ff9bf200640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff9a40a2230 0x7ff9a40a46f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005fd0 tx=0x7ff9b4010040 comp rx=0 tx=0).stop 2024-02-26T06:40:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.630+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8101e90 msgr2=0x7ff9b819f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.630+0000 7ff9bf200640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8101e90 0x7ff9b819f9b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff9a802c0b0 tx=0x7ff9a80045b0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.631+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 shutdown_connections 2024-02-26T06:40:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.631+0000 7ff9bf200640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7ff9a40a2230 0x7ff9a40a46f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.631+0000 7ff9bf200640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b810d830 0x7ff9b8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.631+0000 7ff9bf200640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9b8101e90 0x7ff9b819f9b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.631+0000 7ff9bf200640 1 --2- 172.21.15.49:0/455153866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9b8068820 0x7ff9b819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.631+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 >> 172.21.15.49:0/455153866 conn(0x7ff9b8075a90 msgr2=0x7ff9b80ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.631+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 shutdown_connections 2024-02-26T06:40:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.631+0000 7ff9bf200640 1 -- 172.21.15.49:0/455153866 wait complete. 2024-02-26T06:40:05.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:40:05.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.878+0000 7fe17ce00640 1 -- 172.21.15.49:0/121820092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1781083f0 msgr2=0x7fe1781087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:05.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.878+0000 7fe17ce00640 1 --2- 172.21.15.49:0/121820092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1781083f0 0x7fe1781087d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe168009930 tx=0x7fe16802bc60 comp rx=0 tx=0).stop 2024-02-26T06:40:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.882+0000 7fe17ce00640 1 -- 172.21.15.49:0/121820092 shutdown_connections 2024-02-26T06:40:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.882+0000 7fe17ce00640 1 --2- 172.21.15.49:0/121820092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe17810cca0 0x7fe178114870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.882+0000 7fe17ce00640 1 --2- 172.21.15.49:0/121820092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe178108da0 0x7fe17810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.882+0000 7fe17ce00640 1 --2- 172.21.15.49:0/121820092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1781083f0 0x7fe1781087d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.882+0000 7fe17ce00640 1 -- 172.21.15.49:0/121820092 >> 172.21.15.49:0/121820092 conn(0x7fe17806c060 msgr2=0x7fe17806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.883+0000 7fe17ce00640 1 -- 172.21.15.49:0/121820092 shutdown_connections 2024-02-26T06:40:05.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.883+0000 7fe17ce00640 1 -- 172.21.15.49:0/121820092 wait complete. 2024-02-26T06:40:05.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.886+0000 7fe17ce00640 1 Processor -- start 2024-02-26T06:40:05.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.886+0000 7fe17ce00640 1 -- start start 2024-02-26T06:40:05.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.887+0000 7fe17ce00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1781083f0 0x7fe17819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:05.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.887+0000 7fe17ce00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe17810cca0 0x7fe1781a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:05.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.887+0000 7fe17ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe17819e0b0 0x7fe17819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:05.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.887+0000 7fe17ce00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1781b2ae0 con 0x7fe17810cca0 2024-02-26T06:40:05.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.887+0000 7fe17ce00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1781b2c50 con 0x7fe17819e0b0 2024-02-26T06:40:05.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.887+0000 7fe17ce00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1781b2e10 con 0x7fe1781083f0 2024-02-26T06:40:05.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.888+0000 7fe177400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1781083f0 0x7fe17819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:05.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.888+0000 7fe177400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1781083f0 0x7fe17819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41234/0 (socket says 172.21.15.49:41234) 2024-02-26T06:40:05.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.888+0000 7fe177400640 1 -- 172.21.15.49:0/2383456169 learned_addr learned my addr 172.21.15.49:0/2383456169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:05.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.888+0000 7fe176a00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe17810cca0 0x7fe1781a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:05.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.888+0000 7fe177e00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe17819e0b0 0x7fe17819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:05.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.889+0000 7fe176a00640 1 -- 172.21.15.49:0/2383456169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1781083f0 msgr2=0x7fe17819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:05.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.889+0000 7fe176a00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1781083f0 0x7fe17819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.889+0000 7fe176a00640 1 -- 172.21.15.49:0/2383456169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe17819e0b0 msgr2=0x7fe17819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:05.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.889+0000 7fe176a00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe17819e0b0 0x7fe17819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:05.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.889+0000 7fe176a00640 1 -- 172.21.15.49:0/2383456169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe168009560 con 0x7fe17810cca0 2024-02-26T06:40:05.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.890+0000 7fe176a00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe17810cca0 0x7fe1781a4af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe16400b3c0 tx=0x7fe16400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:05.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.892+0000 7fe177400640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1781083f0 0x7fe17819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:40:05.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.892+0000 7fe16fe00640 1 -- 172.21.15.49:0/2383456169 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164011830 con 0x7fe17810cca0 2024-02-26T06:40:05.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.893+0000 7fe16fe00640 1 -- 172.21.15.49:0/2383456169 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe164011e70 con 0x7fe17810cca0 2024-02-26T06:40:05.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.893+0000 7fe16fe00640 1 -- 172.21.15.49:0/2383456169 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400f2e0 con 0x7fe17810cca0 2024-02-26T06:40:05.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.893+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1781b30f0 con 0x7fe17810cca0 2024-02-26T06:40:05.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.893+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1781b3600 con 0x7fe17810cca0 2024-02-26T06:40:05.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.896+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe178067e90 con 0x7fe17810cca0 2024-02-26T06:40:05.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.906+0000 7fe16fe00640 1 -- 172.21.15.49:0/2383456169 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe1640105d0 con 0x7fe17810cca0 2024-02-26T06:40:05.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.913+0000 7fe16fe00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe15c0a2310 0x7fe15c0a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:05.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.913+0000 7fe16fe00640 1 -- 172.21.15.49:0/2383456169 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fe1640cd730 con 0x7fe17810cca0 2024-02-26T06:40:05.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.913+0000 7fe177400640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe15c0a2310 0x7fe15c0a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:05.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.916+0000 7fe16fe00640 1 -- 172.21.15.49:0/2383456169 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe164096420 con 0x7fe17810cca0 2024-02-26T06:40:05.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:05.917+0000 7fe177400640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe15c0a2310 0x7fe15c0a47d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe168006010 tx=0x7fe16802c600 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:05.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:05 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/455153866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:40:06.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:05 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/455153866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:40:06.343 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:05 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/455153866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:40:06.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.366+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe178117df0 con 0x7fe17810cca0 2024-02-26T06:40:06.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.368+0000 7fe16fe00640 1 -- 172.21.15.49:0/2383456169 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+11342 (secure 0 0 0) 0x7fe164099ae0 con 0x7fe17810cca0 2024-02-26T06:40:06.369 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:40:06.369 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/180447409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":180447409},{"type":"v1","addr":"172.21.15.49:6835","nonce":180447409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1053244563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1053244563},{"type":"v1","addr":"172.21.15.49:6839","nonce":1053244563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2984977662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2984977662},{"type":"v1","addr":"172.21.15.49:6841","nonce":2984977662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/914291308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":914291308},{"type":"v1","addr":"172.21.15.80:6833","nonce":914291308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1451697651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1451697651},{"type":"v1","addr":"172.21.15.115:6833","nonce":1451697651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/879820521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":879820521},{"type":"v1","addr":"172.21.15.49:6837","nonce":879820521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:20:31.555053+0000","modified":"2024-02-26T06:39:33.249841+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6835/1885591642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1885591642},{"type":"v1","addr":"172.21.15.80:6835","nonce":1885591642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.80:6837/2132318232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":2132318232},{"type":"v1","addr":"172.21.15.80:6837","nonce":2132318232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/2578799425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2578799425},{"type":"v1","addr":"172.21.15.115:6835","nonce":2578799425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":144,"addr":"172.21.15.115:6837/3720601525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3720601525},{"type":"v1","addr":"172.21.15.115:6837","nonce":3720601525}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.115:6839/318392314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":318392314},{"type":"v1","addr":"172.21.15.115:6839","nonce":318392314}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6839/3160003047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":3160003047},{"type":"v1","addr":"172.21.15.80:6839","nonce":3160003047}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:40:06.369 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 56 2024-02-26T06:40:06.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.373+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe15c0a2310 msgr2=0x7fe15c0a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:06.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.373+0000 7fe17ce00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe15c0a2310 0x7fe15c0a47d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe168006010 tx=0x7fe16802c600 comp rx=0 tx=0).stop 2024-02-26T06:40:06.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.373+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe17810cca0 msgr2=0x7fe1781a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:06.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.373+0000 7fe17ce00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe17810cca0 0x7fe1781a4af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe16400b3c0 tx=0x7fe16400b880 comp rx=0 tx=0).stop 2024-02-26T06:40:06.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.374+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 shutdown_connections 2024-02-26T06:40:06.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.374+0000 7fe17ce00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe15c0a2310 0x7fe15c0a47d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.374+0000 7fe17ce00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe17819e0b0 0x7fe17819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.374+0000 7fe17ce00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe17810cca0 0x7fe1781a4af0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.374+0000 7fe17ce00640 1 --2- 172.21.15.49:0/2383456169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1781083f0 0x7fe17819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.374+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 >> 172.21.15.49:0/2383456169 conn(0x7fe17806c060 msgr2=0x7fe1780705f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:06.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.374+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 shutdown_connections 2024-02-26T06:40:06.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.374+0000 7fe17ce00640 1 -- 172.21.15.49:0/2383456169 wait complete. 2024-02-26T06:40:06.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:40:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 -- 172.21.15.49:0/1809560863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c1011e0 msgr2=0x7fa61c1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 --2- 172.21.15.49:0/1809560863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c1011e0 0x7fa61c1015c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009900 tx=0x7fa60c02bc40 comp rx=0 tx=0).stop 2024-02-26T06:40:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 -- 172.21.15.49:0/1809560863 shutdown_connections 2024-02-26T06:40:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 --2- 172.21.15.49:0/1809560863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa61c1062b0 0x7fa61c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 --2- 172.21.15.49:0/1809560863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c101b90 0x7fa61c105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 --2- 172.21.15.49:0/1809560863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c1011e0 0x7fa61c1015c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 -- 172.21.15.49:0/1809560863 >> 172.21.15.49:0/1809560863 conn(0x7fa61c0fca60 msgr2=0x7fa61c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:06.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 -- 172.21.15.49:0/1809560863 shutdown_connections 2024-02-26T06:40:06.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.598+0000 7fa61be00640 1 -- 172.21.15.49:0/1809560863 wait complete. 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.599+0000 7fa61be00640 1 Processor -- start 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.599+0000 7fa61be00640 1 -- start start 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa61c1011e0 0x7fa61c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c1062b0 0x7fa61c19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c199630 0x7fa61c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19a100 con 0x7fa61c199630 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19a270 con 0x7fa61c1011e0 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61be00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19a430 con 0x7fa61c1062b0 2024-02-26T06:40:06.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa61c1011e0 0x7fa61c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c1062b0 0x7fa61c19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa61c1011e0 0x7fa61c19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:55952/0 (socket says 172.21.15.49:55952) 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61aa00640 1 -- 172.21.15.49:0/778643566 learned_addr learned my addr 172.21.15.49:0/778643566 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61b400640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c199630 0x7fa61c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61aa00640 1 -- 172.21.15.49:0/778643566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c1062b0 msgr2=0x7fa61c19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61aa00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c1062b0 0x7fa61c19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61aa00640 1 -- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c199630 msgr2=0x7fa61c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61aa00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c199630 0x7fa61c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.600+0000 7fa61aa00640 1 -- 172.21.15.49:0/778643566 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c009560 con 0x7fa61c1011e0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.601+0000 7fa61aa00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa61c1011e0 0x7fa61c19f570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa60c000c00 tx=0x7fa60c02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.601+0000 7fa613400640 1 -- 172.21.15.49:0/778643566 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c004410 con 0x7fa61c1011e0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.601+0000 7fa613400640 1 -- 172.21.15.49:0/778643566 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa60c0052d0 con 0x7fa61c1011e0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.601+0000 7fa613400640 1 -- 172.21.15.49:0/778643566 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c003d20 con 0x7fa61c1011e0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.601+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c1a6370 con 0x7fa61c1011e0 2024-02-26T06:40:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.601+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c1a6800 con 0x7fa61c1011e0 2024-02-26T06:40:06.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.602+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c1026b0 con 0x7fa61c1011e0 2024-02-26T06:40:06.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.606+0000 7fa613400640 1 -- 172.21.15.49:0/778643566 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa60c005440 con 0x7fa61c1011e0 2024-02-26T06:40:06.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.607+0000 7fa613400640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa6000a2230 0x7fa6000a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:06.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.608+0000 7fa613400640 1 -- 172.21.15.49:0/778643566 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8594+0+0 (secure 0 0 0) 0x7fa60c0f0690 con 0x7fa61c1011e0 2024-02-26T06:40:06.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.608+0000 7fa61a000640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa6000a2230 0x7fa6000a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.608+0000 7fa61a000640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa6000a2230 0x7fa6000a46f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa61c19ae80 tx=0x7fa608005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.612+0000 7fa613400640 1 -- 172.21.15.49:0/778643566 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa60c0b9380 con 0x7fa61c1011e0 2024-02-26T06:40:06.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.811+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa61c067e90 con 0x7fa61c1011e0 2024-02-26T06:40:06.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.812+0000 7fa613400640 1 -- 172.21.15.49:0/778643566 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+23244 (secure 0 0 0) 0x7fa60c0bca40 con 0x7fa61c1011e0 2024-02-26T06:40:06.813 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-02-26T06:40:06.814 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":110,"fsid":"a9ae7f00-d46d-11ee-95c3-87774f69a715","created":"2024-02-26T06:11:09.086607+0000","modified":"2024-02-26T06:40:05.842872+0000","last_up_change":"2024-02-26T06:19:33.033894+0000","last_in_change":"2024-02-26T06:19:13.120930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:15:16.187766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:20:29.814371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:20:31.033937+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db966476-ec8d-4000-8d96-2944d08fecca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6803","nonce":3449358654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6805","nonce":3449358654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6809","nonce":3449358654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3449358654},{"type":"v1","addr":"172.21.15.49:6807","nonce":3449358654}]},"public_addr":"172.21.15.49:6803/3449358654","cluster_addr":"172.21.15.49:6805/3449358654","heartbeat_back_addr":"172.21.15.49:6809/3449358654","heartbeat_front_addr":"172.21.15.49:6807/3449358654","state":["exists","up"]},{"osd":1,"uuid":"5cbb4214-1d89-4598-834c-d567d0d63d04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6801","nonce":3019548533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6803","nonce":3019548533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6807","nonce":3019548533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":3019548533},{"type":"v1","addr":"172.21.15.80:6805","nonce":3019548533}]},"public_addr":"172.21.15.80:6801/3019548533","cluster_addr":"172.21.15.80:6803/3019548533","heartbeat_back_addr":"172.21.15.80:6807/3019548533","heartbeat_front_addr":"172.21.15.80:6805/3019548533","state":["exists","up"]},{"osd":2,"uuid":"d416d235-0756-4c38-835b-3d8a8e04d2cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6801","nonce":3987885615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6803","nonce":3987885615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6807","nonce":3987885615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3987885615},{"type":"v1","addr":"172.21.15.115:6805","nonce":3987885615}]},"public_addr":"172.21.15.115:6801/3987885615","cluster_addr":"172.21.15.115:6803/3987885615","heartbeat_back_addr":"172.21.15.115:6807/3987885615","heartbeat_front_addr":"172.21.15.115:6805/3987885615","state":["exists","up"]},{"osd":3,"uuid":"674ab7f1-6ee1-4e0d-9e10-42770a30c5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6811","nonce":2774029183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6813","nonce":2774029183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6817","nonce":2774029183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":2774029183},{"type":"v1","addr":"172.21.15.49:6815","nonce":2774029183}]},"public_addr":"172.21.15.49:6811/2774029183","cluster_addr":"172.21.15.49:6813/2774029183","heartbeat_back_addr":"172.21.15.49:6817/2774029183","heartbeat_front_addr":"172.21.15.49:6815/2774029183","state":["exists","up"]},{"osd":4,"uuid":"41dcb99d-6d5c-4ee1-a461-06944881881c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6809","nonce":1597953633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6811","nonce":1597953633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6815","nonce":1597953633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":1597953633},{"type":"v1","addr":"172.21.15.80:6813","nonce":1597953633}]},"public_addr":"172.21.15.80:6809/1597953633","cluster_addr":"172.21.15.80:6811/1597953633","heartbeat_back_addr":"172.21.15.80:6815/1597953633","heartbeat_front_addr":"172.21.15.80:6813/1597953633","state":["exists","up"]},{"osd":5,"uuid":"53d273fb-162f-4ca7-b995-1387266dacb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6809","nonce":1277586468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6811","nonce":1277586468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6815","nonce":1277586468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1277586468},{"type":"v1","addr":"172.21.15.115:6813","nonce":1277586468}]},"public_addr":"172.21.15.115:6809/1277586468","cluster_addr":"172.21.15.115:6811/1277586468","heartbeat_back_addr":"172.21.15.115:6815/1277586468","heartbeat_front_addr":"172.21.15.115:6813/1277586468","state":["exists","up"]},{"osd":6,"uuid":"570e6f42-9220-477e-b01f-0a4169640324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6819","nonce":2967719452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6821","nonce":2967719452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6825","nonce":2967719452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2967719452},{"type":"v1","addr":"172.21.15.49:6823","nonce":2967719452}]},"public_addr":"172.21.15.49:6819/2967719452","cluster_addr":"172.21.15.49:6821/2967719452","heartbeat_back_addr":"172.21.15.49:6825/2967719452","heartbeat_front_addr":"172.21.15.49:6823/2967719452","state":["exists","up"]},{"osd":7,"uuid":"ecc9f4ec-6db8-44c7-bcf3-4c9d75ee349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6817","nonce":4119828999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6819","nonce":4119828999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6823","nonce":4119828999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4119828999},{"type":"v1","addr":"172.21.15.80:6821","nonce":4119828999}]},"public_addr":"172.21.15.80:6817/4119828999","cluster_addr":"172.21.15.80:6819/4119828999","heartbeat_back_addr":"172.21.15.80:6823/4119828999","heartbeat_front_addr":"172.21.15.80:6821/4119828999","state":["exists","up"]},{"osd":8,"uuid":"c86f9fc9-9e22-4686-83ef-2a7b3b0652cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6817","nonce":2102490724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6819","nonce":2102490724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6823","nonce":2102490724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2102490724},{"type":"v1","addr":"172.21.15.115:6821","nonce":2102490724}]},"public_addr":"172.21.15.115:6817/2102490724","cluster_addr":"172.21.15.115:6819/2102490724","heartbeat_back_addr":"172.21.15.115:6823/2102490724","heartbeat_front_addr":"172.21.15.115:6821/2102490724","state":["exists","up"]},{"osd":9,"uuid":"438322dd-a5d2-4d7a-b7f6-1de27c32fa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6827","nonce":4117635859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6829","nonce":4117635859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6833","nonce":4117635859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":4117635859},{"type":"v1","addr":"172.21.15.49:6831","nonce":4117635859}]},"public_addr":"172.21.15.49:6827/4117635859","cluster_addr":"172.21.15.49:6829/4117635859","heartbeat_back_addr":"172.21.15.49:6833/4117635859","heartbeat_front_addr":"172.21.15.49:6831/4117635859","state":["exists","up"]},{"osd":10,"uuid":"88c87856-9714-45a8-be85-a007457ae094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6825","nonce":1162870008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6827","nonce":1162870008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6831","nonce":1162870008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":1162870008},{"type":"v1","addr":"172.21.15.80:6829","nonce":1162870008}]},"public_addr":"172.21.15.80:6825/1162870008","cluster_addr":"172.21.15.80:6827/1162870008","heartbeat_back_addr":"172.21.15.80:6831/1162870008","heartbeat_front_addr":"172.21.15.80:6829/1162870008","state":["exists","up"]},{"osd":11,"uuid":"46f79795-b13d-4997-b0d2-d89822eedf67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6825","nonce":1497318098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6827","nonce":1497318098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6831","nonce":1497318098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1497318098},{"type":"v1","addr":"172.21.15.115:6829","nonce":1497318098}]},"public_addr":"172.21.15.115:6825/1497318098","cluster_addr":"172.21.15.115:6827/1497318098","heartbeat_back_addr":"172.21.15.115:6831/1497318098","heartbeat_front_addr":"172.21.15.115:6829/1497318098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:23.575031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:14:48.169473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:12.103157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:15:45.171222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:10.883592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:16:35.289551+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:10.774322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:17:37.185604+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:02.553029+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:18:37.917765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:05.441868+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:30.909413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/3617020911":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/3210390195":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2527769498":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/3477045895":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2180727249":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/2102614714":"2024-02-27T06:12:27.274814+0000","172.21.15.49:6801/1431851577":"2024-02-27T06:23:23.289826+0000","172.21.15.49:6801/4089389406":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/3504875682":"2024-02-27T06:12:27.274814+0000","172.21.15.49:0/746416011":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/1431851577":"2024-02-27T06:23:23.289826+0000","172.21.15.49:0/2937664554":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6801/765116076":"2024-02-27T06:11:51.411945+0000","172.21.15.49:0/2168062918":"2024-02-27T06:11:51.411945+0000","172.21.15.49:6800/4089389406":"2024-02-27T06:12:27.274814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:40:06.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa6000a2230 msgr2=0x7fa6000a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:06.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa6000a2230 0x7fa6000a46f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa61c19ae80 tx=0x7fa608005f90 comp rx=0 tx=0).stop 2024-02-26T06:40:06.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa61c1011e0 msgr2=0x7fa61c19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa61c1011e0 0x7fa61c19f570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa60c000c00 tx=0x7fa60c02c6c0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 shutdown_connections 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fa6000a2230 0x7fa6000a46f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa61c199630 0x7fa61c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c1062b0 0x7fa61c19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 --2- 172.21.15.49:0/778643566 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa61c1011e0 0x7fa61c19f570 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 >> 172.21.15.49:0/778643566 conn(0x7fa61c0fca60 msgr2=0x7fa61c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 shutdown_connections 2024-02-26T06:40:06.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:06.815+0000 7fa61be00640 1 -- 172.21.15.49:0/778643566 wait complete. 2024-02-26T06:40:06.835 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-26T06:40:07.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 -- 172.21.15.49:0/1864474231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e0c100740 msgr2=0x7f7e0c10d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:07.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1864474231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e0c100740 0x7f7e0c10d400 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009900 tx=0x7f7df402bc30 comp rx=0 tx=0).stop 2024-02-26T06:40:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 -- 172.21.15.49:0/1864474231 shutdown_connections 2024-02-26T06:40:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1864474231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e0c100740 0x7f7e0c10d400 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1864474231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e0c10d940 0x7f7e0c10fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1864474231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c0ffd90 0x7f7e0c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 -- 172.21.15.49:0/1864474231 >> 172.21.15.49:0/1864474231 conn(0x7f7e0c0fb9d0 msgr2=0x7f7e0c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 -- 172.21.15.49:0/1864474231 shutdown_connections 2024-02-26T06:40:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.027+0000 7f7e0be00640 1 -- 172.21.15.49:0/1864474231 wait complete. 2024-02-26T06:40:07.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.028+0000 7f7e0be00640 1 Processor -- start 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.028+0000 7f7e0be00640 1 -- start start 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0be00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e0c0ffd90 0x7f7e0c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e0c100740 0x7f7e0c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c10d940 0x7f7e0c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c199e20 con 0x7f7e0c10d940 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c199f90 con 0x7f7e0c100740 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0be00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c19a150 con 0x7f7e0c0ffd90 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c10d940 0x7f7e0c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e0c0ffd90 0x7f7e0c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c10d940 0x7f7e0c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51906/0 (socket says 172.21.15.49:51906) 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0b400640 1 -- 172.21.15.49:0/1514001693 learned_addr learned my addr 172.21.15.49:0/1514001693 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e0aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e0c0ffd90 0x7f7e0c19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:41268/0 (socket says 172.21.15.49:41268) 2024-02-26T06:40:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.029+0000 7f7e03e00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e0c100740 0x7f7e0c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0b400640 1 -- 172.21.15.49:0/1514001693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e0c0ffd90 msgr2=0x7f7e0c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0b400640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e0c0ffd90 0x7f7e0c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0b400640 1 -- 172.21.15.49:0/1514001693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e0c100740 msgr2=0x7f7e0c19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0b400640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e0c100740 0x7f7e0c19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0b400640 1 -- 172.21.15.49:0/1514001693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df4009560 con 0x7f7e0c10d940 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0aa00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e0c0ffd90 0x7f7e0c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0b400640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c10d940 0x7f7e0c199690 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400e9d0 tx=0x7f7e0400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e03400640 1 -- 172.21.15.49:0/1514001693 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0400cd80 con 0x7f7e0c10d940 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c19a430 con 0x7f7e0c10d940 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e03400640 1 -- 172.21.15.49:0/1514001693 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e04005020 con 0x7f7e0c10d940 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e03400640 1 -- 172.21.15.49:0/1514001693 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04011670 con 0x7f7e0c10d940 2024-02-26T06:40:07.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.030+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c1a6550 con 0x7f7e0c10d940 2024-02-26T06:40:07.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.032+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c068b60 con 0x7f7e0c10d940 2024-02-26T06:40:07.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.034+0000 7f7e03400640 1 -- 172.21.15.49:0/1514001693 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7e04005500 con 0x7f7e0c10d940 2024-02-26T06:40:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.035+0000 7f7e03400640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7df00a2230 0x7f7df00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.035+0000 7f7e03400640 1 -- 172.21.15.49:0/1514001693 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8594+0+0 (secure 0 0 0) 0x7f7e040ce1a0 con 0x7f7e0c10d940 2024-02-26T06:40:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.035+0000 7f7e0aa00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7df00a2230 0x7f7df00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.036+0000 7f7e0aa00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7df00a2230 0x7f7df00a46f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009fc0 tx=0x7f7dfc005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:07.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.039+0000 7f7e03400640 1 -- 172.21.15.49:0/1514001693 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e04096e90 con 0x7f7e0c10d940 2024-02-26T06:40:07.056 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:06 smithi049 ceph-mon[31696]: pgmap v524: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 27 op/s 2024-02-26T06:40:07.056 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:06 smithi049 ceph-mon[31696]: osdmap e110: 12 total, 12 up, 12 in 2024-02-26T06:40:07.056 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:06 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2383456169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:40:07.056 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:06 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/778643566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:40:07.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.276+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7e0c067e90 con 0x7f7e0c10d940 2024-02-26T06:40:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.279+0000 7f7e03400640 1 -- 172.21.15.49:0/1514001693 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7e0401f090 con 0x7f7e0c10d940 2024-02-26T06:40:07.280 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T06:40:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.282+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7df00a2230 msgr2=0x7f7df00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.282+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7df00a2230 0x7f7df00a46f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009fc0 tx=0x7f7dfc005f90 comp rx=0 tx=0).stop 2024-02-26T06:40:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.282+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c10d940 msgr2=0x7f7e0c199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.282+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c10d940 0x7f7e0c199690 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400e9d0 tx=0x7f7e0400ee90 comp rx=0 tx=0).stop 2024-02-26T06:40:07.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.282+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 shutdown_connections 2024-02-26T06:40:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.282+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f7df00a2230 0x7f7df00a46f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.282+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e0c100740 0x7f7e0c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.283+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c10d940 0x7f7e0c199690 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.283+0000 7f7e0be00640 1 --2- 172.21.15.49:0/1514001693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e0c0ffd90 0x7f7e0c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.283+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 >> 172.21.15.49:0/1514001693 conn(0x7f7e0c0fb9d0 msgr2=0x7f7e0c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.283+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 shutdown_connections 2024-02-26T06:40:07.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.283+0000 7f7e0be00640 1 -- 172.21.15.49:0/1514001693 wait complete. 2024-02-26T06:40:07.302 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-26T06:40:07.324 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:06 smithi115 ceph-mon[36426]: pgmap v524: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 27 op/s 2024-02-26T06:40:07.324 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:06 smithi115 ceph-mon[36426]: osdmap e110: 12 total, 12 up, 12 in 2024-02-26T06:40:07.324 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:06 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2383456169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:40:07.324 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:06 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/778643566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:40:07.325 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:06 smithi080 ceph-mon[36206]: pgmap v524: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 27 op/s 2024-02-26T06:40:07.325 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:06 smithi080 ceph-mon[36206]: osdmap e110: 12 total, 12 up, 12 in 2024-02-26T06:40:07.325 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:06 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2383456169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:40:07.325 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:06 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/778643566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:40:07.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31692]: 2024-02-26T06:40:07.277+0000 7f7519000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 -- 172.21.15.49:0/2851587137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e1c102250 msgr2=0x7f6e1c102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 --2- 172.21.15.49:0/2851587137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e1c102250 0x7f6e1c102630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009860 tx=0x7f6e0c02bc10 comp rx=0 tx=0).stop 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 -- 172.21.15.49:0/2851587137 shutdown_connections 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 --2- 172.21.15.49:0/2851587137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e1c10b620 0x7f6e1c10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 --2- 172.21.15.49:0/2851587137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c102b70 0x7f6e1c10b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 --2- 172.21.15.49:0/2851587137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e1c102250 0x7f6e1c102630 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 -- 172.21.15.49:0/2851587137 >> 172.21.15.49:0/2851587137 conn(0x7f6e1c0fb9d0 msgr2=0x7f6e1c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 -- 172.21.15.49:0/2851587137 shutdown_connections 2024-02-26T06:40:07.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.515+0000 7f6e20a00640 1 -- 172.21.15.49:0/2851587137 wait complete. 2024-02-26T06:40:07.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.516+0000 7f6e20a00640 1 Processor -- start 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.516+0000 7f6e20a00640 1 -- start start 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e20a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e1c102250 0x7f6e1c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e20a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e1c102b70 0x7f6e1c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e20a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c10b620 0x7f6e1c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e20a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c199e50 con 0x7f6e1c10b620 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e20a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c199fc0 con 0x7f6e1c102b70 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e20a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c19a180 con 0x7f6e1c102250 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e1c102b70 0x7f6e1c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c10b620 0x7f6e1c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e1c102b70 0x7f6e1c19fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:55990/0 (socket says 172.21.15.49:55990) 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1aa00640 1 -- 172.21.15.49:0/4099303000 learned_addr learned my addr 172.21.15.49:0/4099303000 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:07.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1b400640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e1c102250 0x7f6e1c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1aa00640 1 -- 172.21.15.49:0/4099303000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e1c102250 msgr2=0x7f6e1c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1aa00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e1c102250 0x7f6e1c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1aa00640 1 -- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c10b620 msgr2=0x7f6e1c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1aa00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c10b620 0x7f6e1c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.517+0000 7f6e1aa00640 1 -- 172.21.15.49:0/4099303000 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08009490 con 0x7f6e1c102b70 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.518+0000 7f6e1aa00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e1c102b70 0x7f6e1c19fb40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800ea60 tx=0x7f6e0800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.518+0000 7f6e13e00640 1 -- 172.21.15.49:0/4099303000 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800cde0 con 0x7f6e1c102b70 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.518+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e14009900 con 0x7f6e1c102b70 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.518+0000 7f6e13e00640 1 -- 172.21.15.49:0/4099303000 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e080035f0 con 0x7f6e1c102b70 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.518+0000 7f6e13e00640 1 -- 172.21.15.49:0/4099303000 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08010690 con 0x7f6e1c102b70 2024-02-26T06:40:07.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.518+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c009490 con 0x7f6e1c102b70 2024-02-26T06:40:07.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.519+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c068b20 con 0x7f6e1c102b70 2024-02-26T06:40:07.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.521+0000 7f6e13e00640 1 -- 172.21.15.49:0/4099303000 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6e08003af0 con 0x7f6e1c102b70 2024-02-26T06:40:07.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.523+0000 7f6e13e00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6e000a22b0 0x7f6e000a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:07.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.523+0000 7f6e13e00640 1 -- 172.21.15.49:0/4099303000 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8808+0+0 (secure 0 0 0) 0x7f6e08014070 con 0x7f6e1c102b70 2024-02-26T06:40:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.523+0000 7f6e1b400640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6e000a22b0 0x7f6e000a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.524+0000 7f6e1b400640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6e000a22b0 0x7f6e000a4770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c02c220 tx=0x7f6e0c0056d0 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:07.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.524+0000 7f6e13e00640 1 -- 172.21.15.49:0/4099303000 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e08097110 con 0x7f6e1c102b70 2024-02-26T06:40:07.580 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e[66308]: ignoring --setuser ceph since I am not root 2024-02-26T06:40:07.580 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e[66308]: ignoring --setgroup ceph since I am not root 2024-02-26T06:40:07.580 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mds[66312]: -- 172.21.15.80:0/3675829211 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558b71e1a1a0 con 0x558b71df1000 2024-02-26T06:40:07.580 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e[66308]: starting mds.e at 2024-02-26T06:40:07.581 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: ignoring --setuser ceph since I am not root 2024-02-26T06:40:07.581 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: ignoring --setgroup ceph since I am not root 2024-02-26T06:40:07.581 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mds[66930]: -- 172.21.15.80:0/1660025050 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5637887fc1a0 con 0x5637887d3000 2024-02-26T06:40:07.581 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: starting mds.b at 2024-02-26T06:40:07.581 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h[66621]: ignoring --setuser ceph since I am not root 2024-02-26T06:40:07.581 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h[66621]: ignoring --setgroup ceph since I am not root 2024-02-26T06:40:07.581 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mds[66625]: -- 172.21.15.80:0/3568186643 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5625c31801a0 con 0x5625c3156800 2024-02-26T06:40:07.581 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h[66621]: starting mds.h at 2024-02-26T06:40:07.590 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: ignoring --setuser ceph since I am not root 2024-02-26T06:40:07.590 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: ignoring --setgroup ceph since I am not root 2024-02-26T06:40:07.590 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mds[67244]: -- 172.21.15.115:0/2174898676 <== mon.2 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e553e81a0 con 0x556e553bec00 2024-02-26T06:40:07.591 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: starting mds.c at 2024-02-26T06:40:07.591 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f[66928]: ignoring --setuser ceph since I am not root 2024-02-26T06:40:07.591 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f[66928]: ignoring --setgroup ceph since I am not root 2024-02-26T06:40:07.591 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mds[66932]: -- 172.21.15.115:0/2756161473 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a92f7601a0 con 0x55a92f736c00 2024-02-26T06:40:07.591 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f[66928]: starting mds.f at 2024-02-26T06:40:07.591 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i[67552]: ignoring --setuser ceph since I am not root 2024-02-26T06:40:07.591 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i[67552]: ignoring --setgroup ceph since I am not root 2024-02-26T06:40:07.591 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mds[67557]: -- 172.21.15.115:0/2143171630 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5566236461a0 con 0x55662361cc00 2024-02-26T06:40:07.592 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i[67552]: starting mds.i at 2024-02-26T06:40:07.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:07.768+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6e1c19abf0 con 0x7f6e1c102b70 2024-02-26T06:40:08.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1514001693' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:40:08.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T06:40:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T06:40:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-26T06:40:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/1514001693' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T06:40:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: osdmap e111: 12 total, 12 up, 12 in 2024-02-26T06:40:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T06:40:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/4099303000' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:40:08.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:07 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:40:08.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.285+0000 7f6e13e00640 1 -- 172.21.15.49:0/4099303000 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6e0809a7d0 con 0x7f6e1c102b70 2024-02-26T06:40:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6e000a22b0 msgr2=0x7f6e000a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6e000a22b0 0x7f6e000a4770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c02c220 tx=0x7f6e0c0056d0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e1c102b70 msgr2=0x7f6e1c19fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e1c102b70 0x7f6e1c19fb40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800ea60 tx=0x7f6e0800ef20 comp rx=0 tx=0).stop 2024-02-26T06:40:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 shutdown_connections 2024-02-26T06:40:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7f6e000a22b0 0x7f6e000a4770 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e1c102b70 0x7f6e1c19fb40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c10b620 0x7f6e1c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 --2- 172.21.15.49:0/4099303000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e1c102250 0x7f6e1c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.289+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 >> 172.21.15.49:0/4099303000 conn(0x7f6e1c0fb9d0 msgr2=0x7f6e1c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:08.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.290+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 shutdown_connections 2024-02-26T06:40:08.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.290+0000 7f6e20a00640 1 -- 172.21.15.49:0/4099303000 wait complete. 2024-02-26T06:40:08.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1514001693' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/1514001693' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: osdmap e111: 12 total, 12 up, 12 in 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/4099303000' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:40:08.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:07 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:40:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1514001693' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:40:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T06:40:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T06:40:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-26T06:40:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/1514001693' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T06:40:08.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: osdmap e111: 12 total, 12 up, 12 in 2024-02-26T06:40:08.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T06:40:08.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/4099303000' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:40:08.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:07 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:40:08.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.532+0000 7fb781e00640 1 -- 172.21.15.49:0/74623135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb77c100ad0 msgr2=0x7fb77c104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.532+0000 7fb781e00640 1 --2- 172.21.15.49:0/74623135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb77c100ad0 0x7fb77c104f90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb768009930 tx=0x7fb76802bcb0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.533+0000 7fb781e00640 1 -- 172.21.15.49:0/74623135 shutdown_connections 2024-02-26T06:40:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.533+0000 7fb781e00640 1 --2- 172.21.15.49:0/74623135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb77c1055b0 0x7fb77c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.533+0000 7fb781e00640 1 --2- 172.21.15.49:0/74623135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb77c100ad0 0x7fb77c104f90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.533+0000 7fb781e00640 1 --2- 172.21.15.49:0/74623135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb77c1001b0 0x7fb77c100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.533+0000 7fb781e00640 1 -- 172.21.15.49:0/74623135 >> 172.21.15.49:0/74623135 conn(0x7fb77c076400 msgr2=0x7fb77c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.533+0000 7fb781e00640 1 -- 172.21.15.49:0/74623135 shutdown_connections 2024-02-26T06:40:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.533+0000 7fb781e00640 1 -- 172.21.15.49:0/74623135 wait complete. 2024-02-26T06:40:08.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.534+0000 7fb781e00640 1 Processor -- start 2024-02-26T06:40:08.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.534+0000 7fb781e00640 1 -- start start 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.534+0000 7fb781e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb77c1001b0 0x7fb77c19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.534+0000 7fb781e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb77c100ad0 0x7fb77c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.534+0000 7fb781e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb77c1055b0 0x7fb77c1a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.534+0000 7fb781e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c1a2cf0 con 0x7fb77c1001b0 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.534+0000 7fb781e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c1a2e60 con 0x7fb77c100ad0 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.534+0000 7fb781e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c1a3020 con 0x7fb77c1055b0 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb77c1001b0 0x7fb77c19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb77c100ad0 0x7fb77c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb77c100ad0 0x7fb77c19e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.49:56016/0 (socket says 172.21.15.49:56016) 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 -- 172.21.15.49:0/3025429174 learned_addr learned my addr 172.21.15.49:0/3025429174 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77b400640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb77c1055b0 0x7fb77c1a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 -- 172.21.15.49:0/3025429174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb77c1055b0 msgr2=0x7fb77c1a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb77c1055b0 0x7fb77c1a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 -- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb77c1001b0 msgr2=0x7fb77c19de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb77c1001b0 0x7fb77c19de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 -- 172.21.15.49:0/3025429174 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb768009560 con 0x7fb77c100ad0 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.535+0000 7fb77a000640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb77c100ad0 0x7fb77c19e370 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb768000c00 tx=0x7fb76802c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.536+0000 7fb780a00640 1 -- 172.21.15.49:0/3025429174 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76802db00 con 0x7fb77c100ad0 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.536+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77c1a32a0 con 0x7fb77c100ad0 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.536+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77c1aab80 con 0x7fb77c100ad0 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.536+0000 7fb780a00640 1 -- 172.21.15.49:0/3025429174 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb76802dc60 con 0x7fb77c100ad0 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.536+0000 7fb780a00640 1 -- 172.21.15.49:0/3025429174 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768003a50 con 0x7fb77c100ad0 2024-02-26T06:40:08.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.537+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c005290 con 0x7fb77c100ad0 2024-02-26T06:40:08.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.539+0000 7fb780a00640 1 -- 172.21.15.49:0/3025429174 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb768003bb0 con 0x7fb77c100ad0 2024-02-26T06:40:08.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.541+0000 7fb780a00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7600a2230 0x7fb7600a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:08.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.541+0000 7fb780a00640 1 -- 172.21.15.49:0/3025429174 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8808+0+0 (secure 0 0 0) 0x7fb7680efea0 con 0x7fb77c100ad0 2024-02-26T06:40:08.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.541+0000 7fb77aa00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7600a2230 0x7fb7600a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:08.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.542+0000 7fb77aa00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7600a2230 0x7fb7600a46f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009bd0 tx=0x7fb76c005f90 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:08.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.545+0000 7fb780a00640 1 -- 172.21.15.49:0/3025429174 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb7680b8ac0 con 0x7fb77c100ad0 2024-02-26T06:40:08.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:08.744+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb75c005540 con 0x7fb77c100ad0 2024-02-26T06:40:09.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: pgmap v527: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-02-26T06:40:09.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: Cluster is now healthy 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6836/3637291833,v1:172.21.15.115:6838/3637291833] up:boot 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6837/1646111828,v1:172.21.15.115:6839/1646111828] up:boot 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.80:6834/2454438771,v1:172.21.15.80:6835/2454438771] up:boot 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.115:6834/1044646314,v1:172.21.15.115:6835/1044646314] up:boot 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.80:6837/2579177097,v1:172.21.15.80:6838/2579177097] up:boot 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: mds.? [v2:172.21.15.80:6836/1555573727,v1:172.21.15.80:6839/1555573727] up:boot 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: fsmap 12 up:standby 2024-02-26T06:40:09.171 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:40:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:40:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:40:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:40:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:40:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:40:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/3025429174' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:09.172 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:08 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.301+0000 7fb780a00640 1 -- 172.21.15.49:0/3025429174 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb768003e60 con 0x7fb77c100ad0 2024-02-26T06:40:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2024-02-26T06:40:09.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7600a2230 msgr2=0x7fb7600a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7600a2230 0x7fb7600a46f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009bd0 tx=0x7fb76c005f90 comp rx=0 tx=0).stop 2024-02-26T06:40:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb77c100ad0 msgr2=0x7fb77c19e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb77c100ad0 0x7fb77c19e370 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb768000c00 tx=0x7fb76802c670 comp rx=0 tx=0).stop 2024-02-26T06:40:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 shutdown_connections 2024-02-26T06:40:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fb7600a2230 0x7fb7600a46f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb77c100ad0 0x7fb77c19e370 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb77c1055b0 0x7fb77c1a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 --2- 172.21.15.49:0/3025429174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb77c1001b0 0x7fb77c19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.306+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 >> 172.21.15.49:0/3025429174 conn(0x7fb77c076400 msgr2=0x7fb77c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.307+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 shutdown_connections 2024-02-26T06:40:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.307+0000 7fb781e00640 1 -- 172.21.15.49:0/3025429174 wait complete. 2024-02-26T06:40:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: pgmap v527: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-02-26T06:40:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:40:09.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T06:40:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T06:40:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: Cluster is now healthy 2024-02-26T06:40:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:40:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6836/3637291833,v1:172.21.15.115:6838/3637291833] up:boot 2024-02-26T06:40:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6837/1646111828,v1:172.21.15.115:6839/1646111828] up:boot 2024-02-26T06:40:09.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.80:6834/2454438771,v1:172.21.15.80:6835/2454438771] up:boot 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.115:6834/1044646314,v1:172.21.15.115:6835/1044646314] up:boot 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.80:6837/2579177097,v1:172.21.15.80:6838/2579177097] up:boot 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: mds.? [v2:172.21.15.80:6836/1555573727,v1:172.21.15.80:6839/1555573727] up:boot 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: fsmap 12 up:standby 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:40:09.332 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:40:09.333 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:40:09.333 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/3025429174' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:09.333 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:08 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:09.334 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-26T06:40:09.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: pgmap v527: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-02-26T06:40:09.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: Cluster is now healthy 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6836/3637291833,v1:172.21.15.115:6838/3637291833] up:boot 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6837/1646111828,v1:172.21.15.115:6839/1646111828] up:boot 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.80:6834/2454438771,v1:172.21.15.80:6835/2454438771] up:boot 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.115:6834/1044646314,v1:172.21.15.115:6835/1044646314] up:boot 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.80:6837/2579177097,v1:172.21.15.80:6838/2579177097] up:boot 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: mds.? [v2:172.21.15.80:6836/1555573727,v1:172.21.15.80:6839/1555573727] up:boot 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: fsmap 12 up:standby 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:40:09.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:40:09.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:40:09.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:40:09.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/3025429174' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:09.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:08 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.557+0000 7fe334600640 1 -- 172.21.15.49:0/2879515806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c10d730 msgr2=0x7fe32c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.557+0000 7fe334600640 1 --2- 172.21.15.49:0/2879515806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c10d730 0x7fe32c10db10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009cd0 tx=0x7fe31c02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.558+0000 7fe334600640 1 -- 172.21.15.49:0/2879515806 shutdown_connections 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.558+0000 7fe334600640 1 --2- 172.21.15.49:0/2879515806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe32c1080e0 0x7fe32c1145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.558+0000 7fe334600640 1 --2- 172.21.15.49:0/2879515806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe32c107720 0x7fe32c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.558+0000 7fe334600640 1 --2- 172.21.15.49:0/2879515806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c10d730 0x7fe32c10db10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.558+0000 7fe334600640 1 -- 172.21.15.49:0/2879515806 >> 172.21.15.49:0/2879515806 conn(0x7fe32c0756c0 msgr2=0x7fe32c077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.558+0000 7fe334600640 1 -- 172.21.15.49:0/2879515806 shutdown_connections 2024-02-26T06:40:09.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.558+0000 7fe334600640 1 -- 172.21.15.49:0/2879515806 wait complete. 2024-02-26T06:40:09.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.559+0000 7fe334600640 1 Processor -- start 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.559+0000 7fe334600640 1 -- start start 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.559+0000 7fe334600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c107720 0x7fe32c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.559+0000 7fe334600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe32c1080e0 0x7fe32c19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.559+0000 7fe334600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe32c10d730 0x7fe32c1a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.559+0000 7fe334600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1a2cc0 con 0x7fe32c107720 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.559+0000 7fe334600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1a2e30 con 0x7fe32c1080e0 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.559+0000 7fe334600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1a2ff0 con 0x7fe32c10d730 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c107720 0x7fe32c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c107720 0x7fe32c19de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51960/0 (socket says 172.21.15.49:51960) 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331c00640 1 -- 172.21.15.49:0/2343988778 learned_addr learned my addr 172.21.15.49:0/2343988778 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331200640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe32c1080e0 0x7fe32c19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe332600640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe32c10d730 0x7fe32c1a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:09.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331200640 1 -- 172.21.15.49:0/2343988778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe32c10d730 msgr2=0x7fe32c1a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331200640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe32c10d730 0x7fe32c1a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331200640 1 -- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c107720 msgr2=0x7fe32c19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331200640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c107720 0x7fe32c19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331200640 1 -- 172.21.15.49:0/2343988778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c009900 con 0x7fe32c1080e0 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.560+0000 7fe331c00640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c107720 0x7fe32c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.561+0000 7fe331200640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe32c1080e0 0x7fe32c19e340 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9f0 tx=0x7fe31800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.561+0000 7fe32a000640 1 -- 172.21.15.49:0/2343988778 <== mon.2 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31800ccc0 con 0x7fe32c1080e0 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.561+0000 7fe32a000640 1 -- 172.21.15.49:0/2343988778 <== mon.2 v2:172.21.15.80:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe31800ce20 con 0x7fe32c1080e0 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.561+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c1a32d0 con 0x7fe32c1080e0 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.561+0000 7fe32a000640 1 -- 172.21.15.49:0/2343988778 <== mon.2 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318018920 con 0x7fe32c1080e0 2024-02-26T06:40:09.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.561+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c1aab50 con 0x7fe32c1080e0 2024-02-26T06:40:09.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.562+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c0683c0 con 0x7fe32c1080e0 2024-02-26T06:40:09.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.564+0000 7fe32a000640 1 -- 172.21.15.49:0/2343988778 <== mon.2 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe318018a80 con 0x7fe32c1080e0 2024-02-26T06:40:09.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.566+0000 7fe32a000640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3100a2230 0x7fe3100a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:40:09.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.566+0000 7fe32a000640 1 -- 172.21.15.49:0/2343988778 <== mon.2 v2:172.21.15.80:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8275+0+0 (secure 0 0 0) 0x7fe318014070 con 0x7fe32c1080e0 2024-02-26T06:40:09.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.566+0000 7fe331c00640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3100a2230 0x7fe3100a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:40:09.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.567+0000 7fe331c00640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3100a2230 0x7fe3100a46f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005fe0 tx=0x7fe31c005f70 comp rx=0 tx=0).ready entity=mgr.25384 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:40:09.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.567+0000 7fe32a000640 1 -- 172.21.15.49:0/2343988778 <== mon.2 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe318097090 con 0x7fe32c1080e0 2024-02-26T06:40:09.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:09.768+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe32c117880 con 0x7fe32c1080e0 2024-02-26T06:40:10.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:09 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:40:10.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:09 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:40:10.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:09 smithi049 ceph-mon[31696]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:40:10.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:09 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T06:40:10.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:09 smithi049 ceph-mon[31696]: osdmap e112: 12 total, 12 up, 12 in 2024-02-26T06:40:10.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:09 smithi049 ceph-mon[31696]: from='client.? 172.21.15.49:0/2343988778' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:10.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:09 smithi049 ceph-mon[31696]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.316+0000 7fe32a000640 1 -- 172.21.15.49:0/2343988778 <== mon.2 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe31809a750 con 0x7fe32c1080e0 2024-02-26T06:40:10.318 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2024-02-26T06:40:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3100a2230 msgr2=0x7fe3100a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3100a2230 0x7fe3100a46f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005fe0 tx=0x7fe31c005f70 comp rx=0 tx=0).stop 2024-02-26T06:40:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe32c1080e0 msgr2=0x7fe32c19e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:40:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe32c1080e0 0x7fe32c19e340 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9f0 tx=0x7fe31800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:40:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 shutdown_connections 2024-02-26T06:40:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:6800/2212175168,v1:172.21.15.49:6801/2212175168] conn(0x7fe3100a2230 0x7fe3100a46f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe32c1080e0 0x7fe32c19e340 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe32c10d730 0x7fe32c1a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 --2- 172.21.15.49:0/2343988778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c107720 0x7fe32c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:40:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.322+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 >> 172.21.15.49:0/2343988778 conn(0x7fe32c0756c0 msgr2=0x7fe32c076040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:40:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.323+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 shutdown_connections 2024-02-26T06:40:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-02-26T06:40:10.323+0000 7fe334600640 1 -- 172.21.15.49:0/2343988778 wait complete. 2024-02-26T06:40:10.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:09 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:40:10.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:09 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:40:10.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:09 smithi080 ceph-mon[36206]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:40:10.330 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:09 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T06:40:10.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:09 smithi080 ceph-mon[36206]: osdmap e112: 12 total, 12 up, 12 in 2024-02-26T06:40:10.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:09 smithi080 ceph-mon[36206]: from='client.? 172.21.15.49:0/2343988778' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:10.331 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:09 smithi080 ceph-mon[36206]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:10.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:09 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:40:10.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:09 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:40:10.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:09 smithi115 ceph-mon[36426]: from='mgr.25384 172.21.15.49:0/21134590' entity='mgr.x' 2024-02-26T06:40:10.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:09 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T06:40:10.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:09 smithi115 ceph-mon[36426]: osdmap e112: 12 total, 12 up, 12 in 2024-02-26T06:40:10.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:09 smithi115 ceph-mon[36426]: from='client.? 172.21.15.49:0/2343988778' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:10.341 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:09 smithi115 ceph-mon[36426]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:40:10.359 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:40:10.453 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:40:10.496 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:40:10.531 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-26T06:40:10.531 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T06:40:10.601 DEBUG:teuthology.orchestra.run.smithi080:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T06:40:10.618 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T06:40:10.636 INFO:tasks.cephadm:Stopping all daemons... 2024-02-26T06:40:10.636 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-26T06:40:10.636 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a 2024-02-26T06:40:11.169 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:10 smithi049 systemd[1]: Stopping Ceph mon.a for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:11.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:11 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31692]: 2024-02-26T06:40:11.207+0000 7f751fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:11.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:11 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a[31692]: 2024-02-26T06:40:11.207+0000 7f751fe00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-26T06:40:11.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:11 smithi049 podman[97378]: 2024-02-26 06:40:11.2801852 +0000 UTC m=+0.224125064 container died 12392c53e5257df8d623e29a315c737dcd2f0fcbc6e5720b3990525108683127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-26T06:40:11.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:11 smithi049 podman[97378]: 2024-02-26 06:40:11.743063893 +0000 UTC m=+0.687003766 container cleanup 12392c53e5257df8d623e29a315c737dcd2f0fcbc6e5720b3990525108683127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.build-date=20240214) 2024-02-26T06:40:11.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:11 smithi049 bash[97378]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a 2024-02-26T06:40:12.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Feb 26 06:40:12 smithi049 podman[97391]: 2024-02-26 06:40:12.044623057 +0000 UTC m=+0.743970150 container remove 12392c53e5257df8d623e29a315c737dcd2f0fcbc6e5720b3990525108683127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-26T06:40:12.535 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.a.service' 2024-02-26T06:40:12.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:12.723 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-26T06:40:12.723 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-26T06:40:12.724 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.b 2024-02-26T06:40:13.080 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:12 smithi080 systemd[1]: Stopping Ceph mon.b for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-b[36202]: 2024-02-26T06:40:15.257+0000 7fac6b160700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:15.580 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-b[36202]: 2024-02-26T06:40:15.257+0000 7fac6b160700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-26T06:40:15.581 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:15 smithi080 podman[74187]: 2024-02-26 06:40:15.401770073 +0000 UTC m=+2.025999821 container died 2b5037e56778e8b615ee479321395b62c0d1ee8a0be999eae68d7ccd4ce9ef72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:40:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:18 smithi080 podman[74187]: 2024-02-26 06:40:18.692453064 +0000 UTC m=+5.316682809 container cleanup 2b5037e56778e8b615ee479321395b62c0d1ee8a0be999eae68d7ccd4ce9ef72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-b, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-26T06:40:18.830 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:18 smithi080 bash[74187]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-b 2024-02-26T06:40:19.221 INFO:journalctl@ceph.mon.b.smithi080.stdout:Feb 26 06:40:19 smithi080 podman[74202]: 2024-02-26 06:40:19.111113523 +0000 UTC m=+3.677922936 container remove 2b5037e56778e8b615ee479321395b62c0d1ee8a0be999eae68d7ccd4ce9ef72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:40:19.240 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.b.service' 2024-02-26T06:40:19.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:19.291 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-26T06:40:19.291 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-26T06:40:19.291 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.c 2024-02-26T06:40:19.590 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:19 smithi115 systemd[1]: Stopping Ceph mon.c for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:19.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-c[36400]: 2024-02-26T06:40:19.641+0000 7f52ae55a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:20.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-c[36400]: 2024-02-26T06:40:19.641+0000 7f52ae55a700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-26T06:40:20.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:19 smithi115 podman[74610]: 2024-02-26 06:40:19.713201589 +0000 UTC m=+0.230310178 container died a8521d9b0855132a294f2ef2d4529930c8be9de7f53a9419468735cdd638fc69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-26T06:40:20.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:20 smithi115 podman[74610]: 2024-02-26 06:40:20.100112372 +0000 UTC m=+0.617220951 container cleanup a8521d9b0855132a294f2ef2d4529930c8be9de7f53a9419468735cdd638fc69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:20.340 INFO:journalctl@ceph.mon.c.smithi115.stdout:Feb 26 06:40:20 smithi115 bash[74610]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mon-c 2024-02-26T06:40:20.518 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mon.c.service' 2024-02-26T06:40:20.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:20.645 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-26T06:40:20.645 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-26T06:40:20.645 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.x 2024-02-26T06:40:21.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:40:20 smithi049 systemd[1]: Stopping Ceph mgr.x for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:21.670 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:40:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:40:21.330+0000 7f6f4ae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:21.670 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:40:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x[31921]: 2024-02-26T06:40:21.330+0000 7f6f4ae00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-26T06:40:21.670 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:40:21 smithi049 podman[97492]: 2024-02-26 06:40:21.426991028 +0000 UTC m=+0.279622056 container died 03ccb02c6201fd4a4c8eceb8c35bc030a2a7d3ba675a7bcee2e0133f6127bb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:22.169 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:40:21 smithi049 podman[97492]: 2024-02-26 06:40:21.947988843 +0000 UTC m=+0.800619864 container cleanup 03ccb02c6201fd4a4c8eceb8c35bc030a2a7d3ba675a7bcee2e0133f6127bb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:22.170 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:40:21 smithi049 bash[97492]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x 2024-02-26T06:40:22.670 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Feb 26 06:40:22 smithi049 podman[97506]: 2024-02-26 06:40:22.241043905 +0000 UTC m=+0.803661061 container remove 03ccb02c6201fd4a4c8eceb8c35bc030a2a7d3ba675a7bcee2e0133f6127bb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:22.749 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.x.service' 2024-02-26T06:40:22.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:22.887 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-26T06:40:22.887 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-26T06:40:22.887 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.y 2024-02-26T06:40:23.234 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:40:22 smithi080 systemd[1]: Stopping Ceph mgr.y for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:23.531 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:40:23 smithi080 podman[74322]: 2024-02-26 06:40:23.234678701 +0000 UTC m=+0.145628050 container died b645b34ca027021dc4e8392c9cf6a7e4033766d4e1eef6e811c034caf0221f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:23.830 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:40:23 smithi080 podman[74322]: 2024-02-26 06:40:23.631361955 +0000 UTC m=+0.542311297 container cleanup b645b34ca027021dc4e8392c9cf6a7e4033766d4e1eef6e811c034caf0221f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-26T06:40:23.830 INFO:journalctl@ceph.mgr.y.smithi080.stdout:Feb 26 06:40:23 smithi080 bash[74322]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-y 2024-02-26T06:40:24.039 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.y.service' 2024-02-26T06:40:24.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:24.078 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-26T06:40:24.078 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-26T06:40:24.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.z 2024-02-26T06:40:24.400 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:40:24 smithi115 systemd[1]: Stopping Ceph mgr.z for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:24.840 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:40:24 smithi115 podman[74731]: 2024-02-26 06:40:24.400283654 +0000 UTC m=+0.132219368 container died 4250a23f67416e92957b7323b3fede75dc114e4e70439d9fcc0a1762d98afd40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True) 2024-02-26T06:40:25.590 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:40:25 smithi115 podman[74731]: 2024-02-26 06:40:25.473012147 +0000 UTC m=+1.204947870 container cleanup 4250a23f67416e92957b7323b3fede75dc114e4e70439d9fcc0a1762d98afd40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:25.590 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Feb 26 06:40:25 smithi115 bash[74731]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mgr-z 2024-02-26T06:40:25.936 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mgr.z.service' 2024-02-26T06:40:25.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:25.973 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-26T06:40:25.973 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-26T06:40:25.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.0 2024-02-26T06:40:26.419 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:26 smithi049 systemd[1]: Stopping Ceph osd.0 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:26.920 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0[43879]: 2024-02-26T06:40:26.644+0000 7fc774e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:26.920 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0[43879]: 2024-02-26T06:40:26.644+0000 7fc774e00700 -1 osd.0 113 *** Got signal Terminated *** 2024-02-26T06:40:26.920 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0[43879]: 2024-02-26T06:40:26.644+0000 7fc774e00700 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:33.170 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:32 smithi049 podman[97611]: 2024-02-26 06:40:32.72853001 +0000 UTC m=+6.276505428 container died e0c4b2c27befa98e239e264a4fd79fa3a660997cefdc5640c07e142779f9ea83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:33.653 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:33 smithi049 podman[97611]: 2024-02-26 06:40:33.402098588 +0000 UTC m=+6.950074000 container cleanup e0c4b2c27befa98e239e264a4fd79fa3a660997cefdc5640c07e142779f9ea83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214) 2024-02-26T06:40:33.653 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:33 smithi049 bash[97611]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0 2024-02-26T06:40:33.919 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:33 smithi049 podman[97632]: 2024-02-26 06:40:33.653163458 +0000 UTC m=+0.914668428 container remove e0c4b2c27befa98e239e264a4fd79fa3a660997cefdc5640c07e142779f9ea83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-26T06:40:34.809 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:34 smithi049 podman[97720]: 2024-02-26 06:40:34.462317603 +0000 UTC m=+0.038587388 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:34.809 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:34 smithi049 podman[97720]: 2024-02-26 06:40:34.587474277 +0000 UTC m=+0.163744060 container create fbdcfb460d7270b76a540c8c22ccba3a93145c262af0c724e21b52ccd082b7cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-26T06:40:35.169 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:34 smithi049 podman[97720]: 2024-02-26 06:40:34.907868869 +0000 UTC m=+0.484138675 container init fbdcfb460d7270b76a540c8c22ccba3a93145c262af0c724e21b52ccd082b7cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:40:35.169 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:34 smithi049 podman[97720]: 2024-02-26 06:40:34.920863185 +0000 UTC m=+0.497132938 container start fbdcfb460d7270b76a540c8c22ccba3a93145c262af0c724e21b52ccd082b7cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-26T06:40:35.169 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:35 smithi049 podman[97720]: 2024-02-26 06:40:35.023697128 +0000 UTC m=+0.599966958 container attach fbdcfb460d7270b76a540c8c22ccba3a93145c262af0c724e21b52ccd082b7cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-26T06:40:35.890 INFO:journalctl@ceph.osd.0.smithi049.stdout:Feb 26 06:40:35 smithi049 podman[97720]: 2024-02-26 06:40:35.49560757 +0000 UTC m=+1.071877378 container died fbdcfb460d7270b76a540c8c22ccba3a93145c262af0c724e21b52ccd082b7cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-26T06:40:36.213 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.0.service' 2024-02-26T06:40:36.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:36.351 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-26T06:40:36.351 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-26T06:40:36.351 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.1 2024-02-26T06:40:36.677 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:36 smithi080 systemd[1]: Stopping Ceph osd.1 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:37.080 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1[40786]: 2024-02-26T06:40:36.677+0000 7f6382d5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:37.080 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1[40786]: 2024-02-26T06:40:36.677+0000 7f6382d5d700 -1 osd.1 113 *** Got signal Terminated *** 2024-02-26T06:40:37.080 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1[40786]: 2024-02-26T06:40:36.677+0000 7f6382d5d700 -1 osd.1 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:42.486 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:42 smithi080 podman[74447]: 2024-02-26 06:40:42.220856184 +0000 UTC m=+5.683697667 container died 481b50436af944dcbb97f9c42c11846df409e626639afaeef9a7a0216d9ee888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True) 2024-02-26T06:40:42.830 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:42 smithi080 podman[74447]: 2024-02-26 06:40:42.593805197 +0000 UTC m=+6.056646705 container cleanup 481b50436af944dcbb97f9c42c11846df409e626639afaeef9a7a0216d9ee888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-26T06:40:42.830 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:42 smithi080 bash[74447]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1 2024-02-26T06:40:43.189 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:42 smithi080 podman[74464]: 2024-02-26 06:40:42.88719102 +0000 UTC m=+0.664213573 container remove 481b50436af944dcbb97f9c42c11846df409e626639afaeef9a7a0216d9ee888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True) 2024-02-26T06:40:43.440 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:43 smithi080 podman[74558]: 2024-02-26 06:40:43.093011241 +0000 UTC m=+0.017711614 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:43.440 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:43 smithi080 podman[74558]: 2024-02-26 06:40:43.235943605 +0000 UTC m=+0.160643967 container create 7a8bc8596d8bf1e1702c248b6da1b8e26444f960f12b86a5cfeaeb4c796f9db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-26T06:40:43.823 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:43 smithi080 podman[74558]: 2024-02-26 06:40:43.441412742 +0000 UTC m=+0.366113122 container init 7a8bc8596d8bf1e1702c248b6da1b8e26444f960f12b86a5cfeaeb4c796f9db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:43.823 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:43 smithi080 podman[74558]: 2024-02-26 06:40:43.444523791 +0000 UTC m=+0.369224149 container start 7a8bc8596d8bf1e1702c248b6da1b8e26444f960f12b86a5cfeaeb4c796f9db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-26T06:40:43.823 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:43 smithi080 podman[74558]: 2024-02-26 06:40:43.541844427 +0000 UTC m=+0.466544803 container attach 7a8bc8596d8bf1e1702c248b6da1b8e26444f960f12b86a5cfeaeb4c796f9db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-02-26T06:40:44.080 INFO:journalctl@ceph.osd.1.smithi080.stdout:Feb 26 06:40:43 smithi080 podman[74558]: 2024-02-26 06:40:43.823816028 +0000 UTC m=+0.748516395 container died 7a8bc8596d8bf1e1702c248b6da1b8e26444f960f12b86a5cfeaeb4c796f9db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-26T06:40:44.509 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.1.service' 2024-02-26T06:40:44.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:44.554 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-26T06:40:44.554 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-26T06:40:44.555 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.2 2024-02-26T06:40:44.840 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:44 smithi115 systemd[1]: Stopping Ceph osd.2 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:45.340 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2[41579]: 2024-02-26T06:40:44.933+0000 7f3865d93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:45.340 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2[41579]: 2024-02-26T06:40:44.933+0000 7f3865d93700 -1 osd.2 113 *** Got signal Terminated *** 2024-02-26T06:40:45.340 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2[41579]: 2024-02-26T06:40:44.933+0000 7f3865d93700 -1 osd.2 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:50.975 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:50 smithi115 podman[74869]: 2024-02-26 06:40:50.670881505 +0000 UTC m=+5.915399463 container died 93deb98ccbce153be0b38d19352475fd8931a9ecd098c19d062a37ed986e8d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-26T06:40:51.268 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:51 smithi115 podman[74869]: 2024-02-26 06:40:51.075186409 +0000 UTC m=+6.319704367 container cleanup 93deb98ccbce153be0b38d19352475fd8931a9ecd098c19d062a37ed986e8d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:51.268 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:51 smithi115 bash[74869]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2 2024-02-26T06:40:51.590 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:51 smithi115 podman[74886]: 2024-02-26 06:40:51.268525686 +0000 UTC m=+0.602540194 container remove 93deb98ccbce153be0b38d19352475fd8931a9ecd098c19d062a37ed986e8d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:51.849 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:51 smithi115 podman[74981]: 2024-02-26 06:40:51.501050521 +0000 UTC m=+0.017830381 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:51.849 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:51 smithi115 podman[74981]: 2024-02-26 06:40:51.610757685 +0000 UTC m=+0.127537579 container create bcaa91416dcaa0924fd3779a6adfb1a9e38f35a7877e2da564df105363bd0d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD) 2024-02-26T06:40:51.850 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:51 smithi115 podman[74981]: 2024-02-26 06:40:51.846909356 +0000 UTC m=+0.363689208 container init bcaa91416dcaa0924fd3779a6adfb1a9e38f35a7877e2da564df105363bd0d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:51.850 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:51 smithi115 podman[74981]: 2024-02-26 06:40:51.85013192 +0000 UTC m=+0.366911771 container start bcaa91416dcaa0924fd3779a6adfb1a9e38f35a7877e2da564df105363bd0d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux ) 2024-02-26T06:40:52.141 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:51 smithi115 podman[74981]: 2024-02-26 06:40:51.946882311 +0000 UTC m=+0.463662171 container attach bcaa91416dcaa0924fd3779a6adfb1a9e38f35a7877e2da564df105363bd0d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:52.405 INFO:journalctl@ceph.osd.2.smithi115.stdout:Feb 26 06:40:52 smithi115 podman[74981]: 2024-02-26 06:40:52.141848582 +0000 UTC m=+0.658628433 container died bcaa91416dcaa0924fd3779a6adfb1a9e38f35a7877e2da564df105363bd0d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-26T06:40:52.803 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.2.service' 2024-02-26T06:40:52.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:52.845 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-26T06:40:52.845 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-26T06:40:52.845 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.3 2024-02-26T06:40:53.419 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:52 smithi049 systemd[1]: Stopping Ceph osd.3 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:40:53.919 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:53 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:40:53.443+0000 7f70b5a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:53.919 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:53 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:40:53.443+0000 7f70b5a00700 -1 osd.3 113 *** Got signal Terminated *** 2024-02-26T06:40:53.920 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:53 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:40:53.443+0000 7f70b5a00700 -1 osd.3 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:53.920 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:53 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:40:53.507+0000 7f70b1200700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.152436+0000 front 2024-02-26T06:40:29.153277+0000 (oldest deadline 2024-02-26T06:40:53.154209+0000) 2024-02-26T06:40:54.730 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:40:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:40:54.358+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:40:54.919 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:54 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:40:54.480+0000 7f70b1200700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.152436+0000 front 2024-02-26T06:40:29.153277+0000 (oldest deadline 2024-02-26T06:40:53.154209+0000) 2024-02-26T06:40:55.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:40:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:40:54.586+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:40:55.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:40:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:40:54.730+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:40:55.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:40:55 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:40:55.164+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:40:55.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:40:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:40:55.334+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:40:55.919 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:55 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:40:55.462+0000 7f70b1200700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.152436+0000 front 2024-02-26T06:40:29.153277+0000 (oldest deadline 2024-02-26T06:40:53.154209+0000) 2024-02-26T06:40:55.919 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:40:55 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:40:55.640+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:40:56.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:40:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:40:55.587+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:40:56.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:40:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:40:55.712+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:40:56.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:40:56 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:40:56.152+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:40:56.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:40:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:40:56.382+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:40:56.919 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:56 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:40:56.456+0000 7f70b1200700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.152436+0000 front 2024-02-26T06:40:29.153277+0000 (oldest deadline 2024-02-26T06:40:53.154209+0000) 2024-02-26T06:40:56.920 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:40:56 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:40:56.685+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:40:57.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:40:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:40:56.596+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:40:57.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:40:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:40:56.687+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:40:57.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:40:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:40:57.125+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:40:57.677 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:40:57 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:40:57.348+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:40:57.919 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3[50368]: 2024-02-26T06:40:57.482+0000 7f70b1200700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.152436+0000 front 2024-02-26T06:40:29.153277+0000 (oldest deadline 2024-02-26T06:40:53.154209+0000) 2024-02-26T06:40:57.919 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:40:57 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:40:57.700+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:40:58.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:40:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:40:57.611+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:40:58.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:40:57 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:40:57.677+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:40:58.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:40:58 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:40:58.084+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:40:58.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:40:58 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:40:58.312+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:40:59.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:40:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:40:58.614+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:40:59.084 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:40:58 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:40:58.743+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:40:59.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:40:58 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:40:58.698+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:40:59.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:40:59 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:40:59.083+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:40:59.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:40:59 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:40:59.272+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:40:59.915 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:40:59 smithi049 podman[97836]: 2024-02-26 06:40:59.557876321 +0000 UTC m=+6.248063234 container died 6ece9138e1002ded1f4dcad70845e95784d56b71f1431b4649031f24799cc504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-26T06:40:59.915 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:40:59 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:40:59.782+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:00.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:40:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:40:59.654+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:00.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:40:59 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:40:59.685+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:00.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:00.080+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:00.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:00 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:00.278+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:00.669 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:00 smithi049 podman[97836]: 2024-02-26 06:41:00.192769036 +0000 UTC m=+6.882955992 container cleanup 6ece9138e1002ded1f4dcad70845e95784d56b71f1431b4649031f24799cc504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:41:00.670 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:00 smithi049 bash[97836]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3 2024-02-26T06:41:00.670 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:00 smithi049 podman[97855]: 2024-02-26 06:41:00.421913063 +0000 UTC m=+0.833672245 container remove 6ece9138e1002ded1f4dcad70845e95784d56b71f1431b4649031f24799cc504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-26T06:41:01.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:00 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:00.619+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:01.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:00 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:00.657+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:01.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:01 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:01.057+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:01.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:00 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:00.824+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:01.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:01.275+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:01.669 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:01 smithi049 podman[97944]: 2024-02-26 06:41:01.23984272 +0000 UTC m=+0.040612877 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:41:01.669 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:01 smithi049 podman[97944]: 2024-02-26 06:41:01.347756978 +0000 UTC m=+0.148527152 container create a02a9f5a2896c2f6e7d25ab2d2c811cb4f9aa09d4b6714d4b2c84d87503df822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:02.072 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:01 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:01.840+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:02.077 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:01 smithi049 podman[97944]: 2024-02-26 06:41:01.67789873 +0000 UTC m=+0.478668948 container init a02a9f5a2896c2f6e7d25ab2d2c811cb4f9aa09d4b6714d4b2c84d87503df822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:41:02.077 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:01 smithi049 podman[97944]: 2024-02-26 06:41:01.692484937 +0000 UTC m=+0.493255078 container start a02a9f5a2896c2f6e7d25ab2d2c811cb4f9aa09d4b6714d4b2c84d87503df822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-26T06:41:02.077 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:01 smithi049 podman[97944]: 2024-02-26 06:41:01.791827525 +0000 UTC m=+0.592597749 container attach a02a9f5a2896c2f6e7d25ab2d2c811cb4f9aa09d4b6714d4b2c84d87503df822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-02-26T06:41:02.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:01 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:01.634+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:02.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:01.610+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:02.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:02 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:02.071+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:02.420 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:02 smithi049 podman[97944]: 2024-02-26 06:41:02.148708343 +0000 UTC m=+0.949478521 container died a02a9f5a2896c2f6e7d25ab2d2c811cb4f9aa09d4b6714d4b2c84d87503df822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-26T06:41:02.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:02.319+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:03.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:02 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:02.611+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:03.092 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:02.660+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:03.103 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:03 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:03.048+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:03.103 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:02 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:02.850+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:03.103 INFO:journalctl@ceph.osd.3.smithi049.stdout:Feb 26 06:41:03 smithi049 podman[97969]: 2024-02-26 06:41:03.087656464 +0000 UTC m=+0.917894075 container remove a02a9f5a2896c2f6e7d25ab2d2c811cb4f9aa09d4b6714d4b2c84d87503df822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-26T06:41:03.146 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.3.service' 2024-02-26T06:41:03.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:03.294 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-26T06:41:03.294 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-26T06:41:03.294 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.4 2024-02-26T06:41:03.580 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:03 smithi080 systemd[1]: Stopping Ceph osd.4 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:41:03.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:03.322+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:03.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:03.322+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:03.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:03.305+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:04.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:03 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:03.597+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:04.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:03 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:03.709+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:04.081 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:03 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:41:03.613+0000 7fbdf356c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:04.081 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:03 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:41:03.613+0000 7fbdf356c700 -1 osd.4 113 *** Got signal Terminated *** 2024-02-26T06:41:04.081 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:03 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:41:03.613+0000 7fbdf356c700 -1 osd.4 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:41:04.081 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:03 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:41:03.782+0000 7fbdefb60700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.221447+0000 front 2024-02-26T06:40:37.221472+0000 (oldest deadline 2024-02-26T06:41:03.121597+0000) 2024-02-26T06:41:04.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:03.664+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:04.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:04 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:04.086+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:04.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:04 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:04.086+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:04.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:03 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:03.853+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:04.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:04.321+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:04.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:04.278+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:04.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:04.278+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:05.080 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:04 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:41:04.763+0000 7fbdefb60700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.221447+0000 front 2024-02-26T06:40:37.221472+0000 (oldest deadline 2024-02-26T06:41:03.121597+0000) 2024-02-26T06:41:05.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:04 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:04.738+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:05.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:04 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:04.594+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:05.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:04.667+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:05.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:04 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:04.901+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:05.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:05 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:05.078+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:05.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:05 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:05.078+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:05.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:05.337+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:05.321+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:05.321+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:05.830 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:05 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:41:05.753+0000 7fbdefb60700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.221447+0000 front 2024-02-26T06:40:37.221472+0000 (oldest deadline 2024-02-26T06:41:03.121597+0000) 2024-02-26T06:41:05.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:05 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:05.757+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:05.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:05 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:05.572+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:06.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:05.715+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:06.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:06 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:06.071+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:06.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:06 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:06.071+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:06.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:05 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:05.856+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:06.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:05 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:05.856+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:06.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:06.320+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:06.363+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:06.363+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:06.830 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:06 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:41:06.790+0000 7fbdefb60700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.221447+0000 front 2024-02-26T06:40:37.221472+0000 (oldest deadline 2024-02-26T06:41:03.121597+0000) 2024-02-26T06:41:06.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:06 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:06.574+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:06.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:06 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:06.721+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:07.079 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:06 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:06.846+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:07.080 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:06 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:06.846+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:07.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:06.688+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:07.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:06.688+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:07.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:07 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:07.079+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:07.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:07 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:07.079+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:07.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:07.333+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:07.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:07.338+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:07.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:07.338+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:07.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:07.573+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:07.830 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4[47739]: 2024-02-26T06:41:07.815+0000 7fbdefb60700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.221447+0000 front 2024-02-26T06:40:37.221472+0000 (oldest deadline 2024-02-26T06:41:03.121597+0000) 2024-02-26T06:41:07.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:07 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:07.759+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:08.057 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:07 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:07.837+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:08.057 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:07 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:07.837+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:08.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:07.727+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:08.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:07.727+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:08.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:08 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:08.056+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:08.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:08 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:08.056+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:08.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:08.332+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:08.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:08.290+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:08.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:08.290+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:08.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:08 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:08.752+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:08.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:08 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:08.576+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:09.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:08.756+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:09.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:08.756+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:09.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:09 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:09.026+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:09.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:09 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:09.026+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:09.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:08 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:08.824+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:09.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:08 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:08.824+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:09.473 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:09 smithi080 podman[74675]: 2024-02-26 06:41:09.19806436 +0000 UTC m=+5.710556399 container died c8e3fca96d65c450f9ebad220d0710d6ce10c940716c4cef9f6861b1b33f1d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:41:09.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:09 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:09.345+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:09.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:09 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:09.315+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:09.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:09 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:09.315+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:09.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:09 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:09.709+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:09.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:09 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:09.622+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:09.831 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:09 smithi080 podman[74675]: 2024-02-26 06:41:09.573000205 +0000 UTC m=+6.085492241 container cleanup c8e3fca96d65c450f9ebad220d0710d6ce10c940716c4cef9f6861b1b33f1d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-02-26T06:41:09.831 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:09 smithi080 bash[74675]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4 2024-02-26T06:41:10.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:09 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:09.767+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:10.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:09 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:09.767+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:10.166 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:09 smithi080 podman[74692]: 2024-02-26 06:41:09.86639448 +0000 UTC m=+0.670469598 container remove c8e3fca96d65c450f9ebad220d0710d6ce10c940716c4cef9f6861b1b33f1d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-26T06:41:10.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:09 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:09.984+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:10.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:09 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:09.984+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:10.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:09 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:09.792+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:10.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:09 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:09.792+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:10.478 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:10 smithi080 podman[74781]: 2024-02-26 06:41:10.07013813 +0000 UTC m=+0.018024742 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:41:10.478 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:10 smithi080 podman[74781]: 2024-02-26 06:41:10.192490043 +0000 UTC m=+0.140376634 container create e7cde29bb44c79b10228a8d22929da44dfddcf01e061f1fa769c56adaa85eb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:41:10.478 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:10 smithi080 podman[74781]: 2024-02-26 06:41:10.478876945 +0000 UTC m=+0.426763545 container init e7cde29bb44c79b10228a8d22929da44dfddcf01e061f1fa769c56adaa85eb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-26T06:41:10.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:10 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:10.330+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:10.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:10 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:10.291+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:10.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:10 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:10.291+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:10.739 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:10 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:10.574+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:10.739 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:10 smithi080 podman[74781]: 2024-02-26 06:41:10.482122522 +0000 UTC m=+0.430009119 container start e7cde29bb44c79b10228a8d22929da44dfddcf01e061f1fa769c56adaa85eb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-26T06:41:10.739 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:10 smithi080 podman[74781]: 2024-02-26 06:41:10.562134149 +0000 UTC m=+0.510020749 container attach e7cde29bb44c79b10228a8d22929da44dfddcf01e061f1fa769c56adaa85eb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:41:11.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:10 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:10.738+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:11.080 INFO:journalctl@ceph.osd.4.smithi080.stdout:Feb 26 06:41:10 smithi080 podman[74781]: 2024-02-26 06:41:10.778409384 +0000 UTC m=+0.726295988 container died e7cde29bb44c79b10228a8d22929da44dfddcf01e061f1fa769c56adaa85eb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-4-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:41:11.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:10 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:10.736+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:11.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:10 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:10.736+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:11.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:11 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:11.009+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:11.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:11 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:11.009+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:11.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:10 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:10.784+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:11.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:10 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:10.784+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:11.425 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.4.service' 2024-02-26T06:41:11.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:11.469 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-26T06:41:11.469 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-26T06:41:11.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.5 2024-02-26T06:41:11.591 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:11 smithi115 systemd[1]: Stopping Ceph osd.5 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:41:11.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:11.298+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:11.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:11.286+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:11.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:11.286+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:11.737 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:11 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:11.554+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:11.738 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:11 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:11.554+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:12.026 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:11 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:11.750+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:12.026 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:11 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:11.750+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:12.026 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:11 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:11.750+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:12.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:11 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:11.737+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:12.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:11.725+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:12.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:11.725+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:12.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:11.785+0000 7fcabc2ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:12.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:11.785+0000 7fcabc2ba700 -1 osd.5 113 *** Got signal Terminated *** 2024-02-26T06:41:12.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:11 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:11.785+0000 7fcabc2ba700 -1 osd.5 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:41:12.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:12.025+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:12.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:12.025+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:12.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:12.025+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:12.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:12 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:12.269+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:12.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:12 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:12.269+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:12.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:12 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:12.325+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:12.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:12 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:12.325+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:12.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:12 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:12.748+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:12.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:12 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:12.577+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:12.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:12 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:12.577+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:13.062 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:12.739+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:13.062 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:12.739+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:13.063 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:12.739+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:13.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:12 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:12.680+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:13.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:12 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:12.680+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:13.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:13 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:13.062+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:13.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:13 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:13.062+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:13.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:13 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:13.062+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:13.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:13.278+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:13.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:13.278+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:13.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:13.346+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:13.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:13.346+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:13.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:13.346+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:14.036 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:13 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:13.743+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:14.036 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:13 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:13.743+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:14.036 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:13 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:13.743+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:14.037 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:13 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:13.729+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:14.037 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:13 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:13.621+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:14.037 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:13 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:13.621+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:14.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:13.706+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:14.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:13.706+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:14.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:13 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:13.706+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.757228+0000 front 2024-02-26T06:40:45.757244+0000 (oldest deadline 2024-02-26T06:41:12.757772+0000) 2024-02-26T06:41:14.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:14.035+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:14.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:14.035+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:14.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:14.035+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:14.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:14.298+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:14.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:14.298+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:14.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:14.338+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:14.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:14.338+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:14.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:14.338+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:15.018 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:14.765+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:15.018 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:14.765+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:15.018 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:14 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:14.765+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:15.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:14 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:14.778+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:15.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:14 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:14.589+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:15.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:14 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:14.589+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:15.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:14.690+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:15.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:14.690+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:15.090 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:14 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:14.690+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.757228+0000 front 2024-02-26T06:40:45.757244+0000 (oldest deadline 2024-02-26T06:41:12.757772+0000) 2024-02-26T06:41:15.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:15 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:15.018+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:15.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:15 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:15.018+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:15.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:15 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:15.018+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:15.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:15.327+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:15.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:15.327+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:15.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:15.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:15.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:15.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:15.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:15.333+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:15.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:15.801+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:15.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:15.573+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:15.832 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:15 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:15.573+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:16.048 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:15 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:15.727+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:16.048 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:15 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:15.727+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:16.048 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:15 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:15.727+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:16.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:15.678+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:16.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:15.678+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:16.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:15 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:15.678+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.757228+0000 front 2024-02-26T06:40:45.757244+0000 (oldest deadline 2024-02-26T06:41:12.757772+0000) 2024-02-26T06:41:16.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:16.046+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:16.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:16.046+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:16.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:16.046+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:16.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:16.334+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:16.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:16.334+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:16.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:16.322+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:16.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:16.322+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:16.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:16.322+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:16.832 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:16 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:16.816+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:16.832 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:16 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:16.816+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:16.832 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:16 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:16.574+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:16.833 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:16 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:16.574+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:17.010 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:16.699+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:17.010 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:16.699+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:17.011 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:16.699+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:17.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:16.652+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.254394+0000 front 2024-02-26T06:40:28.254506+0000 (oldest deadline 2024-02-26T06:40:54.654917+0000) 2024-02-26T06:41:17.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:16.652+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:39.856969+0000 front 2024-02-26T06:40:39.856810+0000 (oldest deadline 2024-02-26T06:41:05.756796+0000) 2024-02-26T06:41:17.091 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:16 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5[48264]: 2024-02-26T06:41:16.652+0000 7fcab48a6700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.757228+0000 front 2024-02-26T06:40:45.757244+0000 (oldest deadline 2024-02-26T06:41:12.757772+0000) 2024-02-26T06:41:17.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:17 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:17.009+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:17.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:17 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:17.009+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:17.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:17 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:17.009+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:17.590 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:17 smithi115 podman[75101]: 2024-02-26 06:41:17.424257403 +0000 UTC m=+5.758717643 container died 1d8f07aef59869e29bb02b7e1573dcd609246aa8defcbd0b66d92113f2814467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-26T06:41:17.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:17.320+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:17.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:17.320+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:17.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:17.354+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:17.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:17.354+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:17.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:17 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:17.354+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:18.056 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:17 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:17.720+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:18.060 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:17 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:17.720+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:18.060 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:17 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:17.720+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:18.077 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:17 smithi115 podman[75101]: 2024-02-26 06:41:17.809967761 +0000 UTC m=+6.144428004 container cleanup 1d8f07aef59869e29bb02b7e1573dcd609246aa8defcbd0b66d92113f2814467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:41:18.077 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:17 smithi115 bash[75101]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5 2024-02-26T06:41:18.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:17 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:17.813+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:18.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:17 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:17.813+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:18.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:17 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:17.590+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:18.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:17 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:17.590+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:18.340 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:18 smithi115 podman[75119]: 2024-02-26 06:41:18.077990302 +0000 UTC m=+0.653986526 container remove 1d8f07aef59869e29bb02b7e1573dcd609246aa8defcbd0b66d92113f2814467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:41:18.341 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:18.318+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:18.341 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:18.318+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:18.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:18.055+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:18.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:18.055+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:18.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:18.055+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:18.639 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:18 smithi115 podman[75213]: 2024-02-26 06:41:18.269392783 +0000 UTC m=+0.017905306 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:41:18.639 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:18 smithi115 podman[75213]: 2024-02-26 06:41:18.413669123 +0000 UTC m=+0.162181672 container create ea200c93b4c25b46758c2238cc91b0cf6d3af085acbf53d4e8da4eea4d2a076a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0) 2024-02-26T06:41:18.640 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:18.394+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:18.640 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:18.394+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:18.640 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:18 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:18.394+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:18.930 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:18 smithi115 podman[75213]: 2024-02-26 06:41:18.640105007 +0000 UTC m=+0.388617547 container init ea200c93b4c25b46758c2238cc91b0cf6d3af085acbf53d4e8da4eea4d2a076a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-26T06:41:18.930 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:18 smithi115 podman[75213]: 2024-02-26 06:41:18.643234322 +0000 UTC m=+0.391746862 container start ea200c93b4c25b46758c2238cc91b0cf6d3af085acbf53d4e8da4eea4d2a076a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True) 2024-02-26T06:41:18.930 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:18 smithi115 podman[75213]: 2024-02-26 06:41:18.740078645 +0000 UTC m=+0.488591176 container attach ea200c93b4c25b46758c2238cc91b0cf6d3af085acbf53d4e8da4eea4d2a076a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:19.043 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:18.747+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:19.043 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:18.747+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:19.043 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:18 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:18.747+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:19.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:18.634+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:19.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:18.634+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:19.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:18.849+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:19.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:18 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:18.849+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:19.271 INFO:journalctl@ceph.osd.5.smithi115.stdout:Feb 26 06:41:18 smithi115 podman[75213]: 2024-02-26 06:41:18.929388108 +0000 UTC m=+0.677900639 container died ea200c93b4c25b46758c2238cc91b0cf6d3af085acbf53d4e8da4eea4d2a076a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-26T06:41:19.419 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:19.042+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:19.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:19.042+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:19.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:19.042+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:19.539 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:19.280+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:19.539 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:19.280+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:19.540 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:19.408+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:19.540 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:19.408+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:19.540 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:19 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:19.408+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:19.549 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.5.service' 2024-02-26T06:41:19.592 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:19.592 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-26T06:41:19.592 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-26T06:41:19.593 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.6 2024-02-26T06:41:20.071 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:19 smithi049 systemd[1]: Stopping Ceph osd.6 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:41:20.071 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:19.769+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:20.071 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:19.769+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:20.072 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:19 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:19.769+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:20.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:19 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:19.593+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:20.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:19 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:19.593+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:20.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:19 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:19.830+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:20.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:19 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:19.830+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:20.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:20.069+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:20.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:20.069+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:20.420 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:20.069+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:20.421 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:20.221+0000 7f911d200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:20.421 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:20.221+0000 7f911d200700 -1 osd.6 113 *** Got signal Terminated *** 2024-02-26T06:41:20.421 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:20.221+0000 7f911d200700 -1 osd.6 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:41:20.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:20.270+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:20.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:20.270+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:20.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:20.270+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:20.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:20.396+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:20.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:20.396+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:20.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:20 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:20.396+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:21.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:20 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:20.618+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:21.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:20 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:20.618+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:21.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:20 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:20.788+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:21.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:20 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:20.788+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:21.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:20 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:20.788+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:21.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:21.021+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:21.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:21.021+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:21.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:21.021+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:21.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:20.790+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:21.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:20.790+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:21.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:20 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:20.790+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:21.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:21 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:21.422+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:21.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:21 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:21.422+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:21.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:21 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:21.422+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:21.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:21 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:21.250+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:21.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:21 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:21.250+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:21.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:21 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:21.250+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:22.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:21 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:21.614+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:22.080 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:21 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:21.614+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:22.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:21 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:21.739+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:22.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:21 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:21.739+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:22.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:21 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:21.739+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:22.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:22.001+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:22.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:22.001+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:22.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:22.001+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:22.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:21.800+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:22.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:21.800+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:22.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:21.800+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:22.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:22 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:22.435+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:22.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:22 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:22.435+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:22.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:22 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:22.435+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:22.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:22 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:22.265+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:22.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:22 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:22.265+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:22.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:22 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:22.265+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:22.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:22 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:22.578+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:22.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:22 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:22.578+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:22.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:22 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:22.692+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:22.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:22 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:22.692+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:22.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:22 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:22.692+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:23.171 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:23.014+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:23.171 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:23.014+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:23.171 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:23.014+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:23.171 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:23.014+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.653262+0000 front 2024-02-26T06:40:57.653431+0000 (oldest deadline 2024-02-26T06:41:22.352453+0000) 2024-02-26T06:41:23.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:22.809+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:23.172 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:22.809+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:23.172 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:22.809+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:23.172 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:22 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:22.809+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:23.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:23 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:23.291+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:23.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:23 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:23.291+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:23.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:23 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:23.291+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:23.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:23 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:23.461+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:23.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:23 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:23.461+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:23.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:23 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:23.461+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:23.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:23 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:23.537+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:23.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:23 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:23.537+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:23.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:23 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:23.694+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:23.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:23 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:23.694+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:23.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:23 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:23.694+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:24.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:24.004+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:24.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:24.004+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:24.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:24.004+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:24.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:24.004+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.653262+0000 front 2024-02-26T06:40:57.653431+0000 (oldest deadline 2024-02-26T06:41:22.352453+0000) 2024-02-26T06:41:24.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:23.829+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:24.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:23.829+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:24.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:23.829+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:24.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:23 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:23.829+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:24.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:24 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:24.289+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:24.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:24 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:24.289+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:24.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:24 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:24.289+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:24.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:24 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:24.491+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:24.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:24 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:24.491+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:24.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:24 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:24.491+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:24.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:24 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:24.648+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:24.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:24 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:24.648+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:24.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:24 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:24.648+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:24.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:24 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:24.515+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:24.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:24 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:24.515+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:24.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:24 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:24.515+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:25.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:25.023+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:29.144262+0000 front 2024-02-26T06:40:29.144996+0000 (oldest deadline 2024-02-26T06:40:55.043929+0000) 2024-02-26T06:41:25.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:25.023+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.545444+0000 front 2024-02-26T06:40:38.545489+0000 (oldest deadline 2024-02-26T06:41:03.245726+0000) 2024-02-26T06:41:25.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:25.023+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.949719+0000 front 2024-02-26T06:40:48.949538+0000 (oldest deadline 2024-02-26T06:41:11.249784+0000) 2024-02-26T06:41:25.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6[57206]: 2024-02-26T06:41:25.023+0000 7f9113a00700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.653262+0000 front 2024-02-26T06:40:57.653431+0000 (oldest deadline 2024-02-26T06:41:22.352453+0000) 2024-02-26T06:41:25.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:24.855+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:25.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:24.855+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:25.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:24.855+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:25.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:24.855+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:25.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:25 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:25.310+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:25.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:25 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:25.310+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:25.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:25 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:25.310+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:25.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:25 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:25.526+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:25.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:25 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:25.526+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:25.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:25 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:25.526+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:25.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:25 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:25.467+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:25.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:25 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:25.467+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:25.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:25 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:25.467+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:25.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:25 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:25.626+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:25.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:25 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:25.626+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:25.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:25 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:25.626+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:26.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:25.866+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:26.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:25.866+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:26.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:25.866+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:26.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:25 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:25.866+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:26.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:26 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:26.296+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:26.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:26 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:26.296+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:26.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:26 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:26.296+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:26.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:26 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:26.506+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:26.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:26 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:26.506+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:26.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:26 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:26.506+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:26.634 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:26 smithi049 podman[98059]: 2024-02-26 06:41:26.284117539 +0000 UTC m=+6.194847714 container died b5e6e42a965963812713a523c6e334d19cb0b1d68062da6ce75053ebc86c7bf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-26T06:41:26.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:26 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:26.644+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:26.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:26 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:26.644+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:26.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:26 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:26.644+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:26.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:26 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:26.472+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:26.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:26 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:26.472+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:26.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:26 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:26.472+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:26.903 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:26 smithi049 podman[98059]: 2024-02-26 06:41:26.747914111 +0000 UTC m=+6.658644297 container cleanup b5e6e42a965963812713a523c6e334d19cb0b1d68062da6ce75053ebc86c7bf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-26T06:41:26.903 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:26 smithi049 bash[98059]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6 2024-02-26T06:41:27.169 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:27 smithi049 podman[98079]: 2024-02-26 06:41:27.007702235 +0000 UTC m=+0.702127341 container remove b5e6e42a965963812713a523c6e334d19cb0b1d68062da6ce75053ebc86c7bf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:41:27.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:26.901+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:27.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:26.901+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:27.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:26.901+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:27.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:26 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:26.901+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:27.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:27 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:27.511+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:27.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:27 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:27.511+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:27.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:27 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:27.511+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:27.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:27 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:27.662+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:27.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:27 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:27.662+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:27.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:27 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:27.662+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:27.840 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:27 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:27.341+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:27.840 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:27 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:27.341+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:27.840 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:27 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:27.341+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:27.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:27 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:27.486+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:27.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:27 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:27.486+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:27.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:27 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:27.486+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:28.221 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:27 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:27.943+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:28.221 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:27 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:27.943+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:28.221 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:27 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:27.943+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:28.221 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:27 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:27.943+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:28.221 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:27 smithi049 podman[98162]: 2024-02-26 06:41:27.8330567 +0000 UTC m=+0.038424911 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:41:28.221 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:27 smithi049 podman[98162]: 2024-02-26 06:41:27.966901159 +0000 UTC m=+0.172269411 container create f4d8c574fafe3059d015fdd69738e633dd86e82230054db1fbd7d31311c2f533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:41:28.221 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:28 smithi049 podman[98162]: 2024-02-26 06:41:28.220695654 +0000 UTC m=+0.426063946 container init f4d8c574fafe3059d015fdd69738e633dd86e82230054db1fbd7d31311c2f533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T06:41:28.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:28.321+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:28.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:28.321+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:28.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:28.321+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:28.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:28.457+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:28.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:28.457+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:28.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:28 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:28.457+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:28.669 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:28 smithi049 podman[98162]: 2024-02-26 06:41:28.234057302 +0000 UTC m=+0.439425503 container start f4d8c574fafe3059d015fdd69738e633dd86e82230054db1fbd7d31311c2f533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-26T06:41:28.669 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:28 smithi049 podman[98162]: 2024-02-26 06:41:28.352823082 +0000 UTC m=+0.558191395 container attach f4d8c574fafe3059d015fdd69738e633dd86e82230054db1fbd7d31311c2f533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-26T06:41:28.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:28 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:28.679+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:28.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:28 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:28.679+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:28.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:28 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:28.679+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:28.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:28 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:28.524+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:28.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:28 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:28.524+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:28.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:28 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:28.524+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:29.004 INFO:journalctl@ceph.osd.6.smithi049.stdout:Feb 26 06:41:28 smithi049 podman[98162]: 2024-02-26 06:41:28.69293726 +0000 UTC m=+0.898305461 container died f4d8c574fafe3059d015fdd69738e633dd86e82230054db1fbd7d31311c2f533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-26T06:41:29.004 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:28 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:28.922+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:29.004 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:28 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:28.922+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:29.004 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:28 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:28.922+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:29.004 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:28 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:28.922+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:29.369 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.6.service' 2024-02-26T06:41:29.512 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:29.515 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-26T06:41:29.515 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-26T06:41:29.515 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.7 2024-02-26T06:41:29.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:29 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:29.310+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:29.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:29 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:29.310+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:29.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:29 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:29.310+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:29.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:29 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:29.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:29.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:29 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:29.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:29.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:29 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:29.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:29.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:29.573+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:29.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:29.573+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:29.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:29.573+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:29.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:29 smithi080 systemd[1]: Stopping Ceph osd.7 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:41:29.832 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:29.668+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:29.832 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:29.668+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:29.832 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:29.668+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:30.330 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:29.834+0000 7f75c3c05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:30.330 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:29.834+0000 7f75c3c05700 -1 osd.7 113 *** Got signal Terminated *** 2024-02-26T06:41:30.330 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:29 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:29.834+0000 7f75c3c05700 -1 osd.7 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:41:30.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:29.959+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:30.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:29.959+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:30.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:29.959+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:30.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:29 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:29.959+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:30.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:30 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:30.277+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:30.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:30 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:30.277+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:30.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:30 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:30.277+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:30.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:30 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:30.418+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:30.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:30 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:30.418+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:30.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:30 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:30.418+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:30.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:30 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:30.562+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:30.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:30 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:30.562+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:30.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:30 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:30.562+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:30.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:30 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:30.680+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:30.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:30 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:30.680+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:30.831 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:30 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:30.680+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:31.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:30.934+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:31.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:30.934+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:31.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:30.934+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:31.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:30.934+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:31.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:30 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:30.934+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:31.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:31 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:31.233+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:31.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:31 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:31.233+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:31.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:31 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:31.233+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:31.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:31 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:31.380+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:31.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:31 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:31.380+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:31.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:31 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:31.380+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:31.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:31 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:31.380+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:32.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:31 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:31.723+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:32.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:31 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:31.723+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:32.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:31 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:31.723+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:32.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:31 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:31.612+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:32.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:31 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:31.612+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:32.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:31 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:31.612+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:32.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:31 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:31.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:32.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:31 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:31.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:32.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:31 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:31.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:32.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:31 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:31.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:32.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:31 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:31.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:32.492 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:32.198+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:32.492 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:32.198+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:32.492 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:32.198+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:32.492 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:32.198+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:32.493 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:32.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:32.493 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:32.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:32.493 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:32.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:32.493 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:32 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:32.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:33.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:32.698+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:33.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:32.698+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:33.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:32.698+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:33.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:32.609+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:33.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:32.609+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:33.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:32.609+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:33.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:32.609+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:33.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:32 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:32.913+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:33.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:32 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:32.913+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:33.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:32 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:32.913+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:33.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:32 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:32.913+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:33.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:32 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:32.913+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:33.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:33 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:33.226+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:33.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:33 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:33.226+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:33.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:33 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:33.226+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:33.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:33 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:33.226+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:33.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:33 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:33.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:33.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:33 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:33.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:33.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:33 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:33.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:33.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:33 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:33.419+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:34.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:33 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:33.712+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:34.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:33 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:33.712+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:34.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:33 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:33.712+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:34.081 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:33 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:33.712+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:07.502412+0000 front 2024-02-26T06:41:07.502340+0000 (oldest deadline 2024-02-26T06:41:33.402489+0000) 2024-02-26T06:41:34.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:33 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:33.641+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:34.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:33 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:33.641+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:34.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:33 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:33.641+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:34.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:33 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:33.641+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:34.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:33.920+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:34.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:33.920+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:34.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:33.920+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:34.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:33.920+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:34.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:33 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:33.920+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:34.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:34 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:34.277+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:34.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:34 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:34.277+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:34.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:34 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:34.277+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:34.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:34 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:34.277+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:34.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:34 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:34.382+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:34.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:34 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:34.382+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:34.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:34 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:34.382+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:34.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:34 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:34.382+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:35.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:34 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:34.731+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.298422+0000 front 2024-02-26T06:40:38.298454+0000 (oldest deadline 2024-02-26T06:41:03.598570+0000) 2024-02-26T06:41:35.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:34 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:34.731+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.899654+0000 front 2024-02-26T06:40:45.899609+0000 (oldest deadline 2024-02-26T06:41:16.500323+0000) 2024-02-26T06:41:35.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:34 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:34.731+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:56.501073+0000 front 2024-02-26T06:40:56.501198+0000 (oldest deadline 2024-02-26T06:41:19.901234+0000) 2024-02-26T06:41:35.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:34 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7[54314]: 2024-02-26T06:41:34.731+0000 7f75c01f9700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:07.502412+0000 front 2024-02-26T06:41:07.502340+0000 (oldest deadline 2024-02-26T06:41:33.402489+0000) 2024-02-26T06:41:35.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:34 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:34.603+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:35.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:34 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:34.603+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:35.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:34 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:34.603+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:35.081 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:34 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:34.603+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:35.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:34 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:34.895+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:35.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:34 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:34.895+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:35.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:34 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:34.895+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:35.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:34 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:34.895+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:35.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:34 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:34.895+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:35.594 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:35 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:35.271+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:35.595 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:35 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:35.271+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:35.595 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:35 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:35.271+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:35.595 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:35 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:35.271+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:35.595 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:35 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:35.337+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:35.595 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:35 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:35.337+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:35.595 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:35 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:35.337+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:35.595 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:35 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:35.337+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:35.752 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:35 smithi080 podman[74899]: 2024-02-26 06:41:35.422613847 +0000 UTC m=+5.707481699 container died a7eec9527b6a1083594fc19b1f762b1da9efe963853bbc072207c064ec700a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-26T06:41:35.752 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:35 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:35.585+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:35.752 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:35 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:35.585+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:35.752 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:35 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:35.585+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:35.752 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:35 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:35.585+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:36.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:35 smithi080 podman[74899]: 2024-02-26 06:41:35.860071647 +0000 UTC m=+6.144939491 container cleanup a7eec9527b6a1083594fc19b1f762b1da9efe963853bbc072207c064ec700a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:36.080 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:35 smithi080 bash[74899]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7 2024-02-26T06:41:36.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:35 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:35.926+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:36.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:35 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:35.926+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:36.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:35 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:35.926+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:36.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:35 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:35.926+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:36.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:35 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:35.926+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:36.519 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:36 smithi080 podman[74916]: 2024-02-26 06:41:36.111802579 +0000 UTC m=+0.674302501 container remove a7eec9527b6a1083594fc19b1f762b1da9efe963853bbc072207c064ec700a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-26T06:41:36.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:36.269+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:36.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:36.269+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:36.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:36.269+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:36.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:36.269+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:36.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:36.381+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:36.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:36.381+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:36.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:36.381+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:36.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:36 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:36.381+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:36.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:36 smithi080 podman[75006]: 2024-02-26 06:41:36.422680814 +0000 UTC m=+0.017935704 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:41:36.830 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:36 smithi080 podman[75006]: 2024-02-26 06:41:36.554578701 +0000 UTC m=+0.149833578 container create 09636e64541ba4c9f0f9a3e062976a973bc9c77f391001ef33562410b1c90bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:41:36.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:36.565+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:36.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:36.565+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:36.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:36.565+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:36.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:36.565+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:37.142 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:36 smithi080 podman[75006]: 2024-02-26 06:41:36.845757989 +0000 UTC m=+0.441012883 container init 09636e64541ba4c9f0f9a3e062976a973bc9c77f391001ef33562410b1c90bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-26T06:41:37.142 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:36 smithi080 podman[75006]: 2024-02-26 06:41:36.848795069 +0000 UTC m=+0.444049948 container start 09636e64541ba4c9f0f9a3e062976a973bc9c77f391001ef33562410b1c90bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T06:41:37.142 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:36 smithi080 podman[75006]: 2024-02-26 06:41:36.951106703 +0000 UTC m=+0.546361591 container attach 09636e64541ba4c9f0f9a3e062976a973bc9c77f391001ef33562410b1c90bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:41:37.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:36 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:36.918+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:37.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:36 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:36.918+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:37.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:36 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:36.918+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:37.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:36 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:36.918+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:37.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:36 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:36.918+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:37.401 INFO:journalctl@ceph.osd.7.smithi080.stdout:Feb 26 06:41:37 smithi080 podman[75006]: 2024-02-26 06:41:37.143036699 +0000 UTC m=+0.738291604 container died 09636e64541ba4c9f0f9a3e062976a973bc9c77f391001ef33562410b1c90bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:37.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:37 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:37.244+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:37.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:37 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:37.244+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:37.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:37 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:37.244+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:37.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:37 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:37.244+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:37.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:37 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:37.379+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:37.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:37 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:37.379+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:37.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:37 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:37.379+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:37.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:37 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:37.379+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:37.789 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:37 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:37.523+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:37.789 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:37 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:37.523+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:37.789 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:37 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:37.523+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:37.789 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:37 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:37.523+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:37.806 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.7.service' 2024-02-26T06:41:37.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:37.847 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-26T06:41:37.847 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-26T06:41:37.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.8 2024-02-26T06:41:38.153 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:37 smithi115 systemd[1]: Stopping Ceph osd.8 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:41:38.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:37 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:37.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:38.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:37 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:37.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:38.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:37 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:37.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:38.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:37 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:37.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:38.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:37 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:37.914+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:38.580 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:38 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:38.548+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:38.580 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:38 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:38.548+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:38.580 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:38 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:38.548+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:38.580 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:38 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:38.548+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:38.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:38.152+0000 7fd4b737f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:38.590 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:38.152+0000 7fd4b737f700 -1 osd.8 113 *** Got signal Terminated *** 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:38.152+0000 7fd4b737f700 -1 osd.8 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:38.200+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:38.200+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:38.200+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:38.200+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:38.395+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:38.395+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:38.395+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:38.395+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:38.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:38 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:38.395+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:39.169 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:38.872+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:39.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:38.872+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:39.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:38.872+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:39.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:38.872+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:39.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:38 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:38.872+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:39.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:39.218+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:39.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:39.218+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:39.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:39.218+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:39.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:39.218+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:39.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:39.218+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:15.972062+0000 front 2024-02-26T06:41:15.972000+0000 (oldest deadline 2024-02-26T06:41:38.872097+0000) 2024-02-26T06:41:39.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:39.399+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:39.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:39.399+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:39.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:39.399+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:39.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:39.399+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:39.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:39 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:39.399+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:39.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:39 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:39.530+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:39.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:39 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:39.530+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:39.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:39 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:39.530+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:39.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:39 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:39.530+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:40.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:39 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:39.897+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:40.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:39 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:39.897+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:40.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:39 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:39.897+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:40.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:39 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:39.897+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:40.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:39 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:39.897+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:40.592 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:40.216+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:40.592 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:40.216+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:40.592 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:40.216+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:40.593 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:40.216+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:40.593 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:40.216+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:15.972062+0000 front 2024-02-26T06:41:15.972000+0000 (oldest deadline 2024-02-26T06:41:38.872097+0000) 2024-02-26T06:41:40.593 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:40.411+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:40.593 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:40.411+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:40.593 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:40.411+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:40.593 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:40.411+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:40.593 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:40 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:40.411+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:40.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:40 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:40.516+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:40.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:40 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:40.516+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:40.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:40 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:40.516+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:40.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:40 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:40.516+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:40.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:40 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:40.516+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:41.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:40 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:40.917+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:41.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:40 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:40.917+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:41.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:40 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:40.917+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:41.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:40 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:40.917+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:41.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:40 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:40.917+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:41.171 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:40 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:40.917+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:41.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:41.261+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:41.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:41.261+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:41.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:41.261+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:41.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:41.261+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:41.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:41.261+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:15.972062+0000 front 2024-02-26T06:41:15.972000+0000 (oldest deadline 2024-02-26T06:41:38.872097+0000) 2024-02-26T06:41:41.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:41.409+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:41.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:41.409+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:41.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:41.409+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:41.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:41.409+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:41.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:41 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:41.409+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:41.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:41 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:41.483+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:41.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:41 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:41.483+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:41.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:41 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:41.483+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:41.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:41 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:41.483+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:41.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:41 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:41.483+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:42.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:41 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:41.967+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:42.421 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:41 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:41.967+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:42.421 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:41 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:41.967+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:42.421 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:41 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:41.967+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:42.421 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:41 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:41.967+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:42.422 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:41 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:41.967+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:42.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:42.278+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:38.765961+0000 front 2024-02-26T06:40:38.766027+0000 (oldest deadline 2024-02-26T06:41:02.866090+0000) 2024-02-26T06:41:42.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:42.278+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:49.768322+0000 front 2024-02-26T06:40:49.768470+0000 (oldest deadline 2024-02-26T06:41:11.468383+0000) 2024-02-26T06:41:42.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:42.278+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.369511+0000 front 2024-02-26T06:40:57.369826+0000 (oldest deadline 2024-02-26T06:41:20.269342+0000) 2024-02-26T06:41:42.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:42.278+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.670888+0000 front 2024-02-26T06:41:06.670784+0000 (oldest deadline 2024-02-26T06:41:31.970690+0000) 2024-02-26T06:41:42.591 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8[55139]: 2024-02-26T06:41:42.278+0000 7fd4b3973700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:15.972062+0000 front 2024-02-26T06:41:15.972000+0000 (oldest deadline 2024-02-26T06:41:38.872097+0000) 2024-02-26T06:41:42.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:42.368+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:42.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:42.368+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:42.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:42.368+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:42.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:42.368+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:42.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:42 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:42.368+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:42.752 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:42 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:42.493+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:42.752 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:42 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:42.493+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:42.752 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:42 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:42.493+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:42.753 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:42 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:42.493+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:42.753 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:42 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:42.493+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:43.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:42.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:43.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:42.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:43.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:42.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:43.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:42.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:43.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:42.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:43.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:42 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:42.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:43.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:43 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:43.328+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:43.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:43 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:43.328+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:43.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:43 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:43.328+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:43.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:43 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:43.328+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:43.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:43 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:43.328+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:43.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:43 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:43.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:43.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:43 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:43.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:43.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:43 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:43.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:43.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:43 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:43.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:43.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:43 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:43.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:44.082 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:43 smithi115 podman[75330]: 2024-02-26 06:41:43.770626661 +0000 UTC m=+5.723971243 container died 6783ff727ff69013761321c3cb50e3148c65fefaaa24cedaec7beca986978f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-26T06:41:44.340 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:44 smithi115 podman[75330]: 2024-02-26 06:41:44.177362198 +0000 UTC m=+6.130706779 container cleanup 6783ff727ff69013761321c3cb50e3148c65fefaaa24cedaec7beca986978f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-26T06:41:44.341 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:44 smithi115 bash[75330]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8 2024-02-26T06:41:44.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:44.288+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:44.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:44.288+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:44.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:44.288+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:44.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:44.288+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:44.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:44.288+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:44.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.006+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:44.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.006+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:44.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.006+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:44.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.006+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:44.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.006+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:44.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.006+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:44.734 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:44 smithi115 podman[75348]: 2024-02-26 06:41:44.412058442 +0000 UTC m=+0.641305139 container remove 6783ff727ff69013761321c3cb50e3148c65fefaaa24cedaec7beca986978f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-26T06:41:44.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:44 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:44.499+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:44.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:44 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:44.499+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:44.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:44 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:44.499+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:44.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:44 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:44.499+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:44.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:44 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:44.499+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:45.081 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:44 smithi115 podman[75441]: 2024-02-26 06:41:44.638151785 +0000 UTC m=+0.017701069 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:41:45.082 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:44 smithi115 podman[75441]: 2024-02-26 06:41:44.746263983 +0000 UTC m=+0.125813251 container create 83a15f926e0a5e91f1de822ec8feb907a34fc07982654ef5a1d1b34b0d85b82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-26T06:41:45.082 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:44 smithi115 podman[75441]: 2024-02-26 06:41:44.982014857 +0000 UTC m=+0.361564146 container init 83a15f926e0a5e91f1de822ec8feb907a34fc07982654ef5a1d1b34b0d85b82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:41:45.082 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:44 smithi115 podman[75441]: 2024-02-26 06:41:44.985479128 +0000 UTC m=+0.365028394 container start 83a15f926e0a5e91f1de822ec8feb907a34fc07982654ef5a1d1b34b0d85b82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-26T06:41:45.082 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:45 smithi115 podman[75441]: 2024-02-26 06:41:45.082181204 +0000 UTC m=+0.461730483 container attach 83a15f926e0a5e91f1de822ec8feb907a34fc07982654ef5a1d1b34b0d85b82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-26T06:41:45.341 INFO:journalctl@ceph.osd.8.smithi115.stdout:Feb 26 06:41:45 smithi115 podman[75441]: 2024-02-26 06:41:45.293025126 +0000 UTC m=+0.672574401 container died 83a15f926e0a5e91f1de822ec8feb907a34fc07982654ef5a1d1b34b0d85b82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-26T06:41:45.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:45 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:45.251+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:45.342 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:45 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:45.251+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:45.342 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:45 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:45.251+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:45.342 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:45 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:45.251+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:45.342 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:45 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:45.251+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:45.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.991+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:45.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.991+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:45.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.991+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:45.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.991+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:45.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.991+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:45.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:44 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:44.991+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:45.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:45 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:45.540+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:45.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:45 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:45.540+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:45.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:45 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:45.540+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:45.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:45 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:45.540+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:45.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:45 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:45.540+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:45.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.8.service' 2024-02-26T06:41:45.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:45.952 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-26T06:41:45.953 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-26T06:41:45.953 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.9 2024-02-26T06:41:46.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:45.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:46.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:45.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:46.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:45.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:46.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:45.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:46.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:45.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:46.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:45 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:45.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:46.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 systemd[1]: Stopping Ceph osd.9 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:41:46.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:46 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:46.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:46.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:46 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:46.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:46.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:46 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:46.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:46.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:46 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:46.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:46.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:46 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:46.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:46.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:46 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:46.505+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:46.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:46 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:46.505+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:46.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:46 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:46.505+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:46.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:46 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:46.505+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:46.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:46 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:46.505+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:46.919 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.591+0000 7fa7a4e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:46.919 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.591+0000 7fa7a4e00700 -1 osd.9 113 *** Got signal Terminated *** 2024-02-26T06:41:46.920 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.591+0000 7fa7a4e00700 -1 osd.9 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:41:47.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.979+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:47.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.979+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:47.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.979+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:47.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.979+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:47.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.979+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:47.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:46 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:46.979+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:47.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:47 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:47.236+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:47.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:47 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:47.236+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:47.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:47 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:47.236+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:47.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:47 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:47.236+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:47.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:47 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:47.236+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:47.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:47 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:47.472+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:47.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:47 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:47.472+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:47.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:47 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:47.472+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:47.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:47 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:47.472+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:47.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:47 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:47.472+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:48.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:47.953+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:48.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:47.953+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:48.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:47.953+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:48.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:47.953+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:48.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:47.953+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:48.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:47 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:47.953+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:48.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:48 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:48.227+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:48.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:48 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:48.227+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:48.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:48 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:48.227+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:48.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:48 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:48.227+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:48.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:48 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:48.227+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:48.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:48 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:48.498+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:48.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:48 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:48.498+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:48.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:48 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:48.498+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:48.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:48 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:48.498+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:48.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:48 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:48.498+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:49.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:48.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:49.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:48.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:49.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:48.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:49.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:48.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:49.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:48.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:49.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:48 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:48.963+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:49.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:49 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:49.215+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:49.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:49 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:49.215+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:49.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:49 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:49.215+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:49.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:49 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:49.215+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:49.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:49 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:49.215+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:49.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:49 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:49.508+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:49.923 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:49 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:49.508+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:49.924 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:49 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:49.508+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:49.924 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:49 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:49.508+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:49.924 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:49 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:49.508+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:49.924 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:49 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:49.508+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:50.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:49 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:49.981+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:50.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:49 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:49.981+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:50.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:49 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:49.981+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:50.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:49 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:49.981+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:50.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:49 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:49.981+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:50.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:49 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:49.981+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:50.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:50 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:50.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:50.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:50 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:50.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:50.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:50 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:50.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:50.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:50 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:50.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:50.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:50 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:50.260+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:50.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:50 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:50.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:50.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:50 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:50.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:50.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:50 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:50.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:50.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:50 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:50.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:50.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:50 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:50.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:50.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:50 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:50.517+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:51.419 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:50.961+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:31.336857+0000 front 2024-02-26T06:40:31.335867+0000 (oldest deadline 2024-02-26T06:40:54.836039+0000) 2024-02-26T06:41:51.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:50.961+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:40.539887+0000 front 2024-02-26T06:40:40.539953+0000 (oldest deadline 2024-02-26T06:41:05.839919+0000) 2024-02-26T06:41:51.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:50.961+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:45.840645+0000 front 2024-02-26T06:40:45.840692+0000 (oldest deadline 2024-02-26T06:41:11.740864+0000) 2024-02-26T06:41:51.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:50.961+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:57.643536+0000 front 2024-02-26T06:40:57.643327+0000 (oldest deadline 2024-02-26T06:41:22.343595+0000) 2024-02-26T06:41:51.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:50.961+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.548923+0000 front 2024-02-26T06:41:08.548849+0000 (oldest deadline 2024-02-26T06:41:30.848726+0000) 2024-02-26T06:41:51.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:50.961+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:14.852049+0000 front 2024-02-26T06:41:14.852136+0000 (oldest deadline 2024-02-26T06:41:40.152342+0000) 2024-02-26T06:41:51.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:50 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9[63815]: 2024-02-26T06:41:50.961+0000 7fa79b600700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.854494+0000 front 2024-02-26T06:41:24.854713+0000 (oldest deadline 2024-02-26T06:41:50.754257+0000) 2024-02-26T06:41:51.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:51.293+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:51.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:51.293+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:51.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:51.293+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:51.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:51.293+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:51.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:51.293+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:51.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:51 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:51.293+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:51.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:51 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:51.489+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:51.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:51 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:51.489+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:51.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:51 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:51.489+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:51.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:51 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:51.489+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:51.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:51 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:51.489+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:51.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:51 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:51.489+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:52.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:52.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:52.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:52.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:52.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:52.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:52.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:52.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:52.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:52.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:52.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:52.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:52.670 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:52 smithi049 podman[98266]: 2024-02-26 06:41:52.378370001 +0000 UTC m=+5.946279774 container stop 55697ca51998c3c555954e641612c4bc653864731740290c23ed362c8ca44af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:41:52.670 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:52 smithi049 podman[98266]: 2024-02-26 06:41:52.406710965 +0000 UTC m=+5.974620738 container died 55697ca51998c3c555954e641612c4bc653864731740290c23ed362c8ca44af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-26T06:41:52.789 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:52 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:52.446+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:52.790 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:52 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:52.446+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:52.790 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:52 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:52.446+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:52.790 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:52 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:52.446+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:52.790 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:52 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:52.446+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:52.790 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:52 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:52.446+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:53.028 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:52 smithi049 podman[98266]: 2024-02-26 06:41:52.799001256 +0000 UTC m=+6.366911079 container cleanup 55697ca51998c3c555954e641612c4bc653864731740290c23ed362c8ca44af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T06:41:53.028 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:52 smithi049 bash[98266]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9 2024-02-26T06:41:53.420 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:53 smithi049 podman[98285]: 2024-02-26 06:41:53.024629312 +0000 UTC m=+0.606043009 container remove 55697ca51998c3c555954e641612c4bc653864731740290c23ed362c8ca44af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-26T06:41:53.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:53.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:53.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:53.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:53.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:53.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:53.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:53.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:53.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:53.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:53.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:53 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:53.297+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:53.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:53.431+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:53.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:53.431+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:53.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:53.431+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:53.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:53.431+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:53.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:53.431+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:53.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:53 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:53.431+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:54.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:53 smithi049 podman[98374]: 2024-02-26 06:41:53.820391284 +0000 UTC m=+0.034652541 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:41:54.170 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:53 smithi049 podman[98374]: 2024-02-26 06:41:53.917113365 +0000 UTC m=+0.131374657 container create 86c9e8b20446e98f6c2838b6dd9c0ab010b027c62d5d04728d385d7bb331277b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:54.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:54.320+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:54.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:54.320+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:54.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:54.320+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:54.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:54.320+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:54.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:54.320+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:54.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:54 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:54.320+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:54.669 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:54 smithi049 podman[98374]: 2024-02-26 06:41:54.246023066 +0000 UTC m=+0.460284378 container init 86c9e8b20446e98f6c2838b6dd9c0ab010b027c62d5d04728d385d7bb331277b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:41:54.669 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:54 smithi049 podman[98374]: 2024-02-26 06:41:54.257736673 +0000 UTC m=+0.471997968 container start 86c9e8b20446e98f6c2838b6dd9c0ab010b027c62d5d04728d385d7bb331277b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-26T06:41:54.669 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:54 smithi049 podman[98374]: 2024-02-26 06:41:54.376031193 +0000 UTC m=+0.590292545 container attach 86c9e8b20446e98f6c2838b6dd9c0ab010b027c62d5d04728d385d7bb331277b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-02-26T06:41:54.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:54.407+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:54.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:54.407+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:54.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:54.407+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:54.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:54.407+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:54.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:54.407+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:54.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:54 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:54.407+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:55.054 INFO:journalctl@ceph.osd.9.smithi049.stdout:Feb 26 06:41:54 smithi049 podman[98374]: 2024-02-26 06:41:54.711274622 +0000 UTC m=+0.925535968 container died 86c9e8b20446e98f6c2838b6dd9c0ab010b027c62d5d04728d385d7bb331277b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:41:55.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:55.273+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:55.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:55.273+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:55.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:55.273+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:55.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:55.273+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:55.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:55.273+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:55.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:55 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:55.273+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:55.620 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.9.service' 2024-02-26T06:41:55.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:55.750 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-26T06:41:55.751 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-26T06:41:55.751 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.10 2024-02-26T06:41:55.779 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:55.426+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:55.780 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:55.426+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:55.780 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:55.426+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:55.780 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:55.426+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:55.780 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:55.426+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:55.780 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:55 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:55.426+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:56.057 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:55 smithi080 systemd[1]: Stopping Ceph osd.10 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:41:56.330 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.057+0000 7f0d7f213700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:56.330 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.057+0000 7f0d7f213700 -1 osd.10 113 *** Got signal Terminated *** 2024-02-26T06:41:56.330 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.057+0000 7f0d7f213700 -1 osd.10 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:41:56.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:56.277+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:56.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:56.277+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:56.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:56.277+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:56.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:56.277+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:56.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:56.277+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:56.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:56.277+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:56.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.447+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:56.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.447+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:56.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.447+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:56.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.447+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:56.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.447+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:56.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:56 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:56.447+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:57.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:57 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:57.268+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:57.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:57 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:57.268+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:57.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:57 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:57.268+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:57.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:57 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:57.268+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:57.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:57 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:57.268+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:57.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:57 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:57.268+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:57.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:57.470+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:57.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:57.470+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:57.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:57.470+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:57.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:57.470+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:57.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:57.470+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:57.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:57.470+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:57.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:57 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:57.470+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.226535+0000 front 2024-02-26T06:41:34.226510+0000 (oldest deadline 2024-02-26T06:41:57.126575+0000) 2024-02-26T06:41:58.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:58 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:58.256+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:58.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:58 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:58.256+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:58.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:58 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:58.256+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:58.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:58 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:58.256+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:58.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:58 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:58.256+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:58.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:58 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:58.256+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:58.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:58.497+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:58.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:58.497+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:58.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:58.497+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:58.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:58.497+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:58.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:58.497+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:58.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:58.497+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:58.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:58 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:58.497+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.226535+0000 front 2024-02-26T06:41:34.226510+0000 (oldest deadline 2024-02-26T06:41:57.126575+0000) 2024-02-26T06:41:59.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:59 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:59.207+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:41:59.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:59 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:59.207+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:41:59.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:59 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:59.207+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:41:59.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:59 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:59.207+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:41:59.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:59 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:59.207+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:41:59.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:41:59 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:41:59.207+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:41:59.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:59.471+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:41:59.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:59.471+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:41:59.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:59.471+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:41:59.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:59.471+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:41:59.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:59.471+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:41:59.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:59.471+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:41:59.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:41:59 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:41:59.471+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.226535+0000 front 2024-02-26T06:41:34.226510+0000 (oldest deadline 2024-02-26T06:41:57.126575+0000) 2024-02-26T06:42:00.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:00 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:00.191+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:00.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:00 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:00.191+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:00.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:00 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:00.191+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:00.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:00 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:00.191+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:00.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:00 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:00.191+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:00.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:00 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:00.191+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:00.830 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:00 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:42:00.436+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:28.813273+0000 front 2024-02-26T06:40:28.813258+0000 (oldest deadline 2024-02-26T06:40:54.113126+0000) 2024-02-26T06:42:00.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:00 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:42:00.436+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:48.517484+0000 front 2024-02-26T06:40:48.517632+0000 (oldest deadline 2024-02-26T06:41:10.817507+0000) 2024-02-26T06:42:00.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:00 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:42:00.436+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-02-26T06:40:58.320017+0000 front 2024-02-26T06:40:58.319934+0000 (oldest deadline 2024-02-26T06:41:24.120394+0000) 2024-02-26T06:42:00.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:00 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:42:00.436+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:06.921891+0000 front 2024-02-26T06:41:06.921849+0000 (oldest deadline 2024-02-26T06:41:31.622133+0000) 2024-02-26T06:42:00.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:00 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:42:00.436+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.923136+0000 front 2024-02-26T06:41:13.923245+0000 (oldest deadline 2024-02-26T06:41:39.822996+0000) 2024-02-26T06:42:00.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:00 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:42:00.436+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:25.124084+0000 front 2024-02-26T06:41:25.124158+0000 (oldest deadline 2024-02-26T06:41:48.623875+0000) 2024-02-26T06:42:00.831 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:00 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10[61253]: 2024-02-26T06:42:00.436+0000 7f0d777ff700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.226535+0000 front 2024-02-26T06:41:34.226510+0000 (oldest deadline 2024-02-26T06:41:57.126575+0000) 2024-02-26T06:42:01.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:01.209+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:01.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:01.209+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:01.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:01.209+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:01.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:01.209+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:01.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:01.209+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:01.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:01.209+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:01.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:01 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:01.209+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.798616+0000 front 2024-02-26T06:41:34.798594+0000 (oldest deadline 2024-02-26T06:42:00.698483+0000) 2024-02-26T06:42:01.948 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:01 smithi080 podman[75124]: 2024-02-26 06:42:01.67803261 +0000 UTC m=+5.737747258 container died 76464ee1ac5df7785c271cf4bdabb093b9dceb8ec501beb1a95e2556ab2e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:42:02.299 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:02 smithi080 podman[75124]: 2024-02-26 06:42:02.031726807 +0000 UTC m=+6.091441474 container cleanup 76464ee1ac5df7785c271cf4bdabb093b9dceb8ec501beb1a95e2556ab2e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-26T06:42:02.299 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:02 smithi080 bash[75124]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10 2024-02-26T06:42:02.492 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:02.197+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:02.492 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:02.197+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:02.492 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:02.197+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:02.492 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:02.197+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:02.492 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:02.197+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:02.493 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:02.197+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:02.493 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:02 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:02.197+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.798616+0000 front 2024-02-26T06:41:34.798594+0000 (oldest deadline 2024-02-26T06:42:00.698483+0000) 2024-02-26T06:42:02.580 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:02 smithi080 podman[75143]: 2024-02-26 06:42:02.299458557 +0000 UTC m=+0.624963971 container remove 76464ee1ac5df7785c271cf4bdabb093b9dceb8ec501beb1a95e2556ab2e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:42:02.902 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:02 smithi080 podman[75233]: 2024-02-26 06:42:02.49540477 +0000 UTC m=+0.017624238 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:42:02.902 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:02 smithi080 podman[75233]: 2024-02-26 06:42:02.600110871 +0000 UTC m=+0.122330316 container create 3a215c6a2cb248dba0567d684311894d3320dbdfdd488eb2cab34158acfd2c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:42:02.902 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:02 smithi080 podman[75233]: 2024-02-26 06:42:02.819499752 +0000 UTC m=+0.341719224 container init 3a215c6a2cb248dba0567d684311894d3320dbdfdd488eb2cab34158acfd2c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-26T06:42:02.902 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:02 smithi080 podman[75233]: 2024-02-26 06:42:02.822553051 +0000 UTC m=+0.344772510 container start 3a215c6a2cb248dba0567d684311894d3320dbdfdd488eb2cab34158acfd2c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-26T06:42:02.902 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:02 smithi080 podman[75233]: 2024-02-26 06:42:02.902653488 +0000 UTC m=+0.424872953 container attach 3a215c6a2cb248dba0567d684311894d3320dbdfdd488eb2cab34158acfd2c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-26T06:42:03.330 INFO:journalctl@ceph.osd.10.smithi080.stdout:Feb 26 06:42:03 smithi080 podman[75233]: 2024-02-26 06:42:03.112258168 +0000 UTC m=+0.634477627 container died 3a215c6a2cb248dba0567d684311894d3320dbdfdd488eb2cab34158acfd2c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True) 2024-02-26T06:42:03.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:03.245+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:03.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:03.245+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:03.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:03.245+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:03.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:03.245+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:03.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:03.245+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:03.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:03.245+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:03.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:03 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:03.245+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.798616+0000 front 2024-02-26T06:41:34.798594+0000 (oldest deadline 2024-02-26T06:42:00.698483+0000) 2024-02-26T06:42:03.847 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.10.service' 2024-02-26T06:42:03.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:03.930 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-26T06:42:03.930 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-26T06:42:03.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.11 2024-02-26T06:42:04.270 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:03 smithi115 systemd[1]: Stopping Ceph osd.11 for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:04.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.270+0000 7fca29fbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:04.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.270+0000 7fca29fbf700 -1 osd.11 113 *** Got signal Terminated *** 2024-02-26T06:42:04.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.270+0000 7fca29fbf700 -1 osd.11 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:42:04.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.284+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:04.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.284+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:04.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.284+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:04.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.284+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:04.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.284+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:04.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.284+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:04.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:04 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:04.284+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.798616+0000 front 2024-02-26T06:41:34.798594+0000 (oldest deadline 2024-02-26T06:42:00.698483+0000) 2024-02-26T06:42:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:05.314+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:05.314+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:05.314+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:05.314+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:05.314+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:05.314+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:05.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:05 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:05.314+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.798616+0000 front 2024-02-26T06:41:34.798594+0000 (oldest deadline 2024-02-26T06:42:00.698483+0000) 2024-02-26T06:42:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:06.336+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:06.336+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:06.336+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:06.336+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:06.336+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:06.336+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:06.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:06 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:06.336+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.798616+0000 front 2024-02-26T06:41:34.798594+0000 (oldest deadline 2024-02-26T06:42:00.698483+0000) 2024-02-26T06:42:07.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:07.358+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:07.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:07.358+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:07.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:07.358+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:07.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:07.358+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:07.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:07.358+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:07.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:07.358+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:07.841 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:07 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:07.358+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.798616+0000 front 2024-02-26T06:41:34.798594+0000 (oldest deadline 2024-02-26T06:42:00.698483+0000) 2024-02-26T06:42:08.591 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:08.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-02-26T06:40:30.287344+0000 front 2024-02-26T06:40:30.287620+0000 (oldest deadline 2024-02-26T06:40:53.787302+0000) 2024-02-26T06:42:08.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:08.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6804 osd.1 since back 2024-02-26T06:40:37.288386+0000 front 2024-02-26T06:40:37.288202+0000 (oldest deadline 2024-02-26T06:41:03.188363+0000) 2024-02-26T06:42:08.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:08.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-02-26T06:40:47.190008+0000 front 2024-02-26T06:40:47.189976+0000 (oldest deadline 2024-02-26T06:41:12.490097+0000) 2024-02-26T06:42:08.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:08.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6812 osd.4 since back 2024-02-26T06:41:08.594457+0000 front 2024-02-26T06:41:08.594537+0000 (oldest deadline 2024-02-26T06:41:30.894417+0000) 2024-02-26T06:42:08.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:08.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-02-26T06:41:13.195376+0000 front 2024-02-26T06:41:13.195410+0000 (oldest deadline 2024-02-26T06:41:37.895372+0000) 2024-02-26T06:42:08.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:08.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-02-26T06:41:24.797833+0000 front 2024-02-26T06:41:24.797740+0000 (oldest deadline 2024-02-26T06:41:50.697512+0000) 2024-02-26T06:42:08.592 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:08 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11[62217]: 2024-02-26T06:42:08.332+0000 7fca225ab700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.80:6820 osd.7 since back 2024-02-26T06:41:34.798616+0000 front 2024-02-26T06:41:34.798594+0000 (oldest deadline 2024-02-26T06:42:00.698483+0000) 2024-02-26T06:42:10.077 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:09 smithi115 podman[75561]: 2024-02-26 06:42:09.820433899 +0000 UTC m=+5.697089921 container died ebb28a3bb5d1fc12f942498d63dca559678c6fe5964c57412a4d3faaade94567 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-26T06:42:10.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:10 smithi115 podman[75561]: 2024-02-26 06:42:10.185594654 +0000 UTC m=+6.062250690 container cleanup ebb28a3bb5d1fc12f942498d63dca559678c6fe5964c57412a4d3faaade94567 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-26T06:42:10.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:10 smithi115 bash[75561]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11 2024-02-26T06:42:10.782 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:10 smithi115 podman[75577]: 2024-02-26 06:42:10.453441632 +0000 UTC m=+0.629425965 container remove ebb28a3bb5d1fc12f942498d63dca559678c6fe5964c57412a4d3faaade94567 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-26T06:42:11.064 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:10 smithi115 podman[75664]: 2024-02-26 06:42:10.686448052 +0000 UTC m=+0.017690110 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:42:11.064 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:10 smithi115 podman[75664]: 2024-02-26 06:42:10.804250372 +0000 UTC m=+0.135492453 container create b8c7744a36931c468f58d0c172a20ad8c7eed2821df74bb35fc8dda642021761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2) 2024-02-26T06:42:11.064 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:10 smithi115 podman[75664]: 2024-02-26 06:42:10.989598144 +0000 UTC m=+0.320840198 container init b8c7744a36931c468f58d0c172a20ad8c7eed2821df74bb35fc8dda642021761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD) 2024-02-26T06:42:11.064 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:10 smithi115 podman[75664]: 2024-02-26 06:42:10.99283971 +0000 UTC m=+0.324081742 container start b8c7744a36931c468f58d0c172a20ad8c7eed2821df74bb35fc8dda642021761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11-deactivate, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:42:11.064 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:11 smithi115 podman[75664]: 2024-02-26 06:42:11.064586675 +0000 UTC m=+0.395828742 container attach b8c7744a36931c468f58d0c172a20ad8c7eed2821df74bb35fc8dda642021761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:42:11.341 INFO:journalctl@ceph.osd.11.smithi115.stdout:Feb 26 06:42:11 smithi115 podman[75664]: 2024-02-26 06:42:11.28358274 +0000 UTC m=+0.614824781 container died b8c7744a36931c468f58d0c172a20ad8c7eed2821df74bb35fc8dda642021761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-26T06:42:11.867 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@osd.11.service' 2024-02-26T06:42:11.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:11.908 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-26T06:42:11.908 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-26T06:42:11.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.a 2024-02-26T06:42:12.419 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:42:12 smithi049 systemd[1]: Stopping Ceph mds.a for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:12.854 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:42:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a[70818]: 2024-02-26T06:42:12.580+0000 7fbb51400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:12.854 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:42:12 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a[70818]: 2024-02-26T06:42:12.580+0000 7fbb51400700 -1 mds.a *** got signal Terminated *** 2024-02-26T06:42:15.046 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:42:14 smithi049 podman[98479]: 2024-02-26 06:42:14.708651398 +0000 UTC m=+2.316358245 container died 5b151ea5f4ef9cb79b05affb183b29091f21be7bc7c6a0c912262ddc74a7046e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:42:15.417 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:42:15 smithi049 podman[98479]: 2024-02-26 06:42:15.207092472 +0000 UTC m=+2.814799329 container cleanup 5b151ea5f4ef9cb79b05affb183b29091f21be7bc7c6a0c912262ddc74a7046e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:42:15.417 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:42:15 smithi049 bash[98479]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a 2024-02-26T06:42:15.669 INFO:journalctl@ceph.mds.a.smithi049.stdout:Feb 26 06:42:15 smithi049 podman[98504]: 2024-02-26 06:42:15.416872768 +0000 UTC m=+0.684305691 container remove 5b151ea5f4ef9cb79b05affb183b29091f21be7bc7c6a0c912262ddc74a7046e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-26T06:42:15.944 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.a.service' 2024-02-26T06:42:16.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:16.086 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-26T06:42:16.086 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-26T06:42:16.086 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.d 2024-02-26T06:42:16.669 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:42:16 smithi049 systemd[1]: Stopping Ceph mds.d for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:17.169 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:42:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d[69793]: 2024-02-26T06:42:16.845+0000 7f12d5c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:17.169 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:42:16 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d[69793]: 2024-02-26T06:42:16.845+0000 7f12d5c00700 -1 mds.d *** got signal Terminated *** 2024-02-26T06:42:19.669 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:42:19 smithi049 podman[98610]: 2024-02-26 06:42:19.257390185 +0000 UTC m=+2.675584835 container died 7ce57060d4c94145dc37d7e323470bdbae1af8f491278605ea32098fd383bf25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:42:20.021 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:42:19 smithi049 podman[98610]: 2024-02-26 06:42:19.802720538 +0000 UTC m=+3.220915225 container cleanup 7ce57060d4c94145dc37d7e323470bdbae1af8f491278605ea32098fd383bf25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-26T06:42:20.022 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:42:19 smithi049 bash[98610]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d 2024-02-26T06:42:20.419 INFO:journalctl@ceph.mds.d.smithi049.stdout:Feb 26 06:42:20 smithi049 podman[98624]: 2024-02-26 06:42:20.020112583 +0000 UTC m=+0.740190554 container remove 7ce57060d4c94145dc37d7e323470bdbae1af8f491278605ea32098fd383bf25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-26T06:42:20.573 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.d.service' 2024-02-26T06:42:20.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:20.709 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-26T06:42:20.709 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-26T06:42:20.709 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.g 2024-02-26T06:42:21.169 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:42:20 smithi049 systemd[1]: Stopping Ceph mds.g for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:21.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:42:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g[70433]: 2024-02-26T06:42:21.296+0000 7fc75f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:21.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:42:21 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g[70433]: 2024-02-26T06:42:21.296+0000 7fc75f400700 -1 mds.g *** got signal Terminated *** 2024-02-26T06:42:22.803 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:42:22 smithi049 podman[98729]: 2024-02-26 06:42:22.494947065 +0000 UTC m=+1.331797954 container died bde75ebc04e5afce76cf8c249704a381f27af90f8a783bc14ee49015da47d259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-26T06:42:23.169 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:42:22 smithi049 podman[98729]: 2024-02-26 06:42:22.914273375 +0000 UTC m=+1.751124258 container cleanup bde75ebc04e5afce76cf8c249704a381f27af90f8a783bc14ee49015da47d259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-26T06:42:23.170 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:42:22 smithi049 bash[98729]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g 2024-02-26T06:42:23.669 INFO:journalctl@ceph.mds.g.smithi049.stdout:Feb 26 06:42:23 smithi049 podman[98743]: 2024-02-26 06:42:23.248829892 +0000 UTC m=+0.737378735 container remove bde75ebc04e5afce76cf8c249704a381f27af90f8a783bc14ee49015da47d259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True) 2024-02-26T06:42:23.766 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.g.service' 2024-02-26T06:42:23.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:23.902 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-26T06:42:23.902 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-26T06:42:23.902 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.j 2024-02-26T06:42:24.419 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:42:24 smithi049 systemd[1]: Stopping Ceph mds.j for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:24.919 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:42:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j[69444]: 2024-02-26T06:42:24.433+0000 7fcb48e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:24.919 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:42:24 smithi049 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j[69444]: 2024-02-26T06:42:24.433+0000 7fcb48e00700 -1 mds.j *** got signal Terminated *** 2024-02-26T06:42:27.419 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:42:27 smithi049 podman[98849]: 2024-02-26 06:42:27.160966664 +0000 UTC m=+2.864857561 container died 2c863d7af9000c504a9d492dd23997034604615a448ba7633958633997a26500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=) 2024-02-26T06:42:27.860 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:42:27 smithi049 podman[98849]: 2024-02-26 06:42:27.592786704 +0000 UTC m=+3.296677627 container cleanup 2c863d7af9000c504a9d492dd23997034604615a448ba7633958633997a26500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-26T06:42:27.860 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:42:27 smithi049 bash[98849]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j 2024-02-26T06:42:28.169 INFO:journalctl@ceph.mds.j.smithi049.stdout:Feb 26 06:42:27 smithi049 podman[98863]: 2024-02-26 06:42:27.860133032 +0000 UTC m=+0.674749464 container remove 2c863d7af9000c504a9d492dd23997034604615a448ba7633958633997a26500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-j, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:42:28.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.j.service' 2024-02-26T06:42:28.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:28.552 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-26T06:42:28.552 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-26T06:42:28.552 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.b 2024-02-26T06:42:28.830 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:42:28 smithi080 systemd[1]: Stopping Ceph mds.b for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:29.088 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:42:28 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: 2024-02-26T06:42:28.848+0000 7f8f2cf1d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:29.088 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:42:28 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b[66926]: 2024-02-26T06:42:28.848+0000 7f8f2cf1d700 -1 mds.b *** got signal Terminated *** 2024-02-26T06:42:31.632 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:42:31 smithi080 podman[75342]: 2024-02-26 06:42:31.356236496 +0000 UTC m=+2.611801444 container died c493e60d4f9a3aeef2f62eef49dba280af207b79a7db6abac1bd2fcd83dc4cf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:42:32.080 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:42:31 smithi080 podman[75342]: 2024-02-26 06:42:31.732227171 +0000 UTC m=+2.987792121 container cleanup c493e60d4f9a3aeef2f62eef49dba280af207b79a7db6abac1bd2fcd83dc4cf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:42:32.080 INFO:journalctl@ceph.mds.b.smithi080.stdout:Feb 26 06:42:31 smithi080 bash[75342]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-b 2024-02-26T06:42:32.261 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.b.service' 2024-02-26T06:42:32.299 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:32.299 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-26T06:42:32.299 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-26T06:42:32.299 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.e 2024-02-26T06:42:32.830 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:42:32 smithi080 systemd[1]: Stopping Ceph mds.e for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:32.830 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:42:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e[66308]: 2024-02-26T06:42:32.626+0000 7feaf15a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:32.831 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:42:32 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e[66308]: 2024-02-26T06:42:32.626+0000 7feaf15a2700 -1 mds.e *** got signal Terminated *** 2024-02-26T06:42:35.621 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:42:35 smithi080 podman[75475]: 2024-02-26 06:42:35.344836707 +0000 UTC m=+2.824936440 container died 24588b772ecb9464dbf3e04fc26e80da7524401881e258df9f9b3bafceb1f8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-26T06:42:36.080 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:42:35 smithi080 podman[75475]: 2024-02-26 06:42:35.68695733 +0000 UTC m=+3.167057056 container cleanup 24588b772ecb9464dbf3e04fc26e80da7524401881e258df9f9b3bafceb1f8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T06:42:36.080 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:42:35 smithi080 bash[75475]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e 2024-02-26T06:42:36.080 INFO:journalctl@ceph.mds.e.smithi080.stdout:Feb 26 06:42:35 smithi080 podman[75489]: 2024-02-26 06:42:35.863101917 +0000 UTC m=+0.510078194 container remove 24588b772ecb9464dbf3e04fc26e80da7524401881e258df9f9b3bafceb1f8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-26T06:42:36.105 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.e.service' 2024-02-26T06:42:36.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:36.145 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-26T06:42:36.145 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-26T06:42:36.145 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.h 2024-02-26T06:42:36.473 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:36 smithi080 systemd[1]: Stopping Ceph mds.h for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:36.830 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h[66621]: 2024-02-26T06:42:36.472+0000 7f5a62e16700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:36.830 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:36 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h[66621]: 2024-02-26T06:42:36.472+0000 7f5a62e16700 -1 mds.h *** got signal Terminated *** 2024-02-26T06:42:39.712 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:39 smithi080 podman[75594]: 2024-02-26 06:42:39.354425294 +0000 UTC m=+2.988564158 container died 8469b681a6d0c308c9d51e9cce5b2dadf0df642f3d954183a4a008a0b318312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-26T06:42:39.975 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:39 smithi080 podman[75594]: 2024-02-26 06:42:39.791093327 +0000 UTC m=+3.425232191 container cleanup 8469b681a6d0c308c9d51e9cce5b2dadf0df642f3d954183a4a008a0b318312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS) 2024-02-26T06:42:39.975 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:39 smithi080 bash[75594]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h 2024-02-26T06:42:40.203 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.h.service' 2024-02-26T06:42:40.226 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:39 smithi080 podman[75608]: 2024-02-26 06:42:39.975733815 +0000 UTC m=+0.601912038 container remove 8469b681a6d0c308c9d51e9cce5b2dadf0df642f3d954183a4a008a0b318312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-26T06:42:40.227 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:40 smithi080 systemd[1]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.h.service: Deactivated successfully. 2024-02-26T06:42:40.227 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:40 smithi080 systemd[1]: Stopped Ceph mds.h for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:42:40.227 INFO:journalctl@ceph.mds.h.smithi080.stdout:Feb 26 06:42:40 smithi080 systemd[1]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.h.service: Consumed 2.935s CPU time. 2024-02-26T06:42:40.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:40.238 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-26T06:42:40.238 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-26T06:42:40.238 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.k 2024-02-26T06:42:40.552 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:42:40 smithi080 systemd[1]: Stopping Ceph mds.k for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:40.830 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:42:40 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k[65988]: 2024-02-26T06:42:40.553+0000 7fcbe2fa8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:40.830 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:42:40 smithi080 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k[65988]: 2024-02-26T06:42:40.553+0000 7fcbe2fa8700 -1 mds.k *** got signal Terminated *** 2024-02-26T06:42:43.577 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:42:43 smithi080 podman[75715]: 2024-02-26 06:42:43.258020024 +0000 UTC m=+2.803325363 container died b45a97587864e9b2149bfc9391c0b43c639deb0a63c40dfaecc498c2bf9e31ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:42:43.577 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:42:43 smithi080 podman[75715]: 2024-02-26 06:42:43.578016949 +0000 UTC m=+3.123322324 container cleanup b45a97587864e9b2149bfc9391c0b43c639deb0a63c40dfaecc498c2bf9e31ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-26T06:42:43.830 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:42:43 smithi080 bash[75715]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k 2024-02-26T06:42:43.830 INFO:journalctl@ceph.mds.k.smithi080.stdout:Feb 26 06:42:43 smithi080 podman[75730]: 2024-02-26 06:42:43.770703108 +0000 UTC m=+0.515952024 container remove b45a97587864e9b2149bfc9391c0b43c639deb0a63c40dfaecc498c2bf9e31ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-k, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:42:43.984 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.k.service' 2024-02-26T06:42:44.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:44.018 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-26T06:42:44.018 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-26T06:42:44.018 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.c 2024-02-26T06:42:44.340 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:42:44 smithi115 systemd[1]: Stopping Ceph mds.c for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:44.601 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:42:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:42:44.439+0000 7f1f1ca58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:44.601 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:42:44 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c[67240]: 2024-02-26T06:42:44.439+0000 7f1f1ca58700 -1 mds.c *** got signal Terminated *** 2024-02-26T06:42:47.605 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:42:47 smithi115 podman[75770]: 2024-02-26 06:42:47.349060396 +0000 UTC m=+3.146525366 container died cd39c9578aaed51fbdcc38546d24bbfc1f1ea8ee9261a011930477c9998959f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-26T06:42:47.864 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:42:47 smithi115 podman[75770]: 2024-02-26 06:42:47.696918969 +0000 UTC m=+3.494383949 container cleanup cd39c9578aaed51fbdcc38546d24bbfc1f1ea8ee9261a011930477c9998959f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:42:47.864 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:42:47 smithi115 bash[75770]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c 2024-02-26T06:42:48.119 INFO:journalctl@ceph.mds.c.smithi115.stdout:Feb 26 06:42:47 smithi115 podman[75795]: 2024-02-26 06:42:47.864498988 +0000 UTC m=+0.513064685 container remove cd39c9578aaed51fbdcc38546d24bbfc1f1ea8ee9261a011930477c9998959f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-26T06:42:48.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.c.service' 2024-02-26T06:42:48.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:48.162 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-26T06:42:48.163 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-26T06:42:48.163 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.f 2024-02-26T06:42:48.484 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:42:48 smithi115 systemd[1]: Stopping Ceph mds.f for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:48.840 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:42:48 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f[66928]: 2024-02-26T06:42:48.483+0000 7fc79011d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:48.840 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:42:48 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f[66928]: 2024-02-26T06:42:48.483+0000 7fc79011d700 -1 mds.f *** got signal Terminated *** 2024-02-26T06:42:51.625 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:42:51 smithi115 podman[75901]: 2024-02-26 06:42:51.35372739 +0000 UTC m=+2.976111842 container died 0eae69f6ee63d22e354534cf01728dfa2dfdd2a80132dd6b043b8264bb09b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-02-26T06:42:51.992 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:42:51 smithi115 podman[75901]: 2024-02-26 06:42:51.708210824 +0000 UTC m=+3.330595268 container cleanup 0eae69f6ee63d22e354534cf01728dfa2dfdd2a80132dd6b043b8264bb09b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T06:42:51.992 INFO:journalctl@ceph.mds.f.smithi115.stdout:Feb 26 06:42:51 smithi115 bash[75901]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-f 2024-02-26T06:42:52.132 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.f.service' 2024-02-26T06:42:52.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:52.168 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-26T06:42:52.169 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-26T06:42:52.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.i 2024-02-26T06:42:52.590 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:52 smithi115 systemd[1]: Stopping Ceph mds.i for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:52.591 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i[67552]: 2024-02-26T06:42:52.494+0000 7f7ddedf2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:52.591 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:52 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i[67552]: 2024-02-26T06:42:52.494+0000 7f7ddedf2700 -1 mds.i *** got signal Terminated *** 2024-02-26T06:42:55.610 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:55 smithi115 podman[76022]: 2024-02-26 06:42:55.353920216 +0000 UTC m=+2.968946128 container died c07093591ce275212ad4edef206a59bf285a4dcac29f7547fdad5e132c2344c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T06:42:55.870 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:55 smithi115 podman[76022]: 2024-02-26 06:42:55.694740436 +0000 UTC m=+3.309766350 container cleanup c07093591ce275212ad4edef206a59bf285a4dcac29f7547fdad5e132c2344c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-26T06:42:55.870 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:55 smithi115 bash[76022]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i 2024-02-26T06:42:56.103 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.i.service' 2024-02-26T06:42:56.127 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:55 smithi115 podman[76036]: 2024-02-26 06:42:55.870828977 +0000 UTC m=+0.512234655 container remove c07093591ce275212ad4edef206a59bf285a4dcac29f7547fdad5e132c2344c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-26T06:42:56.127 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:56 smithi115 systemd[1]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.i.service: Deactivated successfully. 2024-02-26T06:42:56.127 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:56 smithi115 systemd[1]: Stopped Ceph mds.i for a9ae7f00-d46d-11ee-95c3-87774f69a715. 2024-02-26T06:42:56.127 INFO:journalctl@ceph.mds.i.smithi115.stdout:Feb 26 06:42:56 smithi115 systemd[1]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.i.service: Consumed 9.508s CPU time. 2024-02-26T06:42:56.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:42:56.138 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-26T06:42:56.138 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-26T06:42:56.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.l 2024-02-26T06:42:56.448 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:42:56 smithi115 systemd[1]: Stopping Ceph mds.l for a9ae7f00-d46d-11ee-95c3-87774f69a715... 2024-02-26T06:42:56.841 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:42:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l[66616]: 2024-02-26T06:42:56.447+0000 7f5a9f91b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:42:56.841 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:42:56 smithi115 ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l[66616]: 2024-02-26T06:42:56.447+0000 7f5a9f91b700 -1 mds.l *** got signal Terminated *** 2024-02-26T06:43:00.439 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:43:00 smithi115 podman[76145]: 2024-02-26 06:43:00.187905185 +0000 UTC m=+3.834410803 container died 8283382f724e537d1ffd4d0ee81f6a4593ea5c3ae97d1fb1b2f266f95a332d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-26T06:43:00.798 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:43:00 smithi115 podman[76145]: 2024-02-26 06:43:00.539238467 +0000 UTC m=+4.185744074 container cleanup 8283382f724e537d1ffd4d0ee81f6a4593ea5c3ae97d1fb1b2f266f95a332d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:43:00.798 INFO:journalctl@ceph.mds.l.smithi115.stdout:Feb 26 06:43:00 smithi115 bash[76145]: ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715-mds-l 2024-02-26T06:43:00.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9ae7f00-d46d-11ee-95c3-87774f69a715@mds.l.service' 2024-02-26T06:43:00.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:43:00.950 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-26T06:43:00.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T06:43:01.306 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:43:06.557 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T06:43:06.748 INFO:teuthology.orchestra.run.smithi080.stdout:Deleting cluster with fsid: a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:43:09.288 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T06:43:09.479 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:43:11.870 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:43:11.991 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:43:12.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:43:12.055 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-26T06:43:12.057 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi049/crash 2024-02-26T06:43:12.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash -- . 2024-02-26T06:43:12.204 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T06:43:12.204 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2024-02-26T06:43:12.210 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi080/crash 2024-02-26T06:43:12.211 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash -- . 2024-02-26T06:43:12.257 INFO:teuthology.orchestra.run.smithi080.stderr:tar: /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T06:43:12.257 INFO:teuthology.orchestra.run.smithi080.stderr:tar: Error is not recoverable: exiting now 2024-02-26T06:43:12.258 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi115/crash 2024-02-26T06:43:12.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash -- . 2024-02-26T06:43:12.300 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T06:43:12.300 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-02-26T06:43:12.301 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-26T06:43:12.302 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-26T06:43:12.443 INFO:tasks.cephadm:Compressing logs... 2024-02-26T06:43:12.444 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T06:43:12.449 DEBUG:teuthology.orchestra.run.smithi080:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T06:43:12.452 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T06:43:12.481 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T06:43:12.481 INFO:teuthology.orchestra.run.smithi115.stderr:: No such file or directory 2024-02-26T06:43:12.481 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.l.log 2024-02-26T06:43:12.481 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T06:43:12.481 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.i.log 2024-02-26T06:43:12.482 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.11.log 2024-02-26T06:43:12.482 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.z.log 2024-02-26T06:43:12.482 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.i.log: /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.8.log 2024-02-26T06:43:12.482 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.5.log 2024-02-26T06:43:12.482 INFO:teuthology.orchestra.run.smithi115.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T06:43:12.482 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.f.log 2024-02-26T06:43:12.482 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T06:43:12.485 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.5.log: /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.f.log: 90.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T06:43:12.485 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log 2024-02-26T06:43:12.485 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T06:43:12.485 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.2.log 2024-02-26T06:43:12.486 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log: 94.6% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.l.log.gz 2024-02-26T06:43:12.486 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.c.log 2024-02-26T06:43:12.487 INFO:teuthology.orchestra.run.smithi080.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-26T06:43:12.488 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.2.log: 86.8% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T06:43:12.489 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T06:43:12.490 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T06:43:12.490 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.10.log 2024-02-26T06:43:12.490 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T06:43:12.490 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.b.log 2024-02-26T06:43:12.493 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.c.log: 92.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.z.log.gz 2024-02-26T06:43:12.493 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.c.log 2024-02-26T06:43:12.495 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.4.log 2024-02-26T06:43:12.495 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.1.log 2024-02-26T06:43:12.495 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.k.log 2024-02-26T06:43:12.495 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.h.log 2024-02-26T06:43:12.495 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.7.log 2024-02-26T06:43:12.495 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.e.log 2024-02-26T06:43:12.495 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.y.log 2024-02-26T06:43:12.495 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T06:43:12.496 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log 2024-02-26T06:43:12.496 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T06:43:12.496 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.b.log 2024-02-26T06:43:12.496 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.b.log: 94.4% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.k.log.gz 2024-02-26T06:43:12.499 INFO:teuthology.orchestra.run.smithi080.stderr: 92.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.y.log.gz 2024-02-26T06:43:12.505 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T06:43:12.550 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-26T06:43:12.550 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T06:43:12.551 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.6.log 2024-02-26T06:43:12.553 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.a.log 2024-02-26T06:43:12.554 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T06:43:12.556 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.g.log 2024-02-26T06:43:12.557 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.j.log 2024-02-26T06:43:12.559 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.3.log 2024-02-26T06:43:12.560 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.x.log 2024-02-26T06:43:12.561 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.0.log 2024-02-26T06:43:12.563 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T06:43:12.564 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log 2024-02-26T06:43:12.565 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log: 94.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.g.log.gz 2024-02-26T06:43:12.566 INFO:teuthology.orchestra.run.smithi049.stderr: 94.3% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.j.log.gz 2024-02-26T06:43:12.566 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T06:43:12.567 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.9.log 2024-02-26T06:43:12.568 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.d.log 2024-02-26T06:43:12.569 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.a.log 2024-02-26T06:43:12.569 INFO:teuthology.orchestra.run.smithi049.stderr: 86.8% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T06:43:12.575 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.d.log: /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.a.log: 94.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.d.log.gz 2024-02-26T06:43:12.575 INFO:teuthology.orchestra.run.smithi049.stderr: 94.3% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.a.log.gz 2024-02-26T06:43:12.595 INFO:teuthology.orchestra.run.smithi049.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T06:43:12.614 INFO:teuthology.orchestra.run.smithi080.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T06:43:12.620 INFO:teuthology.orchestra.run.smithi049.stderr: 83.7% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T06:43:12.699 INFO:teuthology.orchestra.run.smithi080.stderr: 83.5% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T06:43:12.701 INFO:teuthology.orchestra.run.smithi080.stderr: 90.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T06:43:12.701 INFO:teuthology.orchestra.run.smithi049.stderr: 90.1% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T06:43:12.702 INFO:teuthology.orchestra.run.smithi080.stderr: 86.8% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T06:43:12.929 INFO:teuthology.orchestra.run.smithi080.stderr: 90.9% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T06:43:13.015 INFO:teuthology.orchestra.run.smithi115.stderr: 93.9% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.f.log.gz 2024-02-26T06:43:13.166 INFO:teuthology.orchestra.run.smithi049.stderr: 90.8% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T06:43:13.178 INFO:teuthology.orchestra.run.smithi115.stderr: 91.7% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.c.log.gz 2024-02-26T06:43:13.182 INFO:teuthology.orchestra.run.smithi049.stderr: 90.0% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mgr.x.log.gz 2024-02-26T06:43:13.517 INFO:teuthology.orchestra.run.smithi080.stderr: 91.8% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.b.log.gz 2024-02-26T06:43:13.989 INFO:teuthology.orchestra.run.smithi080.stderr: 93.6% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.e.log.gz 2024-02-26T06:43:14.357 INFO:teuthology.orchestra.run.smithi080.stderr: 93.3% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.h.log.gz 2024-02-26T06:43:14.539 INFO:teuthology.orchestra.run.smithi115.stderr: 93.6% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.i.log.gz 2024-02-26T06:43:14.811 INFO:teuthology.orchestra.run.smithi115.stderr: 93.7% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.c.log.gz 2024-02-26T06:43:14.844 INFO:teuthology.orchestra.run.smithi049.stderr: 90.1% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mon.a.log.gz 2024-02-26T06:43:19.166 INFO:teuthology.orchestra.run.smithi080.stderr: 93.6% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-mds.b.log.gz 2024-02-26T06:43:57.469 INFO:teuthology.orchestra.run.smithi115.stderr: 91.3% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.8.log.gz 2024-02-26T06:44:05.617 INFO:teuthology.orchestra.run.smithi115.stderr: 91.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.11.log.gz 2024-02-26T06:44:06.249 INFO:teuthology.orchestra.run.smithi115.stderr: 91.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.5.log.gz 2024-02-26T06:44:09.556 INFO:teuthology.orchestra.run.smithi115.stderr: 91.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.2.log.gz 2024-02-26T06:44:09.557 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-02-26T06:44:09.558 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m57.091s 2024-02-26T06:44:09.558 INFO:teuthology.orchestra.run.smithi115.stderr:user 3m27.754s 2024-02-26T06:44:09.558 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m6.312s 2024-02-26T06:44:22.514 INFO:teuthology.orchestra.run.smithi049.stderr: 91.4% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.9.log.gz 2024-02-26T06:44:31.702 INFO:teuthology.orchestra.run.smithi080.stderr: 91.5% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.4.log.gz 2024-02-26T06:44:47.489 INFO:teuthology.orchestra.run.smithi080.stderr: 91.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.10.log.gz 2024-02-26T06:44:51.008 INFO:teuthology.orchestra.run.smithi049.stderr: 91.1% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.6.log.gz 2024-02-26T06:45:06.312 INFO:teuthology.orchestra.run.smithi080.stderr: 91.2% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.1.log.gz 2024-02-26T06:45:09.859 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.0.log.gz 2024-02-26T06:45:11.185 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.3.log.gz 2024-02-26T06:45:11.189 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-02-26T06:45:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:real 1m58.699s 2024-02-26T06:45:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:user 4m38.047s 2024-02-26T06:45:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m24.193s 2024-02-26T06:45:12.797 INFO:teuthology.orchestra.run.smithi080.stderr: 91.1% -- replaced with /var/log/ceph/a9ae7f00-d46d-11ee-95c3-87774f69a715/ceph-osd.7.log.gz 2024-02-26T06:45:12.798 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-02-26T06:45:12.799 INFO:teuthology.orchestra.run.smithi080.stderr:real 2m0.332s 2024-02-26T06:45:12.799 INFO:teuthology.orchestra.run.smithi080.stderr:user 3m49.911s 2024-02-26T06:45:12.799 INFO:teuthology.orchestra.run.smithi080.stderr:sys 0m7.447s 2024-02-26T06:45:12.799 INFO:tasks.cephadm:Archiving logs... 2024-02-26T06:45:12.799 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi049/log 2024-02-26T06:45:12.800 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T06:45:53.756 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi080/log 2024-02-26T06:45:53.757 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T06:46:35.018 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi115/log 2024-02-26T06:46:35.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T06:47:13.330 INFO:tasks.cephadm:Removing cluster... 2024-02-26T06:47:13.330 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 --force 2024-02-26T06:47:13.678 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:47:15.141 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 --force 2024-02-26T06:47:15.330 INFO:teuthology.orchestra.run.smithi080.stdout:Deleting cluster with fsid: a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:47:15.860 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9ae7f00-d46d-11ee-95c3-87774f69a715 --force 2024-02-26T06:47:16.055 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: a9ae7f00-d46d-11ee-95c3-87774f69a715 2024-02-26T06:47:16.581 INFO:tasks.cephadm:Removing cephadm ... 2024-02-26T06:47:16.582 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T06:47:16.640 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T06:47:16.658 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T06:47:16.674 INFO:tasks.cephadm:Teardown complete 2024-02-26T06:47:16.675 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-26T06:47:16.690 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-26T06:47:16.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T06:47:16.698 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T06:47:16.701 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T06:47:16.874 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-26T06:47:16.887 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-26T06:47:16.887 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:47:16.895 DEBUG:teuthology.orchestra.run.smithi080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:47:16.898 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:47:16.911 INFO:teuthology.orchestra.run.smithi080.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:47:16.912 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:47:16.915 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:47:16.915 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-02-26T06:47:16.916 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 +611us[ +611us] +/- 132ms 2024-02-26T06:47:16.916 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -406us[ -368us] +/- 50ms 2024-02-26T06:47:16.916 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +246us[ +281us] +/- 70ms 2024-02-26T06:47:16.916 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:47:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:47:16.956 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:47:16.956 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-02-26T06:47:16.957 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 +665us[ +665us] +/- 132ms 2024-02-26T06:47:16.957 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -378us[ -382us] +/- 50ms 2024-02-26T06:47:16.957 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +277us[ +273us] +/- 70ms 2024-02-26T06:47:16.957 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:47:16.972 INFO:teuthology.orchestra.run.smithi080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:47:16.972 INFO:teuthology.orchestra.run.smithi080.stdout:=============================================================================== 2024-02-26T06:47:16.972 INFO:teuthology.orchestra.run.smithi080.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +632us[ +649us] +/- 132ms 2024-02-26T06:47:16.972 INFO:teuthology.orchestra.run.smithi080.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -402us[ -385us] +/- 50ms 2024-02-26T06:47:16.972 INFO:teuthology.orchestra.run.smithi080.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +250us[ +250us] +/- 70ms 2024-02-26T06:47:16.973 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:47:16.978 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-26T06:47:16.989 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-26T06:47:16.990 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-26T06:47:17.001 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T06:47:17.286 DEBUG:teuthology.orchestra.run.smithi080:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T06:47:17.379 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T06:47:17.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:47:17.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:47:17.568 DEBUG:teuthology.orchestra.run.smithi080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:47:17.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:47:17.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:47:17.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:47:17.631 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-26T06:47:17.643 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-26T06:47:17.654 INFO:teuthology.task.internal:Duration was 2957.777241 seconds 2024-02-26T06:47:17.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-26T06:47:17.666 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-26T06:47:17.666 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T06:47:17.674 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T06:47:17.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T06:47:17.719 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:47:17.757 INFO:teuthology.orchestra.run.smithi080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:47:17.912 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:47:18.279 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-26T06:47:18.279 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-02-26T06:47:18.280 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T06:47:18.350 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi080.front.sepia.ceph.com 2024-02-26T06:47:18.350 DEBUG:teuthology.orchestra.run.smithi080:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T06:47:18.382 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-02-26T06:47:18.382 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T06:47:18.405 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-26T06:47:18.405 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T06:47:18.413 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T06:47:18.424 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T06:47:18.514 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-26T06:47:18.515 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T06:47:18.521 DEBUG:teuthology.orchestra.run.smithi080:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T06:47:18.523 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T06:47:19.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-26T06:47:19.124 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-26T06:47:19.124 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T06:47:19.232 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T06:47:19.273 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T06:47:19.301 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-26T06:47:19.313 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:47:19.321 DEBUG:teuthology.orchestra.run.smithi080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:47:19.323 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:47:19.359 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = core 2024-02-26T06:47:19.367 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-02-26T06:47:19.445 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-02-26T06:47:19.513 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:47:19.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:47:19.568 DEBUG:teuthology.orchestra.run.smithi080:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:47:19.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:47:19.582 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:47:19.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:47:19.596 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-26T06:47:19.610 INFO:teuthology.task.internal:Transferring archived files... 2024-02-26T06:47:19.611 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi049 2024-02-26T06:47:19.611 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T06:47:19.739 DEBUG:teuthology.misc:Transferring archived files from smithi080:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi080 2024-02-26T06:47:19.739 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T06:47:19.777 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421/remote/smithi115 2024-02-26T06:47:19.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T06:47:19.817 INFO:teuthology.task.internal:Removing archive directory... 2024-02-26T06:47:19.817 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T06:47:19.825 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T06:47:19.827 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T06:47:19.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-26T06:47:19.893 INFO:teuthology.task.internal:Not uploading archives. 2024-02-26T06:47:19.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-26T06:47:19.904 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-26T06:47:19.905 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T06:47:19.913 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T06:47:19.915 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T06:47:19.927 INFO:teuthology.orchestra.run.smithi080.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 06:47 /home/ubuntu/cephtest 2024-02-26T06:47:19.929 INFO:teuthology.orchestra.run.smithi115.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 06:47 /home/ubuntu/cephtest 2024-02-26T06:47:19.967 INFO:teuthology.orchestra.run.smithi049.stdout: 393590 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 06:47 /home/ubuntu/cephtest 2024-02-26T06:47:19.971 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-26T06:47:19.983 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-26T06:47:20.008 INFO:teuthology.nuke:Checking targets against current locks 2024-02-26T06:47:20.032 DEBUG:teuthology.nuke:shortname: smithi049 2024-02-26T06:47:20.033 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T06:47:20.057 DEBUG:teuthology.nuke:shortname: smithi080 2024-02-26T06:47:20.057 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T06:47:20.062 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:46:04.186119', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrDu0evKNPSsjtnw5GLpAqIRSFCiLkfZE6SMlt9B9o8F6Ou+2RhSpooORwsJMkfew7OsrufFf5jwsomNr6VMrs='} 2024-02-26T06:47:20.080 DEBUG:teuthology.nuke:shortname: smithi115 2024-02-26T06:47:20.081 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T06:47:20.087 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:46:04.184874', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMaAGpLNMrVubKGPygxK0ILlx9L8LVuxUCwiomS/0g/y5jedb+QLejRwrgvB/M60iRatlQ9eCV5ouV5fo/CLULM='} 2024-02-26T06:47:20.105 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:46:04.187169', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLuu6LRYtl16aE5cYvFgLTTTUrx+Bph3hXT94rBMJ2R07130X7x2/ieiMX9PxPQTLULqfn443KJuh6MrPvxex98='} 2024-02-26T06:47:20.117 INFO:teuthology.orchestra.console.smithi049:Power off 2024-02-26T06:47:20.117 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T06:47:20.138 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2024-02-26T06:47:20.139 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:20.153 INFO:teuthology.orchestra.console.smithi080:Power off 2024-02-26T06:47:20.156 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T06:47:20.171 INFO:teuthology.orchestra.console.smithi115:Power off 2024-02-26T06:47:20.171 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T06:47:20.179 DEBUG:teuthology.orchestra.console.smithi080:power off output: Chassis Power Control: Down/Off 2024-02-26T06:47:20.179 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:20.186 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-02-26T06:47:20.192 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off 2024-02-26T06:47:20.192 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:20.202 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is on 2024-02-26T06:47:20.215 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-02-26T06:47:24.188 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:24.203 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:24.215 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:24.299 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-02-26T06:47:24.313 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is on 2024-02-26T06:47:24.325 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-02-26T06:47:28.303 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:28.314 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:28.326 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:28.415 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2024-02-26T06:47:28.424 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is off 2024-02-26T06:47:28.438 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-02-26T06:47:28.516 INFO:teuthology.orchestra.console.smithi049:Power off completed 2024-02-26T06:47:28.524 INFO:teuthology.orchestra.console.smithi080:Power off completed 2024-02-26T06:47:32.443 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:47:32.553 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off 2024-02-26T06:47:32.654 INFO:teuthology.orchestra.console.smithi115:Power off completed 2024-02-26T06:47:32.759 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} duration: 2957.777240753174 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_vshankar@teuthology status: fail success: false 2024-02-26T06:47:32.760 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T06:47:32.855 INFO:teuthology.run:FAIL